tyler.durden/snappy-java
1
0
Fork 0
mirror of https://github.com/xerial/snappy-java.git synced 2025-07-27 16:04:16 +02:00
Code Issues Packages Projects Releases Wiki Activity

Fix doc

Browse Source
This commit is contained in:
xerial 2017-01-19 23:29:17 -08:00
parent 9c89e5c3af
commit 5898066ebd
4 changed files with 146 additions and 152 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

117
BUILD.md Executable file
View File

@ -0,0 +1,117 @@
# Notes for building snappy-java
snappy-java supports Windows, Mac OS X, Linux (x86, x86_64, arm, etc...). If your platform is not supported, you need to build native libraries by yourself.
## Requited Tools
- Java 7 or higher
- Maven3 (mvn)
- GNU make, autotools
## Building snappy-java
To build jar file of snappy-java, type:
$ make
A native library for your machine environment and a jar package target/snappy-java-(version).jar are produced in the target folder.
### Rebuild the native library for your platform
```
$ make clean-native native
```
## Platform specific tips
After snappy-java 1.1.3, we are using docker images of cross compilers. So no longer need to build native libraries by actually running the target OS.
The following notes are obsolete, but preserved here for future references.
### Windows (32/64-bit)
* GNU make
* And also tar, curl, cp, rm, grep commands are needed. (I use Cygwin and MinGW for building snappy-java in Windows)
### Windows (32-bit only)
* Install MinGW http://www.mingw.org/
* Set PATH to the following command in MinGW package
- mingw32-g++
- strip
To build x86 (32bit) dll under 64-bit Windows, use "make win32" target.
### Windows (64-bit only)
* Download MinGW-w64 http://sourceforge.net/projects/mingw-w64/
* Set PATH to the following commands in the downloaded archive:
- x86_64-w64-mingw32-g++
- x86_64-w64-mingw32-strip
NOTICE: Do not use the Cygwin version of MinGW-w64. It fails to build assemblies for 64bit environment.
### Linux (32/64-bit)
* gcc-4.5.x or higher is necessary because snappy-java uses -static-libstdc++ option. It is possible to use gcc-4.3.x but a dependency to libstdc++ remains in the generated jar file; That means if another version of libstdc++ is used, snappy-java might not work correctly.
* You can build 32-bit native library with 64-bit Linux machine (do make linux32)
### Mac
* Install gcc, make, etc. included in Mac OS X install disk. (X Code). And also intall libtool:
```
$ brew install libtool
```
## Building Linux x86\_64 binary
(obsolete: snappy-java now uses a docker image `xerial/centos5-linux-x86_86-pic` which contains g++ built with `-fPIC` option. )
snappy-java tries to static link libstdc++ to increase the availability for various Linux versions. However, standard distributions of 64-bit Linux OS rarely provide libstdc++ compiled with `-fPIC` option. I currently uses custom g++, compiled as follows:
```
$ cd work
$ wget (gcc-4.8.3 source)
$ tar xvfz (gcc-4.8.3.tar.gz)
$ cd gcc-4.8.3
$ ./contrib/download_prerequisites
$ cd ..
$ mkdir objdir
$ cd objdir
$ ../gcc-4.8.3/configure --prefix=$HOME/local/gcc-4.8.3 CXXFLAGS=-fPIC CFLAGS=-fPIC --enable-languages=c,c++
$ make
$ make install
```
This g++ build enables static linking of libstdc++. For more infomation on building GCC, see GCC's home page.
## Building Linux s390/s390x binaries
Older snapshots of snappy contain a buggy config.h.in that does not work properly on some big-endian platforms like Linux on IBM z (s390/s390x). Building snappy-java on s390/s390x requires fetching the snappy source from GitHub, and processing the source with autoconf to obtain a usable config.h. On a RHEL s390x system, these steps produced a working 64-bit snappy-java build (the process should be similar for other distributions):
$ sudo yum install java-1.7.1-ibm-devel libstdc++-static-devel
$ export JAVA_HOME=/usr/lib/jvm/java-1.7.1-ibm-1.7.1.2.10-1jpp.3.el7_0.s390x
$ make USE_GIT=1 GIT_REPO_URL=https://github.com/google/snappy.git GIT_SNAPPY_BRANCH=master IBM_JDK_7=1
## Activating SSE2/AVX2 instructions in BitShuffle
The most of the native libraries that snappy-java contains disable SSE2/AVX2 instructions in terms of portability (SSE2 is enabled only in Linux/x86_64 platforms). To enable AVX2 instructions, you need to compile as follows:
$ make CXXFLAGS_BITSHUFFLE=-mavx2 # -msse2 for SSE2 instructions
## Cross-compiling for other platforms (obsolete)
The Makefile contains rules for cross-compiling the native library for other platforms so that the snappy-java JAR can support multiple platforms. For example, to build the native libraries for x86 Linux, x86 and x86-64 Windows, and soft- and hard-float ARM:
$ make linux32 win32 win64 linux-arm linux-armhf linux-aarch64
If you append `snappy` to the line above, it will also build the native library for the current platform and then build the snappy-java JAR (containing all native libraries built so far).
Of course, you must first have the necessary cross-compilers and development libraries installed for each target CPU and OS. For example, on Ubuntu 12.04 for x86-64, install the following packages for each target:
* linux32: `sudo apt-get install g++-multilib libc6-dev-i386 lib32stdc++6`
* win32: `sudo apt-get install g++-mingw-w64-i686`
* win64: `sudo apt-get install g++-mingw-w64-x86-64`
* arm: `sudo apt-get install g++-arm-linux-gnueabi`
* armhf: `sudo apt-get install g++-arm-linux-gnueabihf`
* aarch64: `sudo apt-get install g++-aarch64-linux`
Unfortunately, cross-compiling for Mac OS X is not currently possible; you must compile within OS X.
If you are using Mac and openjdk7 (or higher), use the following option:
$ make native LIBNAME=libsnappyjava.dylib

78
INSTALL
View File

@ -1,78 +0,0 @@
[Installation note of snappy-java]
If your OS platform is one of the Windows/Linux/Mac OS X (32/64 bit Intel CPUs), the installation process described here is unnecessary. Simply put snappy-java-(version).jar to your Java classpath. If your OS is not listed above, create your own snappy-java-(version).jar file as follows.
= Required tools for building snappy-java =
[For all platforms]
* Java 6 (JDK1.6) http://java.sun.com/
- set JAVA_HOME environment variable to the Java installation folder (e.g. JAVA_HOME=C:/Program Files/Java/jdk1.6.0_24 in Windows)
* Maven 3.x http://maven.apache.org/
- Check mvn command can be used from your command line.
[Windows (32/64-bit)]
* GNU make
* And also tar, curl, cp, rm, grep commands are needed. (I use Cygwin and MinGW for building snappy-java in Windows)
[Windows (32-bit only)]
* Install MinGW http://www.mingw.org/
* Set PATH to the following command in MinGW package
- mingw32-g++
- strip
To build x86 (32bit) dll under 64-bit Windows, use "make win32" target.
[Windows (64-bit only)]
* Download MinGW-w64 http://sourceforge.net/projects/mingw-w64/
* Set PATH to the following commands in the downloaded archive:
- x86_64-w64-mingw32-g++
- x86_64-w64-mingw32-strip
NOTICE: Do not use the Cygwin version of MinGW-w64. It fails to build assemblies for 64bit environment.
[Linux (32/64-bit)]
* gcc-4.5.x or higher is necessary because snappy-java uses -static-libstdc++ option. It is possible to use gcc-4.3.x but a dependency to libstdc++ remains in the generated jar file; That means if another version of libstdc++ is used, snappy-java might not work correctly.
* You can build 32-bit native library with 64-bit Linux machine (do make linux32)
[Mac]
* Install gcc, make, etc. included in Mac OS X install disk. (X Code)
* Install mercurial using Mac Ports http://www.macports.org/
= Building snappy-java =
To build jar file of snappy-java, type:
$ make
A native library for your machine environment and a jar package target/snappy-java-(version).jar are produced in the target folder.
= Building only the native library =
$ make native
= Rebuild the native library for your platform =
$ make clean-native native
= Using system installed libsnappyjava (or snappyjava.dll) =
Set org.xerial.snappy.use.systemlib system property to true:
java -Djava.library.path=(path to the installed snappyjava lib) -Dorg.xerial.snappy.use.systemlib=true ...
With this setting snappy-java does not use bundled native libraries. Insted it tries to load native library installed at the path specified in java.library.path.
= Configure snappy-java using property file =
Prepare org-xerial-snappy.properties file (under the root path of your library) in Java's property file format.
Here is a list of the available properties:
* org.xerial.snappy.lib.path (directory containing a snappyjava's native library)
* org.xerial.snappy.lib.name (library file name)
* org.xerial.snappy.tempdir (temporary directory to extract a native library bundled in snappy-java)
* org.xerial.snappy.use.systemlib (if this value is true, use system installed libsnappyjava.so looking the path specified by java.library.path)

2
Makefile
View File

@ -146,7 +146,7 @@ $(TARGET)/$(snappy-jar-version).jar:
test: $(NATIVE_DLL) test: $(NATIVE_DLL)
$(SBT) test $(SBT) test
DOCKER_RUN_OPTS=--rm DOCKER_RUN_OPTS:=--rm
win32: jni-header win32: jni-header
./docker/dockcross-windows-x86 -a $(DOCKER_RUN_OPTS) bash -c 'make clean-native native CROSS_PREFIX=i686-w64-mingw32.static- OS_NAME=Windows OS_ARCH=x86' ./docker/dockcross-windows-x86 -a $(DOCKER_RUN_OPTS) bash -c 'make clean-native native CROSS_PREFIX=i686-w64-mingw32.static- OS_NAME=Windows OS_ARCH=x86'

101
README.md
View File

@ -1,4 +1,6 @@
The snappy-java is a Java port of the snappy snappy-java
===
snappy-java is a Java port of the snappy
<http://code.google.com/p/snappy/>, a fast C++ compresser/decompresser developed by Google. <http://code.google.com/p/snappy/>, a fast C++ compresser/decompresser developed by Google.
## Features ## Features
@ -94,83 +96,11 @@ If you have snappy-java-(VERSION).jar in the current directory, use `-classpath`
$ javac -classpath ".:snappy-java-(VERSION).jar" Sample.java # in Mac or Linux $ javac -classpath ".:snappy-java-(VERSION).jar" Sample.java # in Mac or Linux
## Public discussion group ## Public discussion group
Post bug reports or feature request to the Issue Tracker: <https://github.com/xerial/snappy-java/issues> Post bug reports or feature request to the Issue Tracker: <https://github.com/xerial/snappy-java/issues>
Public discussion forum is here: [Xerial Public Discussion Group](http://groups.google.com/group/xerial?hl=en) Public discussion forum is here: [Xerial Public Discussion Group](http://groups.google.com/group/xerial?hl=en)
## Building from the source code
See the [installation instruction](https://github.com/xerial/snappy-java/blob/develop/INSTALL). Building from the source code is an option when your OS platform and CPU architecture is not supported. To build snappy-java, you need Git, JDK (1.6 or higher), g++ compiler (mingw in Windows) etc.
$ git clone https://github.com/xerial/snappy-java.git
$ cd snappy-java
$ make
When building on Solaris use
$ gmake
A file `target/snappy-java-$(version).jar` is the product additionally containing the native library built for your platform.
## Building Linux x86\_64 binary
snappy-java tries to static link libstdc++ to increase the availability for various Linux versions. However, standard distributions of 64-bit Linux OS rarely provide libstdc++ compiled with `-fPIC` option. I currently uses custom g++, compiled as follows:
```
$ cd work
$ wget (gcc-4.8.3 source)
$ tar xvfz (gcc-4.8.3.tar.gz)
$ cd gcc-4.8.3
$ ./contrib/download_prerequisites
$ cd ..
$ mkdir objdir
$ cd objdir
$ ../gcc-4.8.3/configure --prefix=$HOME/local/gcc-4.8.3 CXXFLAGS=-fPIC CFLAGS=-fPIC --enable-languages=c,c++
$ make
$ make install
```
This g++ build enables static linking of libstdc++. For more infomation on building GCC, see GCC's home page.
## Building Linux s390/s390x binaries
Older snapshots of snappy contain a buggy config.h.in that does not work properly on some big-endian platforms like Linux on IBM z (s390/s390x). Building snappy-java on s390/s390x requires fetching the snappy source from GitHub, and processing the source with autoconf to obtain a usable config.h. On a RHEL s390x system, these steps produced a working 64-bit snappy-java build (the process should be similar for other distributions):
$ sudo yum install java-1.7.1-ibm-devel libstdc++-static-devel
$ export JAVA_HOME=/usr/lib/jvm/java-1.7.1-ibm-1.7.1.2.10-1jpp.3.el7_0.s390x
$ make USE_GIT=1 GIT_REPO_URL=https://github.com/google/snappy.git GIT_SNAPPY_BRANCH=master IBM_JDK_7=1
## Activating SSE2/AVX2 instructions in BitShuffle
The most of the native libraries that snappy-java contains disable SSE2/AVX2 instructions in terms of portability (SSE2 is enabled only in Linux/x86_64 platforms). To enable AVX2 instructions, you need to compile as follows:
$ make CXXFLAGS_BITSHUFFLE=-mavx2 # -msse2 for SSE2 instructions
## Cross-compiling for other platforms
The Makefile contains rules for cross-compiling the native library for other platforms so that the snappy-java JAR can support multiple platforms. For example, to build the native libraries for x86 Linux, x86 and x86-64 Windows, and soft- and hard-float ARM:
$ make linux32 win32 win64 linux-arm linux-armhf linux-aarch64
If you append `snappy` to the line above, it will also build the native library for the current platform and then build the snappy-java JAR (containing all native libraries built so far).
Of course, you must first have the necessary cross-compilers and development libraries installed for each target CPU and OS. For example, on Ubuntu 12.04 for x86-64, install the following packages for each target:
* linux32: `sudo apt-get install g++-multilib libc6-dev-i386 lib32stdc++6`
* win32: `sudo apt-get install g++-mingw-w64-i686`
* win64: `sudo apt-get install g++-mingw-w64-x86-64`
* arm: `sudo apt-get install g++-arm-linux-gnueabi`
* armhf: `sudo apt-get install g++-arm-linux-gnueabihf`
* aarch64: `sudo apt-get install g++-aarch64-linux`
Unfortunately, cross-compiling for Mac OS X is not currently possible; you must compile within OS X.
If you are using Mac and openjdk7 (or higher), use the following option:
$ make native LIBNAME=libsnappyjava.dylib
## For developers ## For developers
snappy-java uses sbt (simple build tool for Scala) as a build tool. Here is a simple usage snappy-java uses sbt (simple build tool for Scala) as a build tool. Here is a simple usage
@ -191,6 +121,19 @@ $ ./sbt -Dloglevel=debug
For the details of sbt usage, see my blog post: [Building Java Projects with sbt](http://xerial.org/blog/2014/03/24/sbt/) For the details of sbt usage, see my blog post: [Building Java Projects with sbt](http://xerial.org/blog/2014/03/24/sbt/)
### Building from the source code
See the [build instruction](https://github.com/xerial/snappy-java/blob/master/BUILD.md). Building from the source code is an option when your OS platform and CPU architecture is not supported. To build snappy-java, you need Git, JDK (1.6 or higher), g++ compiler (mingw in Windows) etc.
$ git clone https://github.com/xerial/snappy-java.git
$ cd snappy-java
$ make
When building on Solaris, use `gmake`:
$ gmake
A file `target/snappy-java-$(version).jar` is the product additionally containing the native library built for your platform.
## Miscellaneous Notes ## Miscellaneous Notes
### Using snappy-java with Tomcat 6 (or higher) Web Server ### Using snappy-java with Tomcat 6 (or higher) Web Server
@ -198,3 +141,15 @@ Simply put the snappy-java's jar to WEB-INF/lib folder of your web application.
---- ----
Snappy-java is developed by [Taro L. Saito](http://www.xerial.org/leo). Twitter [@taroleo](http://twitter.com/#!/taroleo) Snappy-java is developed by [Taro L. Saito](http://www.xerial.org/leo). Twitter [@taroleo](http://twitter.com/#!/taroleo)
### Configure snappy-java using property file
Prepare org-xerial-snappy.properties file (under the root path of your library) in Java's property file format.
Here is a list of the available properties:
* org.xerial.snappy.lib.path (directory containing a snappyjava's native library)
* org.xerial.snappy.lib.name (library file name)
* org.xerial.snappy.tempdir (temporary directory to extract a native library bundled in snappy-java)
* org.xerial.snappy.use.systemlib (if this value is true, use system installed libsnappyjava.so looking the path specified by java.library.path)