Add README detail around file format compatibility
This commit is contained in:
parent
73c67c7030
commit
b40dffc4b9
13
README.md
13
README.md
|
@ -83,9 +83,20 @@ Stream-based compressor/decompressor `SnappyOutputStream`/`SnappyInputStream` ar
|
||||||
* See also [Javadoc API](https://oss.sonatype.org/service/local/repositories/releases/archive/org/xerial/snappy/snappy-java/1.1.3-M1/snappy-java-1.1.3-M1-javadoc.jar/!/index.html)
|
* See also [Javadoc API](https://oss.sonatype.org/service/local/repositories/releases/archive/org/xerial/snappy/snappy-java/1.1.3-M1/snappy-java-1.1.3-M1-javadoc.jar/!/index.html)
|
||||||
|
|
||||||
#### Compatibility Notes
|
#### Compatibility Notes
|
||||||
* `SnappyOutputStream` and `SnappyInputStream` use `[magic header:16 bytes]([block size:int32][compressed data:byte array])*` format. You can read the result of `Snappy.compress` with `SnappyInputStream`, but you cannot read the compressed data generated by `SnappyOutputStream` with `Snappy.uncompress`. Here is the data format compatibility matrix:
|
|
||||||
|
The original Snappy format definition did not define a file format. It later added
|
||||||
|
a "framing" format to define a file format, but by this point major software was
|
||||||
|
already using an industry standard instead -- represented in this library by the
|
||||||
|
`SnappyOutputStream` and `SnappyInputStream` methods.
|
||||||
|
|
||||||
|
For interoperability with other libraries, check that compatible formats are used.
|
||||||
|
Note that not all libraries support all variants.
|
||||||
|
|
||||||
|
* `SnappyOutputStream` and `SnappyInputStream` use `[magic header:16 bytes]([block size:int32][compressed data:byte array])*` format. You can read the result of `Snappy.compress` with `SnappyInputStream`, but you cannot read the compressed data generated by `SnappyOutputStream` with `Snappy.uncompress`.
|
||||||
* `SnappyHadoopCompatibleOutputStream` does not emit a file header but write out the current block size as a preemble to each block
|
* `SnappyHadoopCompatibleOutputStream` does not emit a file header but write out the current block size as a preemble to each block
|
||||||
|
|
||||||
|
#### Data format compatibility matrix:
|
||||||
|
|
||||||
| Write\Read | `Snappy.uncompress` | `SnappyInputStream` | `SnappyFramedInputStream` | `org.apache.hadoop.io.compress.SnappyCodec` |
|
| Write\Read | `Snappy.uncompress` | `SnappyInputStream` | `SnappyFramedInputStream` | `org.apache.hadoop.io.compress.SnappyCodec` |
|
||||||
| --------------- |:-------------------:|:------------------:|:-----------------------:|:-------------------------------------------:|
|
| --------------- |:-------------------:|:------------------:|:-----------------------:|:-------------------------------------------:|
|
||||||
| `Snappy.compress` | ok | ok | x | x |
|
| `Snappy.compress` | ok | ok | x | x |
|
||||||
|
|
Loading…
Reference in New Issue