Re: [patch 1/3] crypto: Add a zlib crypto module

On Fri, Aug 29, 2008 at 01:41:59PM +0000, Geert Uytterhoeven wrote:
From: Geert Uytterhoeven <Geert.Uytterhoeven@xxxxxxxxxxx>

Add a (de)compression module for the "zlib" format using the crypto API.

I think we can safely conclude that our current compression
interface sucks for what you're trying to achieve :)

While both the "zlib" and "deflate" crypto modules are implemented on top of
the zlib library, they differ in the following aspects:
- The "deflate" crypto module (used by IPSec and UBIFS) does not support
partial decompression, i.e. all compressed data has to be passed at once.
The "zlib" crypto module does support partial decompression;
zlib_decompress() will return -EAGAIN if not all compressed data has been
passed.
- The deflate crypto module uses the raw deflate data format (zlib is
initialized with a windowBits parameter of -DEFLATE_DEF_WINBITS = -11),
while e.g. squashfs and axfs use the zlib data format, with the default
windowBits parameter DEF_WBITS = 15.
Both parameters are incompatible with each other due to the different data
formats, as indicated by the sign of the windowbits parameter.
The absolute value of this parameter is the base two logarithm of the
maximum window size, and larger values are backwards compatible with
smaller values (as far as decompression is concerned).

Therefore I suggest that we change the interface rather than
mutilate the implementations.

So here are a few things we should add:

1) Separate alloc functions for comp/decomp to avoid the memory
wastage you've identified.

2) Provide parameters to these alloc functions through an opaque
pointer. The format of the paramters will be determined by the
name of the algorithm, i.e., if you want to change the parameters
then you should change the name as well (e.g., deflate => deflate2).

This removes the need to dupliate the implementation just because
you want 15 instead of -11.

3) Provide init/update/final (one set each for comp and decomp)
functions similar to crypto_hash, in addition to the current
comp/decomp functions.

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@xxxxxxxxxxxxxxxxxxx>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: How to handle a HTTP::Request with gzip, deflate headers
    ... When checking the headers, Amazon adds the deflate ... What I found out is that they actually output the wrong format here as ... the zlib FAQ to avoid this misunderstanding. ... I'll see if I can hack libwww-perl to retry decoding with the ...
    (comp.lang.perl.misc)
  • [patch 1/3] crypto: Add a zlib crypto module
    ... Add a compression module for the "zlib" format using the crypto API. ... The deflate crypto module uses the raw deflate data format (zlib is ...
    (Linux-Kernel)
  • [PATCH 0/5] Partial (de)compression Crypto API
    ... CRYPTO API, and modifies SquashFS 4 to use this: ... Make SquashFS 4 use the new pcomp crypto interface ... Create a new "zlib" pcomp module instead of modifying the existing ...
    (Linux-Kernel)
  • Re: file as a directory
    ... the kernel. ... compared to .gz format, .tar is much simpler, I guess. ... > needs all that - userland doesn't. ... if we could rely on zlib. ...
    (Linux-Kernel)
  • Re: Get your hands dirty: Help bootstrap Ruby on Windows.
    ... unseekable files like stdin but works on gzipped files by using zlib). ... MinGW packages are distributed in tar.gz format (which I guess map ... which will still need the external tool to uncompress. ... can take a bit slow using a pure-ruby implementation. ...
    (comp.lang.ruby)