Re: [hackers] [quark][PATCH] Add support for precomputed compression

From: Guy Sviry <sviryguy_AT_gmail.com>
Date: Tue, 10 Mar 2020 00:00:20 +0200

First off, I must admit I misunderstood this mailing list's purpose:
I thought this is where we discuss misc patches that are not meant to be
merged into the mainline.

That said, I believe that my patch easily covers compression for the
typical static website deployment, where:
- dirlists are turned off
- large files that require download resumption are already compressed
  (media/tarballs)

This is the reason why I think an online-approach would be more
> suitable (i.e. compressing the stream on the fly during sending, which
> would not discriminate against even range requests or dirlists).
> Gzip is simple enough and modern servers can handle the load.


I agree that an online approach to compression is ideal, it's just I
couldn't
find a simple gzip implementation to power that. Given that we prefer not
to malloc in runtime, things are even trickier.

We could write it ourselves, but even if the implementation is trivial, it
won't be minimalist. I think resorting to zlib is ok in this case. Even
then,
it really might just not worth the effort (the coding effort, that is).

Another, bit crazy approach: Spawning a `gzip -n -` subprocess, using
its stdin for the http handlers, and setting the original fd as gzip's
stdout.

Then we get universal, outsourced compression for relatively little amount
of code.

The only question left now is how to get those exes into the chroot..

Thanks for your feedback!

On Mon, Mar 9, 2020 at 9:55 AM Laslo Hunhold <dev_AT_frign.de> wrote:

> On Mon, 9 Mar 2020 08:30:23 +0100
> Laslo Hunhold <dev_AT_frign.de> wrote:
>
> Dear Guy,
>
> > Even if we consider the "requirement" that we don't want mallocs
> > during serving, I think LZ77 is trivial enough to implement (with a
> > running dictionary and with a CRC to turn it into a DEFLATE-payload)
> > such that we can satisfy these constraints.
>
> please excuse my mistake: Of course I meant a gzip-payload. DEFLATE can
> also be used for HTTP 1.1, but is, I think, too complex to consider it
> as our own implementation.
>
> With best regards
>
> Laslo
>
>
Received on Mon Mar 09 2020 - 23:00:20 CET