Re: [dev] [sbase] [PATCH] Make grep more memory-efficient

From: Dimitris Papastamos <sin_AT_2f30.org>
Date: Mon, 12 May 2014 10:39:04 +0100

On Mon, May 12, 2014 at 12:36:12AM +0100, Dimitris Papastamos wrote:
> On Mon, May 12, 2014 at 12:34:36AM +0200, FRIGN wrote:
> > Hello,
> >
> > previously, we allocated one regex_t for each pattern found.
> > This gets pretty nasty once the tree of patterns grows.
> > Reusing one regex_t and regfreeing it after each use inside grep()
> > implies just one simultaneous element in memory and a drastic decrease
> > in memory-consumption.
> >
> > Please let me know what you think!
>
> Applied, thanks.
>
> Note that I am not aware of real-life cases where so many -e options
> are set in order for this to make a "drastic" decrease in memory consumption.
>
> I like the code though and it feels simpler.

BTW I forgot to mention, the initial thought was to pre-compile the regexes
once and then use the compiled one for all matches.

The memory-decrease is constant really. You are still at O(n) in
regards to the number of search-patterns.

It seems that with your patch, on large inputs it seems that the it is approximately
3 times slower than the previous patch.

We should probably re-think our options here.
Received on Mon May 12 2014 - 11:39:04 CEST