Re: [dev] ii: how to process out in a pipeline and still page with less

From: Markus Wichmann <nullplan_AT_gmx.net>
Date: Sat, 28 May 2022 19:58:40 +0200

On Sat, May 28, 2022 at 08:38:49AM +0000, Hadrien Lacour wrote:
> On Sat, May 28, 2022 at 03:33:16AM +0000, Rodrigo Martins wrote:
> > Hello,
> >
> > The problem here is I/O buffering. I suspect it to happen in the C
> > standard library, specifically on the printf function family.

You know, that is the sort of claim that ought to be researched. First
of all, it is stdio that is providing buffering, if any. For reasons I
won't go into, musl's printf functions will provide a temporary buffer
on unbuffered files, but the buffer is then flushed before the functions
return (else the buffer would be invalid).

> > If I
> > recall, the C standard says stdio is line-buffered when the file is
> > an interactive device and let's it be fully buffered otherwise.

Not quite. I looked it up in both POSIX and C, and found that
    - stderr is not fully buffered
    - stdin and stdout are fully buffered if and only if they are
      determined to not be interactive streams.

This means the standard allows "line buffered" and "unbuffered" for
stderr, and also those two modes for stdin and stdout for interactive
streams.

But yes, in practice we usually see stderr be entirely unbuffered, and
stdin and stdout be line buffered on terminals and fully buffered on
everything else.

> You can use stdbuf(1) to modify that aspect without touching the
> program source itself.
>

Had to look up the source for that. I had heard of stdbuf, but I always
thought that that was impossible. How can one process change a
process-internal detail of another program? One that is not inherited
through fork() or exec(), I mean. Well, turns out it is impossible.
stdbuf sets LD_PRELOAD and execs the command line, and the changing of
the buffer modes happens in the library.

That means that whole thing only works if:
    - you have the target program linked dynamically
    - you have stdbuf and the target program linked against the same libc
    - the target program doesn't change buffering modes later, anyway
    - the target program does not have elevated privilege.

Ciao,
Markus
Received on Sat May 28 2022 - 19:58:40 CEST