Re: [hackers] [blind] update todo: tee is too slow || Mattias Andrée

From: Richard Ipsum <richardipsum_AT_fastmail.co.uk>
Date: Wed, 13 Dec 2017 00:45:18 +0000

I lack community standing to really comment on this.

That being said, as a casual observer it seems fairly obvious to me
that what you're proposing is totally anathema to suckless philosophy.

On Tue, Dec 12, 2017 at 02:49:34PM +0000, Mattias Andrée wrote:
> It may look insane on the surface level. I am not commited to changes
> listed in the TODO, most of them are just ideas that I will have to evalute
> later whether they are worth implementing. Although I'm pretty convinced
> most of them are good.
> ________________________________________
> From: isabella parakiss [izaberina_AT_gmail.com]
> Sent: 12 December 2017 13:31
> To: hackers mail list
> Subject: Re: [hackers] [blind] update todo: tee is too slow || Mattias Andrée
>
> don't you realize how insane the whole thing is?
>
> On 12/12/17, Mattias Andrée <maandree_AT_kth.se> wrote:
> > Perhaps I should clarify that (1) the goal would be to have blind-tee
> > (and blind-cat if that is implemented) to use already existing functions
> > that sends data between two files, and have these functions use splice
> > when possible), so would only use tee explicitly, not splice, and (2)
> > a 1 hour long blind video with the resolution 1920x1080_AT_30 is 6.5TB
> > large, so we are talking about a massive amount of data that is beeing
> > sent between processes. In total, a 1 hour long video could require
> > houndreds of terabytes being sent between process. Double that if it is a
> > 60 fps video (which you will actually on the Internet nowadays) and
> > multiple that by 4 for a 4K video, and double that again for a stereoscopic
> > video (you can find all of these things on Blu-ray videos). So we could
> > be talking about a couple of petabytes of data for a profession video,
> > and 100TB for an amateur video, not just a couple of gigabytes. Try to
> > copy that about of data with dd (in parallell of course), and you might
> > find that even halving that time[1] would be nice, even if most if the time
> > in the rendering process is usually spent on rendering effects, stacking
> > videos, and transcoding.
> >
> > [1] 42 hours per petabyte.
> >
> > ________________________________________
> > From: Mattias Andrée [maandree_AT_kth.se]
> > Sent: 12 December 2017 11:32
> > To: hackers mail list
> > Subject: RE: [hackers] [blind] update todo: tee is too slow || Mattias
> > Andrée
> >
> > When I rendered a video, tee used 100% while the other process was
> > basically at 0, for more than 50% of the rendering time. I ran it multiply
> > times to verify that is was correct. An alternative solution would be
> > to use sockets, but that would require changes to the shell. Optimising
> > tee seems like the sensible alternative. Besides, normally when you use
> > tools like tee and cat, you expect it to finish within milliseconds, even
> > for larger files, here we are talking about time intervals between seconds
> > and a few hours, with most if not all CPU:s at 90% to 100%, so
> > optimisations
> > do not hurt, especially not when as significant as using tee and splice. It
> > may be unfortunate to have to use both tee–splice and read–write (which
> > is required both for platforms not supporting tee and splice, and for file
> > types not supporting them), but considering how little complexity this
> > adds,
> > — it's not much more than a duplication of a function, — I think it is a
> > trade-off worth considering.
> >
> > ________________________________________
> > From: isabella parakiss [izaberina_AT_gmail.com]
> > Sent: 12 December 2017 10:58
> > To: hackers mail list
> > Subject: Re: [hackers] [blind] update todo: tee is too slow || Mattias
> > Andrée
> >
> > bullshit
> >
> > On 12/4/17, git_AT_suckless.org <git_AT_suckless.org> wrote:
> >> commit d8aa45da86d1128149fd7ab6ac3725bf8e88a1b1
> >> Author: Mattias Andrée <maandree_AT_kth.se>
> >> AuthorDate: Mon Dec 4 22:35:59 2017 +0100
> >> Commit: Mattias Andrée <maandree_AT_kth.se>
> >> CommitDate: Mon Dec 4 22:35:59 2017 +0100
> >>
> >> update todo: tee is too slow
> >>
> >> Signed-off-by: Mattias Andrée <maandree_AT_kth.se>
> >>
> >> diff --git a/TODO b/TODO
> >> index b0bbde7..408e942 100644
> >> --- a/TODO
> >> +++ b/TODO
> >> _AT_@ -1,3 +1,7 @@
> >> +blind-tee (and tee(1)) is too slow (bottleneck) and must be
> >> reimplemented
> >> +using tee(2) and splice(2). cat(1) may also be too slow, if this is the
> >> +case, add blind-splice that just copies stdin to stdout using splice(2).
> >> +
> >> blind-transform affine transformation by matrix
> >> multiplication, -[xy] for
> >> tiling, -s for
> >> improve quality on downscaling (pixels'
> >> neighbours must not change)
> >> blind-apply-map remap pixels (distortion) using the X and Y
> >> values, -[xy]
> >> for tiling, -s for
> >>
> >>
> >
> >
> >
> >
>
>

-- 
To be born a god is too easy --
it's hard to turn to the dharma when life is so good.
To be born a demon or an animal is too difficult --
it's hard to practice the way when life is so tough. As a human,
we recognize fairly early that life does not always go our way;
but we are not without some control and capacity for disciplined practice.
It's this combination that allows us to (a) want to practice
and (b) be able to practice.
Received on Wed Dec 13 2017 - 01:45:18 CET

This archive was generated by hypermail 2.3.0 : Wed Dec 13 2017 - 01:48:22 CET