On Sat, 7 Jul 2018 23:29:53 +0100
Dimitris Papastamos <sin_AT_2f30.org> wrote:
> On Sun, Jul 08, 2018 at 12:12:08AM +0200, Mattias Andrée wrote:
> > On Sat, 7 Jul 2018 22:55:28 +0100
> > Dimitris Papastamos <sin_AT_2f30.org> wrote:
> >
> > > This is too intrusive, what's wrong with using a shell script to test
> > > the commands?
> > >
> > > The test framework is more complicated than most sbase commands.
> > >
> > > It would have been nice to discuss this in advance before writing a
> > > 1000 line patch that might not get merged.
> > >
> >
> > Writing all tests in shell isn't the best idea I think.
> > This frameworks makes it easy to write test and it will
> > tell you everything you need to know to figure out what
> > failed. I believe that in the need this will reduce the
> > amount of test code. There are things that are difficult
> > to do in shell: for example create a terminal which is
> > need to test tty(1). Look for example at the tests in
> > https://github.com/maandree/base-util-tests, they aren't
> > that nice, and they even require bash(1), it would be
> > even worse with portable sh(1).
>
> so what you are saying is that your shell code sucks so
> it is better done in C?
>
> sorry im not buying it, this is overkill
>
No, I'm saying that if you want to do it in sh(1) you will
need more test, or make really crappy helper functions, and
you will also need to write special utilities in C just to
test some utilities that cannot be tested entirely in sh(1).
The C code contains:
* a way to print where in a loop a test fails.
this will be useful for example when when adding test
cases for the *sum utilities,
* a way to make tests asynchronously, this could be removed
but will be useful for testing sleep(1), so it does not
take too long,
* a simple way to measure and test how long a test took,
* a set of tiny functions to declare how to program shall
be started,
* a way to create sockets and TTYs. The socket part can
probably be removed but it would be useful for testing
different file types. Support for regular files should
probably be added. TTYs are required to testing tty(1),
* ways to test the exit status with support for both normal
exit and kill by signal,
* ways to check the output to stdout and stderr. In sh(1),
test(1) and grep(1) could be used,
* some code to make to test cases smaller,
* a function to spawn a process with the requested files
and input, and
* a function to read a process's output and wait for it.
I wouldn't say that it's complex, its just a few functions, and
2 or 3 of them is are bit long, but not particularly long. I
think this is worth it for the consistence, short tests and
easily readable tests, a convenient way to locate the failing
test and what's actually wrong, and to not have extra utilities
(also written in C) to do the parts of the tests that cannot be
done in C. I'm not sure how mean that the code is complicated,
it's just long.
Received on Sun Jul 08 2018 - 01:12:59 CEST