Re: [dev] Suckless unit testing in C?

From: Anselm R Garbe <garbeam_AT_gmail.com>
Date: Thu, 26 Feb 2015 08:33:24 +0100

On 25 February 2015 at 20:52, Rian Hunter <rian+suckless-dev_AT_thelig.ht> wrote:
> On Wed, Feb 25, 2015 at 07:45:58AM +0100, Anselm R Garbe wrote:
>> All you need is creating test input, save the test output and whenever
>> you call main() with the same test input, you just check for
>> regressions in the test output compared to the expected test output.
[..]

> If the subtext of the advice here is to do away with interactive programs, except for the shell, then this philosophy makes sense. Unix started on a teletype device without advanced visual ttys. This advice definitely makes things much more scriptable and that is good. I like git, mpc, xdotool, xsetroot, notify-send, sed, awk, sendmail, etc.
>
> Then again, I like my interactive modal programs too. I like emacs, mutt, irssi, elinks, tmux, st, dwm. These tools are necessary for my daily work. How did the early Unixers get along without these tools? ed, mail, write, I'd guess. Should we go back to that? Personally, I'd rather not.

To me this is rather a question of the software architecture.
Interactive programs should consist and/or incorporate lot's of small
noninteractive programs (one for a particular job) that can be
combined interactively like in a similar shell usecase. Each of such a
small noninteractive program can have a particular "test program" or
more complex test suits can be developed for them. Those
noninteractive programs or parts define an interface (perhaps not a C
API interface, but a shell-like interface), but my emphasis is on
'interface'.

I do see sense in developing (unit) tests for interfaces like
libraries (with native lang bindings) or such noninteractive command
line programs. A (unit) test makes sense if the underlying program or
library function is called from various places/contexts with different
inputs and use by more than some internal code line. (Unit) tests
should test interfaces, not concrete implementations. They should
define what the concrete outcome is of a particular interface.

Demanding to write unit tests for internal code lines or hidden
implementations is rather pointless to me. Honestly how on earth would
a unit test suit for dwm's inner workings look like? Checking if the
linked structs attach/detach work correctly? Rather not. That is not
designed as API interface for external use.

Instead I can imagine bad behaving X clients as "unit" tests for dwm
-- as they would challenge the task of dwm -- _window management_.


Speaking out of experience, I would also pretty much object to do TDD
(test driven development) in a sense that you write test cases first
and the real code afterwards. That barely tends to a robust product,
as the test cases will become quite arbitrary.

My recommendation - and the only one - when it comes to unit tests is
this: whenever you or someone else finds a bug that points out a
non-conforming aspect of an implementation of such an interface,
record a unit test for this, in order to avoid that this specific bug
will happen again.

A good example for this is how'd you develop a compiler for some
language: whenever an input program results in a compiler error/bug
that is non-conforming to the language standard, fix the bug in the
compiler implementation and record the specific input program as unit
test for future uses. During time your unit tests will grow an test
real world problems and not artificial programs that the developer
came up with while developing and that haven been rendered invalid
meanwhile.

-Anselm
Received on Thu Feb 26 2015 - 08:33:24 CET