Hi Thomas,
This looks like a very neat test framework. I would however like
the file name to be printed in addition to line number as the
test can cover multiple files: the file with the tests and files
with utilities functions, or replacement for standard functions,
that the tests use.
Regards,
Mattias Andrée
On Tue, 10 Aug 2021 23:45:43 +0200
Thomas Oltmann <thomas.oltmann.hhg_AT_gmail.com> wrote:
> Hi everybody!
>
> Quite some time ago, at slcon19, someone was really interested in my
> very unconventional way of unit testing C code,
> and wanted me to write a post about it on the mailing list. I'm a
> couple years late, but here it is anyway:
>
>
> I've tried multiple well-known C unit testing frameworks in the past,
> but they all suffered from the same issues:
>
> - They're quite large and complicated because they pack lots of
> features for huge projects with huge development teams.
> For small projects like mine (or pretty much all suckless projects)
> most of the offered features are complete overkill.
>
> - they force you to organize your tests into a fixed hierarchy of
> 'test suite' > 'test case' > 'test' or the like.
> This is a bad fit for any project but those of one specific size class.
>
> - they make debugging really hard, because they tend to manage the
> overall control flow themselves,
> spawning all sorts of threads and child processes, so that merely
> attaching a debugger can be a pain.
>
> - they love to spam info logs, often making it difficult to see
> whether anything went wrong or not.
> On a side note, some of them have this absurd notion that failing a
> couple tests is not a big deal ?!
>
>
> So I wrote my own library, which I called 'dh_cuts' - "Dynamic
> Hierarchy C Unit Testing System".
> It is shamefully trivial, but served me well so far.
>
> The central idea is to replace the fixed hierarchy of tests nested in
> suites, as found in other frameworks,
> with a naming hierarchy that is completely dynamic at runtime. This is
> realized as follows:
> When you want your code to enter a new nested level in the hierarchy,
> you call dh_push("<the name of the level>"),
> and when you want to leave it again, you call dh_pop().
>
> An example:
>
> void test_the_flux_capacitor() {
> dh_push("flux capacitor");
> ...
> dh_assert(condition_1 == true);
> ...
> dh_push("turning some knobs");
> ...
> dh_assert(condition_2 == true);
> ...
> dh_pop();
>
> dh_pop();
> }
>
> void run_scifi_testsuite() {
> dh_push("sci-fi");
> test_the_flux_capacitor();
> dh_pop();
> }
>
> int main() {
> dh_init(stderr);
> run_scifi_testsuite();
> }
>
> Per default, if all asserts are passed, this program will print nothing.
> But suppose the test fails because condition_2 is false. In that case, dh_cuts
> will print a trace of the part of the hierarchy where the error occurred:
>
> └ sci-fi
> ․․└ flux capacitor
> ․․․․└ turning some knobs
> ․․․․․․└ triggered assert in line 013: condition_2 == true ← FAIL
>
> This system is great in terms of debuggability, because your tests can convey
> as much diagnostic information via dh_push() as you see fit.
> dh_push() even accepts the same formatted messages as printf(), so you can
> insert things like iteration counts into the hierarchy, and they won't clutter
> the output because they're only shown for asserts that fail:
>
> void monte_carlo_test() {
> dh_push("monte carlo");
> for (int n = 0; n < 1000000; n++) {
> dh_push("iteration %d", n);
> /* perform the n-th round of randomized testing */
> ...
> dh_pop();
> }
> dh_pop();
> }
>
>
> There's a handful of other features to dh_cuts that make it more practical:
>
> - You can use the macro dh_branch like this:
> dh_branch(
> do_some_stuff();
> more *stuff = ...;
> ...
> )
> to sandbox the code in the parentheses, meaning if that code crashes,
> then code following the dh_branch() macro should still be able to execute.
>
> - There's multiple different dh_assert() variations for convenience as
> well as dh_throw() to unconditionally fail with a custom error
> message.
>
> - dh_summarize() can be used to print a one-line summary of executed
> vs failed checks.
>
> - If you don't want the output to include fancy Unicode sequences, you
> can define
> DH_OPTION_ASCII_ONLY as 1 before including dh_cuts.h.
>
> - The entire thing is just a tiny single header library, so you can
> simply copy-paste it if you want, no dependency management neccessary.
>
>
> If you're interested, you can find the code here:
> https://github.com/tomolt/dh_cuts/blob/master/dh_cuts.h
> As a more complete example, I've also attached some testing code for a
> basic hashtable implementation to this post.
> Quite frankly, I don't expect anybody else to start using it, but I
> thought people here might be interested by the idea.
>
> Cheers,
> Thomas Oltmann
Received on Wed Aug 11 2021 - 12:34:10 CEST