Re: [dev] Build system: redo

From: Sergey Matveev <stargrave_AT_stargrave.org>
Date: Fri, 18 Dec 2020 13:08:04 +0300

*** Greg Reagle [2020-12-17 14:39]:
>Makefile implemented in apenwarr redo

Just a remark: when using "basic" features of redo, just what DJB
described, there is no difference between various redo implementations.
I used apenwarr/redo, apenwarr/do, redo-c and goredo with the same
project, with the same .do-files without any modifications and behaviour
differences.

>Like st for example.

st is too simple to see anything interesting :-). But ok, let's convert
its Makefile to redo. Pay attention that in practice you can make single
.do file, instead of bunch of other ones -- you are free to "express"
yourself. But placing each target's description in separate .do
automatically will make a dependency on it, rebulding dependant target
if .do is changed.

Currently if you modify st's Makefile, then nothing happens, because
targets do not depend on Makefile/config.mk itself. If you add
Makefile/config.mk as a dependency to each target, then any modification
will (should) lead to rebulding of every target.

Bunch of .do are also easier to work with by several people. For example
if multiple people want to add new target at the end of Makefile, then
they will get conflict. Adding two different .do won't, obviously.

------------------------ config.rc ------------------------
CC=${CC:-cc}
VERSION=0.8.4
PREFIX=/usr/local
MANPREFIX=${PREFIX}/share/man
X11INC=/usr/X11R6/include
X11LIB=/usr/X11R6/lib
PKG_CONFIG=pkg-config

INCS="-I${X11INC}"
INCS="$INCS `${PKG_CONFIG} --cflags fontconfig`"
INCS="$INCS `${PKG_CONFIG} --cflags freetype2`"

LIBS="-L${X11LIB} -lm -lrt -lX11 -lutil -lXft"
LIBS="$LIBS `${PKG_CONFIG} --libs fontconfig`"
LIBS="$LIBS `${PKG_CONFIG} --libs freetype2`"

STCPPFLAGS="-DVERSION=\"${VERSION}\" -D_XOPEN_SOURCE=600"
STCFLAGS="${INCS} ${STCPPFLAGS} ${CPPFLAGS} ${CFLAGS}"
STLDFLAGS="${LIBS} ${LDFLAGS}"

------------------------ all.do ------------------------
redo-ifchange options st

------------------------ options.do ------------------------
redo-ifchange config.rc
. ./config.rc
cat >&2 <<EOF
st build options:
CFLAGS = $STCFLAGS
LDFLAGS = $STLDFLAGS
CC = $CC
EOF

------------------------ config.h.do ------------------------
cat config.def.h

------------------------ obj.list.do ------------------------
for c in *.c ; do echo ${c%.c}.o ; done | sort

------------------------ clean.do ------------------------
redo-ifchange obj.list config.rc
. ./config.rc
rm -f st `cat obj.list` st-${VERSION}.tar.gz

------------------------ st.do ------------------------
redo-ifchange obj.list
redo-ifchange config.rc `cat obj.list`
. ./config.rc
$CC -o $3 `cat obj.list` $STLDFLAGS

------------------------ default.o.do ------------------------
src=$2.c
redo-ifchange config.rc `sed -n 's/#include "\(.*\)"$/\1/p' $src`
. ./config.rc
$CC $STCFLAGS -o $3 -c $2.c

          ------------------------ >8 ------------------------

I am not attaching "dist", "install", "uninstall" targets, because they
obviously have completely identical contents as Makefile (plus sourcing
of config.rc). No need to use anything like .PHONY, because all that
.PHONY targets are intended to be run by human with "redo" command, that
forcefully rebuild specified targets.

Instead of manually listing st.c and x.c, I used *.c. Instead of manual
specifying of dependant header files for every .c I used sed to
dynamically create that list. By the way, st.c does not depend on
config.h anyway, so current st's Makefile slightly is not correct.

What advantages they already give, except for atomic targets building
(saving output to $3 and then renaming it to target file itself) and
reliable out-of-date detection (https://apenwarr.ca/log/20181113)? If I
change config.mk, then everything will be rebuild. Changing of ".o"
build rules (default.o.do) will rebuild all .o targets. Adding another
.h file and including it in .c -- will make them dependant on that
header and will rebuild if it changes.

If I create new .c file, then current .do won't rebuild obj.list and
"clean"/"st" targets won't know about new required corresponding .o
targets. That can be solved by adding redo-always to obj.list.do. It
will force obj.list rebuilding every time (that is cheap), but good redo
implementations, that check checksum target's contents, will see that it
is not changed, so they won't rebuild ones who depend on obj.list. DJB
exactly suggested to use checksums to determine if target is actually
changed. So, if I add new .h/.c, then no modifications done to .do files
(of course if there is nothing special for some exact targets). However,
due to obj.list target simplicity in my example, it is easier just to
use it as a "function", without any redo-always:

    $ mv obj-list.do obj-list
    and use `. ./obj-list` instead of `cat obj-list`

It is far from ideal, because not everyone depends on CC or LDFLAGS. My
config.rc allows you to change CC through environment variable, that is
not taken in advance too. All of that can be relatively easily solved
too. Unfortunately my big projects with hundreds of .do files are not
free software (not open source), so I can not share. But instead of
depending on some "config.rc" file, most targets depend on exact
"conf/cmd/cc", "conf/cmd/ar", "conf/flags/asn1.rc", "conf/flags/tai.rc"
and so on. For example:

    redo-ifchange ../conf/cmd/cc ../conf/flags/common.rc
    read CC < ../conf/cmd/cc
    . ../conf/flags/common.rc
    $CC $CFLAGS -o $3 -c $name.c

    ------------------------ conf/flags/common.rc ------------------------
    redo-ifchange ../../config
    . ../../config
    cat > $3 <<EOF2
    {
        read CFLAGS
        read LDFLAGS
        read LDLIBS
    } <<EOF
    $CFLAGS $DMALLOC_CFLAGS $MUTEX_CFLAGS
    $LDFLAGS $DMALLOC_LDFLAGS $MUTEX_LDFLAGS
    $LDLIBS $DMALLOC_LDLIBS $MUTEX_LDLIBS
    EOF
    EOF2

and of course all that cmd/*, flags/* are generated with various .do
files, mostly default.do ones. All of them give ability to override some
commands/variables/paths/whatevers through environment variables, take
all of them into account then they are changed. Most (C-related) flags
are determined with pkg-config, that is also actually determined by .do:

    ------------------------ conf/cmd/pkgconf.do ------------------------
    redo-ifchange ../../config
    . ../../config # to be able to override $PKGCONF there
    echo ${PKGCONF:-`command -v pkgconf || command -v pkg-config`} > $3

Practically all of that completely replaces the whole autoconf system,
parallely running all of those pkg-configs. If my $CC, or pkg-config's
output is changed, then only really dependant targets will be rebuild.
And I am silent about ability to generate .do from other .do (.do.do).

https://redo.readthedocs.io/en/latest/ documentation is the best place
to start, in my opinion. However its redoconf autoconf-replacement seems
too complicated for me.

-- 
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: CF60 E89A 5923 1E76 E263  6422 AE1A 8109 E498 57EF
Received on Fri Dec 18 2020 - 11:08:04 CET

This archive was generated by hypermail 2.3.0 : Fri Dec 18 2020 - 11:12:08 CET