[wmii] moving liblitz on [lines broken, for broken mail clients]

Hello there.

So, wmii-3 is out, it's features are nice, and it works more or less
for most people. That's great. What has been accomplished is immense.

wmii-4's todo list is already big. Writing it was like writing the
Christmas wishlist for the next year, yet before this year's christmas
eve, I suppose ;) I hope it will still grow, as there is much to do yet.

There was much talk on IRC about sane apps, be it document viewers
(»web browsers« too), terminals, mail clients. Also, there are issues
with the upcoming tag bars and the current wmiimenu (concerning
especially text input and rendering.) Thirdly, there is the issue of
UTF-8. To approach these issues, imo it is immediately neccessary to
improve and even restructure liblitz. Iirc it was one goal of the
development of liblitz to provide a sane gui library to application
programmers. Atm, liblitz is far from that.

Now, with the beginning development of wmii-4 the discussion about the
further development of liblitz has to start, as there are loads of
non-trivial design decisions.

1) Liblitz should provide a total abstraction from X11 insanity. The
applications should not need to access any X library calls, as long as
they don't need them for explicitely interacting with the X server.
This would make applications easily portable to any platform liblitz is
ported to. Linux is not /the/ OS (some say so at least ;) To accomplish
this task, it is necessary to provide a full-featured window
abstraction and widget model. It must be easy to use and fulfill all
requirements that dwm [dynamic window management ;)] poses upon it.
This is firstly: absolute values for the layout of widgets should not
be needed as long as it is reasonable. It makes sense for multiline
text widgets, but not for single-line text widgets and to buttons only
to a limited extent. Then, widgets should be layouted in a dynamic
manner too, meaning some more or less extreme geometries (which should
be needed/useful for one application) should not make the widgets
unusable. On the other hand, in-window frames and sub-windows shoulb be
omitted, as they interfere with dwm. The window and widget model should
rather encourage to use multiple windows that are layouted by the WM.

2) Text handling is central to any application. On the one hand, text
is so ubiqitous that no-one really sees it as something special. On the
other hand, text is something so extremely complex that there are is an
infinite amount of books about handling it in computers. That's a gap
that is not closed in most widget sets. Both, rendering and input lack
far behind to what is self-evident on paper! This is because handling
text is technically very challenging. For example, many see (or at
least they try to repress the thruth they know too well ;) text as a
one-to-one mapping between bytes. That is true for ASCII or latin-*,
but we won't discuss here that it's not enough. What we really want is
Unicode[1]. (NO, I'M SURE WE DON'T WANT TO DISCUSS THIS HERE.) Most
probably UTF-8, though it is not the most performant representation
form, it is possibly the most rational, at least for us. Thus, we must
handle text as a stream of variable-width characters. No direct access
via C-type arrays is possible. No use of all the C standard library
string functions, like it is done throughout /all of wmii's source!/ As
Unicode is a 21-bit encoding, the amount of characters is gigantic, and
this poses extra difficulties. There are a lot of languages that are
possibly represented, and the amount of features NEEDED is somewhat
overwhelming. Even simple upper-lowercase mappings or algorithms for
line-breaking are not trivial anymore! One has to deal with character
classes, combining characters, grapheme clusters or whatever. It is
absolutely impossible to handle that in C-style arrays for an
application programmer. So, liblitz or some other library has to
provide that functionality. I don't know of any library that can do
that and is not bloated like hell (the only library of that I know that
it can do almost everything in the Unicode standard is IBM's ICU[2],
the binaries are more than 5 MB large oO, and the source is the
incarnation of C Preprocessor). I am convinced that this can be done
better.

I began with extending liblitz by implementing the basic functionality
of a text widget[3]. It is not possible to proceed without further
embedding into liblitz. It is not even near to being finished, not only
technically, but also and especially in design. There is also some
implementing of some text handling routines. This includes the
representation of the text as a gapped array for insertion/deletion of
text in Θ(1) most of the time. Maybe it could be used as the default
representation of text in liblitz, just for simplicity's sake, but
maybe the routines can be rewritten in such a way that they easily work
both with gapped and non-gapped arrays. I didn't find a way to
accomplish this.

I hereby request some discussion on this topic, and especially about
the code I wrote so far.

Greetings,
Denis

[1] http://www.unicode.org
[2] http://www-306.ibm.com/software/globalization/icu/
Open Source, has C- and C++-Bindings, ported to most significant
platforms (that's also what Java uses internally) sloc stats of the
source: cpp: 185897 (54.63%)
ansic: 148004 (43.49%)
sh: 3452 (1.01%)
perl: 2938 (0.86%)
sed: 14 (0.00%)
[3] http://www.ueberl33t.net/tmp/widgets_060506_1146924689.tar.bz2