[dev] [ANNOUNCE] req 1.0 - a gawk und dmenu powered plumberlike

From: Robert Figura <nc-figuraro_AT_netcologne.de>
Date: Wed, 7 May 2014 18:26:49 +0200

A little more than two years ago i started coding some plumberlike in
gawk, and i think it's time for me to seek suggestions and share what
i have. Repository link:


It's also an excellent xdg-open replacement...

What follows is the (lengthy) README, feedback very much appreciated!

Thank you, sincerely yours - Robert Figura

req 0.1 README:

req will harbor your personal unix knowledge in a central place.
Access your commandline snippets and collected wisdom by label,
comment or substring, or filter using data from your current working
context. Bring reason to your history, favorites, scripts, aliases,
abbreviations, and whatnot.

[...plumber?] I seem to remember thinking that it was sort of awk inspired
(Aharon Robbins on comp.lang.awk)

Powered by standard unix interfaces it works with most unix

req is your compact and portable way to program personal workflow
elements in gnu awk. It is a rapid prototyping framework for micro
interaction design to enhance your personal unix experience.

To personalize your new pet you'll want to enter small scripts, shell
commands, and quite possibly regular expressions. If you already wrote
small scripts, perfect! If you're a fan of the commandline you'll love


open file & http & ftp & ssh url, context sensitive start menu &
application launcher, filename extension & content based mime-type
discovery, hotkey daemon, search & download frontend, centralized
history & favorites & bookmarks, bash completion agent, clipboard &
settings manager, xterm background colorizer, automount alternative,
desktop accelerator & time saver & shortcut, inter-application
workflow optimizer, ad-hoc microformat parser, system and process info
& network scanner gui, manual & info page & documentation launcher,
dvb channel select, better xdg_open & dwim, and much more!

Inspired by plan9's plumber. Small source, regexp pattern rules,
inline shell script, commandline & console & x11 interfaces, dmenu
powered, in bash & GNU awk!


see the LICENSE file


Usually you'd bind a hotkey to run req on clipboard contents. Or set
up applications to let those run req, e.g. your filemanager to run req
on the currently selected file. In the following you see the
commandline interface in action:

  # note: -t shows the command instead of executing it, -d dumps the menu

  # mimetype based file launcher
  $ mv trivium1.pdf z.doc
  $ req -t z.doc
  xpdf -cont -z width '/home/rfigura/z.doc'
  # note that it isn't fooled by the "wrong" file name extension

  # it's an url launcher, too
  $ req -t http://localhost/
  surf -n 'http://localhost/'

  # for some urls scripting is turned off:
  $ req -t http://w3c.org/
  surf -p -s 'http://w3c.org/'

  # some urls get downloaded:
  $ req -t http://localhost/test.pdf
  curl -f -J -L -O -b '/home/rfigura/.surf/cookies.txt' 'http://localhost/test.pdf'
  # sorry, i lied. the actual -t facilities can't yet _show_ the above,
  # because of a download wrapper. But it works, and you get the idea

If there is no default action, you can request to get a choice using
the -menu switch. In the following i'll use -d, to get the menu
printed to the console instead. The asterisk "*" indicates that the
auto flag is set, allowing to launch those when interaction isn't

  # search & translate
  $ req -d foobar
  1 - google.com
  2 - translate
  3 - en.wikipedia.org

  # calculator, i do enter formulas in dmenu, or find them on the web
  $ req -d '37864 / 3600'
  1 - * bc
  2 - * pari/gp
  3 - * gnuplot

  # it knows about various date formats, too
  $ req -t '21. Nov 2011'
  1 - cal


Your rules should go to the port.*.awk files. It can be as simple as the
following two-liner suggests:

  # a temporary rule i'd put in port.open.awk...
  get("mimetype") ~ /^video\// && tolower(get("file.name")) ~ /simon.templar/ {
      export("mplayer_args", "-vf crop=526:400:98:4 ")

  # ...so simon templar gets cropped properly in mplayer
  $ req -t simon-templar-12726.mp4
  mplayer -quiet -vf crop=526:400:98:4 '/home/rfigura/simon-templar-12726.mp4'

In case you want to add new data types, or new patterns for existing
data types, you should add code to some class*.awk. Look, req can be a
powerful front-end for unix commands:

  # the following gives a minimalistic process manager:
  # class.awk:
  match($0, /^([0-9]+)$/, m) {
      set("decimal", "" (0 + m[1]))
  (d = get("decimal")) &&
  (c = readFile("/proc/" decimal "/cmdline")) {
      set("pid", d)
  # port.open.awk:
  get("pid") {
      label("SIGTERM"); menu("kill -TERM " Q(get("pid")))

The following is to suggest using req with other scripting languages.
There are many ways to do this, here's an easy one to think of, others
are more fun to look at:

  # a rule to open tk's colorpicker, allowing it to run without choice
  # port.open.awk:
  func wish(script) {
       return "echo " Q(script) " | wish"
  match($0, /^(#[0-9a-fA-F]{6,6})/, m) {
      c = "wm withdraw .\n"
      c = c "puts [tk_chooseColor -parent . -initialcolor \"" m[1] "\" -title \"Tk color picker\"]\n"
      c = c "exit\n"
      label("colorpicker"); run(wish(c))

You get quite some flexibility. You can call internal functions
instead of shell scripts, and read all remaining data when that
happens to be the user's choice:

  func mysql(s) {
      s = collapse(s)
      # todo: is that hinting at a problem in req?:
      s = gensub(/\n/, " ", "g", s)
      runHook(pager("mysql test -e " Q(s)))
      label("mysql"); menu("_AT_mysql " $0)

Suggestion: If you have a comfy working ruleset do `hg commit` to back
it up!

Suggestion: Read the source. There is no easier way to explain the power.

Suggestion: Grep the source for menu labels, commands or namespace


  req [-OPTIONS...] DATA

  -p port (-a port=<port>)
  -f triggering application (-a from=<client>)
  -menu always display menu
  -select select from menu, -select <number>
  -d print menu and exit (-menu -a menu_type=print)
  -a assign attribute, -a <name>=<value>
  -A assign attribute, -A <name> <value>
  -m import header file, -m <file>
  -e|-- rest of commandline is a single data record
  -alias alias mode (-a format=alias)
  -stdin read from stdin, ignore rest of commandline
  -file read files, -file <file1>...
  -rfile reverse read files, -rfile <file1>...
  -percent substitute e.g. %ctx.xwin in data
  -t|-test don't run but print command (-a run_hook=print)
  -v be verbose
  -V|-version show version
  -h|-help display this usage info


- you provide some data via the commandline front end:
  $ req foobar
  $ echo foobar | req -stdin
  # optionally specify to load port.start.awk, to allow running commands
  $ req -p start xterm
  # request a menu
  $ req -menu ...
  # or explicitly add attributes
  $ req -a menu=9menu ...
- the data is then fed to an awk process
- (main.head.awk) stores attributes in attr[]
  - port, from, ...
  - menu_hook, run_hook, ...
- (ctx.awk) gathers context information in ctx[]
  - wdir, wm_name, ...
- awk's main loop is run for each record (line):
  - (main.head.awk) preprocess records (filter_foo, %foo, etc)
  - (class.awk) extracts fields from record into class[]
  - (port.*.awk) these are a sets of rules
    - on match, an inline shell script is recorded in menu[],
      - alternatively an awk function call can be recorded (see below)
      - or anything else be done at once, mostly used for debugging
  - (main.tail.awk) when a menu has been constructed, it is given to the choice agent
    - which may start a menu agent
    - or select automatically based on heuristics
    - or abort, or do something else
  - users choice is then given to the start agent
    - special case: if the command starts with "_AT_", the given awk function
      is called instead (with one argument, the rest of the string)



hg clone http://teslawm.org/req/hg ~/.req

git clone http://github.org/req ~/.req


- gawk:
- bash:

- gawk headers, gnu make, and some c compiler (for exec.so support)
  - supposed to be optional. will be fixed!

optional build dependencies:
- x11 headers, gnu make, and some c compiler (for vmenu)

strongly suggested runtime dependencies, not much fun without:

- file: (file type recognition)
- dmenu: (fast x11 menu)
- xclip: (x11 clipboard support)
- xprop: (x11 window context detection)

optional runtime dependencies, or use your own favorite tools:

- xbindkeys: (for hotkey daemon)
- xterm: (auto-set xterm background)
- less: (the default pager)
- mercurial: (to back up your settings)

A fast tiling windowmanager:

a minimalist browser

add your favorite application here...


$ hg clone teslawm.org/req/hg ~/.req
$ cd ~/.req
$ cp config.mk.sample config.mk
### optionally fetch or locate awk extension header
### edit config.mk
$ make
### maybe edit req.conf
### test it:
$ ./req -d foobar
### copy front-end to someplace in your path
$ cp req ~/bin


- xterm colorization
  $ alias xterm='req -p xterm -alias'
- bash completion
  $ complete -o default -C 'req -p bash-completion -alias' -D
- hotkey daemon
  $ ~/.req/bin/xbindkeys-hotkeyd.sh &
- hook req into environment variables
  $ export BROWSER='req -p open '
  $ export TERMINAL, ...
- replace xdg_open by req (or req -menu)
  - set up your filemanager to let req open all files
- patch your browser to always open urls through req

for more examples see doc/support


- The rules, classifications and context code reflect my personal
  preferences and yours are bound to be very much different. There's
  just too much software out there. So you'll have to gather your own
  rules to make it useful. Please share your finds!

- In many places the code is prototypical, overly simplistic and
  sometimes kind of broken. Much of it can be improved, feel free to
  step in. Other cases might suggest that the whole thing is a hack,
  and maybe it is...

- Gawk is kinda almost fit for this job. It's nice because of its
  small footprint, interesting featureset and notation, and because it
  lives on almost all unix systems. But you have to do things the awk
  way or you'll regret it. Most awk scripts are simple, but awk itself
  isn't. Because of that you'll find hacks, workarounds and


The original quest to get my practical unix knowledge sorted, and to
bring reason to my daily text snippet load has so far been a
tremendous experience. But still, there is lots left to pick up:

- generalize port.hotkey and port.dispatch to an eventd (better performance, too)
- better remote host support. publish, fusermount, system query, ...
  - generalized buddy list
- more agressive data extraction, e.g. contacts, urls, ...
- context sensitive history
- a queue manager and better multiline support
- port to android (reasonable?)
- and, of course, clean everything up, and fix bugs!

On the other hand, the conception could easily support completely new
directions for experimentation:

- remote system monitoring, notification, and instant messageing
- better sticky-notes, log some context along, or use context to find the relevant ones
- support more protocols, e.g. mbox, syslog, dbus, inotifywait, ...
- build bridges to other subsystems, e.g. browser, kernel info, ...
- better local network scanning

I'd also really like to see ideas on how to share the newest req
tricks with the least hazzle. Maybe we need a microformat? twitter? Or
integration with some pastebin? What can we get out of our dvcs? Your


Success stories, ideas, or insults? Send an email: rf_AT_teslawm.org


Plan9's plumber has been there for a long time. I never got to use it,
but i read Rob Pike's paper[1]. Wanted to play with something similar,
so i decided to use a scripting language to specify rules... and
started to code away in awk.

On the other hand, all of it is from scratch, i encountered many
obstacles and invested quite some effort to find useful designs. So
this implementation is by my hands and there is no one else to blame
for it.

And there's vmenu. It is a fork of 9menu by Anselm Garbe which i only
patched to do keyboard handling and to change the input format (the
latter i now regret). You should consider using vintage dmenu in any


[1] "Plumbing and Other Utilities", Rob Pike
Received on Wed May 07 2014 - 18:26:49 CEST

This archive was generated by hypermail 2.3.0 : Wed May 07 2014 - 18:36:06 CEST