[dev] sites linkrot

From: Kai Hendry <hendry_AT_iki.fi>
Date: Sun, 13 Jan 2013 15:40:59 +0800

Hi guys,

Please rip this to shreds https://github.com/kaihendry/linkrot and
perhaps guide me to a better script. Something that can do the http
requests in parallel and hence much faster?

I ran it over sites/
for i in *; do test -d "$i" || continue; linkrot $i > $i.linkrot; done

and the output is over here:
http://s.natalian.org/2013-01-13/

000 means the domain didn't resolve. Definitely have some false
negatives, for e.g. on cat-v. I guess sites sometimes aren't working
and the failures need to be counted/recorded and when it hits a
threshold (e.g. 10 consecutive failures in 10 day daily check), only
then an admin needs to manually intervene?

Kind regards,
Received on Sun Jan 13 2013 - 08:40:59 CET

This archive was generated by hypermail 2.3.0 : Sun Jan 13 2013 - 08:48:05 CET