Barbara Liskov : The Power of Abstraction

Great, great talk

Ho ! By the way, didn't she sort of tell that Checked Exceptions was not a good idea ? (at around 37:30 during the talk). And it was known 30 years ago...

Manages Linux devices with HAL

HAL, the Hardware Abstraction Layer of most recent Linux distros really is an impressive piece of architecture.

I'm impressed by how easy it is to take control of any low level parameter of any hardware.

Two example I've recently used :

A workaround to prevent nautilus in Ubuntu 9.04 from automatically mounting my MTP player :
I've added a no-automount.fdi file in directory /etc/hal/fdi/policy<br /><?xml version="1.0" encoding="UTF-8"?> <!-- -*- SGML -*- --><br /><deviceinfo version="0.2"><br /> <device><br /> <match key="info.product" string="ZEN V Plus"><br /> <remove key="info.capabilities" type="strlist">portable_audio_player</remove><br /> </match><br /> </device><br /></deviceinfo><br />(See my answer "stephg said on 2009-11-09" in this post for more details)

An other example to slow down the cursor speed of my high resolution mouse. As the xorg configuration file is not used anymore in Ubuntu, the X11 options can be set with a HAL configuration file.
I've added a 10-x11-input.fdi file in directory /etc/hal/fdi/policy : <br /><?xml version="1.0" encoding="UTF-8"?> <!-- -*- SGML -*- --><br /><deviceinfo version="0.2"><br /> <device><br /> <match key="info.product" string="Logitech USB Laser Mouse"><br /> <merge key="input.x11_options.ConstantDeceleration" type="string">2</merge><br /> </match><br /> </device><br /></deviceinfo><br />This "merge" scheme really is brilliant because it allows to go back to a default configuration by deleting a file.

My "Hello World" Scala script

My first Scala project, a simple Scala script to download podcasts and sync with an Mp3 player : Jonction

Delete all tracks from MTP device

My MTP MP3 player does not allow to delete all albums with one command (yes I really wanted to delete all tracks ! )

Bash to the rescue :
mtp-tracks | grep 'Track ID' | awk '{print $3}' | xargs mtp-delfile -n

Of course you must install the mtp-tools, on Ubuntu :
sudo apt-get install mtp-tools

Cloned Git repository in DropBox folder

Nice and easy way to setup a Git server : a cloned git repository in a DropBox folder. This is brilliant and It works very well (see the Collaboration part of the article).

File parsing in Scala with Pattern Matching and Tail-Call optimization

In the process of learning Scala, I had to find a subject for my very first "Non Hello World" program in Scala. It was a good opportunity to code a script I wanted to write a long time ago in order to automatize the sync of the podcasts I download for my Mp3 player (I have very specific use cases that are not addressed by the usual podcast downloader software).

As my player is an MTP player,the script launches some mtp-tools command line utilities of libmtp to drive the player.

In particular the mtp-tracks command lists some information for the tracks stored into the player. The output of the command looks like :

Attempting to connect device(s)
mtp-tracks: Successfully connected
Friendly name: My Zen
Track ID: 40337
Title: Java Posse #273 - Roundup 09 - Managing Technical Debt
Artist: Tor Norbye, Carl Quinn, Joe Nuxoll, Dick Wall
Genre: Podcast
Album: The Java Posse
Origfilename: Tor Norbye, Carl Quinn, Joe Nuxoll, Dick Wall-Java Posse #273 - Roundup 09 - Managing Technical Debt.mp3
Track number: 0
Duration: 3809000 milliseconds
File size 45766104 bytes
Filetype: ISO MPEG-1 Audio Layer 3
Use count: 9 times
Track ID: 41737
Title: Java Posse #272 - Newscast for July 30th 2009
Artist: Tor Norbye, Carl Quinn, Joe Nuxoll, Dick Wall
Genre: Podcast
...

So basically, in order to parse the file I must skip the header (there's a footer too) and find each track information. It looks like a job for Pattern Matching on list with a mix of RegExp matching. And as he goal is also to learn Scala (And improve my Functional Programming skills) I wanted to code the parsing without any mutable object (yes, I'm trying to follow the rules described in Programming in Scala)

So here is the part of the code that parses the mtp-tracks output :
<br />//let's define some RegExp<br />Expval TrackId = """Track ID: ([0-9]+)""".r<br />val Title = """ *Title: (.+)""".r<br />val Album = """ *Album: (.+)""".r<br />val Filename = """ *Origfilename: (.+)""".r<br /> <br />//The recursive function<br />def collect(lines: List[String],tracks: List[MtpTrack]): List[MtpTrack] = {<br /> lines match{<br /> case TrackId(id) :: Title(title) :: _ :: _:: Album(album) :: Filename(filename) :: rest => <br /> collect(rest,new MtpTrack(id,album,title,filename) :: tracks)<br /> case _ :: rest => collect(rest,tracks)<br /> case Nil => tracks<br /> }<br />}<br />
The list of lines of the mtp-tracks command output are stored in the lines list. This recursive function can then be called with collect(lines,Nil), it returns a list of parsed MtpTrack object.

To explain the three pattern matching cases of the collect function, I must first explain how pattern matching works for regexp.

In this code, four matchers are defined for four regexp : TrackId, Title, Album and Filename. For instance, if one line matches on the expression Track(id) the first group of the regexp is captures in the id variable.

So, now, the description of the three cases after lines match

• case TrackId(id) :: Title(title) :: _ :: _:: Album(album) :: Filename(filename) :: rest
  this one says "match on list if list first element matches the regexp TrackId, then regexp Title, then whatever twice, then regexp Album, then regexp Filename. And also, if it matches extract the rest of the list in the rest variable. Also, capture all regexp groups in the declared variables id,title,album and filename". Rather powerfull.
  Then, with the matched information we create an MtpTrack object that is added to tracks, the already collected list of MtpTrack (actually a new list is created as a List in Scala is immutable).
  
• case _ :: rest
  this one is tried when the previous case does not match. It says "if list starts with any element, extract the rest of the list after this element in a rest variable". rest may also by empty.
• case Nil
  this one is the stop clause for the recursion, it matches when the list is empty.
  

Thanks to the power of pattern matching composition (RexgExp in list), header and footer are automatically ignored. And as the function ends with a direct recursive call, the Scala compiler has compiled the code with tail-call optimisation, so not stack overflow, even with very large command output.

I must say that I was amazed by how much can be expressed in Scala with so little lines of code. And as I'm a beginner in Scala, there are surely even smaller and idiomatic ways to code the same functionnality.

Backup with rsync and cygwin on Windows

I recently had to setup an automatic backup scheme for a desktop Windows PC used for office tasks. I usually try to avoid complicated backup software that tend to store the backups in a proprietary format. I want to be able to browse the files in the backup, and copy one file to recover the content.

On UNIX/Linux I use rsync and thanks to Cygwin I was able to do the same thing on Windows. The goal for the script was :

• Must backup two directories into one directory located on an USB key drive
• Must not delete any file in the backup
• Must always keep the old versions of any updated file
  

Here is a bash script that backups two directories foo1 and foo2
<br />#!/bin/sh<br />SRCDIR="/cygdrive/c/links/foo1 /cygdrive/c/links/foo2"<br />DESTDIR="/cygdrive/J/backupdir"<br /><br />OLDBDIR="old."`date +%F`<br /><br />rsync -avb --backup-dir=$OLDBDIR --modify-window=2 $SRCDIR $DESTDIR<br />

• links actually is a symlink to the directory that contains foo1 and foo2. I created this symlink with the ln -s command as the path to the parent directory of foo1 and foo2 had spaces and I didn't want to deal with path escaping issues.
• --modify-window=2 is necessary on windows, else the whole set of file is saved all the time.
• --delete can be added to the rsync command if we want the deleted files to be removed from the backup
  

Then a .bat file is created in order to run the script with a double click :
set PATH=%PATH%;/cygdrive/c/cygwin/bin<br />bash.exe backup.sh<br />

Test Driven Wall Tiling

Last week I was tiling a kitchen backsplash and countertop and as I was using the usual tile alignment tricks, I was struck by the parallel with the Test-driven development techniques for application development.

This picture shows the wooden cleat (the dark brown area on the picture) that was temporarily screwed to the wall to obtain a perfect alignment for the second row of tiles above the countertop.

It made me think of TDD because :

• The screwed wooden cleat is a temporary built artifact that has nothing to do with what we want to construct (that's to say, a tiled backsplash), but we built it nonetheless, and with care, as it's going to direct the alignment of the tile. That's what we do with unit test, they are a programs designed to unit test other programs.
  
• The wooden cleat is attached to the wall using the same tools and technique that are used to build the main "noble" artifacts (wood, screw, you name it). A unit test is build with the same skills than the tested program.
  
• When the cleat is fixed we do not have to think anymore about the tile alignment. The alignment is forced by the wooden rule, we can forget about it and concentrate on the quality of the tiling. The unit test creates design or behavior constraints on the code that is written. You can then forget about these constraints (you'll get a red test anyway if you don't respect these constraints)
  
• The wooden cleat is set at the beginning of the tiling for the row, you do not verify the alignment at the end. Test first
  
• Nobody told you to screw a wooden cleat on the wall. Its an act of craftsmanship. We do it because we think that it will improve the quality of the tiling. You have to write unit tests, even when nobody tells you to do so.
  
• Sometimes, it is not perfectly clear where the tiling must start, so you just set the tiles on the countertop without adhesive, and then you know where the wooden cleat must be fixed. Can be seen as exploratory testing.
• I need the second row to be perfectly aligned since the first row above the contertop can not be aligned on the countertop itself (tiles thickness may vary). When parts of a system have good unit tests, the integration of these parts is easier.
  

So why do we still see large software systems that are written without any sort of unit tests whereas this seems to be a widespread technique for building physical artifacts (tiling, walls, etc...) ?

I think it is because programming is like tiling without adhesive. You may always be able to move the tiles. But even though we may try to align all the tiles at the end of the project, we would not completely succeed, and the quality of the alignment would suffer.

We should rather use all the available engineering techniques during the life of the project in order to be sure that the alignment is perfect, all the time. From start to end.

OPML import for Rhythmbox, kind of

Update : it seems that Rhythmbox now has support for OPML import, you may type the path to the OPML import in the new feed dialog box. I didn't try it, though

I've just installed the brand new Ubuntu 9.04 to replace my good old 6.06, and now I must find a way to import all my podcast feeds exported in opml format from Juice, to Rhythmbox, the official Ubuntu music manager.

However, it seems that Rhythmbox does not manage opml imports. I don't feel like copying manually 70 rss feeds...

An opportunity to learn the XML library of Groovy !

Save this groovy script as opml2rhythmbox.groovy
<br />#!/usr/bin/groovy<br /><br />if(args.size() != 1){<br /> println("Usage : opml2rhythmbox.groovy myfile.opml")<br /> System.exit 1<br />}<br /><br />File opmlFile = new File(args[0])<br />def opml = new XmlSlurper().parse(opmlFile)<br /><br />def builder = new groovy.xml.MarkupBuilder()<br />opml.body.depthFirst().findAll { it.@type == "rss"}.each { rss -><br /> builder.entry(type:'podcast-feed') {<br /> title rss.@title.text()<br /> location rss.@xmlUrl.text()<br /> status '1'<br /> } <br />}<br />
Then if it is not already done, install Groovy
<br />sudo apt-get install groovy<br />
Execute the script on your exported opml file
chmod +x opml2rhythmbox.groovy<br />./opml2rhythmbox.groovy exported.opml > generated.xml<br />
Now, you must manually copy the content of the generated generated.xml file, into the ~/local/share/rhythmbox/rhythmdb.xml db file of Rhythmbox (make a backup of the rhythmdb.xml file before !)

The generated file content must be inserted after the rhythmdb opening tag at the begining of the rhythmdb.xml file
<br /><?xml version="1.0" standalone="yes"?><br /><rhythmdb version="1.6"><br /><!-- Insert here --><br />
Then start Rhythmbox and run "Update all Feeds"

You're done !

OOP vs Procedural : a definition

In my opinion one of the best definition :

Procedural code gets information then makes decisions. Object-oriented code tells objects to do things. Alec Sharp

That's an explanation for the Tell, don't ask advice for OOP.