skilldrick.co.uk

Thou Shalt Rend Software and Religion Asunder

Steve McConnell, Code Complete Second Edition

There are numerous infamous religious wars in the world of programming. Windows vs. Unix, Windows vs. Mac, IDEs vs. text editors, Emacs vs. vi, Python vs. Ruby, where to put the curly brace, spaces vs. tabs, 2 spaces vs. 4 spaces; the list goes on ad nauseum. The fact that these arguments never end shows that there is no “true” answer. Usually it comes down to personal preference, which is fine. The problem comes when programmers become bogged down in the arguments, and begin to see their side as “the one true way”. Anyone using the other method is an infidel and must be converted or crushed. Of course, this is often just tongue-in-cheek banter, but all too often it goes further than this.

Most of my friends probably have a preference either way for Coke/Pepsi, and some have explained to me their reasoning. But none of them have ever tried to tell me that I’m drinking the wrong drink, because that would be silly (I hardly ever drink Coke or Pepsi, but that’s a moot point).

Alex Martelli wrote a fantastic essay about the differences between Ruby and Python on comp.lang.python. He makes a thorough comparison, noting what is shared and what is different. He also gives his opinion on why he prefers the Python way, while noting that it is only his opinion (speaking on implicit calling of functions vs. requiring empty parentheses: “I do understand why others would thing otherwise, even though I could hardly disagree more vehemently with them:-). “).

The upshot of all this is my vow to avoid all religion in programming. This is something I’ve been doing gradually for a long time. At one point I only ever used Windows, but reading The Pragmatic Programmer convinced me to try Linux. I started off using IDEs (NetBeans, Visual Studio), but then moved to Emacs. I’ve recently started learning Vim properly, and haven’t decided yet which I’ll pick as my primary editor (because you still need to have favourites!). I’d got to a point where I’d virtually declared Python the One True Language — this led me to start learning Ruby. I use 4 spaces for indentation in Python, and 2 in Ruby. I use spaces instead of tabs because tabs are silly and anyone who doesn’t think so is silly too . When I use JavaScript I put braces on the same line as the control statement, because that’s what Douglas Crockford told me to do. I do the same thing everywhere else because I prefer it visually.

Now, I know there are a lot of enlightened programmers out there who already work in this way, so good work. But to all the others, have a look through the list of wars at the top of this post, and see whether you might be able to cross over to the dark side, however briefly, and try out your enemy for a bit. At least then you can give an objective argument about why you prefer one over the other (which always beats “x is teh suck”). Hopefully though, you’ll find that these things just aren’t worth arguing over.

I like to run, and I like tech, so obviously I like to measure and graph my runs. So I use Nike+. Unfortunately, Nike+ has a rubbish pure-flash website (rant: do people still make pure-flash sites? This is 2010!).

I want a way to get to my run data without having to use Nike’s website, so I’ve decided to code something up. I’m learning Rails at the moment, so I’ve decided to do this as a Rails app. In this post, and probably some subsequent posts, I’ll be showing my progress.

Step 1: getting the data

First off, we need some way of getting the data from Nike. This post on labs.interfacedigital.co.uk shows how to get the raw run data through Nike’s public API. The two URLs we’ll be using are this one for getting a summary of all runs:

http://nikerunning.nike.com/nikeplus/v1/services/app/run_list.jsp?userID=#{user_id}

and this one for getting the details of a single run:

http://nikerunning.nike.com/nikeplus/v1/services/app/get_run.jsp?id=#{run_id}&userID=#{user_id}

The all runs XML looks like this:

<plusService>
  <status>success</status>
  <runList endIndex="-1" startIndex="0">
    <run id="1752070113" workoutType="standard">
      <startTime>2010-02-28T13:24:40+00:00</startTime>
      <distance>6.264</distance>
      <duration>4067000</duration>
      <syncTime>2010-02-28T14:34:16+00:00</syncTime>
      <calories>470.0</calories>
      <name/>
      <description/>
      <howFelt>3</howFelt>
      <weather>2</weather>
      <terrain>1</terrain>
      <intensity/>
      <gpxId/>
      <equipmentType>sportband</equipmentType>
    </run>
    ...
  </runList>
</plusService>

Step 2: read the data

I’m doing the following in a Rails model called Run. The underlying table has fields for the necessary run attributes. I’m not going to go into this in detail because this isn’t a Rails tutorial.

require 'open-uri'
require 'rexml/document'

...

attributes = {
 #:attr         #xml tagname
  :distance   => "distance",
  :start_time => "startTime",
  :duration   => "duration",
  :calories   => "calories"
}

open(all_runs_url) do |f|  #open the XML file using the open-uri library
  doc = REXML::Document.new f.read  #create a new REXML object
  doc.elements.each("plusService/runList/run") do |run_info|  #iterate over 'run' elements
    run_id = run_info.attributes["id"]  #get the Nike id of the run
    run = find_or_initialize_by_run_id(run_id)  #get a new Run object, either from an existing record
                                                #in the database or create a new one

    attributes.each do |key, value|  #iterate through attributes hash, which maps object attrs to XML tags
      run.write_attribute(key, run_info.elements[value].text)
    end

    run.calculate_avg_pace  #work out the average pace
    run.save
  end
end

calculate_avg_pace looks like this:

def calculate_avg_pace
  self.avg_pace = duration_in_minutes / distance_in_miles
end

and uses the following virtual attributes:

def duration_in_minutes
  duration / (60.0 * 1000)
end

def distance_in_miles
  distance * 0.62
end

At some point I’ll refactor so miles/km is an option, but for now I just want to get something working. So far though, so good:

Next steps will be to make the numbers look a bit nicer, for example by showing average pace in mm:ss. Other than that, who knows?

Links

• Github page
• Demo version on Heroku
• SlowGeek – Rasmus Lerdorf’s version of the same thing (discovered while writing this post)
• All your base are belong to us – for those scratching their heads over the title

I’ve been using Python for a while and really like it. I’d heard a lot of good stuff about Ruby so decided to have a go to see what all the fuss was about. The list of differences between Python and Ruby on ruby-lang.org was useful in writing this post. Here are my first impressions:

• One of the things that always bugged me about Python is its lack of proper lambdas. Sure, you can use a locally defined function instead, but it doesn’t give you the kind of fluency you can achieve programming in Scheme or JavaScript, say. I haven’t done much Ruby yet, but it seems there are various ways to do lambdas/closure, in a more flexible way than Python. This wouldn’t have mattered to me, but since I’ve been working through SICP and using JavaScript more, I’ve been thinking in closures.

• This doesn’t really set it apart from Python, but it does seem to have good consistent coding conventions, which I think are important for a language. And I like the fact that capitalised class names and lowercase variables are enforced by the language.

• I used to think it sounded dangerous, but I like the fact that parentheses are optional for function calls. I can see how this can make Ruby a very expressive language.

• I’ve never understood why in Python objects have a __len__() method, but the standard way is to use the len() function to access it. Why not treat objects like objects and use method calls?

• One type of array, not lists and tuples. Minor headache resolved!

• Parsed and unparsed strings like in PHP, making use of the distinction between single and double quotes. Useful.

• Private methods – thank you!

• 0 as a truthy value – not sure about this. Although I suppose it’s clearer to say if count > 0 rather than if count.

• elsif instead of elif. That’s going to cause me a lot of typos.

At work I’m stuck on Windows (XP). At home I use Linux (Arch) and Windows (7). I still use Windows because I need to check that software I write runs on Windows, and certain things like iTunes and Spotify still work much better on Windows. I hate developing on Windows though.

I, like many others, would like to be able to have a Unixy (I didn’t think it was a real word, but look!) experience on Windows, so I (like many others) installed Cygwin.

Installing Cygwin

Installing Cygwin’s pretty easy. Just download setup.exe (using the link “Install or update now! (using setup.exe)”) and run it. Follow the instructions, using the suggested values, until you get to Select Packages. Select Packages is a bit like apt-get on Debian or Ubuntu, or Pacman on Arch Linux. It provides an interface for installing and uninstalling packages for your Cygwin installation. For now, go with the defaults, but add Shells -> chere (see below for why). To install chere, click on the text that says “Skip”. This will change to the latest version number of chere. You can run setup.exe again any time without reinstalling Cygwin, so don’t feel like you need to find everything you’re ever going to need. Now click next, and be prepared to wait a looooong time (it could take longer than an hour).

Open Cygwin Here!

There’s a PowerToy for Windows XP which will allow you to right click on a folder and open a command window. On Windows Vista and 7 this is built in. It’s hidden unless you hold Ctrl+Shift when right clicking. It would be even more useful if you could open a Cygwin shell in this way too. Luckily, you can! That’s what chere is for. Just installing chere in Cygwin won’t cut it though, you need to install it into Windows. To do this, run chere -i (for install) in Cygwin. On Windows Vista or 7 you’ll need to run Cygwin as administrator (right click the icon and select “Run as administrator”) for this to work.

Open Explorer Here!

I’d like to be able to do the opposite – that is, open an explorer.exe window with the working directory of the Cygwin shell. This is fairly easy to do as a bash script. Make a new file called openexp or something equally cryptic and save it in /bin (this is a Cygwin dir). Make it executable (chmod +x /bin/openexp) and put the following line in it:

explorer $(cygpath --windows $(pwd))

Now, when you want to open explorer from Cygwin just run openexp.

If you’re new to Bash you may need this explained. Like most codey things, it’s best explained from the inside out.

pwd: pwd stands for present working directory, and outputs the current path.

$(pwd): $() is used for command substitution. $(pwd) is replaced by `/home“ or whatever

cygpath --windows $(pwd): cygpath –windows converts its path argument from Cygwin (Unix) form to Windows form. Its argument will be the working directory in Unix form.

explorer $(cygpath --windows $(pwd)): explorer.exe is Windows Explorer. It takes a directory as an argument – this will be the Windows formatted version of the working directory.

I’ve got a new Samsung R530 laptop. The first thing I did when I got it was to install Arch Linux (my Linux distro du jour). I nearly broke everything when I installed Grub in the Windows boot partition (I think Grub and NTLDR were fighting), but it all turned out well in the end.

I really like developing in Linux – it really is the perfect development environment. It’s just a pain that everything doesn’t work out of the box. The thing I’ve had the most trouble with is the keyboard ‘Fn’ hotkeys. By this I mean combinations like Fn + Up (brightness up), Fn + F4 (switch display) etc. I’m going to walk through the steps I took to get these keys working in Arch Linux in the hope that someone in the same situation as me may be saved the hours of faffing I’ve had to do.

First, I tried finding out the keycodes using xev and acpi_listen, as suggested by this Super User question. xev showed the volume hotkeys working (which were doing their job anyway), but neither showed anything for the brightness keys. I asked the Arch Linux forum for help and was directed to the Extra Keyboard Keys Arch Wiki page. There I learnt my first lesson:

A scancode is the lowest identification number for a key. If a key doesn’t have a scancode then we can’t do anything because it means that the kernel doesn’t see it.

A keycode is the second level of identification for a key, a keycode corresponds to a function.

A symbol is the third level of identification for a key, it is the way Xorg refers to keys.

I used showkey to find out if the problem keys had keycodes, and discovered that they didn’t. If a key doesn’t have a keycode, it can still have a scancode. If it doesn’t have a scancode then the kernel just doesn’t see it, and you’re screwed. To find out if a key has a scancode, run dmesg | tail -5 after pressing one of the keys. This should tell you what the scancode is, and tell you how to map the scancode to a keycode.

So, at this stage I was able to discover the scancodes of all the non-functioning hotkeys. I made a list a bit like this:

• Fn + F2: Show battery – e003
• Fn + F4: Switch display – e002
• Fn + F5: Backlight on/off – e004
• etc…

The next step was to follow the instructions in Map scancodes to keycodes. This told me to look in /lib/udev/keymaps. I found samsung-other in there, which contained the following:

0x82 switchvideomode # Fn+F4 CRT/LCD (high keycode: "displaytoggle")
0x83 battery # Fn+F2
0x84 prog1 # Fn+F5 backlight on/off

In fact, every item in samsung-other matched the keycodes I’d found. Great! The next step is to tell udev to use these keycodes. To do this, make a new file in /etc/udev/rules.d/ called 10-local.rules (as suggested in Writing udev rules), and write:

SUBSYSTEM=="input", ATTRS{name}=="AT Translated Set 2 keyboard", RUN+="keymap $name samsung-other"

Replace samsung-other with your keymap if you’re using a different one. Reboot, then run showkey and see if the keys have keycodes (they should do now). If not, you’ll need to look to Map scancodes to keycodes for advice.

At this point, I’ve got working keycodes, but it seems like X is receiving constant key presses when I press one of them, and it’s causing the computer to crash. I don’t actually have a solution to this at the moment. I thought I’d post this in case my working helps anyone else with a similar problem. Maybe it’ll work for you!

I was in a position where I wanted to rsync some files from my Linux box to a NAS drive. The drive in question is a Lacie Internet Space (like this but white), which runs Linux. I guess because it’s designed mainly for Windows use, it uses the SMB/CIFS protocol.

In order to rsync files to the NAS, I mounted the drive with fstab. This worked OK, but when I tried to do rsync -va --del, I got a load of permissions errors, whether I ran it as me or root. The errors meant that file metadata wasn’t being transferred, but also that locally deleted files weren’t being deleted on the NAS (making the –del option useless).

I searched and searched for the solution, and everybody seemed to have it, but nothing worked. Then I came across KaZim’s fstab page. The answer which was eluding me was to add the noperm option to the relevant line in fstab. (Apparently it’s a problem with smb shares on Linux servers.) Here’s my working fstab entry:

//HIPSERV/MyBackup/Archie /mnt/backup cifs credentials=/root/.eve,nocase,noperm,file_mode=0777,dir_mode=0777 0 0

[It may be obvious to most people, but to mount everything in fstab do mount -a as root.]

And here’s the working rsync:

rsync  -va --del /home/nick/ /mnt/backup/

I was listening to Software Engineering Radio recently; the interview with Gilad Bracha on Newspeak. In it, Gilad talks at length about dynamic typing, and its superiority over static typing. This was the best argument for dynamic typing I’d heard, and it slotted perfectly with my growing respect for dynamic typing.

Some background information: Gilad Bracha co-wrote the second and third Java Language Specifications. He is the creator of Newspeak, a dynamically typed languaged influenced by Self and Smalltalk. While a statically typed language has metadata in the form of type information, Newspeak types can have rich and arbitrary metadata, but its use is not enforced by a type-checker. Tools can be used to read this metadata and give the programmer useful information about the state of the program, but there are no rules its use.

It’s best if I just let Gilad do the talking. I could paraphrase his points but I wouldn’t say it as well.

Gilad Bracha I have a long-standing religious war going on this thing [typing]. I’ve spent a lot of my time working with types; unlike most proponents of dynamic languages I’ve spent a lot of work building static type systems, and I know what they are. And I know what they are good for: they’re mainly good for documentation — documentation for both man and machine. The idea that they make your programs more reliable is a myth. They also have the advantage that it is easier to optimise, but that’s not essential, that’s been proven time again. Most of the hotspot technology that makes Java fast was developed for Self and Smalltalk. It doesn’t necessarily rely on the typing. Some things do, because the typing’s there and people tend to use that. But the real value is that humans who read this can quickly get an idea of what this thing does, and machines can do that, so you can do better name completion, better refactoring, etc., etc.

Markus Völter So with machines you don’t necessarily mean the compiler, but the tooling for the language.

GB Yes. And so I’m all for having what I call pluggable, or optional type systems. The difference is the conventional approach is the type system is part of the language. If you don’t pass the type-checker your program isn’t legal; it won’t run; the compiler will spit it back in your face. The idea with an optional type system is, OK, you write these types; you can run a type-checker; it can tell you what it thinks; it does not affect your semantics; it doesn’t change what the program does, and it certainly doesn’t prevent you from running the program just as if it had no types at all.

MV Why would you run a program whose type-checker, even if it’s optional, tells you your types are wrong?

GB Because what type-checkers do is they attempt to prove that your program conforms to the type system. The type system guarantees you that, if you conform to it, certain types of bad thing won’t happen. That doesn’t mean that if you don’t conform to it bad things will happen. They do not prove that your program is bad; they prove that your program isn’t good, according to their definition of good. And their definition of good is always too restrictive inherently, because the only way to tell if it’s really good is the runtime semantics.

MV But if I try to add 2 to a string, that will never make sense, right?

GB If you add 2 to a string, it may never make sense … but that isn’t really the problem. This is the kind of toy example that theoreticians give you, but that really isn’t the problem. The problems are much more subtle than that. And there are a lot of things that do make sense. It’s sort of like the Anglo-Saxon legal system versus I believe the Napoleonic one: are you innocent until proven guilty or are you guilty until proven innocent? We generally don’t assume that someone should be arrested because he can’t prove that he didn’t do anything wrong. A type system is essentially Napoleonic law. It arrests you because you can’t show that you’re worthy. That’s one issue, the flexibility. There are other issues, which really boil down to documentation.

One of the amusing arguments that people bring up often nowadays: they’ve learned to appreciate IDEs — something that comes from the Smalltalk world, essentially, and the early Lisp world, and so forth — and now they tell you “our IDEs can do such a wonderful job refactoring, and without type information you probably can’t do as well”. And they sort of neglect the fact that where did refactoring come from? It came from Smalltalk. And the Java world, they didn’t invent this, because one of the things these type systems do is it makes it hard to invent things. It makes it hard to do meta-programming, it makes it hard to do things that the language designer didn’t anticipate as typical use-cases. Because then your type-checker says “no, I can’t prove that that’s OK” and it’s very hard to be creative. The wonderful thing about dynamic typing is it lets you express anything that is computable. And type systems don’t — type systems are typically decidable, and they restrict you to a subset. People who favour static type systems say “it’s fine, it’s good enough; all the interesting programs you want to write will work as types”. But that’s ridiculous — once you have a type system, you don’t even know what interesting programs are there.

Edit: Although the following approach works on a small sample application, as the app gets more complex it’s less likely to work. I’ve moved to just using PyQt images in the GUI, and PIL images in the backend. If anyone has any ideas on how to make this work let me know in the comments.

---

Here’s a fun one. If you’re using PIL and PyQt4, you may want to display a PIL image in your PyQt GUI. This is a multi-step process.

1. Create your PIL image (you can use my kitten if you want).

2. Create an ImageQt image from your PIL image (this is PIL’s version of a QImage).

3. Create a QPixmap from your ImageQt image.

4. Create a QLabel and set its QPixmap to be the one just created.

And the above in code:

im = Image.open('kitten.jpg')
myQtImage = ImageQt.ImageQt(PilImage)
pixmap = QtGui.QPixmap.fromImage(myQtImage)
label = QtGui.QLabel('', self)
label.setPixmap(pixmap)

This works fine on Linux, but on Windows it crashes python.exe. I think the problem is with the ImageQt object. According to its documentation. To quote:

Creates an ImageQt object from a PIL image object. This class is a subclass of QtGui.QImage, which means that you can pass the resulting objects directly to PyQt4 API functions and methods.

On Windows it seems that it isn’t quite this easy. There is a simple solution though. Just make a new QImage from the ImageQt object:

myQtImage2 = QtGui.QImage(myQtImage1)

and this seems to turn the fake QImage into a real QImage, which can now be happily turned into a QPixmap.

Here’s a fully working program to illustrate the above in context (this is Python 2.6):

import sys
from PyQt4 import QtCore, QtGui
import Image
import ImageQt


class MyWidget(QtGui.QWidget):
    def __init__(self, parent=None):
        QtGui.QWidget.__init__(self, parent)

        self.setGeometry(300, 300, 400, 293)
        self.setWindowTitle('My Widget!')


        PilImage = Image.open('kitten.jpg')
        QtImage1 = ImageQt.ImageQt(PilImage)
        QtImage2 = QtGui.QImage(QtImage1)
        pixmap = QtGui.QPixmap.fromImage(QtImage2)
        label = QtGui.QLabel('', self)
        label.setPixmap(pixmap)


if __name__ == '__main__':
    app = QtGui.QApplication(sys.argv)
    myWidget = MyWidget()
    myWidget.show()

    sys.exit(app.exec_())

This is the first of a series of posts on the differences between static and dynamic typing in programming languages. This post is intended as an introduction to the topic.

What is a type?

The following definition comes from artima.com:

My definition is that a type is metadata about a chunk of memory that classifies the kind of data stored there. This classification usually implicitly specifies what kinds of operations may be performed on the data.

Types are a way of classifying data in computer programs. Getting down to the nitty gritty, the computer’s processor doesn’t care whether a given variable is a number or a string; it’s just a bunch of bits. In fact, in C, the type of a single character and the type of an integer using one byte are the same: char. It is interesting to note that C does not even have a type for strings; instead it uses a sequence of chars arranged sequentially in memory. The only way the computer knows that the string has finished is by ending it with a null byte, written as \0.

C++ is a newer language, and so it has a type for strings, called string. This gives you certain benefits. In C, you need to allocate enough memory to hold a given string; otherwise errors and security vulnerabilities may occur. The C++ string class abstracts this away, meaning that these security holes can be a thing of the past.

Types are a way of telling the computer the purpose of your data. Multiplying two floats together works differently to multiplying two ints – the processor must know whether the values it is multiplying are floats or ints in order to correctly calculate the result. They are also a way of telling other programmers (and yourself in a few days) the purpose of your code. For example, using a date type instead of using an int to hold a datestamp makes it very obvious that the value being manipulated represents a date, and not just an arbitrary number. Of course, while the computer couldn’t care less whether you are incrementing an integer by one or going forward by a day, to a human being this is a crucial distinction.

What is static typing?

Static typing is used by programming languages as diverse as C, Java, F# and Pascal. In a statically typed language, if x is declared as an int, x will always be an int (in the current scope at least). In other words, types are associated with variables. Older statically typed languages require the programmer to declare the type of every variable, which leads to wastes of typing like the following:

Button myButton = new Button();

Newer statically typed languages use type inference to allow the compiler to ascertain the type of a variable implicitly. Note that while the following is possible in a statically typed language with type inference:

myButton = new Button(); //myButton is inferred to be a Button

this is not:

myThing = new Button(); //myThing is inferred to be a Button
myThing = new Frame();  //myThing is a Button, not a Frame

A statically typed language will check types at compile-time. When the compiler gets to the second line in the snippet above, it will fail, because it will not be able to assign an object of type Frame to a variable of type Button. The compiler will check types wherever they are used, and ensure that no inconsistencies occur in the program.

What is dynamic typing?

Amongst the better known dynamically typed languages are Python, Ruby, JavaScript, Erlang and Smalltalk. In a dynamically typed language, any value can be assigned to any variable. There are still types, but the type is associated with the value rather than the variable. Using Python as an example, we could do the following:

myFile = open('foo.txt', 'r') #myFile is a File object
myFile = myFile.read() #myFile is now a string object
myFile = len(myFile) #myFile is now a number

In a dynamically typed language, type checking occurs at run-time. Before every operation is carried out, the interpreter must check to see if it is possible. Here’s an example in Python:

x = 10
y = 5
z = x * y

When the interpreter gets to the third line, it must first check to see if x and y are compatible with the * operator. If the above example had been in C, both x and y would have had to be declared as ints, or the program would not have compiled. The statement x * y would have been compiled to a single multiply instruction, so at run-time, this multiply instruction is carried out with no type checking.

Summary

So, types are basically just metadata about memory being used by your program. When the program is compiled or interpreted, the compiler or interpreter can maintain strict rules about how types can interact, or it can be a bit more laissez faire. There are good reasons for both approaches, and generally the choice is a trade-off between safety and flexibility.

The only way to get a proper idea of the different approaches and feels is to start coding. Python and Ruby are both great examples of modern dynamically typed languages. Have a look at C++, Java or Haskell for statically typed languages.

My next posts on this topic are going to look more deeply into some of the issues surrounding static and dynamic typing.

Over the next few weeks, I’m going to be writing a number of posts on the subject of static vs. dynamic typing. I think it’s a fascinating area to explore, and it’s one that affects our lives as programmers every day, yet it seems that a lot of people don’t really understand the distinction.

I won’t pretend to be an expert on the subject – I see this as much a learning experience for me as for the reader.

I’m also not going to pretend to be completely neutral – although I started programming in C++, at the moment I mainly program in Python and JavaScript. I’m aiming to keep as neutral as possible, as I don’t believe religion has any place in software development, and I truly believe that there are advantages to both systems.

Coming up

• A basic introduction to static and dynamic typing
• Static typing: the presumption of guilt
• Dynamic typing: a better model of life?