ProjectDrinks

Way back in September I tried to get together a group of programmers around Montreal to hang out, have drinks, and talk about various projects that we might be working on at the time. Unfortunately I only did the event once so it didn't really pick up any steam and ended up fizzling out.

I've decided to give it another go, but this time I'm giving it a name and a website: ProjectDrinks, largely inspired by StartupDrinks but without the startup aspect - these are projects and other fun little apps that may or may not have any commercial value, they're purely for enjoyment.

The meetups will be the last Monday of every month starting next Monday (March 28, 2011) at 6:30pm. The location will be Trois-Brasseurs at the corner of St. Catherine and Crescent in downtown Montreal, chosen largely because it's the first place I found that has both beer and wireless - at least according to Île sans fil.

I'll be heading out there with my laptop on Monday evening, having a few beers, and fiddle with a little project I've been messing around with. If anybody wants to come out and chat about any projects you might have on the go feel free to head on down!

Why We Have Financial Crises

How economists view the world:
1) Individuals are all walking in a line
2) Individual i drops a $20 bill
3) Individual i + 1 says, "oh sweet, a $20 bill!"
3) Individual i + 2 do nothing, since such a state of dis-equilibrium normally doesn't exist

A more realistic viewpoint:
1) Repeat step 1 and 2 from before
2) Individual i + 1 says, "hmm, that guy dropped a $20 bill. Does he know something I don't know?" Individual i + 1 drops a $20 as well.
3) Individual i + 2 says, "whoa, those two guys just dropped a $20 bill. That must be a good investment strategy!" Individual i + 2 drops a $20 bill as well.
4) Repeat n times.
5) Individual i + n says, "whoa, what a bunch of suckers!" Individual i + n starts grabbing up $20 bills.
6) Individuals i + n + 1 to i + n + k start grabbing up $20 bills.
7) Individuals i + n + k + 1 start thinking, "whoa, those guys made a ton of cash picking up $20 bills. I'm going to take out a bank loan and expect that I'll grab enough $20 bills from other suckers to pay it back and make fat stacks of cash!

And so it progresses...

Controlling Rhythmbox with your Wiimote

I've been fiddling with one of my Wiimotes tonight to try and get it to control Rhythmbox. Turns out it's actually really easy! You can do it via the cwiid library for Python, and with a Rhythmbox plugin.

For this to work, you need a Wiimote and some sort of Bluetooth device for your computer that works with Ubuntu. You'll also need the correct package:

sudo apt-get install python-cwiid

Here's the step-by-step guide to getting it to work:

1. Create a folder at ~/.gnome2/rhythmbox/plugins if it doesn't already exist.
Create a file in there called WiimoteControl.rb-plugin, and put the following in it:

[RB Plugin]
Loader=python
Module=WiimoteControl
IAge=1
Name=Wiimote Control
Description=A way to control Rhythmbox with your Wiimote
Authors=Rob Britton 
Copyright=Copyright (c) 2011 Rob Britton
Website=http://lovehateubuntu.blogspot.com/

You can replace my name with yours if you want, I won't mind ;)
2. Create a file called __init__.py, and put the following in it:

   import rb
   import cwiid
   
   rbshell = None
   
   def callback(mesg_list, time):
     # Called whenever a button is pushed on the Wiimote
     global rbshell
     for mesg in mesg_list:
       if mesg[0] == cwiid.MESG_BTN:
         if mesg[1] == cwiid.BTN_A:
           rbshell.props.shell_player.playpause()
         if mesg[1] == cwiid.BTN_LEFT:
           rbshell.props.shell_player.do_previous()
         if mesg[1] == cwiid.BTN_RIGHT:
           rbshell.props.shell_player.do_next()
   
   
   class WiimoteControlPlugin (rb.Plugin):
     def __init__(self):
       rb.Plugin.__init__(self)
   
     def activate(self, shell):
       # called when Rhythmbox starts
       global rbshell, bus
       self.shell = shell
       rbshell = self.shell
       self.wiimote = None
   
       print "looking for wiimote..."
       self.wiimote = cwiid.Wiimote()
   
       print "found wiimote"
       self.wiimote.enable(cwiid.FLAG_MESG_IFC)
       self.wiimote.mesg_callback = callback
       self.wiimote.rpt_mode = cwiid.RPT_BTN
   
     def deactivate(self, shell):
       # called when Rhythmbox closes
       self.wiimote.close()

   This code sets up the Wiimote and binds left/right to previous/next song, and the A button to Play/Pause.
3. Open Rhythmbox, and while it is opening push the 1 and 2 buttons on the Wiimote. It will take a sec for Rhythmbox to open since it is looking for the Wiimote.

All this should make it so that your Wiimote can connect and control Rhythmbox. To get debug messages, you can run Rhythmbox from the command line:

rhythmbox -D WiimoteControl

The program isn't ideal, since if you don't connect the Wiimote in time it will not properly set everything. It might be better to put some sort of timer that goes periodically to check to see if the Wiimote is there and if it is not, attempts to connect it. Also I think it would be neat to be able to control the volume using the up and down buttons, but I'm not quite sure how to do that yet. It doesn't seem to be in the Rhythmbox plugin guide, and the other plugins don't really do that sort of thing.

Impressed

This canvas test runs at the same speed on my machine with IE9 and with Firefox 3.6. The difference? The IE9 one was running in VirtualBox.

I always thought it would be a strange day when I was congratulating Internet Explorer, but today I gotta say, not bad! Maybe I should try looking at various other canvas examples to see if this trend is not just confined to this one example.

If2

I was browsing on Rosetta Code this evening in an attempt to find some unfinished tasks to fiddle with, when I found a rather interesting example. The task is to create a new "if2" statement that takes two conditional statements rather than one. In addition to the regular "if" block, it takes three "else" statements that are executed depending on which of the conditions are true. I've actually had this situation come up when I've been programming and at the time it never occurred to me that I could extend Ruby to allow for this scenario.

It turns out you can! It takes a little bit of combinator and anonymous class hackery but it does the job:

class HopelesslyEgocentric
  def method_missing what, *args; self; end;
end
 
def if2 cond1, cond2
  if cond1 and cond2
    yield
    HopelesslyEgocentric.new
  elsif cond1
    Class.new(HopelesslyEgocentric) do
      def else1; yield; HopelesslyEgocentric.new; end
    end.new
  elsif cond2
    Class.new(HopelesslyEgocentric) do
      def else2; yield; HopelesslyEgocentric.new; end
    end.new
  else
    Class.new(HopelesslyEgocentric) do
      def neither; yield; end
    end.new
  end
end

Then you can go ahead and use it like this:

if2(5 < x, x < 7) do
  puts "both true"
end.else1 do
  puts "first is true"
end.else2 do
  puts "second is true"
end.neither do
  puts "neither is true"
end

I thought that was pretty nifty. I doubt performance is amazing, but it is fairly easy to use!

3D Turtles and L-Systems

A while back I posted about these things called L-Systems, which are a way of programmatically generating images based on some fairly simple rules.

As it turns out, you can do some pretty nifty things. The initial images that I put up in my last post used a turtle-graphics type of rendering based on the output of the L-System after a certain number of iterations. I'm doing the same thing here, except it is now a 3D turtle with a number of other things it can do like change colour, change the width of the line, draw polygons, etc.

Here's a few examples. A tree (this one is generated completely deterministically, which is kinda cool):

Here's a nice little flower bush. This one uses stochastic rules for colouring the flowers:

These programs are all in 3D, so if you download the actual code you can circle around the plant, zoom in, etc. Unfortunately I was a bit lazy with the camera system so it is a bit annoying sometimes, but it works well enough.

You can check it out yourself here at the Github repo. I'll be putting up a little guide in the Wiki shortly, so check back there if you want to know how the "language" for the system works.

Testing Real-Time Software

I've come to an interesting problem at work and I've decided to ask you all for your opinion on the matter. Since it's likely that many of you are better software testers than I am, maybe some of you will have some advice.

My issue is that the software I'm writing has to take timing into consideration when doing its calculations: it's a program that analyzes stock behaviour. For example, if I call foo() twice with 2 seconds in between, the result might be different than if I call foo() twice with 4 seconds in between, even if all the inputs are exactly the same. What I want to do is make an automated testing system to verify that the code is doing what it is supposed to be doing.

The first step is to pull out any timing related code into a separate module, so that when running an automated test the script that is supposed to do something over the course of 30 minutes doesn't actually have to wait 30 minutes for the test to pass. Instead, there is a test module for timing that will tell the program that 30 minutes has passed, even if it's only been a few milliseconds.
This also means that the program can't use the traditional timing methods available (since the code is in .NET, that means no System.Timers.Timer or DateTime.Now). This makes things a little bit tricky.

My first thought to try and test this sort of stuff was to use a script like this:

call foo()
simulate 5 second pause
call foo()
assert that result is as expected

The main issue with this is that while the test process does simulate time passing, it does not send the program data during that time. An example of something that wouldn't be easily testable in this case is if I have a program that says, "if the data received over the last minute follows pattern X, do Y." In this example the program is active during the entire minute, it is just watching and waiting for something interesting to happen.

My second thought is what I am thinking of implementing now: I basically have a CSV file for the data. The first column is the time passed since the last amount of data is received, in milliseconds. The other are just arbitrary data, with the first row being a name given to the data. For example I'll have something like this:

Time  Bid BMO  Ask BMO
0     61.73    61.75
500   61.73    61.76
500   61.74    61.77

Then to trigger a test, I have something called a checkpoint. When a checkpoint is hit, it is telling the system to trigger a series of tests. Those tests are run, the results will be reported, and any errors/exceptions thrown will halt this particular test. After that the data will continue until the end of the file is hit.

To me it seems like the testing system is much more data-driven than a normal testing suite, so I'm not entirely certain I'm doing it the right way. However at the same time the nature of the data is not quite the same as with other software, so maybe this sort of approach is a good one. What do you guys think?

Empirical SoEn is Hard: Unobservability

One thing that I find interesting is attempting to apply some of the methods of econometrics and labour economics to measure productivity within software development. At the moment it seems like many of us (myself included) base our opinions of "what works" from our own perspectives and from anecdotal examples (which I've said before doesn't really count as evidence for something) - although this here is an anecdote which may or may not be true.
It would be nice if we could come up with some good analysis techniques to give real support to our claims of what works and what doesn't, and better yet when something works and when it doesn't.

It turns out though that tackling the problem of measuring productivity within software development is quite hard. There are all sorts of problems that can arise here that make analyzing data and testing hypotheses rather difficult. This article will be the first in a series talking about some of the problems that I've though of, and I'd be more than happy to hear about what you guys might think about these issues.

The first problem is that of unobservability (also known as latent variables), which is where you are unable to measure a certain variable because you can't really see the variable directly the same way we can with observed variables. An example of this is ability: it is common knowledge (I think) that there is varying degrees of ability when it comes to developing software. But can we give somebody a stamp saying, "this person has productivity X?" Sure, we can use some indirect measures like lines of code produced per hour or bugs closed per day or some junk like that, but these are simply proxy measures that are the result of ability, not ability itself. Compare this with observed variables like years of experience, or language/methodology used, or team size: we can directly give a value for these variables with some specified unit, so therefore they are observed.

These types of variables are problematic because it is difficult to hold them fixed. Since we can't observe them, we often end up with some omitted variable bias as these variables are often correlated with our other observable variables. This happens when we see an increase in productivity and think it is due to one of our observed variables, when it is actually caused by an unobserved variable that is also influencing an observed variable.

I've thought of a few unobserved variables in software development, feel free to add any more that you think of:
1) Ability (the obvious one). I've already talked about this one to death, so I won't go into much more detail here.
2) Team Chemistry. You can throw a bunch of people into a room together, who individually are extremely good at what they do, but doesn't mean they'll get a lot done - if they all sit there bickering over testing frameworks or variable names or other things like that, they're not going to get a lot done. Likewise you can put a group of people together who may not be super geniuses, but if they work well together you still end up getting some good results. This is an important factor in a team's productivity, and you can't really stamp a number on it.
3) Productivity itself. All this time I've been talking about measuring the effects of various factors on productivity, but we don't actually have a way to directly measure productivity. You can see the indirect effects of the productivity: milestones get reached sooner, bugs get fixed faster, less bugs get introduced in the first place, etc. etc. But we don't have a measure to say something like, "the combination of development methodology X with N programmers of E experience and ... gives us P units of productivity."

I thought about putting the complexity of the project in, however I'm not sure if that affects actual productivity. It can affect some metrics used to measure productivity, but if you view productivity as how much someone is getting done per unit time, then I don't think complexity makes a difference in the same way that say for example, having two monitors does. Maybe I'm also on crack and need to go to bed soon, so this point is up for debate.

All these unobserved factors in software development make things rather difficult to do real quantitative analysis. One possible solution is using experiments to analyze various factors, however those come with their own set of issues that I will discuss in future posts.