Non-deterministic Programming - Amb

I've been very slowly plowing through SICP and I've recently read through their chapter on non-deterministic programming. When you program this way your variables no longer have just one value, they can take on all of their possible values until stated otherwise. An example:

x = amb(1, 2, 3, 4)

In this case x is all of 1, 2, 3, and 4. If you try to print out x it will print out 1 because the act of printing it temporarily forces a value, but otherwise you can treat the variable as though it had all of those values.

You can then force certain subsets of the values with assertions:

assert x.odd?

In this case x would become just 1 and 3. If you then added a final assertion that x > 2 you would force a single value and x would be 3. If you instead added the assertion x > 3 then x would have no values: an exception would be thrown saying that x is basically "impossible".

This is useful when you are searching for something. Suppose you're trying to find numbers that satisfy Pythagorus' theorem:

a = amb(*1..10)
b = amb(*1..10)
c = amb(*1..10)

assert a**2 + b**2 == c**2

puts a, b, c
next_value
puts a, b, c

This code would print out 3, 4, and 5 on the first output, followed by 6, 8, and 10 on the second. The next_value function would tell the amb system to find another solution to the set of variables that satisfy the assertions we specified. If nothing else is found, an exception will be thrown.

Even more interesting, there is a library in Ruby called amb that implements this stuff. Unfortunately it doesn't work just like the above example, you can only get the either the first set of values that fit, or all of them:

require 'rubygems'
require 'amb'
include Amb::Operator

a = amb(*1..10)
b = amb(*1..10)
c = amb(*1..10)

# calling amb with no arguments causes it
# to backtrack until the criteria is met
amb unless a*a + b*b == c*c

# prints out the first match
puts "#{a}, #{b}, #{c}"

# prints out every match and then crashes
amb
puts "#{a}, #{b}, #{c}"

This is a pretty cool way to program, and it would be interesting to know when it might be practical to use. I tried it with a couple of problems on Project Euler but unfortunately since the backtracking method that amb uses isn't always the most efficient approach it would choke a bit. Perhaps if this gem gets some attention and some love, it might end up as something a bit more performant!

Trading with IronRuby

Over the last year I've been using Ruby (more specifically IronRuby) to write algorithmic trading scripts. These aren't high-frequency algorithms, more implementations of various longer-term strategies using Ruby.

There was poll over in an algorithmic trading forum on EliteTrader asking about what was the programming language of the future among Java, Scala, C#, F#, C++ and OCaml. A number of other people posted about other languages such as Python or Lisp, and how well they satisfied four criteria:

1. memory management (GC vs unmanaged)


2. concurrency model


3. static typing vs duck typing (or better yet, type inference)


4. object-oriented programming vs functional programming

I wrote a post on there about my experiences using IronRuby for trading, so I decided I would share that here as well:

I've been using IronRuby for the last year, I've found it works very well as I'm able to go from idea to a working script in no time flat. Performance is not amazing, but the majority of my ideas do not require performance.

For the evaluation (note that pretty much every point also applies to JRuby, the implementation of the language within the Java virtual machine):

1. memory management (GC vs unmanaged)

Uses .NET garbage collector, which is pretty solid. It tends to use a bit more memory than the equivalent VB.NET or C# script, but that's alright. RAM is cheaper than my time.

2. concurrency model

Again, uses .NET threads which are fairly solid. You can use any of the classes in the .NET library for concurrency.

3. static typing vs duck typing (or better yet, type inference)

Ruby uses duck typing. Mixins (aka traits in Scala) means you can write code in chunks and mix-and-match them per script. For example, you could have a SpreadTrade mixin that you just drop into a script and it will make the script do spread trading.

There is a bit of clunkiness when interacting between C#/VB.NET code and IronRuby code due to type conversions and what-not - since Ruby collections can contain an arbitrary mix of types, collections are always treated as collections of Object when passing into C# code.

4. object-oriented programming vs functional programming

Ruby has a very nice blend of both. Absolutely everything is an object (including classes and primitives like integers), existing classes are open (you can add methods to existing classes), and closures passed to methods have a rather succinct syntax. For example, it is possible to build a library where the following code is valid:

symbol.changes_by 2.5.percent do
  # now within a closure with some code to execute when the stock changes by 2.5%
end

At the same time, you still have all your nice functional programming techniques:

# higher order functions:
(1..5).map { |i| i * 2 }   # produces: [2, 4, 6, 8, 10]
(1..100).select &:odd?     # gets all the odd numbers from 1 to 100
(1..100).inject &:+        # produces the sum of 1 to 100

# currying
f = lambda { |a, b| a + b }
g = f.curry[1]
g[2]          # produces 3

Two new criteria that could be added:

5. Meta-programming. In Ruby not only are things dynamically typed, but types themselves are dynamic. Rather than manually coding something every time, I can write one line within a class that will automatically generate the code that I want:

class SomeOrder
  # this code will automatically generate code to cause orders of type SomeOrder to be
  # hedged by an object of SomeOtherOrder
  hedged_by SomeOtherOrder

  # generate code that will cancel the order on some condition
  cancel_if {
    # market is tanking
  }

  # etc.
end

This allows you to write code in a more declarative way, which can lead to less bugs as there are less moving parts you have to manually specify each time.

6. Readability: As demonstrated in the examples above, I can show my scripts to non-programming traders and they are able to follow the strategy fairly easily. They may not understand the meaning of the punctuation, but the flow of the logic can be set up in a way similar to the way they might describe the strategy in English.

Some downsides:
- Like I said, performance is not amazing, it's on par with Python and the other scripting languages.
- When coming from static languages it is inevitable you will get frustrated the first time you stub your toe on the dynamic typing system - you can get some weird bugs when you accidentally pass in a string instead of an array or something like that.
- Those nice declarative meta-programming things can be rather tricky to implement within a library.
- Finally, the language assumes that everybody knows what they're doing. It is possible for some people to write very bad code in Ruby that messes things up - for example, overriding the + operator for arrays to implement vector addition when the standard library has it defined as the array concatenation operator. Imagine your surprise when [1, 2] + [3, 4] produces [4, 6] instead of the expected [1, 2, 3, 4].

Out of the readers who are also Rubyists, what do you guys think about this topic?

Ruby's instance_eval and Structured Configuration Files

Recently at work I was setting up a section of the program involving a user-configurable GUI element. Initially I thought about using something like XML to handle this, however if I went that route I'd have to manage parsing the XML through some library, traversing the XML tree, and create some sort of GUI element based on what type of node it was, what the attributes were, etc.

On top of that it was highly likely that the requirements for this GUI system would change and the level of sophistication behind the GUI system would increase - buttons would need to do more complicated tasks that could potentially be arbitrary. Having a hard-coded system in C# would not be feasible since tweaking logic would require a re-compile and an application restart.

I decided that since the system already had an IronRuby install built in, perhaps I could do this configuration file in Ruby as well. Given an XML configuration file that looks like this:

<group name="Buttons">
  <line>
    <button text="Cool Script">
      <something_crazy_and_awesome />
    </button>
  </line>
</group>

I could translate this to Ruby that looks like this:

group "Buttons" do
  line do
    button "Cool Script" do
      something_crazy_and_awesome
    end
  end
end

On top of that in order to use this configuration file in a program I don't actually need to parse the file, I can just execute it within the context of different objects. For the dialog box that allows the user to set up the configuration I can just execute the Ruby code within that dialog box object and it will automatically create the elements that allow the user to change the configuration. Then in the part of the application where the user actually uses the configuration, I can just execute the file again but with a different definition of group, line, etc. that construct the proper GUI elements.

This is not just applicable to GUI elements, you can use this for anything that uses some sort of structured configuration. Rake uses this to great effect with tasks. But how would you go about implementing this pattern?

Turns out it's really simple using instance_eval. Here's an example that constructs the GUI element (this is in
IronRuby so it uses .NET for GUI construction):

class GroupBuilder
  def initialize parent, name, &block
    @group_box = GroupBox.new(name)
    parent.Controls.Add(@group_box)
    instance_eval &block
  end

  def line &block
    LineBuilder.new(@group_box, &block)
  end
end

class LineBuilder
  attr_accessor :panel

  def initialize parent, &block
    # in the GUI environment we use a panel for adding things
    @panel = Panel.new
    parent.Controls.Add @panel

    instance_eval &block
  end

  def button text, &block
    ButtonBuilder.new(@panel, text, &block)
  end

  # ... anything else that can be placed in a line ...
end

class ButtonBuilder
  def initialize parent, text, &block
    # create a button
    btn = Button.new(text)
    # bind the click event of the button to execute the
    # Ruby code within the block
    btn.Click { self.instance_eval &block }
    parent.Controls.Add btn
  end

  def something_crazy_and_awesome
    # do something crazy and awesome
  end
end

class ScriptProcessor
  def initialize control, script
    @control = control
    instance_eval File.read(script)
  end

  def group name, &block
    GroupBuilder.new(@control, name, &block)
  end
end

To do all this stuff, you can just create a ScriptProcessor object, pass in a .NET control and a script filename:

f = Form.new
ScriptProcessor.new(f, "my_config_file.rb")
f.Show

For each type of nested element you can create a class which define a method for the various types of sub-elements that you are allowed to have. Each of those methods will then handle the processing that needs to happen when the system sees an element of that type.

I think you could probably do this without classes and just use lambdas, but I think the code is clearer when you have objects since it is very explicit as to what each object is for.

Doing this with C# is a bit trickier since C# doesn't have instance_eval, but it turns out you can have a bit of a hack in order to get it to work quite well. I'll write up a quick post about this at a later date.

If2

I was browsing on Rosetta Code this evening in an attempt to find some unfinished tasks to fiddle with, when I found a rather interesting example. The task is to create a new "if2" statement that takes two conditional statements rather than one. In addition to the regular "if" block, it takes three "else" statements that are executed depending on which of the conditions are true. I've actually had this situation come up when I've been programming and at the time it never occurred to me that I could extend Ruby to allow for this scenario.

It turns out you can! It takes a little bit of combinator and anonymous class hackery but it does the job:

class HopelesslyEgocentric
  def method_missing what, *args; self; end;
end
 
def if2 cond1, cond2
  if cond1 and cond2
    yield
    HopelesslyEgocentric.new
  elsif cond1
    Class.new(HopelesslyEgocentric) do
      def else1; yield; HopelesslyEgocentric.new; end
    end.new
  elsif cond2
    Class.new(HopelesslyEgocentric) do
      def else2; yield; HopelesslyEgocentric.new; end
    end.new
  else
    Class.new(HopelesslyEgocentric) do
      def neither; yield; end
    end.new
  end
end

Then you can go ahead and use it like this:

if2(5 < x, x < 7) do
  puts "both true"
end.else1 do
  puts "first is true"
end.else2 do
  puts "second is true"
end.neither do
  puts "neither is true"
end

I thought that was pretty nifty. I doubt performance is amazing, but it is fairly easy to use!

3D Turtles and L-Systems

A while back I posted about these things called L-Systems, which are a way of programmatically generating images based on some fairly simple rules.

As it turns out, you can do some pretty nifty things. The initial images that I put up in my last post used a turtle-graphics type of rendering based on the output of the L-System after a certain number of iterations. I'm doing the same thing here, except it is now a 3D turtle with a number of other things it can do like change colour, change the width of the line, draw polygons, etc.

Here's a few examples. A tree (this one is generated completely deterministically, which is kinda cool):

Here's a nice little flower bush. This one uses stochastic rules for colouring the flowers:

These programs are all in 3D, so if you download the actual code you can circle around the plant, zoom in, etc. Unfortunately I was a bit lazy with the camera system so it is a bit annoying sometimes, but it works well enough.

You can check it out yourself here at the Github repo. I'll be putting up a little guide in the Wiki shortly, so check back there if you want to know how the "language" for the system works.

Rug

Over the last few months I've been working with Concordia's CART CGD group, which is a group of computer art students who want to make video games. We did a few things called "game jams", which is where a group of people meet together and try and churn out a game in a few hours. We didn't actually come up with anything that great this last semester, but then again there was only two game jams (if you're interested in participating, they have them every Saturday starting January 15th, send me an email and I can give you more details).

We did the game jams with Lua and a game framework called Löve2D, which helps simplify game development.

I found after learning a bit of Lua that it is a lot like Javascript (although I have to say I still prefer Javascript). It isn't an object-oriented language in the same way that C++ or Ruby are, but it is still possible to create objects with methods and inheritance and so on. I found in the end that I need a little bit more structure when I'm programming than is offered by Lua - I actually also found the same thing with Javascript when starting to build some larger Javascript apps like Colonial.

However, I found that I really liked Löve2D. A lot of things were very simple and straight-forward. Before that the nicest gaming library I worked with was pygame which is a wrapper around the C version of SDL and a few extra bells and whistles like collision detection and sprite handling. There's a port to Ruby called RubyGame, but I find it suffers from the same problem as pygame: it's just a wrapper around SDL with a few bells and whistles.

I decided to go and roll out my own Ruby gaming library called Rug. It's mostly written in C++ for speed, but the API is much more Ruby-ish than Rubygame or Pygame with a few hints from Löve2D. At the moment it doesn't have a lot of features; it supports some rudimentary collision detection and animations. It doesn't stick completely to Löve2D's API, there were a few things in there that I found kinda clunky when it came to animations that I fixed up.

You can see an example of Pong here with a screenshot:

Or a very simple platformer here, with a screenshot:

The library is still very new, and there are a few things that I'm not happy with that will change in the near future (in other words if you choose to fiddle with this, don't be surprised if things change - but then again if you've had any experience with Rails you will be used to this sort of thing ;) ). Documentation still has a ways to go, I started documenting the features religiously but in the end I found that I changed the way I wanted the library to work often enough that documenting can wait until things are a little bit solid. For example the physics module for collision detection has undergone quite a few API changes, and after having to update the documentation several times I decided that I will wait until I am happy with how the module works before telling everyone else how to use it.

So anyway, feel free to play around with it and tell me what you think. Installing it is pretty easy in Linux, however in Windows it can be a bit annoying with all the Ruby headers and the various SDL libraries. I managed to get it to compile in Windows, so shortly I'll put up a zip with all the various DLLs that you need to run Rug.
I have no idea how easy/hard it will be to set up on a Mac, if it works anything like Linux you can probably just use MacPorts to install SDL and the Ruby development headers.

Second Thoughts on IronRuby

Unfortunately after using IronRuby at work a bit, I've found that there are still a few too many bugs with the thing for me to want to depend on it for production software. The bugs are already reported, however they are tagged as low priority and will not likely be fixed any time soon. Rather than spending lots of time fixing the bugs myself (I get paid to make profit, not to work on open-source pet projects), I'll just continue doing things the way that I have been!

Hopefully someone else will find IronRuby useful!

In case you're wondering, the bug that really killed it for me was the inability to use gems with RVM under Linux. The following command fails with issues installing RubyGems:

rvm install ironruby

No Rubygems drastically reduces the usefulness of Ruby, enough that I'm not really wanting to use it anymore.

Embedding a Ruby REPL in a .NET App

I've finally gotten fed up with using VB.NET as a scripting language at work, so I decided to try dropping an IronRuby interpreter into the system. It's fairly simple to do, I'll describe here how to build a REPL within your app.

Step 1: Make a window for the REPL. Give it a box for your input and a box for your output. How you do all this is up to you, the interesting part is how to use Ruby within it.

Step 2: Initialize the scripting engine. You need a config file, I call it ironruby.config:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
 <configSections>
   <section name="microsoft.scripting" type="Microsoft.Scripting.Hosting.Configuration.Section, Microsoft.Scripting,
     Version=1.0.0.5000, Culture=neutral, PublicKeyToken=31bf3856ad364e35" requirePermission="false" />
 </configSections>

 <microsoft.scripting>
   <languages>
     <language names="IronRuby;Ruby;rb" extensions=".rb" displayName="IronRuby 1.0" type="IronRuby.Runtime.RubyContext, IronRuby,
       Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
   </languages>
 </microsoft.scripting>
</configuration>

You need to make sure you have the DLLs available that IronRuby comes with. Just put them in the same folder as your app.

Now the code to load in your Ruby stuff. Initialize a few objects:

var runtime = new ScriptRuntime(ScriptRuntimeSetup.ReadConfiguration("ironruby.config"));
var engine = runtime.GetEngine("Ruby");

// create a scope - you need to do this in order to preserve variables declared across executions
var globalScope = engine.CreateScope();

// I'm assuming you have a textbox or something named command, just adjust
// this to what you want if you have something else
var result = engine.CreateScriptSourceFromString("(" + command.Text + ").inspect").Execute(globalScope);

And, jackpot! The result variable is a string that is the result of whatever is in the command.Text variable.

This code could be a little bit more robust, so make sure to handle exceptions nicely. An exception will be thrown if say the syntax is invalid, or there are other problems.

The nice thing about this is that you can tinker with the classes within your own application. In the REPL if you put:

# replace MyApp.exe with the .exe or .dll file that your app uses
require "MyApp.exe"

You can then access your namespaces and classes within your own app, and fiddle with them while the app is running!

Ruby Syntax Gem

I've been using a gem called syntax to transform Ruby code into something that looks nice in HTML. It is quite easy to install:

sudo gem install syntax

And easy to use:

require 'rubygems'
require 'syntax/convertors/html'

convertor = Syntax::Convertors::HTML.for_syntax "ruby"
html = convertor.convert(File.read(ARGV[0]))

By default it supports Ruby, XML and YAML, and there might be a bunch of other ones out there on the Internet somewhere.

Speaking of syntax, I'm wondering if anybody knows of a good colour scheme for Gvim. I've been using one called pablo that comes with it for a while since it is pretty much the only dark theme that they have that doesn't make my eyes bleed, but it still isn't amazing. Does anybody know of one that looks really good that they are willing to share?

Case Classes in Ruby

After working a bit in Scala and using the case classes and pattern matching, I've found it's pretty nice to have and it would go well in Ruby! So I've whipped up a little something basic, you can check out the Git repo here: http://github.com/robbrit/ruby_case_classes

What are case classes? They're basically simple classes that don't really do much other than store a set of values (possibly, they don't even need to do that). They're good on their own if you have very basic classes that you want to use, for example exceptions.

With my code, you define a case class like this:

case_class :ClassName, BaseClass, <parameters>

The parameters you pass at the end are the fields of the class and when you instantiate the object you can set them:

case_class :Hello, Object, :title
...
h = Hello.new(:title => "some title")
h.title  # => "some title"

Unfortunately at the moment you have to specify the name of the field.

Case classes are handy on their own to save a few keystrokes, but where they help a lot more is with pattern matching. Pattern matching comes from functional programming and was adopted by Scala. It is a handy feature to have, so that's why it is here.

To do pattern matching:

case_class :Blah, Object, :title
case_class :Blih, Object, :title
...
blah = Blah.new(:title => "hello")

match blah do
  # Type matching
  for_case Blih do
    ...
  end

  # Guard and type matching
  for_case Blah, :title => "hello" do
    ...
  end

  # Just guard matching
  for_case "_", :title => "hello" do
    ...
  end

  # Multiple guards, one of which is a method call
  for_case "_", :title => "hello", :to_i => 0 do
    ...
  end

  # Match anything - you could put "_" here, but it's optional
  for_case do
    ...
  end
end

Here we have four examples of things you can match against. You can match against just a type, you can match against a type with a guard, you can match against just a guard, and you can have a "default" case if it doesn't match anything.
Guards are basic equality conditions that you can use to match against. If the object fits the guards, then the block is executed.
Note there is a fall-through here. The case that is executed is the first one to match the object, later ones that match are not executed.

Some possible improvements:
- match against values instead of just types. For example:

for_case [] do ...

- inequalities instead of strict equalities in guards
- case class new() doesn't need named parameters
Any other suggestions are welcome.

There's one bug (that I know of). It doesn't work at the top-level. So if this is your entire file, it will fail:

include CaseClasses

case_class :Blah, Object

You need to wrap it in some object. I'm probably having a brain-fart or something as to how to fix this, but for now you'll have this limitation. If anybody has a solution around that, feel free to let me know.

Jaunty and RMagick

If you're doing Ruby development under the Jaunty and you're using RMagick, make sure that after you upgrade to reinstall the libmagickwand-dev package, and then reinstall the RMagick gem:

sudo apt-get install libmagickwand-dev
sudo gem uninstall rmagick
sudo gem install rmagick --no-ri --no-rdoc

I can imagine there may be issues with other languages too, it seems like Jaunty replaces/upgrades some ImageMagick libraries, so you'll have to patch it up after you upgrade.

UPDATE(Apr. 27, 2010): The latest version of RMagick does not work with the most recent version of Imagemagick in the Ubuntu repository. To install the version of RMagick that does work, you need to use version 2.11:

sudo gem install rmagick --no-ri --no-rdoc --version=2.11

Hungarian Notation in Ruby?

Way back when I was first learning to program, I was quickly introduced to the Win32 C API, which uses a lot of Hungarian notation on the variable names. This got really annoying really quickly, especially back then when there was a difference between a "long pointer" and a "short pointer", you'd get stuff like "g_lpasz_..." which means "long pointer to an array of zero-terminated strings, which happens to be a global variable". It can get nastier.
Eventually I dropped this, because in C/C++ you explicitly declare the type of a variable anyway, and the IDE that I used at the time (Visual Studio) would display the type anyway. It was both redundant and slightly verbose, so it didn't really come as a surprise that when I came to Java in university and saw that it did not use this notation that maybe it was worth ditching (it's one of the few things that Java ditched from C++ that I actually agreed with). So now I don't use it when I'm programming C++.

Most of the time these days, I'm not programming in C++. While I still like the language - probably because I haven't found a language that does what it does well better, namely execution speed with OOP/generics - most of the projects I seem to do are better suited to Ruby - there are probably better languages to use for some of the things I do (OORegress would probably be better off in Scala since it does number crunching) but whatever.

As my apps and web apps get more complex however, I'm thinking some sort of very basic type system would be in order. Since in Ruby you can never be really certain what type a variable is (I've been bitten in the ass in Rails by Hash vs. HashWithIndifferentAccess), having some sort of hint to other programmers what type the variable should be would be quite helpful. Just a thought.

Like most things, it is best to use it sparingly. If your Hungarian notation portion has more than say 5 characters in it, it's getting a little long. Plus since in Ruby a parameter may have more than one expected type - in Rails, ActiveRecord::Base#find can take an integer or a symbol for the first parameter - and lists can be heterogeneous there are situations where the system may not be comprehensive enough for what you need. However for 90% of the times, I think it would be an improvement.

The Freaky Sandbox

One thing that can always be useful is the ability to run "unsafe" code inside a sandbox, where it can do limited damage. For a project I am working on, I wanted to do this very thing. Users would be able to upload their own code for execution on the server.

Naturally, this leads to a whole whack of security issues. If you just run code unchecked, users could get access to files, threads, processes, all sorts of things.
Fortunately there is this gem called the Freaky Sandbox which lets you run Ruby code, and it disables access to many security sensitive areas.

Unfortunately the gem doesn't work anymore. If you check out the source and install it, all sorts of bad things happen when you actually try to use the gem.
So before diving in to fix this, I decided to see if JRuby had a similar gem. Fortunately, it does. Now when I first tried it with JRuby 1.1.something it also didn't work, but I made some tweaks to it and now it does. Also with JRuby 1.2 RC1 it works (and also with RC2).
UPDATE (Mar. 28/10): The gem in the repositories no longer works, however you can check out the code to the javasand gem and install it using these instructions.

So now you can do fancy things like this:

require 'sandbox'

s = Sandbox.safe

File.open("untrusted_code.rb") do |f|
  s.eval f.read
end

What it does is filters out things like Thread and Kernel.fork, and file manipulation, things like that. It also doesn't see any classes that you have defined unless you import them into the sandbox.

So how could this be useful? One thing I'm thinking of is to allow users to write custom scripts for a site. Of course most users wouldn't be able to take advantage of this feature, but if script writers made their scripts available then they could just grab it and install it.

You could try something like this:

sandbox = Sandbox.safe

sandbox.import RandomAPIClassThatIWrote

res = s.eval untrusted_code

# do stuff with res

Unfortunately importing modules doesn't seem to work, it gives a NullPointerException when you try. If I get some free time soon I may look into it and try to fix it, but for now I'll just have to live with importing classes.

Now the question is: is this perfectly safe? Well, no. The code you run in the sandbox can still have infinite loops in it which will halt your program. Unfortunately it is impossible to tell if arbitrary code will have infinite loops in it or not, so we're out of luck for fixing this problem. However what you can try is the following:

sb_thread = Thread.new do
  s.eval untrusted_code
end

sleep TIMEOUT

sb_thread.raise if sb_thread.alive? # use raise since kill doesn't seem to work

Normally this is unsafe, because of blocking operations like synchronous IO and things like that. However with the sandbox, you cannot use the require keyword. You also have no access to the File or Dir classes (they are not defined inside the sandbox). So correct me if I'm wrong but unless you go and import these classes into the sandbox there are no unsafe things that could happen by just killing the thread.

Erb Comments in JRuby vs. Ruby

With my current job, we built up most of the software with JRuby. More recently I discovered that development is faster using Ruby and then deploying on JRuby.

However then we discovered some bugs that were in code that hadn't changed in a long time. For some reason, certain chunks of our view templates were just disappearing. Nobody could figure out why.

This is where Vim came unexpectedly to the rescue. You can write comments in erb by writing something like this:

<%# this is a comment %>

You can also go like this:

<% # this is a comment %>

However with the latter type of comment, it would always mess up the syntax highlighting in Vim. It wouldn't recognize that the %> was closing the erb comment tag, because it thought it was part of the comment. I had noticed this before but didn't really care that much, because it wasn't my code and I didn't really want to mess with it. But this time in order to find the bug I figured I should have correct syntax highlighting.
Anyway after fixing the comments, I couldn't find anything wrong with the file. I was still baffled. So I figured I'd just go and look at the page in the browser again, and boom! It worked!

My hypothesis is that Ruby and Vim probably use the same logic for parsing those erb comments, but JRuby uses something different. JRuby would behave exactly the same if we had <%# or <% #. Ruby on the other hand will think that anything after the # is part of a comment, and therefore will ignore the %> that is on the same line.

JRuby on Rails and Development Efficiency

I've been working with JRuby for about six months now, and it has been pretty good. It has native thread support, and you have a number of different options available from the Java world.

As a deployment strategy, JRuby is pretty solid IMO. However from a development perspective it is a bit slower than MRI. The biggest one you notice is that JRuby takes a while for the JVM to warm up. This is fine if you're just running Mongrel or WEBrick or something, but when you have a bunch of small scripts or Rake tasks to run or something to play around with in irb, it is quite annoying to have to use JRuby and wait that extra few seconds for the JVM to load. Also for some reason it takes way longer for my test suite to run in JRuby than with MRI. Oh well, whatever.

Another problem is that many gems are native, and therefore not available to JRuby. At this point JRuby has enough of a following for popular gems to have a JRuby port somewhere, but the matter of finding it and getting it to work on all your developers' machines is a pain in the ass. Better to just do it once on the deployment machine(s) and be done with it. Some examples of gems that don't work in JRuby: rcov, RMagick, mysql, anything to do with datamapper. The memcache-client gem used to work, if you use version 1.5.0 it works fine but the latest one fails.
EDIT: There's been a bit of confusion by what I meant here. What I mean is that the gems in the repository do not work with JRuby, so going 'jgem install GEM' does not work, you have to find the port online. This isn't usually that difficult, but a bit more time-consuming than the standard way of doing things.

However I'd say JRuby is great for production for a few reasons. First off, it has access to native threads. I believe Ruby 1.9 uses native threads, but my Rails app currently does not work with the Ruby 1.9 available in the Ubuntu repositories and I'd rather not have to maintain a new Ruby install unless absolutely necessary.
JRuby also has access to a wider range of application servers. Mongrel works well with JRuby, and any other web server written in Ruby should work fine as well. JRuby can also be deployed as a WAR with any application server that uses WAR files. We're using Glassfish, but I think you can do it with Tomcat and others too.
Finally, JRuby has access to Java libraries. Say what you will about Java the language, there are a ton of Java libraries out there. For basic stuff, Ruby has pretty much everything it needs, but when you want to move outside of web development things get sparse quickly. Want to write an OpenOffice plugin? JRuby can do it by using OpenOffice's Java API. Want to use a NLP tool like GATE? The API is in Java. Where are things like this for Ruby?

Anyway, IMO ideal setup is:
development - Ruby, unless you're using some Java libraries like I mentioned above
production - JRuby
This may change as Ruby 1.9 gets better, but at the moment I'm liking the above setup.

Peach

A coworker referred me to this little Ruby gem called Peach, which is a parallel processing gem designed to speed up each/map/delete_if by dividing up the work among several threads - something always good.

I did some checks to see how much faster it is. Unfortunately I couldn't see a speed improvement, and for the basic map: i => 2i the peach versions were actually much slower, likely due to the overhead of splitting up the collection and merging the results.

A couple gotchas - make sure to set $peach_default_threads, or you'll be sorry when working with massive arrays. The default is to use one thread for each element in the array. This is fine for arrays with like 3 elements and the operation takes a long long time, but for arrays with 100 000+ elements, that's just insane - the overhead is not worth the gain.
The gem doesn't show much improvement on MRI, probably because MRI uses green threads. Also I can't test this, but it may not show too much improvement on a single-core processor, depending on what you're doing. So basically it is much better to use JRuby for this on a multi-core machine, since JRuby uses native threads and can actually take advantage of the hardware available.

Digging through the code a bit, I can see some points of potential optimization due to the natures of the operations. Right now the code splits up the array into a number of sub-arrays based on the number of threads to run, then executes the function, and finally merges the results. For Array#each, this can be done in place as Array#each just returns itself - no need to split the array and remerge it afterward.
Array#map on the other hand cannot be done in place. However since the size of the output is the same as the size of the input, the new array can be allocated before the threads begin and each thread works with indices. This saves a massive amount of time merging the arrays afterward, since Array#+ creates a new array and copies all the elements from the old arrays into the new one.

The gem is still in a young state, there are plenty of places for optimization (I think I will try my hand at this). It only works with Array, so most other enumerable types are not supported yet. Also there are only the three operations that are supported, no inject or anything yet. Finally, you must use your brain when doing parallel processing. Side effects = bad. They introduce all sorts of problems with race conditions. So try to avoid them when using peach.

All in all, this is an awesome idea and with some publicity, the open-source community will improve this gem big time.

UPDATE: I made my own version of pmap which allocates a new array and modifies that array directly, and it doesn't provide a huge boost in speed for larger collection sizes. For smaller sizes it is faster (by a lot) but later on not so much. I would gather this is because the overhead of Peach is a much smaller chunk of the processing time in the long run than the processing of each sub-array. So the tweak shaves off a bit of time, but not a lot. I will post my tweaks at a later date because I want to add some more functionality like inject.

Ruby and Recursive Send

Some dynamic languages have the ability to call an arbitrary method of an object. The name of the method to be called is usually stored inside a string.

Ruby is no different. You use Object#send to call a method:

"5".send(:to_i)   # => 5

This isn't that useful for a small thing like that example there, but for more sophisticated applications this can save you a lot of typing.

For something I'm working on, I have some code that gets repeated a lot on different return results of an object. So I figured I'd just stick all the things I had to do in a string, iterate over the string calling send. However some of the properties I wanted were a little complex, as in they were calling the methods of a return result of a method. For example (something I may have used on my freegamage site):

["genre", "user.name"].each do |method|
  game.send(method)
end

Unfortunately the "user.name" property is not actually a method of game, so it fails.

We have a solution though:

class Object
  def send_r(method)
    method.split(".").inject(self) { |ob, meth| ob.send(meth) }
  end
end

This little method assumes you're using something like "method1.method2.method3..." when you call it, and then recursively calls each method on the previous method's return value.

Win/Fail in Ruby

Very simple yet quite amusing:

def win
  true
end

FAIL = nil

Put this somewhere in your Ruby code, and you'll be able to write:

if something == win
...
if something == FAIL

Originally I had it with a lowercase fail, but that might conflict with some library somewhere. Plus it is cooler if it is in all caps.

This was inspired by a tweet from Giles Bowkett.

Ruby Array#to_hash

The number of times I've had to convert an array to a hash is amazing. And the sad thing is that the standard Ruby array doesn't have any simple way to do it! This is probably because there are a lot of ways you can convert an array to a hash.

Here's the way I do it a lot:

hash = Hash[*keys.zip(values).flatten]

This works, unless your values array has nested arrays in which case you're screwed. A better alternative is:

hash = keys.zip(values).inject({}) { |h, nvp| h[nvp[0]] = nvp[1]; h }

This handles nested arrays, but it makes Ruby elegance freaks cry. But that's not my problem, so let's go like this:

class Array
  def to_hash
    self.inject({}) { |h, nvp| h[nvp[0]] = nvp[1]; h }
  end
end

Assuming that the array is in name-value pair format, this should work fine. So you have:

[ [:a, 5], [:b, "hello"] ].to_hash   # => {:a => 5, :b => "hello" }

There are probably problems with this, if you know of any please comment!

Shrink your Pictures

There may come a time when you copy your images off your digital camera and realize that they are at some ridiculous dimension like 2000x1700, which is not really a nice size to be emailing to your friends and family.
It might be ok if you just have one to send, but when you have like 30, it is not so easy.

So here's a little something to shrink them all:

Dir.mkdir "small" unless File.exists?("small")

Dir.foreach(".") do |file|
  next if file !~ /\.(jpg|png)$/i

  puts "Shrinking #{file}"
  `convert #{file} -resize "300x300>" small/#{file}`
end

This shrinks all the jpgs and pngs in a folder to a maximum of 300x300 while maintaining the aspect ratio. If the image is smaller, then it is not resized. It dumps all the images in a folder called small.

Or if you don't like Ruby, here it is in bash:

mkdir small
for i in $(ls | egrep -i "\.(jpg|png)$"); do
  echo "Shrinking " $i
  convert $i -resize "300x300>" small/$i
done

Now you can send your pictures to all your friends! Note that in order to use these, you'll need ImageMagick installed. As for doing this in Windows, I have no idea. A good bet might be to use RMagick and then use rubyscript2exe to make it work.

UPDATE: Suppose there are also some pictures that you want to rotate too. Luckily for us, ImageMagick handles this too. Unfortunately I don't know bash well enough to update the bash one, but for Ruby:

Dir.mkdir "small" unless File.exists?("small")

Dir.foreach(".") do |file|
  next if file !~ /\.(jpg|png)$/i

  out_file = ""
  commands = ""

  if file =~ /^rotate(\d+)_/
    commands += "-rotate \"#{$1}\" "
    out_file = file[/_(.*)$/, 1]
  end

  puts "Shrinking #{file} to small/#{out_file}"
  `convert #{file} -resize "300x300>" #{commands} small/#{out_file}`
end

Now if you prefix any files you want to rotate with "rotateX_", it will rotate your files clockwise by X degrees. I'd recommend using 90.
It's great with any file browser that shows the thumbnails of the image, so you can just look, rename (by pressing F2 in Nautilus) and go.

This is also pretty easy to add on to if you want to add more filters. ImageMagick has a lot of stuff it can do - you can even make things into polaroids if you want.