I've been messing around with something called RUDL, which is supposed to be a more Ruby-like port of SDL than Ruby/SDL. This is a pretty good idea, it worked really well with pygame in my opinion, and so I would like to see this kind of thing in Ruby.
Unfortunately the coders for RUDL have done something that is extremely annoying. In some parts of the code, they assume that ints and pointer types are compatible. This is true in a 32-bit environment, since ints and pointers are both 32-bit, however in a 64-bit environment this is not the case. An int is still 32-bit, but pointers are 64-bit. This means that if you are converting from a pointer to an int back to a pointer (or something of that nature), things will not always behave as you expect.
I've seen this kind of thing before, and the situations in which it arises can be subtle. One time I saw someone using the varargs() system and they passed in 0 as a pointer value. This is fine in a 32-bit system, but with the untyped nature of the varargs stuff gcc was only zeroing the lower 32-bits of the value. So setting the pointer to 0 was not actually setting the pointer to zero, so later on when they did if(!pointer) it didn't work and caused the system to crash.
In RUDL the issue is less subtle. Within SDL's audio mixing system, you can label groups with an integer. So the system is using pointers to index these groups. In a 32-bit world this works fine since there is a 1-to-1 conversion for pointers to ints. Unfortunately in a 64-bit world, your pointers do not have a 1-to-1 conversion, so sometimes you will get conflicts and things will act weird.
So yes, if you can avoid it, do not treat an int like it is a pointer! This will make your code non-portable. And even if you do decide to do this, please heed your compilers warnings! gcc does give out warnings for this scenario, and you should listen to them.