Flutterby™! : Co-worker is debugging a Ruby deployment 2013-12-11 22:21:06.993102+00

Co-worker is debugging a Ruby deployment

2013-12-11 22:21:06.993102+00 by Dan Lyke 4 comments

Co-worker is debugging a Ruby deployment, reinforcing that if your development environment doesn't involve C and an ICE, it's too abstract.

#Comment Re: made: 2013-12-11 23:57:17.872456+00 by: Jack William Bell

Also, if your linker is calculating your process headers you have too much automation going on.

I'm not averse to automation, but when things go wrong, I want to be able to track down where and how.

This sort of accountability for code does not seem to be part of modern software development trends.

My comment grew out of an experience I had writing C code for an embedded device. We had to use the linker with the automation turned off, passing in a map of the executable code addresses, because we needed to swap code in and out of flash at runtime. (Not enough RAM to load everything at once.)

The problem was that we couldn't turn off some of the automation; it was all or nothing. This meant we had to build the process headers for the memory map from code as well. Pain in the ass, but I figured it out and everything was good.

Then I started getting really weird crashing bugs. After a LOT of debugging I found that the execution pointer was getting corrupted. Only to random addresses at random times. I couldn't figure it out. We ended up flying in a hardware engineer to work with me on the problem and between us we figured it out...

Turns out the linker was supposed to align on 16 byte boundaries, no matter what we had in the memory map. One of my process boundaries was off by 4 bytes, just enough to put the process stack pointer into the mapped area for the previous process. Which should have been OK because the linker would fix it, only it didn't because that was an un-documented feature of the automation.

This meant that, every now and then, the stack pointer for that process would get overwritten with a value from another process. Then whatever function was running in my code would attempt to jump to the return address in the stack frame, setting the instruction pointer to a random value.

Ouch! Luckily all of the times I had to do that were back in the 386 "memory extender" days (.RTLink for the win), or I've been able to steal boilerplate for it.