Tools sometimes define the problem

May 23, 2013

When I have a bug of the sort “nothing is working” or “the peripheral is acting like the processor isn’t talking to it”, the digital multimeter (DMM or DVM) is the tool I start with. It sits on my desk, all the time, even after a clutter purged. It is easy to use but a very yes/no, working/not working sort of tool. It is kind of binary, like a hammer. It is either hitting the nail or not.

If something is not working, the next tool is an oscilloscope. Like a microscope for electrons, an oscilloscope will let me see what is going on. However, to get an oscope, I need to call an EE friend and ask if I couldn’t please borrow his scope (he always says yes). Then I head over to his house after he gets home from work (I think his wife is afraid of the garage) and pick up the scope. I usually get it for a week or two, depending on whether he’s got a weekend project planned. (This is Phil of Weekend Engineering so he often has weekend projects planned.)

Where I’m going here… using an oscope requires planning. It requires me to admit my bug can’t be solved by just trying something else, that typing another line of code or doing a recompile isn’t enough. An embarrassing amount of my time is tweaking my code to do one thing just a little different. Admitting I can’t do that until it magically works, well, it takes a little while.

Even when I have 24/7 access to a scope, using it still means figuring out where to attach the probes to the board and configuring the scope. Admitting I have a problem, a real problem and not just a typo, is oddly difficult. I should just be able to figure it out.*

*This is a myth. A complete and total myth.

When I worked at HP, there were plenty of scopes around. But they all weighed 40-60 lbs (and dropping it would have been *bad* because they were hideously expensive (like 1/2 year salary expensive)). So in addition to admitting to myself I needed help, I had to find a big strong man to carry the thing for me. Who would inevitably offer to help me connect it to my board and then take credit for the solution. (Why didn’t we just put the darn thing on wheels? As I look back, I’m a little frustrated by that idiocy.)

There was another tool I used a few times there, one that I think was on wheels: the logic analyzer. It hooked up to dozens of digital signals and would help me figure out what was going on all over the system. But it was a very difficult tool to use (its manual was about four times longer than the oscilloscope manual and it cost a whole year’s salary). So to use the logic analyzer, I had to admit the problem was big enough to stop my normal work for three days, get someone to help me transport it, get another someone to modify my board so the signals were available, set up the analyzer, and then, within a few hours of getting the information, solve my bug.

Once I admitted the scale of the problem warranted a tool of that magnitude, and put in the diligent effort of setting up the tool, the solution was always obvious.

Now logic analyzers are tiny widgets that plug into USB ports and take all of ten minutes to set up. (I did it (almost) live unboxing of the Saleae Logic on my new podcast.)

But I still have to admit I need it. I wonder if I’m going about this wrong. In the past, I’ve always waited for a problem to happen. Then I’ve waited until I determined I couldn’t solve it the easy way (poke, poke, poke). Then I grudgingly admit it won’t fall to trivial debugging. Then I pull out whatever tool will help, grudgingly (still) hook it up and configure it. Then (usually), the problem falls fairly quickly.

First, the grudgingly parts… the fact that I can’t type my way out of a problem doesn’t make me stupid. I know that and yet… this is what programming is to many people.

Once I got my itty-bitty, super cheap new analyzer set up, I left it in place. And now I’ve moved on to other issues, just adding more signals. It is really handy. I can look at things whenever I want… before I start tippy typing randomly and pressing the recompile clean button for no reason.

But some tools take time and they don’t look like forward progress. It is hard to know when to throw in the towel on being a monkey typing randomly (and when to stop┬áhoping Google, Stack Overflow, and caffeine can solve your problem). On the other hand, reading the manual or getting the right tool or taking a class… well, sometimes it is necessary to take a step to the side to get on to the fast track to the solution.