Striped Hats and Algorithms

While I was at the opthamologist's the other day, squinting through dilated eyes at an issue of Forbes, and about fifteen minutes away from getting moderately bad news (I guess it's a good thing that we don't know its coming), I read an interesting article about improvements that Norfolk Southern had made to the way that they assemble and route trains. Of course, since this kind of magazine is oriented to financial people, decision makers, and the like, the description of what they did and how it was done was along the lines of 'and then a miracle occurred', but, in a nutshell, they hired a company (I think this is them, but I'm not sure) to chart the connectivity and usage of track between major assembling points. The company also was charged to develop a way to show which pieces of freight (at the boxcar level) had waited longer than the average.

One of the railroad men said that it was an epiphany (my word) to see what this piece of software could show them. He said that he came from the school of thought which said 'We're the railroad; we'll get to your stuff when we're good and ready.' For the first time, they had an objective way to show how long it took to assemble a train, play with what-ifs, and generally try to get the elapsed time down. And, of course, they did (hardly anyone writes about the failures); the average time between two nodes went from a day and a half to three quarters of a day, and freight which had to be expedited could be clearly seen in the master computer system. All of this, you'd assume, had been around for years, but not here. Here, it was fast, new, and sexy.

Computer systems are like that. They can pop up graphic images that amaze you -- not because the detail wasn't known before, but because the detail wasn't known to you before. If you saw The Hunt for Red October, you remember the scene wherein Ryan is on the aircraft carrier, fiddling with the tabletop display that showed where the US submarine fleet was; by clicking on one icon, he saw the name of the sub pop up. 'That's the Dallas', the admiral says. "Bart Mancuso's boat.'

Now think about how a computer system thats set up to execute jobs works. We tell it 'here's the general rules of the road' -- ie, this kind of job is more important than any other; this kind of job should be billed like this. Then we give it a list of jobs to execute, and we give it some more rules -- run these jobs start to finish, one at a time; run those jobs as time permits. We hand crank all that material in. Then we hit Start, and away she goes. The best monitors we have will give us snapshots of how things are going, will tell us when we are in trouble, let us know what work we're running when, and the best schedulers will let us know what we can expect to run in the coming hours, and, if they run as they normally do, how long they can be expected to run. Its up to us to figure out how to run them -- which runs in what order, or which one can be held to let the Cannonball barrel through. The system doesn't do that.

Why not?