We’ve all been there: it’s 5am and you’re up to your humerus in a network failure that no amount of coffee and swearing seems to fix. People are hovering on your shoulder, presumably to observe the ferocious look of concentration on your face. How do you escape this 3rd Circle of Troubleshooting Hell and ride the Unicorn of Success to victory? Here are five tips to help you escape with your sanity (mostly) intact.
- Be methodical. Start at the point closest to the most likely source of the problem and step through the possible trouble points one by one.
- Draw diagrams of the problem, and write down the steps you’ve already tried. Whiteboards are useful for this, especially when you’re working in a team, but a notebook is also a good idea. Make sure to keep any notes you make for the next time you have to troubleshoot the same system. Writing down what you’ve already tried also stops you going into a despair cycle, trying the same thing over and over again. Keep your notebooks, too. I’ve found mine invaluable for solving problems years after solving a similar fault.
- Make sure you have something decent to eat and keep drinking (preferably not whisky – no matter how tempting it might be). It’s important to remember that your brain needs to be refueled regularly, especially if you haven’t had much sleep. Make time to grab something decent to eat and keep yourself hydrated.
- Get a human shield. People hanging around your desk asking for updates every 20 seconds doesn’t help matters. Find someone to deflect them and deploy them in a tactical position to intercept wingnuts. This leaves you free to concentrate on actually fixing this problem at hand. If you must, give them a title such as “Meatshield Puppet-Master” to make them feel important.
- Recognise when you’re not making progress. Sometimes you’ll just run out of ideas. At this point, you can continue to bang your head off the wall, or you can take 5 minutes to get some fresh air and give your subconscious brain a chance to work on the problem in the background. If the problem isn’t critical, but you’re still having problems solving it, try sleeping on it. Just don’t do what I’ve done in the past and lurch upright at an unreasonably anti-social time of night shouting the network engineer’s version of “Eureka!”. Trying to explain to an irate spouse why you’re making so much noise can be tricky, especially if they “don’t do geek-talk.”
I think these are fairly common sense ways to try and make yourself as effective and efficient as possible, but they’re easy to forget when you’re under pressure to fix a fault quickly.
Over to you guys – what have I missed? What was your worst troubleshooting experience? Mine involved a certain firewall vendor, a failed “upgrade”, and a 26 hour working day.