This article is Part 4 in the 6-part series “The Bulletproof Maintenance Window”. For the rest of the story, see the links at the bottom of the page.
The maintenance window has opened, it’s time to start. At this point, it might be tempting to switch things around or skip some of your carefully-documented procedures. Maybe you thought of a better way to do it after the change control people signed off, and it was too much of a hassle to file a new change record. Or maybe you think that the purpose of the runbook is to simply show the change control committee that you know what you’re doing, and that you don’t really need to follow it. Don’t fall in to this trap. Following your plan provides some pretty important benefits:
- You’ll have some degree of muscle memory when working through the change, so that you can save your brain cycles for the exceptions that will inevitably come up.
- The actual configuration steps will likely go faster since you can just copy and paste. The more mature version of this is to automate the config steps, which obviously have to be followed exactly by your automation machinery.
- Large, complex maintenances that require multiple senior engineers are most effective when the steps are divided logically among the engineers. If somebody doesn’t follow their part of the plan, chaos can quickly ensue.
Communication During the Work
Changes with large scope or high impact will often be monitored in real time by business leaders. Following the plan can inspire confidence from those leaders and help keep them off your back. Perhaps the most critical component of communication, though, is with your peers that are executing the change with you, and with the other members of the team who are supporting you, such as the NOC and operations teams.
A colleague of mine who is also a private pilot once explained to me the hierarchy of activities that American pilots engage in while they fly. The mantra they follow is “Aviate, then Navigate, then Communicate”. This emphasizes that a pilot’s first job is to keep the aircraft in the air, and the next job is to keep it pointed in the right direction. Last on the list is communication, not because it’s not important but because humans can only really do one thing at a time. If I am a passenger in one of those planes, I’d rather my pilot fly than shoot the bull with somebody – particularly if there is some kind of problem happening. The written maintenance plan provides valuable communication that can be consumed by others while you are heads-down executing the maintenance or troubleshooting some unforeseen issue that has come up. You can quickly pop up and tell people, “we’re on step 5, and have run in to a problem that we are troubleshooting, will provide updates when we get them”. This preempts the constant “So, what’s our status?” and gives you a quick and efficient way to communicate the progress of the change.
Creating and then systematically following a rigorous maintenance plan is a hallmark of professionalism in network engineering.
Next, in Part 5, we’ll take a look at the role of validation in the success of a planned maintenance activity.
This article is Part 4 in the 6-part series “Bulletproof Maintenance Windows”. For the rest of the story, check out the following: