As a consultant, I have done several network assessments for clients. One of the biggest items that is almost always missing is documentation. In my assessment reports, I can’t just say, “You are missing documentation,” and leave it at that. I have to be more specific. I have to specifically call out what should have been documented, how it should be documented, and why it should be documented. These are my opinions of best practice for documenting your network.
L1/L2 drawings
Let’s start off with network diagrams. You should have a L1/L2 drawing of the physical connectivity and layout of your network. The drawing should consist of all of your network devices and firewalls at a minimum. If you have an EtherChannel/LAG between two devices, make sure your drawing reflects that. Include interfaces on each end of the link. If you don’t currently have a L1/L2 network drawing, CDP on Cisco devices is great for helping you mapping out the network.
One of my pet peeves is ‘pretty’ drawings that have the product icon of the device or the PowerPoint symbol of the device’s function. That’s fine if you are in sales and you draw in PowerPoint. For us engineers, the object representing the device needs to be simple and useful. The objects I use in my drawings are typically three different shapes; rectangles for switches (and other devices), circles for routers, and hexagrams for firewalls. This keeps it easy to draw and consistent. Since I use these simple shapes, I can put details about that device within the object itself. The information I typically include the hostname, the management IP, and device model. Here is a very simple example.
A drawing like this is simple, clean, and informative. You don’t need ‘pretty.’ I also try to stay away from using colors to code objects because it will be useless to someone who is color blind.
You need to have this type of documentation so you know how your network is connected. If you bring on a new team member or bring in a consultant to help with something, these drawings can quickly get that person up to speed on your network. Also, it seems that there is a fear of everyone in IT getting hit by a bus, so these drawings can help the new guy understand your network in the event of your untimely demise.
L3 drawings
Your Layer 3 drawing should include every device that is involved with routing in your network. If you have L2 only devices hanging off of a router, you don’t need to include them on this drawing. The drawing should again use simple objects that include the same details, except for using the router ID of the device instead of the management IP.
Jaakko Rautenen wrote an article about how to draw L3 network diagrams, so I am not going to get into the details of how.
Again, this helps the new guy, a consultant, or even a new vendor understand your network quickly. You can also use these drawings to help a sysadmin or a helpdesk person understand what is going on in the network.
Circuit inventory
While the Se0/0 interface configured with the circuit ID in the description on that interface is good, it is not sufficient documentation of circuit information. I believe this is a critical piece of documentation on the equipment itself, there is so much more information that you should be capturing. Here is a list of things you need to capture:
- Circuit ID (12/ABCD/123456/MC)
- Circuit Type (T1/Ethernet/OC3)
- Bandwidth (1.5Mbps, 50Mbps, 75Mbps CIR)
- WAN Type (P2P, MPLS, Internet)
- Termination equipment (RTR-BRANCH1)
- Carrier (AT&T, Verizon)
- Carrier NOC # (888-555-1234)
- ALOC (123 Some St, Dallas, TX)
- ZLOC (456 Same St, San Antonio, TX)
- Turn up date (1/1/2013)
- Contract expiration (1/1/2014)
- Notes
The ALOC and ZLOC are terms carriers use to identify the two ends of a circuit and come from the days when I worked for a carrier. Use whatever terms you want, but document the termination locations of the circuits. I would also recommend including a notes field to put things like ‘Extended demarc from 1st floor, west closet’. Of course, you can use a spreadsheet to collect this information. Here is a template I use. There are alternatives, for better or worse, like SharePoint or a wiki. Whatever you use, you need something to organize and search within it.
When you have a WAN circuit down, you want the information to bring it back up as quickly as possible. If all you know is that it is a circuit with carrier X, then you have to find their NOC number to report the outage, and you may have to look for a while or end up getting transferred many times. When you talk to the tech, you should have at least a circuit ID to give them. If you just say it is the one in Houston, TX, they have to search through their records to try to find it. I have seen carriers take down the wrong circuit because they thought they had the one you were talking about since they did not get a circuit ID. Give them the circuit ID; when they find the circuit, verify the addresses with them. Knowing when the circuits were turned up and when they expire can help you to plan on when you need to start looking at renewing the circuits or seek new alternatives.
IP addresses
All of the IP addresses in your network should be documented. You should have a list of all of the subnets that are in use, then another list for each subnet broken down to document what IP addresses are DHCP and which are statically assigned. If you are reserving IP addresses for network use (I like to use 0-15) those should be listed and marked as reserved.
Most organizations start out with a spreadsheet; if you are the only person managing the network, that may work. Here is a template I use to start the documentation with a spreadsheet. For larger networks, you may to look at an IP Address Management (IPAM) system. I have used Infoblox in the past, and it is a really good solution.
Using a ping scan of a subnet to determine what IP addresses are available to statically assign a new device is not acceptable. Period! I have seen this done in even large multinational organizations and it has cost them hours of troubleshooting and eventually a flight to the site to determine why an assigned IP is not working. ALL IP addresses should be documented.
Inventory
Your inventory list should include manufacturer, model, serial number, hostname, location, and closet (if more than one). If your company requires an asset tag, that should be included as well. Another piece of information that I like to include is previous serial numbers of devices. For example you had a 2960 switch, but it failed and you swapped it out with a new one Cisco sent you, document what the old/previous serial number was. You can also include a notes field so you can tie any additional information to the device.
You can use a spreadsheet to document this, but if you have something like Sharepoint or a Wiki, then you can start linking the this information with some of the previous information. Perhaps even create a page of previous devices so you can start tracking failures to determine if they are environmentally related or model specific.
In a previous job, we had this information. When hurricane Ike came through Houston, it severely damaged one of the buildings. The business asked for the manufacturer, model, and serial number of all of our equipment at the site to report to insurance. It took 5 minutes to get them that information, and we didn’t have to go on site. If we didn’t have this information, we would have been stuck because we were not allowed to go in the building due to the damage. If your equipment is stolen, how are you going to get this information for police or insurance if you don’t have it documented already? Or what about reconciling your equipment to what the vendor says you have on the vendor’s maintenance program (i.e. SmartNet). I have used my inventory list many times to do this comparison and found that the vendor thought we had some devices we didn’t, plus some others not on maintenance that should have been.
Firewall rules
The documentation for your firewall should include your NAT rules along with your filtering rules. For NAT rules, you need to include the original source, NATed source, original destination, NATed destination, source/destination ports if they are being translated as well, who requested the rule (person and/or business group), ticket number from your ticket system that originated the request, date the rule was installed, last date the rule was reviewed, when the rule should be removed (if temporary), who put the rule in the firewall, and a description of what the rule is for (to email server). Filtering rules should include all of the above with the exception of the NAT specifics, but with more details on the individual ports that are opened.
I have not seen a good system to document this information, so my recommendation is a spreadsheet. If you put this on SharePoint or a Wiki, restrict who can view this information to those that really need to see it. If you know of a good way to document this information, I am open to hearing about it.
Whenever I do a firewall upgrade, I always hear, “There are probably some rules in there we need to clean out and don’t use any more.” Why is this? No one knows why they are in there; in case they are still needed, so no one wants to touch them. This is probably one of @MrsYisWhy‘s biggest pet peeves as well. If you have this documented, you can review your firewall policies on a regular basis and remove what is not needed.
Rack layout
A rack layout diagram should have a drawing of the rack itself and the equipment that is in the rack. As stated before, pretty icons are more trouble than they are worth. Simple rectangles to show where the equipment is located in the racks is all you need.
I use Omnigraffle or Visio to do the rack layout drawings. I have a Visio template and an OmniGraffle template that includes 5 racks and use this to start every rack layout drawing.
This has been helpful to me in the past at far remote sites where you need someone to look at a device or plug something in. You can tell someone to look at U15, then move over three ports and plug in that cable there. Also, if you go on site or can get someone there to take pictures of everything, this is a great way to help guide someone when remote troubleshooting.
Configuration Template
You should have a configuration template that defines all of your standard configurations. I have written an article before on building a template, so I am not going to go in detail. Your template may be vastly different from anyone else’s, because your environment is different.
The template should not be in a Word document. At the worst case, use Notepad. My preferred text editor is UltraEdit as you can set it up to do syntax highlighting for Cisco IOS.
A template gives you a starting point for every new device in your network, as well as something to compare the existing configurations to in order to see what may have been removed. Network configuration management systems can use this template to verify current configurations and alert you when out of policy.
Conclusion
This was a quick rundown of what I believe as best practices to have for your network documentation. No matter what your network size is, you need to have these items as a bare minimum. If you can use an application like Sharepoint (if you have to) or a Wiki to store your documentation you will be able start linking all of these separate documents together and it will become much easier to find the information you need. Please leave me feedback about your thoughts and what other documentation you include for your network.
About Charles Galler
