Network Documentation Best Practices: What’s Important & How To Track It

As a consultant, I have done several network assessments for clients. One of the biggest items that is almost always missing is documentation. In my assessment reports, I can’t just say, “You are missing documentation,” and leave it at that. I have to be more specific. I have to specifically call out what should have been documented, how it should be documented, and why it should be documented. These are my opinions of best practice for documenting your network.

L1/L2 drawings

Let’s start off with network diagrams. You should have a L1/L2 drawing of the physical connectivity and layout of your network. The drawing should consist of all of your network devices and firewalls at a minimum. If you have an EtherChannel/LAG between two devices, make sure your drawing reflects that. Include interfaces on each end of the link. If you don’t currently have a L1/L2 network drawing, CDP on Cisco devices is great for helping you mapping out the network.

One of my pet peeves is ‘pretty’ drawings that have the product icon of the device or the PowerPoint symbol of the device’s function. That’s fine if you are in sales and you draw in PowerPoint. For us engineers, the object representing the device needs to be simple and useful. The objects I use in my drawings are typically three different shapes; rectangles for switches (and other devices), circles for routers, and hexagrams for firewalls. This keeps it easy to draw and consistent. Since I use these simple shapes, I can put details about that device within the object itself. The information I typically include the hostname, the management IP, and device model. Here is a very simple example.

documentation_network_drawing

A drawing like this is simple, clean, and informative. You don’t need ‘pretty.’ I also try to stay away from using colors to code objects because it will be useless to someone who is color blind.

You need to have this type of documentation so you know how your network is connected. If you bring on a new team member or bring in a consultant to help with something, these drawings can quickly get that person up to speed on your network. Also, it seems that there is a fear of everyone in IT getting hit by a bus, so these drawings can help the new guy understand your network in the event of your untimely demise.

L3 drawings

Your Layer 3 drawing should include every device that is involved with routing in your network. If you have L2 only devices hanging off of a router, you don’t need to include them on this drawing. The drawing should again use simple objects that include the same details, except for using the router ID of the device instead of the management IP.

Jaakko Rautenen wrote an article about how to draw L3 network diagrams, so I am not going to get into the details of how.

Again, this helps the new guy, a consultant, or even a new vendor understand your network quickly. You can also use these drawings to help a sysadmin or a helpdesk person understand what is going on in the network.

Circuit inventory

While the Se0/0 interface configured with the circuit ID in the description on that interface is good, it is not sufficient documentation of circuit information. I believe this is a critical piece of documentation on the equipment itself, there is so much more information that you should be capturing. Here is a list of things you need to capture:

  • Circuit ID (12/ABCD/123456/MC)
  • Circuit Type (T1/Ethernet/OC3)
  • Bandwidth (1.5Mbps, 50Mbps, 75Mbps CIR)
  • WAN Type (P2P, MPLS, Internet)
  • Termination equipment (RTR-BRANCH1)
  • Carrier (AT&T, Verizon)
  • Carrier NOC # (888-555-1234)
  • ALOC (123 Some St, Dallas, TX)
  • ZLOC  (456 Same St, San Antonio, TX)
  • Turn up date (1/1/2013)
  • Contract expiration (1/1/2014)
  • Notes

The ALOC and ZLOC are terms carriers use to identify the two ends of a circuit and come from the days when I worked for a carrier. Use whatever terms you want, but document the termination locations of the circuits. I would also recommend including a notes field to put things like ‘Extended demarc from 1st floor, west closet’. Of course, you can use a spreadsheet to collect this information. Here is a template I use. There are alternatives, for better or worse, like SharePoint or a wiki. Whatever you use, you need something to organize and search within it.

When you have a WAN circuit down, you want the information to bring it back up as quickly as possible. If all you know is that it is a circuit with carrier X, then you have to find their NOC number to report the outage, and you may have to look for a while or end up getting transferred many times. When you talk to the tech, you should have at least a circuit ID to give them. If you just say it is the one in Houston, TX, they have to search through their records to try to find it. I have seen carriers take down the wrong circuit because they thought they had the one you were talking about since they did not get a circuit ID. Give them the circuit ID; when they find the circuit, verify the addresses with them. Knowing when the circuits were turned up and when they expire can help you to plan on when you need to start looking at renewing the circuits or seek new alternatives.

IP addresses

All of the IP addresses in your network should be documented. You should have a list of all of the subnets that are in use, then another list for each subnet broken down to document what IP addresses are DHCP and which are statically assigned. If you are reserving IP addresses for network use (I like to use 0-15) those should be listed and marked as reserved.

Most organizations start out with a spreadsheet; if you are the only person managing the network, that may work. Here is a template I use to start the documentation with a spreadsheet. For larger networks, you may to look at an IP Address Management (IPAM) system. I have used Infoblox in the past, and it is a really good solution.

Using a ping scan of a subnet to determine what IP addresses are available to statically assign a new device is not acceptable. Period! I have seen this done in even large multinational organizations and it has cost them hours of troubleshooting and eventually a flight to the site to determine why an assigned IP is not working. ALL IP addresses should be documented.

Inventory

Your inventory list should include manufacturer, model, serial number, hostname, location, and closet (if more than one). If your company requires an asset tag, that should be included as well. Another piece of information that I like to include is previous serial numbers of devices. For example you had a 2960 switch, but it failed and you swapped it out with a new one Cisco sent you, document what the old/previous serial number was. You can also include a notes field so you can tie any additional information to the device.

You can use a spreadsheet to document this, but if you have something like Sharepoint or a Wiki, then you can start linking the this information with some of the previous information. Perhaps even create a page of previous devices so you can start tracking failures to determine if they are environmentally related or model specific.

In a previous job, we had this information. When hurricane Ike came through Houston, it severely damaged one of the buildings. The business asked for the manufacturer, model, and serial number of all of our equipment at the site to report to insurance. It took 5 minutes to get them that information, and we didn’t have to go on site. If we didn’t have this information, we would have been stuck because we were not allowed to go in the building due to the damage. If your equipment is stolen, how are you going to get this information for police or insurance if you don’t have it documented already? Or what about reconciling your equipment to what the vendor says you have on the vendor’s maintenance program (i.e. SmartNet). I have used my inventory list many times to do this comparison and found that the vendor thought we had some devices we didn’t, plus some others not on maintenance that should have been.

Firewall rules

The documentation for your firewall should include your NAT rules along with your filtering rules. For NAT rules, you need to include the original source, NATed source, original destination, NATed destination, source/destination ports if they are being translated as well, who requested the rule (person and/or business group), ticket number from your ticket system that originated the request, date the rule was installed, last date the rule was reviewed, when the rule should be removed (if temporary), who put the rule in the firewall, and a description of what the rule is for (to email server). Filtering rules should include all of the above with the exception of the NAT specifics, but with more details on the individual ports that are opened.

I have not seen a good system to document this information, so my recommendation is a spreadsheet. If you put this on SharePoint or a Wiki, restrict who can view this information to those that really need to see it. If you know of a good way to document this information, I am open to hearing about it.

Whenever I do a firewall upgrade, I always hear, “There are probably some rules in there we need to clean out and don’t use any more.” Why is this? No one knows why they are in there; in case they are still needed, so no one wants to touch them. This is probably one of @MrsYisWhy‘s biggest pet peeves as well. If you have this documented, you can review your firewall policies on a regular basis and remove what is not needed.

Rack layout

A rack layout diagram should have a drawing of the rack itself and the equipment that is in the rack. As stated before, pretty icons are more trouble than they are worth. Simple rectangles to show where the equipment is located in the racks is all you need.

I use Omnigraffle or Visio to do the rack layout drawings. I have a Visio template and an OmniGraffle template that includes 5 racks and use this to start every rack layout drawing.

documentation_rack_layout

 

This has been helpful to me in the past at far remote sites where you need someone to look at a device or plug something in. You can tell someone to look at U15, then move over three ports and plug in that cable there. Also, if you go on site or can get someone there to take pictures of everything, this is a great way to help guide someone when remote troubleshooting.

Configuration Template

You should have a configuration template that defines all of your standard configurations. I have written an article before on building a template, so I am not going to go in detail. Your template may be vastly different from anyone else’s, because your environment is different.

The template should not be in a Word document. At the worst case, use Notepad. My preferred text editor is UltraEdit as you can set it up to do syntax highlighting for Cisco IOS.

A template gives you a starting point for every new device in your network, as well as something to compare the existing configurations to in order to see what may have been removed.  Network configuration management systems can use this template to verify current configurations and alert you when out of policy.

 

Conclusion

This was a quick rundown of what I believe as best practices to have for your network documentation. No matter what your network size is, you need to have these items as a bare minimum. If you can use an application like Sharepoint (if you have to) or a Wiki to store your documentation you will be able start linking all of these separate documents together and it will become much easier to find the information you need. Please leave me feedback about your thoughts and what other documentation you include for your network.

 

Charles Galler

Charles Galler

Charles is a network and UC engineer for a mainly Cisco reseller. He has worked in the networking industry for about 13 years. He started as a network administrator for a small CLEC (carrier) where he did it all in IT and worked on the carrier network. After the CLEC, Charles went to work for a large healthcare organization in the Houston area and stayed with them for about three and a half years. Now he works for a reseller in the professional services part of the organization. He is currently studying for his CCIE in Routing and Switching and plans on passing it before the end of 2014. You can find him on the Twitter @twidfeki.
Charles Galler
Charles Galler

Latest posts by Charles Galler (see all)

  • Dale

    I did some network diagrams and rack layout drawings using Visio for my company and it is an excellent program for this task. We have HP equipment and I was able to get some Visio templates of the equipment from the vendor to use with my Visio drawings. Excellent article and thanks for sharing!

  • http://www.facebook.com/flavio.j.silva.9 Flavio Jose Silva

    Every project manager, technical design and all the people involved in network project team must read this article.

    A good documentation save a life and a lot of waste time !!!

    This is the dream of the the analyst.

  • Lucas

    The problem is to keep it updated, mainly when lots of hands are involved and changes are frecuent.

    Anybody using a cmdb tool?

    • Charles Galler

      When you make changes, you have to build into your change window or post-support time to update the documentation. The more simple the documentation is, the easier it is to update. Another reason why I don’t like using the product icons.

      I have used Solarwinds NCM when I managed a network and liked it. Other than that I have written scripts. Infoblox has a solution for managing configurations as well. I have heard people using RANCID as a free option.

    • Jaakko Rautanen

      When you are planning a change you must draw your plannings into L2 and L3 diagrams. Take a copy of your existing diagrams make changes to them. After a change migrate new and old diagrams.

  • Fernando Montenegro

    Good article! Quick question: how do you suggest capturing non-R/S/FW equipment in your diagrams? I’m thinking WAN accelerators, Load Balancers, IDS/IPS sensors, SBCs, … all those things that are not your typical R&S&FW but still mess with packets/frames/flows in some way, shape or form…
    Thanks!

    • Charles Galler

      If it participants in L3 routing, then it should go on that drawing. Otherwise put it on the L2 drawing. WAN accelerators, IDS/IPS I would draw as a rectangle. Load Balancers I would draw as a triangle. It’s really up to you as to what you want to use.

      For the SBC’s I would create a separate drawing for the voice network and put them on there.

      Just keep it clean, simple, and easy to update.

  • BMZ17

    Excellent guideline

  • http://twitter.com/bjkirton Ben Kirton

    Hi Chrales,

    Great article and i will be passing it around my company.

    The Omnigraffle diagram link seems to go to an empty page at cubby. Could you update it please? I would love to grab a copy.

    • Charles Galler

      Sorry about that. I have updated the links with the correct locations. Let me know if they still don’t work.

  • TJ Evans

    Great recommendations!

    IME it gets noticeable harder when dealing with ‘dense’ deployments … trunks carrying lots (but not all) VLANs, virtualized components, etc.

    An important consideration is if these diagrams are to be viewed as a soft-copies (where you can zoom in) or to be printed (where old eyes let their age be known!).

    Sidenote – track not just your IPv4 addresses, but also your IPv6 addresses … takes a bit more space, makes life harder on ‘busy’ diagrams :).

  • http://www.knill.net/ David Knill

    Good article. Just catching up on PP, but figured I’d add my $0.02.

    When it comes to drawing diagrams as a consultant, I highly
    recommend looking at NetBrain…it does a good job of drawing L2/L3 diagrams
    either through SNMP/SSH discovery or offline configurations. The tool can draw a
    diagram in seconds and show relationships such as routing protocol neighbors
    and it exports to Visio. It can also make it easier to keep the diagrams
    updated and will provide inventory reports. Saves a lot of my very expensive
    time ;)

    NetBrain is by no means perfect, but I can get good feel for an
    unknown topology very quickly. I know other vendors have similar tools, but
    NetBrain is the best I have used so far (they are not paying me for the plug).

    I often use Excel for configuration scripts – with one
    column for the script and a 2nd column for comments. In some cases, I can also
    place modeled configurations (from a lab environment) in a 3rd column so it is
    easy to translate a modeled configuration to a real one (for example, when the
    lab configuration uses different interface numbers). I’ve never had a problem
    with copy/pasting from Excel into SecureCRT. FWIW – I’m sure OpenOffice or
    equivalent would probably work OK as well.

    One last thing – when it comes to larger environments, I’m
    not sure documenting the specific firewall rules is worth the trouble – the firewalls
    tend to be managed with a central provisioning tool (say, like Panorama on Palo
    Alto firewalls) and get updated frequently. All of the rules are pretty much
    there and somewhat self-documenting. Removing rules that aren’t needed is still
    a pain, but using firewall counters is likely a good way to figure that out. This
    doesn’t negate the need to have the general security policy documented (on
    which the specific rules are based).

  • Glen Turner

    The rack diagram should include max weight and typical and maximum watts. You also want front and rear columns, since cable bars can be mounted at either end. Finally, record the door key number. If the rack has identifiers, then these should be noted. Photos are invaluable.

    L2 diagrams should include the nominal and actual loss of fixed fiber links. The power budget calculation for links longer than 2000m should be attached. The exact model of Sfp should be noted. The service ID reported by the monitoring system should be highlighed, and all other service Ids, tiedowns, and order numbers recorded.

    For remote hands sites all patching should be in the diagram, you’d usually have multiple diagrams split by function (external, internal, mgnt) and power cabling too. The length and connectors of patch leads should be noted. This allows clear and certain instructions to be given to remote hands.

  • Joanna Wales

    Hello, I’m looking for a UK consultant to draw up the network diagrams for our site. Does anyone know any?

  • Arnaud Lafon

    Excellent article. Thanks for Omnigraffle template btw :)

  • Hossein Ghaffar Armaghani

    It was very practical and effective. Thank you.

  • Chris

    Great article.
    It is amazing how many companies one would think have their act together, start drawing the network on a white board whenever one asks about their configuration.

    Many companies I have seen opt for one of three: NMS systems, Visio diagrams and homegrown spreadsheets or databases. NMS systems are not real documentation tools, as they only show partial (discoverable) data, and their graphical capabilities are limited. Visio diagrams or the like have nice drawing features, but they almost always end up being static out-of-date diagrams and they are not really collaborative unless you up the ante with some SharePoint, which is still convoluted (and probably expensive). No need to comment on spreadsheets or databases.

    I have done a lot of documentation myself in the past for other clients and I usually utilized a combination of netViz for the diagrams, and other tools for the icons. Now that netViz is end of life I would recommend Graphical Networks’ netTerrain tool, which can also give you rack elevation views and DCIM features (at a cost though). It can also integration with external NMS systems and has a nice modeling engine that automatically creates subcomponents for each device.

    netBrain was also mentioned here, which is a good alternative if you want to create ad-hoc dynamic topological diagrams. There is also Opnet, but I am not too familiar with that.

  • RyGuy

    This is a very useful blog! Any chance the links could be updated? Thank you