So In this blog, I wanted to share two things. Number One, A high level overview of how I approached about creating a PCE prototype, but more importantly the motivation behind it and the things I learned during the process.
So almost, a little bit over an year ago, when I wasn’t working for a Vendor and was looking for a PCE to fool around with RSVP-TE/SR-TE but I wasn’t able to find anything. At that time, Only thing I was able to found was Open daylight but its PCEP implementation didn’t have Segment Routing support, so I was out of luck. At the same time I was also working on improving my programmatic skills. So I thought why not combine both problems together and write my own PCEP prototype with the idea that going through that process I will
- Improve my programmatic skills.
- Understand the protocol and technology better
- Pick up the capability of creating Prototypes.
- Most importantly, it will be so much fun.
So that goal resulted in https://github.com/Dipsingh/mpls-pce where one can use PCEP to send RSVP-TE or SR based tunnels which was given as a JSON input from the user.
During that process I learned a few more things in Python, Google Protocol Buffers (GPB’s) for sending data between IPC and of course gained deeper understanding of PCEP as a protocol. You may have already seen this picture and at that time I truly felt like that.
By the time I was finished, I was working for a Vendor and had access to there PCE product, but the learnings I got through writing the prototype was very valuable to me and just by using a product, I wouldn’t have gained the understanding I have now.
Then I thought of extending just from having the PCEP module to have some Constrained based SPF for doing computation based on a given Topology. In a PCE, one of the critical components is the Algorithms which is responsible for doing the computation based on various constraints. This is typically going to be the secret sauce for any vendor Implementation.
In order to achieve the goal,there were two major things I had to solve:
- First was to get an Input Topology from the network to build TED.
- Create Algorithms for CSPF computation.
Initially for getting the input topology, I relied on ODL. ODL had the support for BGP-LS and it peered with the a BGP-LS speaker to get the topology over BGP which i was able to pull over RESTConf in a JSON format. Then I parsed the JSON file , created a Graph Database of Nodes and Links including their attributes.
For solving the CSPF computation part, I thought of modifying Dijkstra to add more constraints. Which took me back to my engineering days and I refreshed on various Algorithms out there, including Dijkstra (https://packetpushers.net/back-basics-dijkstra-spf/). I also looked at various data structures to implement Dijkstra and their algorithmic complexity. Later I modified the Dijkstra to add various constraints and came up with the following CSPF’s
- No Constraint (Simple Heap based Dijkstra)
- Avoid or Include a certain Link as a Constraint.
- Available BW as a Constraint.
- Available BW + (Avoid or Include a certain Link as a constraint).
- Avoid a particular Node
After completing the CSPF and TED part, I wanted to create a user command line shell where a user can interact with the PCE and send things like looking at topology details and issue commands like find a Path from Point A to Point B with or without constraints. Also If the user wishes, then he can provision the tunnel at the Headend via PCEP for SR-TE or RSVP-TE.
Below is a high level design.
Tying all the three Modules:
Once I was finished with User command line shell, The next problem I faced was that I had three independent modules, and the question was how can I tie them together.
- User shell module,
- Core PCE (CSPF+TED) portion based on Networkx
- PCEP module.
This problem led me to explore various messaging buses out there like ZMQ, RabbitMQ, Kafka etc… I was initially thinking of using RabitMQ but a Facebook engineer recommended me to look at ZMQ. So I looked deeply into ZMQ and I really liked it that how lightweight it was. Needless to say I ended up in using ZMQ to tie all the three separate modules together.
After tying all the three modules, the prototype was almost complete except the fact that I didn’t really like how heavy the ODL was and I wanted something light weight. I looked at some open source BGP implementations out there, but none of them has implemented BGP-LS. Again for this the same engineer recommended me to take a look at using GoBGP (Written in GoLang). GoBGP didn’t had the BGP-LS implementation either, which meant I had to write my own. Which led me to learn GoLang (Which by the way is my fav. Language now) and ended up implementing BGP-LS functionality in GoBGP for ISIS-Level 2,TE attributes and some SR support. At this point I want to give a shout out to GoBGP devs on how clean and organized GoBGP code is. This well organized code made this a very straightforward process. https://github.com/Dipsingh/gobgp
Then I used ZMQ to tie the GoBGP piece with the existing code. I still have to make some changes here to clean some GRPC interaction and remove some hacks I did but that’s after this blog. So that’s just the high level overview of the process I went through and I hope you can see how much I learned during this process which otherwise may not have been possible.
Let’s see a Quick Demo:
So lets take a look a brief demo. Below is the reference topology we have. The IGP in use is ISIS-Level 2 with Segment Routing and RSVPE-TE enabled. Prefix-Sid’s and Adjacency SID’s are shown in the topology.
Each Blue link IGP cost is 1 except the Red link which has the IGP cost of 10. All the Topology data, including SR attributes are exported via BGP-LS to the PCE from the POR router. On the server side, we have GoBGP daemon which grabs the topology data over BGP-LS and which is later exported via a PCE main process using ZMQ.
Below is the relevant configuration on POR router.
We have GoBGP Linux binary running in the background.
Now we will start our PCE process on the server.
We are all set here. Now lets query how many Nodes we have in our topology from the command shell.
Now Let’s ask for a path from SEA to DAL.
So you can see, it found two shortest paths in the topology and they avoid the SEA-MIN high cost link. Now at this time I will like to provision this path as an RSVP-TE on the SEA headend. So I choose Path ID #1 and say False for SR-TE which means we will provision this tunnel as an RSVP -TE Tunnel.
Lets see if the tunnel is up on the SEA router.
Okay, now let’s ask the PCE to add a constraint by avoiding the SFC node and see if it can find a path. This time it found the path via SEA –> MIN –> KCY –> DAL. Now let’s push this path via SR-TE on the SEA tunnel.
I am hoping by this time you can see how combining the domain interest and implementing something in that area can push you to learn things which you may not learn otherwise. Rather than wasting your time with all the mumbo jumbo of SDN hype and debating if its going to take your job or not, if you focus on picking up programmatic skill as another skill, You will realize it can make you more efficient, help you in getting a better understanding of Protocols/Technology and build stronger domain expertise.
And it’s very well possible that you may start hating your day to day Networking job and you would like to spend more time writing Code.:)