I suppose I should have expected it, but my tongue in cheek post on my failure to get a Microcell running in my house has generated a number of comments, mostly ideas on troubleshooting. So, if you really want to know…
Several points to note up front. First, this is going to be a bit longer, because, well, the story is actually longer than what I first posted. Second, everyone involved has been very helpful. The problem isn’t really the people, it’s our hubris as engineers in the face of complexity, particularly when combined with the desire to build devices that are completely opaque to the user. Opaqueness and complexity aren’t like peanut butter and chocolate.
Now, for the longer (and boring) story…
When I first brought the Microcell home, I just stuck it behind my e4200 and assumed it would work. It didn’t, so I poked around on the ‘net, and found a 4 or 5 page sheet detailing a bunch of ports that need to be “open” for it to work. Given there’s no console, I pulled my local ARP table, found the right entry, and pulled it into the permanent list on the e4200. Then I went through the process of mapping all the ports listed to the (now fixed) IP address, and opening all of them.
Reboot, still no dice. There’s always more than one way to skin a cat, of course. e4200′s have a DMZ mode. When you push a device into the DMZ, it opens all the well known ports to the device by mapping all these ports to the device’s IP address, and turning off any stateful checking. So I put the (now fixed) IP address (verified in the ARP cache at each step in this process) into the DMZ, turned off all the port forwarding I’d configured.
Reboot, no dice. Okay, so… I pulled the static DHCP mapping, and put the MAC address directly into the DMZ table. It won’t work in this configuration, either.
Since I have no console port, I hooked it all up through a hub, and pulled a packet trace onto one of the various laptops laying around my house. The first packet the Microcell sent is an IGMP join. What multicast tree could a device inside my house be trying to join? Something just didn’t seem right, so I shipped the trace off to some friends inside Cisco. “Please send us a working trace to compare this too,” came the reply. Obviously, there’s no help in that direction. I did learn one thing in this process that was useful —the Microcell should be building an SA back to AT&T’s server. It’s obviously not doing so for some reason, because the IGMP join should be inside the SA, not on my network.
Maybe my little house network just happens to be using the same address range as some hardcoded address within the microcell, so it thinks it’s already on the SA? Maybe… So I renumbered my internal network.
Reboot, still no dice. Okay, maybe my Cisco 800, which is sitting on the same network (running a hardware VPN back to the local Cisco office), is confusing the Microcell. Let’s try disconnecting my entire home network from the back side of the e4200, leaving just the Microcell and the laptop (still connected so I can run packet traces) to the network. Nope, it still won’t work.
From this point forward, I left all the other devices off my network, just to be certain something else wasn’t confusing the Microcell.
I replaced the Microcell, and repeated the entire process. None of those attempts work, either. Maybe it’s the e4200? I ran down to the local office supply store and picked up a Netgear and repeated the entire process.
Finally, after a long conversation with AT&T, I decided I was just going to have to get a second IP address. This was a long and frustrating process, but I finally did get the magical second IP address, and plugged the Microcell directly into the cable modem itself —only to discover the provider had provisioned the second IP address on my other cable modem. Okay, rewire and try again.
Reboot everything —and it still won’t work.
But at least the symptoms have changed. Packet traces now show me that the IPsec SA is, in fact, being built. No more IGMP joins. Yay!
But now, once the IPsec SA is built, it works for about 20 minutes, and reboots itself. After this reboot, there’s no traffic coming out of the Microcell. It does a DHCP request, then just stops. Nothing. Nada.
At this point, I called AT&T, and they told me to put the Microcell behind a router. Sure, like I’m going through that again. “Okay, it needs to have a router behind it.” How does that make any sense at all? But, just to humor the tech, I did put a laptop behind it, and I could browse, ping, trace, whois, nslookup, etc., without any problems. Then another tech told me it won’t work with a Surfboard, and I happen to have two Surfboards…
But this longer (more boring) story, brings me to the same place. Configuring a radio to VOIP connection is complex. Configuring VOIP is complex. Configuring an IPsec SA is complex. Pulling location services off a device and comparing it to a physical address to make certain e911 works is complex. There’s not a single thing here that’s simple in real life.
And yet there’s no console, and there’s no process to actually troubleshoot any of this stuff. I’m reduced to making up theories and trying different things to see if they might work. Random acts of engineering.
So for those who were curious…
About Russ White
