Automation – Poor Mans Style

There has been a lot of discussion recently in networking circles surrounding automation especially in discussions about Software Defined Networking (SDN). While automation means different things to different people I would define it as any tool or solution that automates repetitive tasks (making the job easier) while making the output more consistent and ultimately the network more reliable. I’m a huge proponent of having the computer do the work, I guess that could be defined as automation.

The purpose of this post is to provide some simple examples of how you can start automating today. These are not glamorous solutions hence the poor man slogan but they should help provide some idea of what’s possible. There are plenty of open-source and commercial solutions out there, one that’s been receiving some extra press these past few months is Puppet.

In my current organization we deploy a lot of equipment and we usually do so on a very tight timetable where we have hours, not days or weeks to turn up a closet or a remote site. So our time is extremely precious but more so we can’t afford to be troubleshooting erroneous configuration errors that could easily be avoided with some simple automation. Like numerous organizations before us we too had Microsoft Word Templates and Excel macros and formulas but we almost always ran into problems with the human element of the equation.

I took a small 1Gbps CentOS Linux guest with a LAMP (Linux, Apache, MySQL, PHP) stack and started throwing together some Perl, PHP and JavaScript code. The outcome was a pretty powerful example of what’s possible without a big capital investment or some consulting company reaching their quarterly sales goal on your dime.

Here are three simple examples which are adoptions of each other, adding additional features as time allowed and the solutions matured.

Juniper SRX – VPN Branch Offices

While we were migrating our remote branch offices (31+ locations in all) to Juniper SRX Service Gateways we quickly realized we needed a more reliable solution than building the configuration by hand.  We had a Microsoft Word template that had various fields marked {RED}, the field engineer would perform a search-n-replace to ultimately build the configuration. In our first few conversions we had a number of typos in the configuration that caused use to overrun our scheduled maintenance window. How can we make configuring the Juniper SRX easier for our field engineers? What about a web based portal that takes in the assorted variables and outputs a working configuration?

The solution was really quite easy and has been done by others before. The field engineer plugs in a few values and the Perl/PHP application spits back a complete configuration for both the branch office Juniper SRX 210H and the main office Juniper SRX 650. The initial version of the application required the field engineer to enter a random 128 character shared key, later versions of the application automatically generated a random shared key for use in the configuration. This approach completely eliminated any other configuration issues during the migration project and is now part of our standard process for a new greenfield site.

Avaya Ethernet Routing Switch 4850GTS-PWR+

On the heals of that migration we had a very large expansion project underway at our largest facility. The physical construction called for the installation of about 63+ Avaya Ethernet Routing Switch 4850GTS-PWR+ switches. In order to help streamline the configuration process and help eliminate configuration errors I built an adaption of the earlier application above to fit the requirements for this project. In this project I expanded the functionality of the original application by adding JavaScript code to perform client side data validation. If the field called for an IP address, then the JavaScript code would only submit the data to the server if the field passed validation. It was pretty straight forward and simple but we took the original solution and improved on it.

APC UPS/PDU Management Cards

In that same expansion project we also identified the need to streamline the configuration of the American Power Conversion (APC) UPS’s and PDUs that we were deploying throughout the infrastructure. If you’ve ever worked with them you know they can be somewhat difficult to quickly and easily configure. Our field engineers were spending on average 1 hour to configure each device and often there were inconsistencies in the configuration depending on which field engineer had performed the configuration. So we came up with a new streamlined process which allows the engineer to complete the task in about 15 minutes. The field engineer manually configures a DHCP reservation (manual DHCP) utilizing the MAC address of the management card within our Infoblox IP address management solution. Once the UPS or PDU is online and communicating with the network the field engineer plugs in a number of variables into the web browser and the Perl application will output the configuration. In this case we decided to take this solution one step further by having the Perl application actually program the configuration into the device. The Perl application will generate the configuration and then will make a FTP call to the actual asset and upload the configuration. The only thing left for the field engineer was to perform some simple tests once the task was complete, to verify that the asset was reporting, sending SNMP traps, to our management platform. And even that last step could have probably been easily automated.

My Thoughts

There are a number of frameworks that I could have used in writing these applications but I decided to keep it simple (this time around). The point here is to just provide an example of what’s possible. There are quite a few tools and solutions in the market place that already leverage SNMP, NET-CONF, XML, SOAP APIs, etc to help provide integration between systems as well as management and automation.

Wouldn’t it be great if the last application accepted the MAC address of the APC UPS/PDU and made an automated call to Infoblox and automatically created a DHCP reservation for that asset? Thereby streamlining the process even further? There’s nothing stopping me from doing that other than the time and energy it takes to code the solution and then test it appropriately.

I’m not ready right now to release the actual code but if enough people request I will work to creating sanitized copies and release the code under a GPL license.

Let me know what your doing around automation.

I recall a number of interesting posts a few years back where some folks had completely automated how they inventory and on-board their IP phones. They were using bar code scanners to collect the information from the outside of the box and then had an automated process for taking that information and creating the necessary configuration files for a zero-touch installation, including the actual node and TN information for the Avaya Communication Server 1000. That was a pretty neat example of automation in my opinion and obviously saved them a lot of time and effort.

 Cheers!