Michael McNamara https://blog.michaelfmcnamara.com technology, networking, virtualization and IP telephony Sun, 14 Mar 2021 15:50:44 +0000 en-US hourly 1 https://wordpress.org/?v=5.7.2 How to start blogging in 2021? https://blog.michaelfmcnamara.com/2021/03/how-to-start-blogging-in-2021/ https://blog.michaelfmcnamara.com/2021/03/how-to-start-blogging-in-2021/#respond Sun, 14 Mar 2021 15:50:44 +0000 https://blog.michaelfmcnamara.com/?p=6798 It’s interesting how many people still ask this basic question, wanting to know how much it costs and what it takes or more specifically “how to do it”. I get the question from college students, colleagues and more often neighbors who stumble upon my not so secret digital identify. While there’s a lot more social media around today than there was back when I started blogging in 2007 and I believe there’s still a space for blogging. You’d be surprised that many of the reasons people start blogs are commonly similar. Whether it’s for professional exposure or experience, personal interests or curiosity there are no shortage of tools or solutions available today to help a budding creator.

I started with Blogger back in 2007 and then in 2008 I migrated to a self-hosted installation of WordPress. While there are a number of great managed solutions available today I’m one of those guys that enjoys the challenges of learning by building it yourself and then managing it day to day. The self-hosted WordPress or WordPress.org as some refer to it, requires a server to run the software stack. In my case I’m using a Linux Virtual Private Server (VPS) rented/leased from a hosting provider in order to run WordPress. This was traditionally done with what is referred to as a LAMP stack, Linux, Apache, MySQL and PHP. These days I’m running a LEMP stack which includes Linux, Nginx, MariaDB and PHP. I’ve gone through a few hosting providers in my days, starting with RimuHosting, then Linode and today I’m using DigitalOcean. I’m also still using GoDaddy as my domain registrar. While I’ve heard a lot of horror stories from GoDaddy customers I haven’t experienced any issues myself. I have heard really good stores from customers of Gandi.net.

You can still find my original site on Blogger today at http://michaelfmcnamara.blogspot.com/.

If you are looking to test out blogging I would strongly suggest you start with Blogger or perhaps WordPress.com – not to be confused with WordPress.org. Whether you decide to try Blogger or WordPress.com both solutions make it incredibly easy to get up and running quickly and easily. If you later find that you enjoy blogging and you want to delve into all the features and options then you can migrate your content to any number of solutions, both commercial and other.

Since I run a self-hosted WordPress site I needed to purchase the following components separately;

Domain Name (michaelfmcnamara.com)GoDaddy$56.32/2 years
Virtual Private Server (Linux CentOS 7.6 x64)DigitalOcean$240/2 years
SSL Certificate (Wildcard)RapidSSL$258.00/2 years
$554.32 Total (2 years)

As you can see the costs quickly add up, on average $23/monthly. I advise anyone just jumping into blogging to start out with a free solution until you are ready to commit your hard earned $$$. I use my server to host multiple websites (and more recently a Minecraft server) so the costs presented above are a little skewed so don’t go postal on me in the comments. There are definitely cheaper alternatives out there, this is just what I’m doing these days and it works for me. As another example if you used a WordPress.com Premium account that would run you $8/monthly or $192 over 2 years.

You can look to use advertising to help offset some of the costs above. For a number of years there I was earning about $130/monthly from Google Adsense and directly contracted banner ads which helped offset the costs. It takes quiet a bit of effort to get beyond anything more than “beer” money so keep that in mind if you think you’ll be able to launch a blog or even a YouTube channel and it will start paying for itself in six months.

In the end it’s not Blogger or WordPress that’s going to make your blog successful, it will be the content that you share!

If you have any questions drop them below and I’ll do my best to answer them.


https://blog.michaelfmcnamara.com/2021/03/how-to-start-blogging-in-2021/feed/ 0
LastPass – Internet Upheaval https://blog.michaelfmcnamara.com/2021/03/lastpass-internet-upheaval/ https://blog.michaelfmcnamara.com/2021/03/lastpass-internet-upheaval/#comments Mon, 08 Mar 2021 04:48:54 +0000 https://blog.michaelfmcnamara.com/?p=6903

It seems that everyone and anyone wants to talk about LastPass since their announcement on February 16th that they were going to limit their free tier product offering. The vast majority of videos and articles haven’t been kind to LastPass or their current owners, LogMeIn.

I haven’t really mentioned LastPass since I first talked about them in December of 2014. I’m a paying LastPass customer since 2013. At the time a LastPass premium account was $12/year. A small cost for any IT professional that values their time (and productivity) and security in trying to keep the passwords for every application they use or every system they manage in their head. I currently have 763 passwords in my vault.

It seems that anytime a vendor takes away something that was free the Internet masses take to their media of choice to rail against the injustice. A large number of tech savvy users already scowl at the mention of LogMeIn. The company eliminated it’s free account offering of the popular remote control application by the same name in 2014. In 2016 the company acquired GoToMyPC, the largest competitor to LogMeIn, and subsequently raised the pricing on that service.

I’m no fan of LogMeIn, but I support paying for products that provide a value and service in my day to day life. As an Information Technology professional a Password Manager should be an essential part of your kit. Thankfully there are plenty to choose from and they all have their own strengths and weaknesses.

I believe prior to the LogMeIn acquisition you needed a Premium LastPass account to use the mobile application on either Android or iOS. Someone feel free to correct me in the comments below. I’m not sure where or when that changed was made but somewhere along the line they started allowing non-Premium users to use the mobile app. The timing here is important because it does feel like a potential bait and switch play. Opening the mobile app for a few years and then squeezing that group in hopes of getting some percentage to switch to a Premium account.

If I had to choose a password manager today I wouldn’t necessarily jump at spending $36/year – the current pricing for new LastPass Premium customers. However, I might be convinced to purchase their new LastPass Family for 6 family members at $48/year. That said I’ve been pretty happy with LastPass to date.

What password manager are you using? Hopefully you are using a password manager!


https://blog.michaelfmcnamara.com/2021/03/lastpass-internet-upheaval/feed/ 1
VMware VeloCloud SD-WAN Orchestrator API and Python – Part 3 https://blog.michaelfmcnamara.com/2021/03/vmware-velocloud-sd-wan-orchestrator-api-and-python-part-3/ https://blog.michaelfmcnamara.com/2021/03/vmware-velocloud-sd-wan-orchestrator-api-and-python-part-3/#respond Tue, 02 Mar 2021 03:31:11 +0000 https://blog.michaelfmcnamara.com/?p=6886 It looks like this project is going to be moving forward again… time to dust off the Python code and finish out the last few pieces to the puzzle.

Interestingly enough I ran into a quick problem testing my original code. It looks like something had changed with the “Profile” that we’re using for each Edge. When I run my original Python script I’m getting a HTTP/400 returned along with the following response code, Interface “CELL1” present in Profile but not in Edge. Looking through some of the JSON data it would appear that something has changed with the Profile that I’m using in the configuration. The error I’m getting when calling rest/configuration/updateConfigurationModule likely means that I’m missing some required data in my Jinja templates that the VMware VeloCloud Orchestrator is now expecting.

There is a Chrome extension called VeloCloud Developer Assistant, that can help you break down the JSON data and make it a little easier to visually consume and troubleshoot. I personally prefer just going into the Chrome developer tools and copying out the entire JSON data block that’s being posted and then running that through some JSON formatting tool to help clean it up for human consumption. If you go through the steps in the web UI with the Chrome developer tools open, can you go back and extract all the JSON data that is being sent to the VeloCloud Orchestrator, and in short you can easily reverse engineering the calls and the JSON data.

In the end I was able to find the missing CELL1 interface under the routedInterfaces element. I added the missing data elements to the Jinja template and everything started working again. I ended up writing a few other supporting scripts to help with the overall project goal. I wrote a Perl script to poll the existing hardware to gather up all the IP configuration details from each VLAN and interface which then can be fed into the Python script to build the configuration within the VeloCloud Orchestrator. There’s also a management IP required, so I used a snippet of Perl code that I wrote back in 2016 to call the Infoblox API to assign the next available IP address in the management subnet.

With the Jinja templates it’s relatively easy to put this code onto a web server and build a simple WebUI around some Python or PHP code to generate new configurations when needed.


https://blog.michaelfmcnamara.com/2021/03/vmware-velocloud-sd-wan-orchestrator-api-and-python-part-3/feed/ 0
Working from home upgrades https://blog.michaelfmcnamara.com/2021/02/working-from-home-upgrades/ https://blog.michaelfmcnamara.com/2021/02/working-from-home-upgrades/#respond Fri, 19 Feb 2021 01:00:00 +0000 https://blog.michaelfmcnamara.com/?p=6868 I made some purchases over the past few months to help improve my work from home environment and thought I’d share my thoughts on those items. I purchased all the items below from from my local BestBuy using the BestBuy Android mobile app with curbside pickup. As someone who works in retail I was really impressed with how well the checkout and curbside pickup process works at BestBuy and how effortless it was, a real technology win in my honest opinion. Kudos to their team on an incredibly frictionless process and to all their store associates. The pricing for each item was in line with pricing from online resellers so I wasn’t really sacrificing anything by purchasing from a traditional brick-n-mortar business and I was happy to support my local store.

LG – 34WL500-B 34″ IPS LED UltraWide FHD FreeSync Monitor with HDR (HDMI) – Black

When you upgrade to an UltraWide display you won’t ever want to go back. I desperately needed the additional desktop space on my work laptop to help improve my general productivity. I usually have 15-30 windows open at any time and having to switch back and forth, or worse yet go hunting for individual windows can be an incredible productivity drain. This display only has a max resolution of 2560 x 1080 but that’s fine for my aging eyes and provides me all the desktop real estate I need to work efficiently. The included stand isn’t overly large and brightness levels from the display are great. This monitor is currently on-sale at BestBuy for $300, a great price for a 34″ wide monitor. The 29″ version LG 29WL500-B is an even better deal at BestBuy for $200. I would recommend either of these for a work from home environment. I don’t play any games on this display so I can’t comment about game performance.

Logitech – G PRO X Wireless DTS Headphone:X 2.0 Gaming Headset for Windows with Blue VO!CE Mic Filter Tech and LIGHTSPEED Wireless – Black

I’ve traditionally used relatively cheap Plantronics headsets on my home desktop but I decided it was time to cut the cord and go with a premium wireless headset that would allow me to move around on long conference and video calls. Having the ability to move it around between my personal desktop and my corporate laptop was also extremely beneficial. I’m not yet sold on the Blue VOICE feature, I didn’t particularly like how I sounded with that feature enabled so I need to-do some additional testing and validation. I’m still up in the air about this headset, I’ll need a little more time before I decide if it was a good purchase.

Bose – Companion 2 Series III Multimedia Speaker System (2-Piece) – Black

I’ve often opt for the cheap Insignia speakers but this time I wanted a quality set of speakers to use when I wasn’t using my wireless headset and so I chose the Bose Companion 2 Series III speakers. I’m not a high-fidelity guy but these sound incredibly better than any other computer speakers I’ve ever owned and easily rival the sound put out from the Onkyo receiver and speakers in my basement surround sound system. These speakers get a solid buy rating from me. There are likely better options available for the audiophiles out there but I couldn’t justify spending $200 or $300 on desktop speakers.

Have you made any purchases lately? Anything fun?


https://blog.michaelfmcnamara.com/2021/02/working-from-home-upgrades/feed/ 0
The Swedes are coming! https://blog.michaelfmcnamara.com/2021/02/the-swedes-are-coming/ https://blog.michaelfmcnamara.com/2021/02/the-swedes-are-coming/#respond Thu, 18 Feb 2021 03:17:37 +0000 https://blog.michaelfmcnamara.com/?p=6860 No, I was hacked with some stolen user credentials.

I was surprised today when I noticed that someone had posted a new article to this site at 6:36AM this morning titled “3 Reasons to Start Using Dealspaces”. Interestingly enough the user account used to post the article was a test account under my wife’s name that I probably haven’t used in years.

I went looking at the nginx access.log files and found the relevant entires; - - [17/Feb/2021:11:36:17 +0000] "POST //xmlrpc.php HTTP/1.1" 200 141 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0" - - [17/Feb/2021:11:36:18 +0000] "POST //xmlrpc.php HTTP/1.1" 200 2253 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0" - - [17/Feb/2021:11:36:19 +0000] "GET /2021/02/3-reasons-to-start-using-dealspaces/ HTTP/1.1" 200 9985 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0"

The IP address belongs to a Swedish Internet Service Provider named Bahnhof, not particularly helpful as it could have also been a Tor endpoint or exit node. I can tell from the time stamps that the action was likely scripted as there was exactly one second between each request.

Needless to say I immediately deleted the post and the user account that was used to make the post and then changed my own password out of an abundance of caution. I then scoured the entire WordPress filesystem using the recent backup I had to try and make sure that nothing else was changed. I even dumped the database and ran a quick comparision against a recent backup, again looking for any changes or any obfuscated code.

My Thoughts?

Old user accounts are becoming a bigger and bigger problem as the longer they hang around in the wild they will eventually end up being compromised. This is why IT security professionals plead with users to use different passwords on every single website and to frequently change those passwords. Unfortunately in this case I’m going to guess that the password used for this account likely wasn’t very secure (Test123) and that’s likely how the hacker was able to login to WordPress and post the article. So shame on me for yet again falling into the roll of a user.

Are you curious if your user credentials have ever been leaked? Check out have i been pwned?


https://blog.michaelfmcnamara.com/2021/02/the-swedes-are-coming/feed/ 0
School Music Concerts, Copyrights, Live Streaming and Zoom https://blog.michaelfmcnamara.com/2021/02/school-music-concerts-copyrights-live-streaming-and-zoom/ https://blog.michaelfmcnamara.com/2021/02/school-music-concerts-copyrights-live-streaming-and-zoom/#respond Wed, 17 Feb 2021 03:10:20 +0000 https://blog.michaelfmcnamara.com/?p=6824 I attended a wonderfully orchestrated virtual performance this evening put together by my daughters’ high school music director. The event was streamed using Zoom and it was extremely well done, with the exception that I couldn’t hear the actual music performances.

I could hear other people on the call and I could hear the music director as he spoke but I couldn’t hear any of the audio he was “sharing” from his desktop or laptop. Thankfully it turned out I wasn’t alone as other people quickly reported the same issue in chat… but oddly enough it turned out there were other folks in the meeting that could hear the performance fine so I was stumped.

How is it that some attendees in the meeting could hear the audio but other attendees couldn’t hear the audio?

In the ensuing conversation I heard how one couple had an Apple iPad and they couldn’t hear anything. I learned that even my parents who were using their new Windows desktop that I built for them in January had no issues hearing the performances. I even connected to the meeting from a second Windows device and confirmed that it too had the same problem. I’m not sure if it’s relevant but I did discover that both of my Windows devices were running the same (latest) version of the Zoom client, Version: 5.5.2 (12494.0204). I can only guess that there is a Zoom bug out there that we stumbled into, perhaps it has something to-do with encryption as I noticed that the teacher’s audio stream was alerting as “not encrypted” during the meeting. I felt really bad for the teacher but there wasn’t anything he could do and it definitely wasn’t his fault. Technology had failed him, just like it has failed so many of us so many times. I likely suspect Zoom was the culprit in this specific instance – I actually submitted a support ticket to Zoom from my corporate account so we’ll see where that goes if anywhere. It was just another sign of the times in this new COVID-19 reality that we’re all living in.


The teacher did make a comment that while LIVE streaming was allowed for educational use thanks to waivers from the music publishers, recording was still not permitted. I did some quick searching and found some relevant articles from the National Association for Music Education in an article titled, Music Publishers Agree to Allow Educational Use of Copyrighted Music. If you’ve ever tried to upload your child’s school performance to YouTube you’ll quickly run into issues if the recording has copyrighted music in it. I’ve been there done that, fun times getting a copyright strike against a middle school band performance.

My Thoughts?

I’m very disappointed to say that unfortunately technology failed again today, through no fault of the users. I’m usually pretty harsh on users (the word user is a dirty word in my house) but in this case it was the technology itself that failed. I like finding answers to these mysteries and hopefully Zoom will respond and we can fix it so the next concert can go on without any issues.


https://blog.michaelfmcnamara.com/2021/02/school-music-concerts-copyrights-live-streaming-and-zoom/feed/ 0
Discussion Forums – Closing https://blog.michaelfmcnamara.com/2021/02/discussion-forums-closing/ https://blog.michaelfmcnamara.com/2021/02/discussion-forums-closing/#comments Fri, 12 Feb 2021 02:38:35 +0000 https://blog.michaelfmcnamara.com/?p=6762

It’s hard to believe that the Network Infrastructure Forums have been running for over 11 years. In July of 2009 I installed Simple Machines Forum software on a virtual private server from RimuHosting with the purpose of setting up an open and free discussion forum for network engineers and system administrators. At the time I was extremely frustrated with a number of vendors and manufacturers supporting their users (outside of professional services). I felt that I could help fill the void… and in the case of Bay Networks, Nortel Networks, Avaya and Symbol, Motorola, Zebra I wasn’t far off. I went out and registered the domain networkinfrastructure.info and set out to create a place for IT professionals (including resellers) to share ideas, problems and solutions.

Like most everything in life though, it’s time has come and gone and it is time to more forward. The majority of the information on the forum is now extremely dated and with the rise of other solutions, user traffic has dropped off significantly in the past few years. At it’s peak the forum had a very robust community around Nortel switching and Symbol, Motorola, Zebra wireless equipment serving both end-users and resellers.

If you are interested in some statistics;

  • 49,023 registered members
    • 23,000 legitimate members when you remove all the bots
  • 20,891 posts
  • 4,383 topics
  • 62,621,940 page views

Special thanks to the moderators who helped curate the discussions, I couldn’t have done half as well without your contributions.

  • Dominik
  • Flintstone
  • Paul L
  • Telair

Thank you to everyone who participated in the discussions!


https://blog.michaelfmcnamara.com/2021/02/discussion-forums-closing/feed/ 4
Desktop Remote Control – A new option? https://blog.michaelfmcnamara.com/2021/02/desktop-remote-control-a-new-option/ https://blog.michaelfmcnamara.com/2021/02/desktop-remote-control-a-new-option/#comments Wed, 10 Feb 2021 03:30:00 +0000 https://blog.michaelfmcnamara.com/?p=6772 It can be trying and difficult providing technical support to friends or family members remotely in this COVID-19 reality. A good desktop remote control solution can really make the difference between helping to resolve a problem or everyone walking away extremely frustrated.

I recently had to assist my 75 year old father with an issue he was having and it was a struggle to get through the “Buy Now” banner ad that was popping up from Teamviewer. I probably use Teamviewer 1-2 times a month for 20-30 minutes so I can’t justify dropping $49/monthly on that solution. So I did what any techie would do and I took to Google in search of a new solution.

I stumbled across a solution called AweSun by AweRay. I’ve never heard about AweRay so I dug a little deeper and found the domain name was registered in May 2019 and their first news release was in April 2020. So they are a fairly new player in the space from what I could learn. I also noticed that their infrastructure is housed in Google Cloud Platform.

I’ve spent a few days with the solution and it worked really well. I was able to quickly and easily connect to my parents computer using their Device ID and Passcode very similar to how Teamviewer works. The product also supports copying clipboard data between the local computer and the computer you are remote controlling, so you can cut and paste between computers with ease. When I rebooted my parents computer AweSun properly started itself back up and allowed me to remotely connect after the reboot without any issues or problems.

Looking through their pricing and features list it seems like the “free” version is extremely functional.

The pricing is definitely much better than many of the competing solutions on the market. Obviously you can’t beat free, but even at $9/month – it’s definitely feasible to throw the company some business for a month or two in order to support the product without breaking your wallet.

I’m curious if anyone else has tested AweSun?


https://blog.michaelfmcnamara.com/2021/02/desktop-remote-control-a-new-option/feed/ 3
Microsoft Windows Server 2019 NPS Firewall Bug? https://blog.michaelfmcnamara.com/2021/02/microsoft-windows-server-2019-nps-firewall-bug/ Sat, 06 Feb 2021 00:10:18 +0000 https://blog.michaelfmcnamara.com/?p=6751 I do some consulting on the side, helping end-users and resellers with technical hurdles or issues in their environments. It’s been a pretty good side hustle for me over the years and it can be a welcome distraction from the daily grind.

A reseller recently asked me for assistance with an issue they were having setting up 802.1X authentication for their wireless users and devices. In the early Windows 95 days you needed to make sure you had the correct patches and drivers to get the built-in WPA supplicant (Wireless Zero Configuration) to work properly but these days this solution is pretty well documented across the net and most client devices work right out of the box.

I had assumed that the problem would be something simple but after 2 hours of troubleshooting I too was stumped by a little but apparently well known issue on Windows Server 2019 with NPS (Network Policy Server) which replaced IAS (Internet Authentication Service) starting back in Windows Server 2008. Apparently the default firewall rules added during the NPS server role installation don’t work!

It turns out that this bug goes all the way back to November of 2018. I found a post written by Richard M. Hicks titled, Always On VPN and Windows Server 2019 NPS Bug. That’s just crazy… that’s more than two years ago and apparently Microsoft still hasn’t decided to correct the issue.

Here’s a tip for all those budding network or system administrators trying to troubleshoot 802.1X wireless authentication requests. Whether you are using Microsoft’s NPS or HPE/Aruba ClearPass or Cisco Identity Services Engine (ISE). I find having a tool to generate some RADIUS authentication requests to validate that your RADIUS server is responding and working properly is invaluable. I personally like NTRadPing as it’s easy to use, just drop it in a folder and launch it on a Windows desktop or laptop. Occasionally you might need to hack the RADIUS dictionary file (raddict.dat) that accompanies the application but that’s pretty easy as well.

Have you got any stories to share?


I had a screw loose !$%&# https://blog.michaelfmcnamara.com/2021/02/i-had-a-screw-loose/ Wed, 03 Feb 2021 02:27:42 +0000 https://blog.michaelfmcnamara.com/?p=6728

If you live in the Northeastern United States you’re probably still digging out from the massive winter storm that blew through the area over the past three days. Here in my local area of Pennsylvania we managed to get just over 12 inches of snow, although towns not far away managed well over 23 inches and my family in northern New Jersey are estimating that they had around 18 inches of white stuff.

Thankfully I have a 27″ Briggs & Stratton 1227MD Snow Blower to help clean my 2,000 square foot driveway. I bought this unit back in 2017 and while it hasn’t had too much work in the past few years it’s always been reliable and easy to operate. On Monday afternoon my trusty steed stopped working and a bit of panic set in as the second wave of snow starting falling. The engine was fine, but the snow blower would frequently stop moving forward and the wheels would lock up. The snow blower would move in reverse but it would not move forward. Using a 10mm socket and socket wrench I was able to remove the lower panel that covers the drive train and a screw literally fell out. Looking at the bottom panel it was quite clear that the screw was rolling around at the bottom of the panel and was occasionally getting wedged between the drive gear and the external sheet metal causing everything to “lock up”.

I looked around to see if I could determine where the “extra” screw had come from but I wasn’t able to find anything missing or out of place. I put the machine back together and this morning it ran like a champ for 2+ hours clearing my driveway and my neighbors.

My Thoughts?

Thankfully I was able to quickly troubleshoot the problem and determine the issue. While I enjoy the occasional manual labor, I wasn’t looking forward to the thought of having the shovel 12 inches of snow from my long driveway, so I was sufficiently motivated to “figure it out“. Are you handy? If so great! If not, don’t be afraid to branch out and try new things, whether you are a “handy” person or not. It’s not rocket science!


PS: Thanks for the cake Anita, it was delicious!

Troubleshooting Application Performance and Monitoring with Selenium https://blog.michaelfmcnamara.com/2021/01/troubleshooting-application-performance-and-monitoring-with-selenium/ Fri, 29 Jan 2021 00:27:53 +0000 https://blog.michaelfmcnamara.com/?p=6620 It was yet another exciting week…

When Cloud or SaaS application performance starts impacting user productivity how do you go about troubleshooting? Performance can be extremely subjective… what is fast to some people is slow to others and vice versa. How do you even measure performance? Invariably people want to blame the network because that’s the simplest answer. However, it can take a lot of effort and due diligence to dig down and find the actual culprit.

In this specific case we had ~ 8,000 miles between the users and the server infrastructure. So I’m personally expecting additional challenges due to the extreme round trip times (220ms) and latency that may play some roll in any possible issue or issues.

Let’s try to frame the issue;

  • Is the issue persistent or intermittent? Intermittent
  • Is the issue occurring with any regularity? Yes, 11:00AM – 12:30PM local time daily
  • Is the issue impacting every user or just specific users? Multiple users, not clear if every user is impacted but a majority of users
  • Is there anything common among the impacted users? They are all using the same VPN and proxy server infrastructure, they are all located in the same country.
  • When did the problem start? Users have been working for 3+ months without issue, but this problem is fresh within the past 2 weeks.

The last point is likely key… so what’s changed in the past 2 weeks that’s causing this issue? Let’s get to that later but those simple facts are key in driving your investigation.

We start with the simple baseline network tests;

  • ping – good with minimal pack loss
  • traceroute (mtr) – looks like pathways with multiple ISPs
  • speed tests – generally good
  • packet capture – in general looks good, some out of order packets, some dupe ACKs, these are likely the result of the ~ 8,000 miles between the endpoints.

In the baseline results there are no smoking guns but there are some suspect data points in there, although we need to remember that this isn’t a LAN based application. This is an Internet based application with 8,000 miles between the endpoints so there is going to be some noise in the packet trace.

Note: I’ve seen all sorts of interesting Internet issues since March 2020 when the pandemic lock-down first kicked off here in the US, and again recently at the beginning of September 2020 when the majority of US school students returned to remote learning. I observed a large number of my US users had better latency to our UK VPN gateways than to our local US VPN gateways. Ultimately we found a number of Internet peering points between the different Internet Service Providers (I’m being nice here and not naming names) were getting completely blasted and was adding 75-125ms to every packet. Eventually the providers addressed this problem with additional peering but it was a painful couple of weeks.

Now what we need are some additional data points that can be collected during the issue;

  • HAR (HTTP Archive) from Chrome web browser collected from user experiencing issue – this was a key piece of data that helped move the issue forward
  • packet capture – wasn’t able to be captured due to locked down computers

What can we do to monitor the performance of the cloud application?

  • ping – We setup pings monitors from a number of data centers globally to monitor for basic availability
  • curl – We setup some simple HTTP/HTTPS monitoring using cURL
  • selenium – At the recommendation of the application provider we setup ThousandEyes and a transaction monitor to generate synthetic transactions by logging into the application and working through a few different functions which themselves have dependencies on external REST and SOAP APIs.

The application itself has a number of dependencies from external microservices, so initially we were concerned that these external services might be having performance issues themselves which might be impacting the application itself. So we had to setup additional monitoring to try and validate the performance of those REST and SOAP APIs during the reported timeframes.

This was my first foray into working with Selenium and ThousandEyes but I was able to kludge my way through the solution after about 2 days. I did run into a few problems with the application website using dynamic Class IDs but eventually I got some basic tests working properly. The solution itself worked fairly well… we had some decent “front door” statistics within hours and the synthetic transaction data gave us a good idea that the application was performing properly during the reported timeframes the users were experiencing issues.

The application vendor was extremely helpful in examining the HAR data, and quickly determined from the HAR and their own internal logs that HTTP/HTTPS requests from the clients were being queued up and delayed from reaching their back-end infrastructure (Chrome only allows 6 concurrent connections to a single hostname). Within the HAR data the vendor observed some fairly aggressive custom polling within the application that was making unconditional Javascript calls every 2 seconds that resulted in a 12Kb data set being transferred to the client. The initial theory was that some Internet slowdown was causing the client requests to slowdown and eventually fall behind which then coupled with the unconditional Javascript calls and the six connection limit in Chrome led to an extremely poor user experience.

We eventually learned that the infrastructure the users were riding had recently switched Internet Service Providers two weeks earlier. Hmmm… hadn’t the issues started 2 weeks earlier? Yes they had! Ultimately we determined that there was enough occasionally packet loss and packet retransmissions over this new Internet link that it was impacting this specific application. The infrastructure was switched back to the original Internet link and the issue hasn’t been observed since.

My Thoughts?

In this specific case the intermittent packet loss and retransmissions were causing the application to fall behind in it’s communications with the backend infrastructure which was resulting in an extremely poor user experience. It’s relatively safe to argue that if the application code wasn’t as aggressive in it’s polling that it could potentially “tolerate” a certain amount of packet loss and retransmissions.

I personally believe as a network engineer it’s invaluable to learn why something doesn’t work instead of just accepting that it doesn’t work. Inevitably there will be things that we can’t explain but I’m a huge advocate of spending the effort to make sure you understand the vast majority, it’s really the only way you’ll make the environment around you better and ultimately more resilient.


Weight Loss and Personal Health https://blog.michaelfmcnamara.com/2021/01/weight-loss-and-personal-health/ Fri, 22 Jan 2021 23:00:00 +0000 https://blog.michaelfmcnamara.com/?p=6698

In 2020 my diet and personal health choices finally caught up to me and I had to make some drastic changes. Since that September day I’ve lost more than 60lbs using a mixed low-carb / keto diet. Unfortunately I’ve also had to account for my gout diagnosis which has left me with some very restricted menu options.

The data in the graph below is from a Withings WiFi Scale and makes it pretty easy to see how quickly the weight can catch up to you, or me I guess I should say, over the years.

In July of 2018 I broke my ankle while playing ice hockey and that literally sidelined me for more than six months. The only good thing during that timeframe? I literally couldn’t get to the kitchen for months so I didn’t put on any weight. I owe that feat to my wonderful wife, she loves telling me “no!“. You can see in the graph above that 2018 was relatively flat, until I got my mobility back and started snacking again. :(

How did I do it?

In my specific case, the hunger wasn’t the big issue. The pain from the gout attack was pretty severe and lasted almost 4 weeks, and it literally masked my general hunger for the first few weeks. I was religious about keeping my fluid levels up, drinking 160oz of water daily trying to flush the uric acid out of my body. I believe the water kept me feeling full, but the numerous trips to the bathroom can be super annoying. However, it promotes getting up from the desk frequently which is a good thing.

My job for the past 7 years required me to commute almost 100 miles each day keeping me in my car for ~ 3 hours each day about 15 hours each week. Thankfully that’s changed dramatically with COVID-19 and I’ve made use of the extra free time to walk my dog, Bucket, twice daily. The Fitbit Versa 2 I wear helps track the 10,000+ steps I try to tally daily. Not surprisingly my resting heart rate has dropped from an average of 63bpm to 53bpm at my current weight.

What’s the future hold?

In December I had another round of blood tests and there was “excellent improvement“, as noted by my physician. I still need to have another round of blood tests again this month, so here’s hoping that things are continuing to improve.

As for my personal goals, I would like to get down to around 240lbs. The trick will be adopting a diet and eating behaviors that I can use to maintain my weight and not start piling it all back on once I hit my goal. Ask me in six months where I am. Don’t be so focused on work, or the family or everyone else around you that you forget to take care of your own health!


Desktop Build Fails – Time for Upgrade https://blog.michaelfmcnamara.com/2021/01/desktop-build-fails-time-for-upgrade/ Sat, 09 Jan 2021 17:54:15 +0000 https://blog.michaelfmcnamara.com/?p=6687

In November I got a call from my parents that the desktop I built for them back in 2014 was not booting up. After talking my father through opening the case and sending me some video, it appeared that the case and CPU fans would start but then stop, only to start again and then stop again, rinse and repeat over and over. I had seen this symptom before and thought the issue might be with the power supply so I ordered a new EVGA 500W power supply and had it shipped to their house. Kudos to my 70 year old father for replacing the power supply by himself, unfortunately the problem persisted so that likely meant that the motherboard had failed, although the motherboard looked fine and there were no obvious failed capacitors or other damage.

That machine had lasted them six years, so that was a pretty good investment in my honest opinion and I could likely reuse the case, power supply, CPU and memory from that computer on other builds or projects my daughters were working on. In the end I decided to just build them a new machine, swapping the SSD from their old machine, this way there would be minimal change to them. All their email and shortcuts would be there, all the software would be the same, the icons would all be in the “right” place. It would just make things super simple for them as the user, especially since both of my parents are in their 70s.

I went and ordered a bunch of new components to build them a new machine.

With the help of my youngest daughter we assembled the the pieces of the new machine and then tested that everything was working properly using a spare SSD that I had available.

My parents brought the old machine over to my house, I removed the SSD and installed it into the new machine, cleaned up some of the drivers, installed a new Windows 10 license key (OEM version would not re-activate) and they were back up and running with minimal fuss to them.

Let’s see how long that machine lasts them.


Twelve Days of Cooking and Baking https://blog.michaelfmcnamara.com/2021/01/twelve-days-of-cooking-and-baking/ Mon, 04 Jan 2021 23:00:00 +0000 https://blog.michaelfmcnamara.com/?p=6656 Over the holidays I ended up doing a fair bit of cooking and baking with relatively impressive results even though the family is fairly hard to satisfy as they all have different tastes and preferences. As always I need to give credit to my loving wife and assistant.

Here are a few of the recipes that really stood out….

Smash Burger
Classic Smashed Burgers Recipe from Serious Eats

The kids and wife really like Five Guys and In-N-Out Burger so I often enjoy recreating their favorite burgers here at home… it’s really not that hard, all you need is a decent cast iron pan and a penchant for smashing things. ;) If you want to go all the way, you can even wrap the burger in tinfoil if you want the “soggy” effect.

Dad’s Famous Potato Wedges
Crispy Garlic Baked Potato Wedges by Cafe Delites

There’s nothing famous about them… other than me calling them famous. I like to soak the potatoes in water for an hour after cutting them up, helps make them nice and soft on the inside while crispy on the outside.

Beer Braised Chicken and Bacon
Beer-Braised Chicken by Food Network

This is one of the newer recipes I’ve been cooking and it has been delicious. The inclusion of mustard gives it that little bite… remember to cook the bacon ahead of time, lay it out on parchment paper on a baking tray and let it do it’s thing at 350F, then just add it back into the dish just before you serve.

Chocolate Chip Cookies
Best Chocolate Chip Cookies by allrecipes

Chocolate chip cookies are a staple in any American household, unfortunately they don’t last long in my household so you better get one before they are gone.

The Best Pizza Dough Recipe by Sugar Spun Run

I like using bread flour instead of all purpose flour, and the King Arthur brand of bread flour has worked well for me in the past although I need to continue to try and perfect my hoagie rolls. I feel like a good pizza stone is the key to making good home made pizza. The pizza itself was delicious, the problems arose when trying to transfer the pizza to the hot pizza stone in the oven, that first attempt didn’t go so well. I guess I need to pickup a pizza peel.

French Onion Soup
French Onion Soup by Simply Recipes

This recipe takes a little bit of time to get the onions to caramelize. In the end I cheated by using croutons instead of making some Crostini – in my defense I was hungry!

Chicken Parmesan
One-Pan Crispy Parmesan Chicken Cutlets by Kitchn

The family just loves the crispy fried chicken cutlets. I like to quickly fry them in a cast iron skillet and then move them to the oven for ~ 10 minutes at 350F to finish cooking while I get everything else ready. Don’t cover them, else they’ll end up getting soggy. And Margaret doesn’t like soggy chicken cutlets.

Apple Puffs
Apple Cinnamon Pastries by Entertaining with Beth

I like to peel the apples and fry them ahead of time in some butter – I want super soft apples in my apple pie and apple puffs. I would suggest you roll out the puff pastry a bit, else it will be very thick. The next time around I think I’ll try to make an Apple Strudel instead of Apple Puffs or Apple Turnovers.

Cinnamon Rolls
The Best Cinnamon Rolls You’ll Ever Eat by Ambitious Kitchen

These weren’t as big a hit as I thought they would be with the daughters. The wife loved them and I thought they were pretty good as well. The second time around I skipped the cream cheese and definitely preferred the all sugar glaze/icing.

Let me know what your cooking!


Merry Christmas and Happy New Year 2021 https://blog.michaelfmcnamara.com/2020/12/merry-christmas-and-happy-new-year-2021/ Thu, 24 Dec 2020 23:40:37 +0000 https://blog.michaelfmcnamara.com/?p=6649 I’m sure many of you, like myself, are eager to put this year behind us…. and likely many more are missing loved ones that are no longer with us. There’s no doubting that 2020 will be remembered along with all those that have left us too soon.

Wishing you and your family a Merry Christmas and Happy New Year!


Herman Miller Aeron Office Chair at Home? https://blog.michaelfmcnamara.com/2020/10/herman-miller-aeron-office-chair-at-home/ https://blog.michaelfmcnamara.com/2020/10/herman-miller-aeron-office-chair-at-home/#comments Sun, 04 Oct 2020 13:33:13 +0000 https://blog.michaelfmcnamara.com/?p=6629 How many of us gave in and bought a new office or desk chair in 2020?

Like so many others, I was extremely fortunate to be able to work from home during a time when so many others were out of work and financially struggling. Unfortunately working from home has brought it’s own set of challenges and hurdles.

I’ve had back and neck issues in the past so I generally need to be extremely thoughtful of my posture and how square I am when sitting at a desk or I’ll end up paying the price. Unfortunately I started having pain in my lower back from spending so much time in my relatively cheap Staples office chair in May and so I re-purposed an old wood workbench I built almost a dozen years ago from 2×4 lumber and some Oak plywood sheets. It happened to be just the right height for a standing desk and allowed my back time to recover.

Ultimately I broke down and ordered a Herman Miller Aeron office chair which I finally received on July 3rd. While it was relatively expensive at $1,200 – I felt like it was an investment in my health and well being. I would guess that I probably spend at least 60 hours a week in this chair. It’s been almost three months and thankfully my back hasn’t bothered me at all since I started using the Herman Miller. I continue to be mindful of my posture at the desk, and I try to take more frequent breaks to get up from the desk itself.

It seems there’s been a rush on desks, chairs and webcams in this new COVID-19 era. I’m curious what everyone else is seeing in their area. Stay safe everyone!


https://blog.michaelfmcnamara.com/2020/10/herman-miller-aeron-office-chair-at-home/feed/ 2
Epson Printer Firmware Update Restricts Third-Party Ink Cartridges https://blog.michaelfmcnamara.com/2020/09/epson-printer-firmware-update-restricts-third-party-ink-cartridges/ Wed, 02 Sep 2020 22:50:24 +0000 https://blog.michaelfmcnamara.com/?p=6601 I was recently working with an Epson XP-15000 I had lying around the office, cleaning it up so I could send it off with my daughter as she heads back to college and her off-campus apartment (even though all classes are now via remote learning thanks to COVID-19). I had everything working perfectly but the printer kept prompting about some firmware update, so I eventually relented because I knew the firmware update prompt would likely cause confusion and questions down the road from the “user”.

Well I got more than I bargained for in that firmware update. !@$# %e

Let me just point out that this is no $100 printer, but instead sells for $350 – $450 retail. After the firmware update the XP-1500 printer would no longer recognize the third-party ink cartridges that I had installed in the printer no matter what I would do. Likely needless to mention at this point, but I quickly found I wasn’t alone as there were dozens if not hundreds of posts all over the Internet reporting the same issue. Here’s one, another, another and another… you get the idea.

It seems there was a class action lawsuit filled last October in California against Epson for this exact behavior as mentioned in an article on TheRegister titled, US customers kick up class-action stink over Epson’s kyboshing of third-party ink. Although it appears that case was dismissed back in February 2020.

I did briefly entertained the possibility of using InkChip, a third-party located in Hong Kong that provides a firmware for the printer that removes the “ink cartridge” check from the printer’s software. Although in the end I went out to my local Staples and spend over $130 in Epson ink cartridges, this printer has six ink cartridges.

Interestingly enough I couldn’t find any clean reviews of InkChip – I can only imagine that the printer manufacturers have a plethora of lawyers just ready to pounce one any mention of this company on the Internet – I wouldn’t be surprised if I was contacted – remember it’s 2020 and anything can happen this year!

I’ve been an Epson customer for almost 15+ years, I’ve probably bought more than 7 Epson printers in those past 15 years easy. I’m not sure I’ll be buying another Epson printer in the future.

The moral of the story… don’t upgrade the firmware in your printer if you use third-party ink cartridges.


CenturyLink/Level 3 Internet meltdown followed by Reddit moderator madness https://blog.michaelfmcnamara.com/2020/08/centurylink-level-3-internet-meltdown-followed-by-reddit-moderator-madness/ Sun, 30 Aug 2020 20:05:56 +0000 https://blog.michaelfmcnamara.com/?p=6602 It was another exciting morning around the Internet. Seems that CenturyLink(Level 3) had a meltdown that caused all sorts of issues for ~ 5 hours this morning starting around 6:04AM EDT and lasting until around 11:12AM EDT.

It started as it always does with reports of DNS issues, then CDN issues (Cloudflare) and eventually CenturyLink was identified as the culprit, or to be more precise any packets traversing the CenturyLink (Level3) network.

Thankfully Reddit was a great community resource and reports quickly started rolling in on these two threads;

For reasons that still aren’t 100% clear the moderators for r/networking decided to delete the first thread. So the refugees from r/networking went to r/sysadmin to escape the persecution only to have the moderators of r/networking admit their mistake sometime later and un-delete the post.

I’ll admit I was floored when I found the original thread was deleted. There were hundreds of us struggling to source what was actually going on and trying to understand how we could mitigate the impact to our employers and some moderator deletes the thread?!? @$%#

The refugees eventually made their feelings known in a thread titled, META: I guess major news-worthy outages are off topic here?


VMware VeloCloud SD-WAN Orchestrator API and Python – Part 2 https://blog.michaelfmcnamara.com/2020/08/vmware-velocloud-sd-wan-orchestrator-api-and-python-part-2/ Sun, 02 Aug 2020 14:26:20 +0000 https://blog.michaelfmcnamara.com/?p=6517 Update: July 2020 – unfortunately COVID-19 halted my VeloCloud roll out just as it was starting. It’s difficult being a retailer when you can’t have your 650 stores open for business.

In my previous post I detailed how I was setting out to programatically create 650 VeloCloud Edge profiles using VMware VeloCloud’s Orchestrator API.

I had a few hours to dedicate to the quest last weekend and I was able to complete a working Python script. It turns out I was missing an additional parameter in one of my calls that took a few hours to track down. The modulesId needs to be determined from getEdgeConfigurationStack, this appears to be a set of device specific configurations that will override the default “Profile” settings.

I now I have a working script that will build an edge configuration using a set “Profile” within the Orchestrator and then passing it a template using Jinja2 with the device name, VLANs, IPv4 addressing, etc.

Here are the updates steps I’m taking in my script;

  • Step 1. Login via rest/login/enterpriseLogin (store authentication cookie)
  • Step 2. Call rest/enterprise/getEnterpriseConfigurationsPolicies to get the profileId that we’ll be using for all the devices (this is the equivalent of the Profiles in the web UI)
  • Step 3. Call rest/edge/edgeProvision with template passing device name along with profileId (again this is the Profiles in the web UI), result will be edgeId and activation key
  • Step 4. Call rest/enterprise/getEnterpriseEdges passing edgeId to confirm
  • Step 5. Call rest/edge/getEdgeConfigurationStack to get modulesId of new edge profile (this is the device specific profile for anything that is overridden from the “Profile” set in the device configuration)
  • Step 6. Call rest/configuration/updateConfigurationModule with template replacing edgeId, profileId and modulesId along with IPv4 addresses, etc parse result to confirm – THE MAJORITY OF WORK IS ACCOMPLISHED IN THIS STEP
  • Step 7. Logout via /rest/logout

Those are the major steps… now I need to write some accompanying code to parse a list of stores and ultimately dump all of this into a database or CSV so we can store and track the activation code for each physical device.

I will also work on publishing the code I’m using so others can follow in my footsteps… it really wasn’t that hard, it took a few days to figure out the REST API calls and then the relationships between the different ‘modules’ and then track down the missing pieces to get everything working properly.

If there’s interest in me releasing the code, drop a note below… depending on the interest, I’ll see if I can make time to clean up the code and publish it to Github.


YouTube TV 2+ Years Later https://blog.michaelfmcnamara.com/2020/07/youtube-tv-2-years-later/ https://blog.michaelfmcnamara.com/2020/07/youtube-tv-2-years-later/#comments Sun, 26 Jul 2020 13:29:12 +0000 https://blog.michaelfmcnamara.com/?p=6576

In February 2018 I made the switch to YouTube TV, dropping the very expensive Verizon FiOS TV. When I first picked up YouTube TV the subscription was $35/monthly and with my Verizon FiOS 1Gbps Internet at $130/monthly I was saving around $65/monthly over what I had been paying Verizon for both Internet and TV. That was a huge savings when you think about that year over year… $780/yearly. And it enabled my family to watch TV on whatever device they wanted to use.. TV (Roku), Tablet, Smartphone, etc.

Last month YouTube announced that the price for new and existing members would be increasing to $64.99/monthly, although they would be adding 8 additional channels. As many have pointed out that’s a significant increase, almost $180 more yearly. The “cheaper” alternative in YouTube TV is actually now one of the more expensive options available to consumers.

There are hundreds of posts on the Internet that detail the alternatives to YouTube TV, – Sling TV, Hulu Live, Philo, etc. I’m not going to go through all the different options… why are you here anyway?

The six accounts are very handy for my large family… the limitation of three streams though can make for some interesting hallway discussion when the kids are constantly kicking each other out. It would have been nice if YouTube TV had increased the number of streams when increasing the price. Perhaps if YouTube were to bundle YouTube Premium, now that’s a feature that I would really enjoy. I would probably estimate that I watch more YouTube than “TV” these days… and in a COVID-19 world, with no sports to watch, I don’t see that changing anytime soon.

I’ll keep YouTube TV for now… but I’ll probably need to evaluate my options in the near future.


https://blog.michaelfmcnamara.com/2020/07/youtube-tv-2-years-later/feed/ 2
Your certificate expires in 1 day!!! https://blog.michaelfmcnamara.com/2020/07/your-certificate-expires-in-1-day/ Fri, 17 Jul 2020 01:07:15 +0000 https://blog.michaelfmcnamara.com/?p=6579

We’ve all had SSL certificates expire… however, some are more important than others and have a much broader impact. In February 2020 Microsoft had a certificate expire that brought down Teams as reported by the Engadget in an article titled, Microsoft Teams went down because of an expired certificate. The SSL certificate on the discussion forums that I still maintain was set to expire tomorrow so I thought I should replace it. I could have used Let’s Encrypt for free but I chose to use RapidSSLonline, as it was only $14.99/year for 2 years. I’ve seen too many websites with expired Let’s Encrypt SSL certificates when certbot goes awry or someone forgets to manually renew the certificate every 90 days.

There are hundreds of posts, articles, blogs, video on this topic so I’m going to be extremely brief here. In this case I decided to generate a new private key and essentially a new certificate for forums.networkinfrastructure.info even though I was technically renewing the certificate. I purposely like to keep the FQDNs in the certificate filenames and I keep the files in /etc/ssl/certs for reference in any Apache or Nginx configuration files. For the example below I’ll use my.domain.com to keep the text manageable.

openssl genrsa -out my.domain.com.key 4096
openssl req -new -key my.domain.com.key -out my.domain.com.csr
cat my.domain.com.csr

I went online to RapidSSLonline and ordered a SSL certificate, completed the domain validation via email and then cut-n-pasted the contents of the certificate signing request (my.domain.com.csr) into their portal. Waited for the validation and generation and then downloaded the SSL certificate which I stored in a file called my.domain.com.crt. I also downloaded the certificate for the intermediate root, which in this case was for RapidSSL RSA CA 2018. The intermediate root, if any, will change depending on who you use to purchase the SSL certificate.

In order to make sure the certificate chaining was correct, I had to copy the intermediate root certificate along with the certificate I had just downloaded in a single file (bundle).

cat my.domain.com.crt > bundle.my.domain.com.crt
cat intermediate.crt >> bundle.my.domain.com.crt

I used the filename bundle-my.domain.com.crt and then copied the private key and the bundle to my server so I could update the Nginx configuration.

ssl_certificate     /etc/ssl/certs/bundle-my.domain.com.crt;
ssl_certificate_key /etc/ssl/certs/my.domain.com.key;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/ssl/certs/dhparams.pem;

With that done I checked the Nginx configuration file;

service nginx configtest

And then performed a restart to push the change into production;

service nginx restart

The last step was to visit SSL Shopper to verify the certificate and chaining was all good.

Any questions, let me know.


How to install and setup Ansible to manage Junos on CentOS https://blog.michaelfmcnamara.com/2020/07/how-to-install-and-setup-ansible-to-manage-junos-on-centos/ https://blog.michaelfmcnamara.com/2020/07/how-to-install-and-setup-ansible-to-manage-junos-on-centos/#comments Fri, 03 Jul 2020 12:01:48 +0000 https://blog.michaelfmcnamara.com/?p=6563

If you Google “Ansible” and “Junos” you’ll find literally hundreds of articles, posts and videos… some covering pre 2.0 Ansible, some covering Ansible 2.5, or 2.6 or later and almost all of them are completely different – and a great many of the instructions no longer work!

I recently wanted to test out the Ansible Junos modules put out by Juniper but first I had to spend a good hour figuring out all the inter dependencies to get everything working on a CentOS 7 server. The Juniper DAY ONE: AUTOMATING JUNOS WITH ANSIBLE written by Sean Sawtell is a great starting point but I ran into problems just getting my local environment running. The hundreds if not thousands of posts and videos were extremely confusing and I quickly grew frustrated.

What follows is a quick guide on how to get everything working on a minimal CentOS 7 server. Depending on your requirements, it might be more advisable to look at running a fully prepared Docker container, where all the needed software is ready to run. You just need to provide the Ansible configuration and playbooks.

Here’s what you need to-do from root or a root equivalent account using sudo. Since I built this test VM on a VMware ESXi 6.5 server I wanted to install the open-source VMware tools and perform any updates.

yum install open-vm-tools
yum update
init 6

yum install epel-release
yum install python3 jxmlease

pip3 install ncclient
pip3 install junos-eznc
pip3 install ansible

ansible-galaxy install Juniper.junos

That’s all you need and you are ready to go… if you want to play around with Netmiko or Napalm you only need to use PIP to install those Python modules.

pip3 install netmiko
pip3 install napalm


https://blog.michaelfmcnamara.com/2020/07/how-to-install-and-setup-ansible-to-manage-junos-on-centos/feed/ 2
Aruba ClearPass – userPrincipalName and samAccountName https://blog.michaelfmcnamara.com/2020/06/aruba-clearpass-userprincipalname-and-samaccountname/ Sat, 27 Jun 2020 13:53:15 +0000 https://blog.michaelfmcnamara.com/?p=6556 I’ve recently been standing up a number of virtual Aruba ClearPass appliances to provide 802.1X RADIUS authentication for both wired and wireless clients. If you are using Windows Active Directory as an authentication source, here’s a quick trick to allow your users to authenticate using either the userPrincipalName (email address) or their samAccountName (username). In my current environment, we’re a multi-brand organization with multiple @brand.com email domains where users are more likely to know their email address than their AD username. In it’s default configuration Aruba ClearPass will only authenticate against the username (samAccountName).

Log into Aruba ClearPass and go to the Policy Manager and select Configuration -> Authentication -> Sources and select your Windows Active Directory source – see the example below;

You need to update the filters on the source such as follows.

Original ClearPass Filter Query:
Updated ClearPass Filter Query:

And then don’t forget to Save the changes and now you should be good to go!


COVID-19 The War Waged by Information Technology Professionals https://blog.michaelfmcnamara.com/2020/03/covid-19-the-war-waged-by-information-technology-professionals/ Fri, 27 Mar 2020 01:29:02 +0000 https://blog.michaelfmcnamara.com/?p=6516 The past few weeks have been extremely exhausting both professionally and personally. Coronavirus (COVID-19) has taken the world by storm and is literally upending people’s daily lives and ruining businesses large and small. Let’s not forget the large number of people that have lost their lives to this virus. My thoughts and prayers for all those who have lost love ones. My thanks and admiration to all those medical professionals on the front lines treating the sick.

While very few of us have planned and organized days these past few weeks have been unlike anything I’ve ever experienced, running from one fire to another, one disaster to another. Whether it’s a power failure in a data center or someone deciding to water the potted plant that they hung over the network switch, there’s always some new emergency or problem that requires IT to jump in and save the day. This event was no different but the scale and duration was a whole new experience for everyone.

We started mobilizing our disaster preparedness plan around the middle of February. The initial request from the leadership team was pretty straight forward, “How do we prepare to have our home office employees and call centers agents work remotely?”. Like most large-medium sized enterprises we have a couple of hundred people working remotely every day, however we were talking about going from 200-300 daily remote users to potentially 3,000-4,000 daily remote users in a very short time span. And a significant portion of those users still had desktop devices.

In the span of a week we had ordered, imaged, configured and deployed (shipped or handed out) over 400 laptops to over 400 employees and call center agents. We also spun up a new Virtual Private Network (VPN) solution using Palo Alto Network’s GlobalProtect to help supplement our existing Pulse Secure and Microsoft Direct Access solutions.

I should note that I reached out to Pulse Secure and they offered us a temporary 60 day license to help us cope with the additional users – kudos to Pulse Secure.

Like everyone we’re in the middle of our second week and the Internet itself is starting to show it’s cracks. This past Monday and Tuesday we experienced connectivity issues across 30 stores in and around London, UK for ~ 45-60 minutes at a time. We later learned that Monday was the first day in the UK with all schools closed and British Telecom (BT) wasn’t handling the strain well. I’m sure it’s not helping BT that Disney+ just launched in the UK and Ireland on Wednesday.

We’ve had a number of issues with Microsoft, Slack and Zoom over the past two weeks and expect those issues will likely continue as more and more people around the nation and globe transition to working remotely.

Nobody’s really sure what the future holds… hopefully things will start to improve as we work to flatten the curve.

Thanks to all the IT folks that are continuing to carry on the struggle, be it onsite or from the confines of your own home…we know what what your’re going through and we appreciate your efforts!

If you have story to share, let us know below.

Stay safe! Cheers!

Juniper EX4300 & EX2300 J-Web Authentication via TACACS+ https://blog.michaelfmcnamara.com/2020/03/juniper-ex4300-ex2300-j-web-authentication-via-tacacs/ https://blog.michaelfmcnamara.com/2020/03/juniper-ex4300-ex2300-j-web-authentication-via-tacacs/#comments Thu, 26 Mar 2020 01:56:37 +0000 https://blog.michaelfmcnamara.com/?p=6531 About 6 weeks back now I thought this was going to be a quick configuration and I’d be done… this was all back before the global pandemic. Unfortunately, a few minutes turned into six week journey.

We were looking to provide our 24×7 and IT support teams with read-only access to the CLI and J-Web interfaces on our EX4300 and EX2300 switches. We were going to start with using TAC_PLUS but we would eventually integrate with our HPE/Aruba ClearPass instances down the road (authenticating against Windows Active Directory).

I quickly found out that authenticating against TACACS+ while logging in via J-Web was broken, SSH worked fine but logging in via the web browser was broken. The error, “Invalid username or password specified” would always be returned. Some quick troubleshooting showed that the switches weren’t even reaching out to the TACACS+ servers so we decided to reach out to JTAC. We were running Junos 18.2R3-S2 for the EX2300 and Junos 18.4R2-S2 for the EX4300, these were the recommended software releases for each platform at the time I started this adventure.

This past week Juniper let me know that there was a PR raised for the following;

Logging into JWEB fails with “Invalid username or password specified”, but same credentials work for SSH access to CLI when authentication-order is configured

The issue was resolved in the following software releases;

  • EX4300 – Junos 18.4R3
  • EX2300 – Junos 18.3R3-S1

I upgraded some switches in order to test and wouldn’t you know it.

It works!


https://blog.michaelfmcnamara.com/2020/03/juniper-ex4300-ex2300-j-web-authentication-via-tacacs/feed/ 2