Cisco WLC Bonjour Process Task and Expired Certificates

It’s been a crazy for weeks for me… vacation, consulting engagements, traveling to Reno, NV to stand up a new network – rack, stack, install, configure, test and turnover. So I thought after returning to Philadelphia this past week that things would slow down a little, boy was I wrong. I had a number of challenges and what follows is just one of them involving wireless – I also have another one involving the Lenovo Thinkpad T460 and the Intel AC 8260 Wireless adapter having issues with 802.11n over a Cisco 1262N AP but that’s another story.

On Wednesday morning I had two Cisco 5508 Wireless LAN Controllers both crash with a “Bonjour_Process_Task” taking too much cpu: 100% error message. It turns out that this is a known issue (CSCux78464 WLC crashes in Process Bonjour_Process_Task) that is resolved in 8.0.135.1, an engineering release which you need to contact Cisco TAC to obtain. If that wasn’t enough excitement for the morning I quickly noticed that of 120 APs that we usually have connected to the WLC we only had about 70 APs connected.  A quick examination of the debug logs (debug capwap errors enable) showed that multiple APs were failed to join the controller with messages like “Discarding non-ClientHello Handshake or DTLS encrypted packet” and “DTLS session is not established”. A quick call to Cisco TAC revealed that there are built-in certificates into the APs that can expire over time and that’s what had essentially happened. The certificates had expired since the APs had last joined the WLC and now that the certificates were expired they were not able to join the controller. Thankfully there’s a command in the CLI to ignore the certificate expiration;

config ap cert-expiry-ignore mic enable

With that command configured on the WLC the APs starting joining the controller and all was well again.

The field notice from Cisco providing all the details can be found here.

Cheers!