While recently troubleshooting a potential network connectivity issue I noticed some odd behavior from a Windows 10 desktop while doing a trace route to www.google.com. I quickly discovered the problem was only evident from my home Verizon FiOS Internet connection.

I highly doubt there’s a Google caching server 2 hops away from my desktop… so I ran a few trace routes to endpoints that I know are in Ireland and Singapore and they to responded in under 1-5ms. That response wasn’t physically possible given the distance that ICMP packet would need to traverse. I ran a quick test by setting up a Linux CentOS Linux VM on Digital Ocean in their Singapore Data Center and ran a quick traceroute while performing a packet trace on the CentOS Linux VM – no ICMP traffic was observed from my Windows 10 desktop. So there was something else responding on behalf of the destination IP.

I ran a quick scan with Norton Internet Security – not entirely sure how well that product protects Windows machines today as Norton seems to be more of an advertising platform for LifeLock but that’s a discussion for another day and time. In the end Norton didn’t reveal any surprises and I couldn’t find anything on my end after replicating the same behavior on multiple other machines, both Windows and Linux (CentOS/Ubuntu) on my home network. I did noticed that UDP and TCP traffic seem unaffected, just ICMP traffic. So I did what any good engineer would do, I took to Google Search and found that I wasn’t alone in my discovery;

Verizon Community Forums – TRACE1 ROUTE Odd Results FiOS Network

There was also a post and reply over on the r/networking reddit sub from reddigel suggesting that the issue was a DNAT rule that was forwarding ICMP traffic with a TTL > 1 to the ONT .

The DNAT theory seems the most plausible explanation because something is replying to the ICMP packets with a TTL > 2 and it’s fairly close to the source.

Anyone have any other theories?

Cheers!