SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!
I was looking at some DNS stats and noticed one host was by far top of the most DNS lookups list, with more than 3x the next highest host... I was curious and went looking to find out what it was - only to see that it is my SG2005P-PD, doing mltiple DNS lookups of multiple NTP hosts every few seconds - why would it be doing this, and how do I fix it so it stops? Every omada device on the network should have the same NTP config, so it is really odd that this one device is behaving so badly on its own.
Small snapshot of multiple DNS requests for multiple NTP hosts every few seconds from 192.168.4.92
Confirming that 192.168.4.92 is this Omada managed switch:
By far this one switch is dominating DNS lookup, all for these NTP hosts (30% of ALL DNS requests):
My Site NTP config only has the single "time-dot-nist-dot-gov" (using "-dot-" to prevent illegal link blocking) host specified, so I don't even know where it is getting the other "ntp1-dot-glb-dot-nist-dot-gov" NTP hostname from... and it shouldn't be looking up either multiple times a second (it shouldn't be attemptig to sync time multiple times a second either)!
- Copy Link
- Subscribe
- Bookmark
- Report Inappropriate Content
Hi @daubstep
Thanks for posting in our business forum.
It has nothing to do with any of the devices. It is a problem with the domain, your NTP server.
It was first resolved into ntp1. domain, and then ntp1 was resolved into the IP.
So, it is not a problem with the switch or the Adguard you've deployed. Normal behavior.
- Copy Link
- Report Inappropriate Content
Hi @daubstep
Thanks for posting in our business forum.
It has nothing to do with any of the devices. It is a problem with the domain, your NTP server.
It was first resolved into ntp1. domain, and then ntp1 was resolved into the IP.
So, it is not a problem with the switch or the Adguard you've deployed. Normal behavior.
- Copy Link
- Report Inappropriate Content
@Clive_A But why are omada switches (I removed this one, now two others are showing the same behavior) checking time multiple times per second? Surely this isn't normal behavior... once an hour, once a minute even if being super-aggressive... but I've never seen an ntp setup query to check the time multiple times a second all day long???
Current top two DNS calling clients are two of the three omada switches on the network (note: not any of the other omada gear, all of which should be using the SDN ntp config site-wide) - so there is definitely something wrong here...
These two switches are running:
- SG2008P: 3.20.2 Build 20240920 Rel.36048
- SG2210MP: 4.20.3 Build 20240920 Rel.36048
- Copy Link
- Report Inappropriate Content
Hi @daubstep
Thanks for posting in our business forum.
daubstep wrote
@Clive_A But why are omada switches (I removed this one, now two others are showing the same behavior) checking time multiple times per second? Surely this isn't normal behavior... once an hour, once a minute even if being super-aggressive... but I've never seen an ntp setup query to check the time multiple times a second all day long???
Current top two DNS calling clients are two of the three omada switches on the network (note: not any of the other omada gear, all of which should be using the SDN ntp config site-wide) - so there is definitely something wrong here...
These two switches are running:
- SG2008P: 3.20.2 Build 20240920 Rel.36048
- SG2210MP: 4.20.3 Build 20240920 Rel.36048
30,000 are all NTP? No other domains involved?
- Copy Link
- Report Inappropriate Content
30,000 are all NTP? No other domains involved
Yes, all NTP hostname lookups and nothing else (as expected, as these switches shouldn't be originating any other traffic)
There must be at least two things wrong here:
- No need to attempt to hit ntp anywhere near this frequently - time should be stable enough for more than 8 seconds
- Even if needing to hit ntp often, no need to resolve the host every time - client should cache DNS resolution for some time
Each device issues 3 requests for NTP domains in a second every ~8 seconds - this is the only DNS traffic from one of these two switches (showing all DNS queries from this host):
- Copy Link
- Report Inappropriate Content
Hi @daubstep
Thanks for posting in our business forum.
daubstep wrote
30,000 are all NTP? No other domains involved
Yes, all NTP hostname lookups and nothing else (as expected, as these switches shouldn't be originating any other traffic)
There must be at least two things wrong here:
- No need to attempt to hit ntp anywhere near this frequently - time should be stable enough for more than 8 seconds
- Even if needing to hit ntp often, no need to resolve the host every time - client should cache DNS resolution for some time
Each device issues 3 requests for NTP domains in a second every ~8 seconds - this is the only DNS traffic from one of these two switches (showing all DNS queries from this host):
As I am consulting with this team, I can answer you the second "wrong" here.
Regardless of what NTP server you set, as long as it is a domain, it needs to be resolved. The switch or any of the products do not cache the IP address. It queries instead of caching it.
- Copy Link
- Report Inappropriate Content
> As I am consulting with this team
Has the team come up with any additional ideas here? This solved itself, or so it seemed, for almost a week without my doing or changing anything, however, today, after not showing up much at all in stats over the past week, and also after not doing or changing anything, DNS requests are once again totally dominated by one of the two TP-Link Omada switches I have running in this topology (a TL-SG2210MP) all for the same NTP hosts again.
- Copy Link
- Report Inappropriate Content
Hi @daubstep
Thanks for posting in our business forum.
daubstep wrote
> As I am consulting with this team
Has the team come up with any additional ideas here? This solved itself, or so it seemed, for almost a week without my doing or changing anything, however, today, after not showing up much at all in stats over the past week, and also after not doing or changing anything, DNS requests are once again totally dominated by one of the two TP-Link Omada switches I have running in this topology (a TL-SG2210MP) all for the same NTP hosts again.
No reply.
I did a test and I am using the Software Controller. The router nor the switch sends the DNS query for the defined NTP servers.
It is the controller who queries these servers.
I am using the same SG2210MP V4 and firmware.
What would be the controller you use?
Wireshark results? Do the port mirroring as you see these come from the switch.
Mine's under the port mirroring and I captured from the router.
- Copy Link
- Report Inappropriate Content
Information
Helpful: 0
Views: 231
Replies: 7
Voters 0
No one has voted for it yet.