SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!

SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!

SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!
SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!
2024-10-03 20:37:05 - last edited 2024-10-08 03:25:35
Model: SG2005P-PD  
Hardware Version: V1
Firmware Version: 1.0.3 Build 20240807 Rel.60731

I was looking at some DNS stats and noticed one host was by far top of the most DNS lookups list, with more than 3x the next highest host... I was curious and went looking to find out what it was - only to see that it is my SG2005P-PD, doing mltiple DNS lookups of multiple NTP hosts every few seconds - why would it be doing this, and how do I fix it so it stops? Every omada device on the network should have the same NTP config, so it is really odd that this one device is behaving so badly on its own.

 

Small snapshot of multiple DNS requests for multiple NTP hosts every few seconds from 192.168.4.92

Many ntp lookups

 

Confirming that 192.168.4.92 is this Omada managed switch:

 

By far this one switch is dominating DNS lookup, all for these NTP hosts (30% of ALL DNS requests):

 

My Site NTP config only has the single "time-dot-nist-dot-gov" (using "-dot-" to prevent illegal link blocking) host specified, so I don't even know where it is getting the other "ntp1-dot-glb-dot-nist-dot-gov" NTP hostname from... and it shouldn't be looking up either multiple times a second (it shouldn't be attemptig to sync time multiple times a second either)!

  0      
  0      
#1
Options
1 Accepted Solution
Re:SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!-Solution
2024-10-08 03:24:10 - last edited 2024-10-08 03:24:14

Hi @daubstep 

Thanks for posting in our business forum.

It has nothing to do with any of the devices. It is a problem with the domain, your NTP server.

It was first resolved into ntp1. domain, and then ntp1 was resolved into the IP.

So, it is not a problem with the switch or the Adguard you've deployed. Normal behavior.

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update!Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
Recommended Solution
  1  
  1  
#2
Options
7 Reply
Re:SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!-Solution
2024-10-08 03:24:10 - last edited 2024-10-08 03:24:14

Hi @daubstep 

Thanks for posting in our business forum.

It has nothing to do with any of the devices. It is a problem with the domain, your NTP server.

It was first resolved into ntp1. domain, and then ntp1 was resolved into the IP.

So, it is not a problem with the switch or the Adguard you've deployed. Normal behavior.

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update!Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
Recommended Solution
  1  
  1  
#2
Options
Re:SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!
2024-10-17 01:51:53

  @Clive_A But why are omada switches (I removed this one, now two others are showing the same behavior) checking time multiple times per second? Surely this isn't normal behavior... once an hour, once a minute even if being super-aggressive... but I've never seen an ntp setup query to check the time multiple times a second all day long???

 

Current top two DNS calling clients are two of the three omada switches on the network (note: not any of the other omada gear, all of which should be using the SDN ntp config site-wide)  - so there is definitely something wrong here...

 

These two switches are running:

  • SG2008P: 3.20.2 Build 20240920 Rel.36048
  • SG2210MP: 4.20.3 Build 20240920 Rel.36048

 

  0  
  0  
#3
Options
Re:SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!
2024-10-17 02:04:45

Hi @daubstep 

Thanks for posting in our business forum.

daubstep wrote

  @Clive_A But why are omada switches (I removed this one, now two others are showing the same behavior) checking time multiple times per second? Surely this isn't normal behavior... once an hour, once a minute even if being super-aggressive... but I've never seen an ntp setup query to check the time multiple times a second all day long???

 

Current top two DNS calling clients are two of the three omada switches on the network (note: not any of the other omada gear, all of which should be using the SDN ntp config site-wide)  - so there is definitely something wrong here...

 

These two switches are running:

  • SG2008P: 3.20.2 Build 20240920 Rel.36048
  • SG2210MP: 4.20.3 Build 20240920 Rel.36048

 

30,000 are all NTP? No other domains involved?

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update!Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
  0  
  0  
#4
Options
Re:SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!
2024-10-17 02:13:57 - last edited 2024-10-17 02:33:57

  @Clive_A 

30,000 are all NTP? No other domains involved

 

Yes, all NTP hostname lookups and nothing else (as expected, as these switches shouldn't be originating any other traffic)

 

There must be at least two things wrong here:

  1. No need to attempt to hit ntp anywhere near this frequently - time should be stable enough for more than 8 seconds
  2. Even if needing to hit ntp often, no need to resolve the host every time - client should cache DNS resolution for some time

 

Each device issues 3 requests for NTP domains in a second every ~8 seconds - this is the only DNS traffic from one of these two switches (showing all DNS queries from this host):

  0  
  0  
#5
Options
Re:SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!
2024-10-17 08:05:20

Hi @daubstep 

Thanks for posting in our business forum.

daubstep wrote

  @Clive_A 

30,000 are all NTP? No other domains involved

 

Yes, all NTP hostname lookups and nothing else (as expected, as these switches shouldn't be originating any other traffic)

 

There must be at least two things wrong here:

  1. No need to attempt to hit ntp anywhere near this frequently - time should be stable enough for more than 8 seconds
  2. Even if needing to hit ntp often, no need to resolve the host every time - client should cache DNS resolution for some time

 

Each device issues 3 requests for NTP domains in a second every ~8 seconds - this is the only DNS traffic from one of these two switches (showing all DNS queries from this host):

 

As I am consulting with this team, I can answer you the second "wrong" here.

Regardless of what NTP server you set, as long as it is a domain, it needs to be resolved. The switch or any of the products do not cache the IP address. It queries instead of caching it.

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update!Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
  0  
  0  
#6
Options
Re:SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!
2024-10-30 17:05:10

  @Clive_A 

> As I am consulting with this team

 

Has the team come up with any additional ideas here? This solved itself, or so it seemed, for almost a week without my doing or changing anything, however, today, after not showing up much at all in stats over the past week, and also after not doing or changing anything, DNS requests are once again totally dominated by one of the two TP-Link Omada switches I have running in this topology (a TL-SG2210MP) all for the same NTP hosts again.

  0  
  0  
#7
Options
Re:SG2005P-PD Switch is the top (by >3x) DNS querier in the entire network!
2024-10-31 01:44:40

Hi @daubstep 

Thanks for posting in our business forum.

daubstep wrote

  @Clive_A 

> As I am consulting with this team

 

Has the team come up with any additional ideas here? This solved itself, or so it seemed, for almost a week without my doing or changing anything, however, today, after not showing up much at all in stats over the past week, and also after not doing or changing anything, DNS requests are once again totally dominated by one of the two TP-Link Omada switches I have running in this topology (a TL-SG2210MP) all for the same NTP hosts again.

No reply.

 

I did a test and I am using the Software Controller. The router nor the switch sends the DNS query for the defined NTP servers.

It is the controller who queries these servers.

I am using the same SG2210MP V4 and firmware.

 

What would be the controller you use?

Wireshark results? Do the port mirroring as you see these come from the switch.

 

Mine's under the port mirroring and I captured from the router.

 

 

 

 

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update!Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
  1  
  1  
#8
Options