@Duble08 Thanks for the thorough and thoughtful post.

I want to bring this to @Hank21 and @Clive_A attention as I think it certainly warrants some investigation.

Hi @Duble08 

Thanks for posting in our business forum.

Duble08 wrote

I've been having these issues for about a year now. I've gone through several different firmware versions and even replaced routers.

 

I'm currently running a remote software controller (5.13.22) on my own in-house, self hosted Windows 2019 Server. I'm an IT professional that engineers, designs, and builds networks for homes and businesses. I install SDN products from both TP-Link and Ubiquiti. Lately, due to supply issues (and quality control issues) I've been seeing from Ubiquiti, I have been installing Omada systems. I currently have 28 Omada sites managed by my controller (I had to do a memory adustment through 64 bit Java to be able to acheive this).

 

With all that said, I feel I have some input that others might not, seeing that I'm managing nearly 30 sites rather than one.

 

Moving on - The routers that I have installed on these jobs sites are mostly ER605 V2's, but I also have some V1's. Out of my 28 sites, I only have these issues with 3. One of those sites runs a V1, the other two are V2's.

 

Something to point out about the 3 "trouble" sites, is that they have unstable internet connections at those locations. It seems to me, that when the internet becomes unstable and disconnects "x" amount of times, the router stops communication with the controller for some reason - Maybe it's because it reobtains a different IP from the ISP when it reconnects and is conflicting in some way (I haven't confirmed that this is happening, nor can I find detailed logs to try to verify), or maybe the router just gives up on it's communication after so many failed attempts. Those sites still reconnect to the internet properly, and the AP's reprovision and connect back up, but the router doesn't. By physically rebooting the router (unplugging power and plugging it back in), it becomes seen by my controller again.

 

Something I want to point out, is one of these sites originally had a V1 router. It was the first site I started noticing this issue with. It's firmware was still at the factory firmware (I think it was 1.0.0). All other sites worked, except this one (I think I had about 12-15 sites going at this time). First thing I tried to do is update the firmware. This didn't change anything. I figured since everybody else was working fine, it must've just been a faulty router - so I replaced it with a V2, running on the firmware 2.0.0. Surprisingly, the issue still persisted. So I upgraded the firmware on the router (now it's 2.1.2). Issue still persisted. I upgraded my controller. Issue still persisted. The only issue the client at this site notices is when their internet goes out momentarily (this is an ISP issue and has been verified). They have no clue when it's not talking to the controller, and lack of communication with the controller is not affecting their connectivity (as it shouldn't, since this is SDN).

 

Since installing more of these systems, I now have 3 sites of the 28 that no longer connect. One of them drops off after a couple days, the other two after a few weeks. I tried the reboot schedule on one of the sites, but then the customer started complaining that their internet was down. This would happen the day after it was supposed to reboot - and I still wouldn't see it active on the controller - so I'd have the customer manually power cycle the router to get it back online. I stopped doing the scheduled reboots because it seemed to cause more issues than it resolved.

 

So now, I just deal with the router becoming "disconnected" and only request that the client at the site reboots it if I have to make changes. This keeps them connected, but creates an inconvienience for me, being the one managing the sites. What makes this a really bad thing, is when the routers not connected to the controller, it's not reporting any issues to it, either. So I can't determine what went "weird" on their network if an issue arrises and the router wasn't communicating with my controller at the time.

 

I believe there's something messed up in the routers firmware about reconnection attempts after losing internet connectivity. This really needs to be resolved, as it's been an issue for every firmware version since the first one.

Let me tell you what I know. There was an issue in the firmware back in July and was quickly withdrawn and a fix was provided.

The OP issued this problem back last year. So it has nothing to do with the July firmware issue.

If you wanna dig into when this happened, it would be the first day when Omada SDN was introduced. Ever since the controller was introduced, there have been tons of threads/tickets about this issue.

And I will unequivocally say your issue would result from your network environment. Unless you have ruled them out and checked everything.
Let me tell you what I have experienced with the device, I have set it up next to my office desk and it runs 24/7 and very few log records tell me it is disconnected. Barely not. I leave it there and it only disconnects when I reconnect the Controller PC to a different router to perform some sort of tests. I have a full setup and none of them disconnect unless it is me unplug them or move the controller from the network.

I would not buy your story if you told me you even replaced(RMA) it. Then it is definitely not the hardware issue or software. It gotta be your network issue.

Why not start a new thread and paste what you have done to correct the issue? Issue symptoms and frequency?

It is super easy for layman to change something and cause the network to disconnect. This is why there is an IT department in any type of company. If you are dealing with your customers, and you say you see a lot of disconnection issues, I'd say quite normal. What else would you run into? Generally, as a support guy, you'd see config and disconnection issues all the time. But what steps have you tried to rule out the variables that affect the connections? And what you should tell your customers to do and not to do if they don't understand whether a parameter would cause a disconnection?