Omada controler offline for minutes
Hi,
I have two EAP225 access points and I'm running the Omada Controller as a docker image on my Synology NAS. Every couple of days, the controller and EAP's go offline. I have no WiFi signal for a couple of minutes and then everything comes back online again. I checked that I still have internet access on the wired network during this time.
Below's a snippet of the log during the most recent outage. (Note that it is in reversed order: the most recent log line is at the top.)
Any help in resolving this issue would be highly appreciated!
Best regards,
Bart
2021-01-06 08:03:41,stdout,2021-01-06 09:03:41 [scheduled-pool-2] [INFO]-[SourceFile:80] - Start pushing new connected devices. 2021-01-06 08:03:35,stdout,2021-01-06 09:03:35 [Thread-15] [INFO]-[SourceFile:366] - The result of connection is true. 2021-01-06 08:03:11,stdout,2021-01-06 09:03:11 [discovery-handler-pool-0] [INFO]-[SourceFile:129] - old connection Adopting of 1C-3B-F3-A1-4C-A0 is not timeout 2021-01-06 08:03:11,stdout,2021-01-06 09:03:11 [discovery-handler-pool-1] [INFO]-[SourceFile:129] - old connection Adopting of 1C-3B-F3-A1-4C-A0 is not timeout 2021-01-06 08:03:11,stdout,2021-01-06 09:03:11 [discovery-handler-pool-1] [INFO]-[SourceFile:129] - old connection Adopting of 1C-3B-F3-A1-4C-A0 is not timeout 2021-01-06 08:01:19,stdout,2021-01-06 09:01:19 [net-event-handler-pool-11] [WARN]-[SourceFile:133] - adopt timeout for 74-DA-88-D8-93-D4. 2021-01-06 08:01:19,stdout,2021-01-06 09:01:19 [device-timeout-service-0] [WARN]-[SourceFile:94] - Adopting 74-DA-88-D8-93-D4 time out 2021-01-06 08:00:01,stdout,2021-01-06 09:00:00 [net-event-handler-pool-12] [WARN]-[SourceFile:133] - adopt timeout for 74-DA-88-D8-93-D4. 2021-01-06 08:00:00,stdout,2021-01-06 08:59:59 [device-timeout-service-0] [WARN]-[SourceFile:94] - Adopting 74-DA-88-D8-93-D4 time out 2021-01-06 07:58:09,stdout,2021-01-06 08:58:09 [net-event-handler-pool-4] [WARN]-[SourceFile:133] - adopt timeout for 74-DA-88-D8-93-D4. 2021-01-06 07:58:09,stdout,2021-01-06 08:58:09 [device-timeout-service-0] [WARN]-[SourceFile:94] - Adopting 74-DA-88-D8-93-D4 time out 2021-01-06 07:58:06,stdout,2021-01-06 08:58:06 [Thread-15] [INFO]-[SourceFile:320] - Exception occurs when connecting service server:java.net.UnknownHostException: n-devs-smb.tplinkcloud.com: Temporary failure in name resolution 2021-01-06 07:56:49,stdout,2021-01-06 08:56:49 [net-event-handler-pool-5] [WARN]-[SourceFile:165] - Failed to auto adopt 74-DA-88-D8-93-D4 5 times as timeout 2021-01-06 07:56:49,stdout,2021-01-06 08:56:49 [net-event-handler-pool-5] [WARN]-[SourceFile:133] - adopt timeout for 74-DA-88-D8-93-D4. 2021-01-06 07:56:49,stdout,2021-01-06 08:56:49 [device-timeout-service-0] [WARN]-[SourceFile:94] - Adopting 74-DA-88-D8-93-D4 time out 2021-01-06 07:56:41,stdout, at java.lang.Thread.run(Thread.java:748) [?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_272] 2021-01-06 07:56:41,stdout, at com.tplink.omada.common.b.a.a.run(SourceFile:40) [omada-common-4.1.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.omada.cloud.common.device.task.DeviceStatusPushTask.b(SourceFile:57) [omada-cloud-4.1.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.omada.cloud.common.device.task.DeviceStatusPushTask.f(SourceFile:85) [omada-cloud-4.1.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.omada.cloud.platform.a.b.h.a(SourceFile:144) [omada-cloud-4.1.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.omada.cloud.platform.a.b.h.a(SourceFile:209) [omada-cloud-4.1.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.omada.cloud.platform.b.a(SourceFile:89) ~[omada-cloud-4.1.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.eap.cloudsdk.httpclient.ApiHttpClient.sendPostRequest(SourceFile:204) ~[cloudsdk-1.0.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.eap.cloudsdk.httpclient.ApiHttpClient.sendPostRequest(SourceFile:259) ~[cloudsdk-1.0.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.eap.cloudsdk.httpclient.ApiHttpClient.sendPostRequest(SourceFile:289) ~[cloudsdk-1.0.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.eap.cloudsdk.httpclient.ApiHttpClient.a(SourceFile:325) ~[cloudsdk-1.0.5.jar:?] 2021-01-06 07:56:41,stdout, at com.tplink.eap.cloudsdk.httpclient.a.sendPostRequest(SourceFile:148) ~[cloudsdk-1.0.5.jar:?] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45) ~[httpclient-4.5.7.jar:4.5.7] 2021-01-06 07:56:41,stdout, at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.net.InetAddress.getAllByName0(InetAddress.java:1277) ~[?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324) ~[?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929) ~[?:1.8.0_272] 2021-01-06 07:56:41,stdout, at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) ~[?:1.8.0_272] 2021-01-06 07:56:41,stdout,java.net.UnknownHostException: n-smb-api.tplinkcloud.com: Temporary failure in name resolution 2021-01-06 07:56:41,stdout,2021-01-06 08:56:41 [scheduled-pool-9] [WARN]-[SourceFile:214] - Failed to push message [omadaEAPStatus]. 2021-01-06 07:56:20,stdout,2021-01-06 08:56:20 [scheduled-pool-9] [INFO]-[SourceFile:84] - Start pushing new disconnected devices. 2021-01-06 07:55:29,stdout,2021-01-06 08:55:29 [net-event-handler-pool-6] [WARN]-[SourceFile:133] - adopt timeout for 74-DA-88-D8-93-D4. 2021-01-06 07:55:29,stdout,2021-01-06 08:55:29 [device-timeout-service-0] [WARN]-[SourceFile:94] - Adopting 74-DA-88-D8-93-D4 time out 2021-01-06 07:55:19,stdout,2021-01-06 08:55:19 [net-event-handler-pool-9] [INFO]-[SourceFile:185] - handleConnectedTimeout for 1C-3B-F3-A1-4C-A0 2021-01-06 07:55:19,stdout,2021-01-06 08:55:19 [device-timeout-service-0] [INFO]-[SourceFile:121] - Connected device time out for 1C-3B-F3-A1-4C-A0 2021-01-06 07:55:02,stdout,2021-01-06 08:55:02 [Thread-15] [INFO]-[SourceFile:320] - Exception occurs when connecting service server:java.net.UnknownHostException: n-devs-smb.tplinkcloud.com: Temporary failure in name resolution 2021-01-06 07:53:59,stdout,2021-01-06 08:53:59 [net-event-handler-pool-2] [WARN]-[SourceFile:133] - adopt timeout for 74-DA-88-D8-93-D4. 2021-01-06 07:53:59,stdout,2021-01-06 08:53:59 [device-timeout-service-0] [WARN]-[SourceFile:94] - Adopting 74-DA-88-D8-93-D4 time out 2021-01-06 07:53:19,stdout,2021-01-06 08:53:19 [Thread-15] [INFO]-[SourceFile:320] - Exception occurs when connecting service server:java.net.UnknownHostException: n-devs-smb.tplinkcloud.com: Temporary failure in name resolution 2021-01-06 07:52:29,stdout,2021-01-06 08:52:29 [net-event-handler-pool-7] [WARN]-[SourceFile:133] - adopt timeout for 74-DA-88-D8-93-D4. 2021-01-06 07:52:29,stdout,2021-01-06 08:52:29 [device-timeout-service-0] [WARN]-[SourceFile:94] - Adopting 74-DA-88-D8-93-D4 time out 2021-01-06 07:52:18,stdout,2021-01-06 08:52:18 [Thread-15] [INFO]-[SourceFile:320] - Exception occurs when connecting service server:java.net.UnknownHostException: n-devs-smb.tplinkcloud.com: Temporary failure in name resolution 2021-01-06 07:51:58,stdout,"2021-01-06 08:51:58 [Thread-15] [INFO]-[SourceFile:127] - Connect service server automatically, ConnectionType is PERSISTENT_CONNECTION. " 2021-01-06 07:51:58,stdout,2021-01-06 08:51:58 [Thread-15] [INFO]-[SourceFile:114] - Close connection to SEF server. 2021-01-06 07:51:58,stdout,"2021-01-06 08:51:58 [Thread-15] [INFO]-[SourceFile:194] - The result of connection is false, service host is null, port is 0. " 2021-01-06 07:51:58,stdout,2021-01-06 08:51:58 [Thread-15] [INFO]-[SourceFile:155] - Exception occurs when connecting SEF server:java.net.UnknownHostException: n-deventry-smb.tplinkcloud.com: Temporary failure in name resolution 2021-01-06 07:51:28,stdout,2021-01-06 08:51:28 [Thread-15] [INFO]-[SourceFile:155] - Exception occurs when connecting SEF server:java.net.UnknownHostException: n-deventry-smb.tplinkcloud.com: Temporary failure in name resolution 2021-01-06 07:51:10,stdout,2021-01-06 08:51:10 [net-event-handler-pool-0] [WARN]-[SourceFile:133] - adopt timeout for 74-DA-88-D8-93-D4. 2021-01-06 07:51:09,stdout,2021-01-06 08:51:09 [device-timeout-service-0] [WARN]-[SourceFile:94] - Adopting 74-DA-88-D8-93-D4 time out 2021-01-06 07:51:03,stdout,2021-01-06 08:51:03 [Thread-15] [INFO]-[SourceFile:155] - Exception occurs when connecting SEF server:java.net.NoRouteToHostException: No route to host (Host unreachable) 2021-01-06 07:51:02,stdout,2021-01-06 08:51:02 [Thread-19] [INFO]-[SourceFile:839] - Thread 'recvThread' is stopped 2021-01-06 07:51:02,stdout, ... 3 more 2021-01-06 07:51:02,stdout, at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:815) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.SSLSocketImpl.access$200(SSLSocketImpl.java:72) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1095) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:457) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout,Caused by: java.net.SocketException: Socket closed 2021-01-06 07:51:02,stdout, at java.lang.Thread.run(Thread.java:748) [?:1.8.0_272] 2021-01-06 07:51:02,stdout, at com.tplink.eap.cloudsdk.client.s.run(SourceFile:830) [cloudsdk-1.0.5.jar:?] 2021-01-06 07:51:02,stdout, at com.tplink.eap.cloudsdk.client.p.a(SourceFile:120) ~[cloudsdk-1.0.5.jar:?] 2021-01-06 07:51:02,stdout, at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:831) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.SSLSocketImpl.access$300(SSLSocketImpl.java:72) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1303) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.TransportContext.fatal(TransportContext.java:262) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.TransportContext.fatal(TransportContext.java:267) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.TransportContext.fatal(TransportContext.java:324) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout, at sun.security.ssl.Alert.createSSLException(Alert.java:127) ~[?:1.8.0_272] 2021-01-06 07:51:02,stdout,javax.net.ssl.SSLException: Socket closed 2021-01-06 07:51:02,stdout,2021-01-06 08:51:02 [Thread-19] [INFO]-[SourceFile:837] - javax.net.ssl.SSLException: Socket closed 2021-01-06 07:51:02,stdout,2021-01-06 08:51:02 [Thread-15] [INFO]-[SourceFile:110] - Connect SEF server automatically. 2021-01-06 07:51:02,stdout,2021-01-06 08:51:02 [Thread-15] [INFO]-[SourceFile:146] - Close connection to service server. 2021-01-06 07:51:02,stdout,2021-01-06 08:51:02 [Thread-15] [INFO]-[SourceFile:366] - The result of connection is false. 2021-01-06 07:51:02,stdout,"2021-01-06 08:51:02 [Thread-15] [INFO]-[SourceFile:127] - Connect service server automatically, ConnectionType is PERSISTENT_CONNECTION. " 2021-01-06 07:51:02,stdout,2021-01-06 08:51:02 [Thread-20] [INFO]-[SourceFile:796] - Thread 'expiredRequestCleanThread' is stopped 2021-01-06 07:51:02,stdout,2021-01-06 08:51:02 [Thread-20] [INFO]-[SourceFile:792] - expiredRequestCleanThread is interrupted. 2021-01-06 07:51:02,stdout,2021-01-06 08:51:02 [Thread-15] [INFO]-[SourceFile:146] - Close connection to service server. 2021-01-06 07:51:02,stdout,2021-01-06 08:51:02 [Thread-21] [INFO]-[SourceFile:921] - Thread 'heartBeatThread' is stopped 2021-01-06 07:50:44,stdout,2021-01-06 08:50:44 [device-monitor-timeout-task-0] [INFO]-[SourceFile:263] - Device 1C-3B-F3-A1-4C-A0 status change from Connected to Heartbeat Missed 2021-01-06 07:49:54,stdout,2021-01-06 08:49:54 [discovery-handler-pool-0] [INFO]-[SourceFile:569] - Old connected device 74-DA-88-D8-93-D4 reconnect to controller 2021-01-06 07:49:53,stdout,"2021-01-06 08:49:53 [nioEventLoopGroup-5-1] [INFO]-[SourceFile:225] - 74-DA-88-D8-93-D4 reset to discovery status from CONNECTED, close old connection " 2021-01-06 07:49:04,stdout,2021-01-06 08:49:04 [device-monitor-timeout-task-0] [INFO]-[SourceFile:263] - Device 74-DA-88-D8-93-D4 status change from Connected to Heartbeat Missed 2021-01-06 02:02:40,stdout,2021-01-06 03:02:40 [discovery-handler-pool-1] [INFO]-[SourceFile:569] - Old connected device 74-DA-88-D8-93-D4 reconnect to controller 2021-01-06 02:02:07,stdout,2021-01-06 03:02:07 [discovery-handler-pool-0] [INFO]-[SourceFile:569] - Old connected device 1C-3B-F3-A1-4C-A0 reconnect to controller 2021-01-06 02:01:23,stdout,2021-01-06 03:01:23 [device-monitor-timeout-task-0] [INFO]-[SourceFile:263] - Device 1C-3B-F3-A1-4C-A0 status change from Connected to Heartbeat Missed 2021-01-06 02:01:04,stdout,2021-01-06 03:01:04 [nioEventLoopGroup-7-2] [WARN]-[SourceFile:61] - handle message failed 2021-01-06 02:01:04,stdout,2021-01-06 03:01:04 [nioEventLoopGroup-7-1] [WARN]-[SourceFile:61] - handle message failed 2021-01-06 02:01:04,stdout,2021-01-06 03:01:04 [nioEventLoopGroup-7-2] [WARN]-[SourceFile:98] - Failed to updateManageRebuildDeviceContext for 74-DA-88-D8-93-D4 2021-01-06 02:01:04,stdout,2021-01-06 03:01:04 [nioEventLoopGroup-7-1] [WARN]-[SourceFile:98] - Failed to updateManageRebuildDeviceContext for 1C-3B-F3-A1-4C-A0 2021-01-06 02:01:04,stdout,2021-01-06 03:01:04 [nioEventLoopGroup-7-2] [ERROR]-[SourceFile:189] - Failed to find device context for 74-DA-88-D8-93-D4 after receive REBUILD_REQUEST during manage 2021-01-06 02:01:04,stdout,2021-01-06 03:01:04 [nioEventLoopGroup-7-1] [ERROR]-[SourceFile:189] - Failed to find device context for 1C-3B-F3-A1-4C-A0 after receive REBUILD_REQUEST during manage 2021-01-06 02:01:04,stdout,tail: '/opt/tplink/EAPController/logs/server.log' has appeared; following new file 2021-01-06 02:01:04,stdout,tail: '/opt/tplink/EAPController/logs/server.log' has become inaccessible: No such file or directory