I see the same problem to a local server running the home automation system Home Assistant. From outside the network it's really fast but going from local wifi or wired network to the external...