Dear @d0ugmac1 ，

d0ugmac1 wrote

Routing performs perfectly, but i had to had reduce the Branch MTU (Wired Network->WAN->Advanced Settings) to 1400 to get anything more than pings across the link (I just picked a number I knew would make a difference if this was the issue...I'm sure 1460 might be possible too).  This seemed to cause the router to reboot or at least go offline for a painful period of time. I looked for any kind of FW rule that might not permit fragmentation, but I couldn't see any that were active.

When you turned down the MTU was it only dropped once?

If yes, this is normal; as the WAN will reconnect after the MTU is adjusted to the setting and the VPN will naturally disconnect.

 

Now, I also cannot for the life of me make the IPSEC encryption work.  I was able to make this work when HQ was still operating in standalone mode and Branch was under Omada control, but once HQ converted to a controller, I was no longer able to use IPSEC over the L2TP tunnel.  The SA comes up fine every time, but the tunnel no longer establishes...and there are zero error messages to give a hint as to why in the event logs.

At your headquarters, you can try to pre-set up the VPN on the controller with the same settings as standalone and adopt router.

My main purpose is to VLAN a port on my switch and route it back over the L2TP connection to HQ.  This seems quite easy to do, and I've test configured the Routing Policy to do this and the VPN 'WAN' interface shows up like you would expect.  I tried to do the reverse, ie default route a switch port at HQ back over the VPN...however, this was not possible as the Omada SDN doesn't see the 'Server' end as a WAN option.

After you have managed the router with the controller, when you set up the headquarters side as a VPN server, Local Networks can select a certain Interface within the LAN (here you can just select the VLAN you want).

Best Regards!

  @Hank21 

Thanks for your thoughtful reply.

1. Yes, only once.  My point was more around warning users that there will be a period of loss of connectivity before applying.  I was also thinking that if MTU needs to change for the VPN to work as a result of the header payload, then the Controller should know and it should make this part of the reconfiguration when adding a VPN....which is crude...or fix the Path MTU discovery mechanisms employed maybe?

2. That is exactly what I did, but now IPSEC refuses to establish a connection (SA up, no tunnel).  Has anyone been able to make this config work?

3. It's actually the other way around, I have no issue with adding local networks on the server side at HQ.  What I would like to do is create a Wired Network subnet, map it to a port, and then forward all traffic via the VPN instead of the default WAN...this is not given as an option.  Basically change the default route from 0.0.0.0 for this subnet and forward it via the 172.16.1.1 IP assigned to the tunnel endpoint.  I can definitely do this in reverse (ie from Branch to HQ) just not from HQ to Branch.  I suspect this is because of the client/server context...but since branch to branch isn't an option given my NAT situation...I have no alternative.

So I was finally able to encrypt my L2TP connection from Branch->HQ.  I had to reduce the Site MTU's at both ends before the encrypted link would come up, if either side was left at 1500, then only the SA would establish.  However, nowit doesn't reliably pass traffic...back to the drawing board.

Now, there is no reason for me to artificially set my sites MTU's to less than 1,500 as that is supported by my provider.  It seems like there is something wrong internally when creating L2TP tunnels...and it can only be fixed through the external change to the entire site MTU.  From what I could see, the OpenVPN client/server implementation works just fine, however, since it will not operate in 'routing' mode, it's of little use to me.  At the very least the controller should be aware that this setting needs to be changed and prompt the user (along with a warning that the WAN will go down for a minute or so).

We need some kind of Path MTU discovery fix/???

I'm talking to myself here...but it looks like a larger journey.  Full disclosure, my 'branch' is hanging off a Starlink system.  I've just now learned of the following limitations from Marcus in this link https://www.reddit.com/r/Starlink/comments/osq7hh/starlink_ipsec_tunnel_issues/ 

Starlink blocks ESP packets (on ipv4 and ipv6). So if your VPN uses ESP, it's not going to work. It might set up, but no traffic goes through.

Starlink ipv4 is NATed. If you use the Starlink router, it is double NATted. If you connect directly to the user terminal, you still won't have a public ipv4 reachable from the internet.

If you use UDP encapsulation of ESP (port 4500) your tunnels should work.

If you use ipv4, you will only be able to initiate a vpn connection.

If you use ipv6, you can be an initiator or a responder. Keep in mind that while your Starlink ipv6 is globally routable, it is not static and it does change occasionally.

So looks like the ESP blocking was my unexpected first problem.

I'm going to see if I can make IPv6 work...hopefully I don't paint myself into an IP corner....