(posted here because its a controller centric guide, mods - feel free to move) - This is an enhancement of a previous guide of mine, updated with new information

 

Guide:

If you have a complex network and multiple vlans, you can optimise traffic flows with the following benefits

• Remove the router from all intervlan routing
• Have all intervlan routing performed on your L2+ and L3 Switches at line-rate
• Have all internet outbound and return traffic traverse a dedicated vlan
• Have all intervlan traffic traverse a dedicated vlan
• Retain Gateway controlled DHCP services including reservations
• Retain Gateway DNS proxy
• Gateway can still enforce policy routing and bandwidth controls as the IP header of packets remains intact from its actual source network, and isn’t changed while it is routed by the switches

 

Reasons for doing this:

• Higher performance intervlan switching and routing
• Less congestion of interswitch and switch>gateway uplinks
• Less network latency
• Reduce traffic on management vlan that would normally be used for intervlan routing
• Better and more granular security with switch ACLs

 

Prerequisites:

• L2+ / L3 Omada switches
• Omada Controller
• Omada Gateway
• More than 2 vlans serving users

 

 

 

How to configure:

STEP 1

First, set up gateway interfaces for all your vlans.  Assign an IP and enable DHCP as necessary on each one.  No need to enable DHCP on the transit vlan, nothing will live there other than switches with fixed IPs.

In the image below, “Core” is my management vlan, “Data Transit” is the inter-vlan and internet transit vlan, which we will get to later

It is important that you set DHCP to point clients to the switch SVI of each vlan as their gateway, which we will set up later.  You can set DNS to whatever you like.  If you use gateway DNS proxy, you can still enter the gateway IP in the dns fields

 

Router interfaces:

Set the DHCP default gateway to the IP of switch SVIs we will be configuring next:

Suggested:

Create a switch profile that includes all VLANs rather than relying on “All” that you can apply to trunking and uplink switch ports

 

STEP 2

Set up the switch interfaces for each vlan, giving them a static IP in each (remember to set them to what you set the DHCP default gateways to, per interface.  To be double sure of DHCP working properly, its worth setting the DHCP relay to point to the router interface IP for that vlan.  Remember to include the transit vlan and give it an IP.

 

Step 3

Now, set up switch default route 0.0.0.0/0 next hop Gateway Transit Vlan IP

 

Step 4

Set up Gateway default route so switch-routed VLANs get internet traffic returned to them.  The next-hop is set to the data-transit vlan IP of the routing switch.  You need to include all vlans in the “destination” field, it is OK to supernet them into one massive subnet to save entering them individually

 

 

Step 5

Set up switch ACLs to control inter-vlan routing as per your needs

(for anyone curious, the top ACL in my list here effectively “cloaks” the network infrastructure from Pings and Traceroutes on clients inside my “Restriced LAN” groups, by preventing ICMP response from my switches (set in the MAC group by MAC address per switch) – but still allowing traffic to flow)

 

Step 6

Reboot everything so all the switches and gateway are forced to learn the new routing paths

 

Step 7

Monitor for effectiveness.  Here you can see that all traffic is flowing over the Transit vlan.  The rest of the interfaces are showing very small amounts which is DHCP and DNS traffic directly to and from the gateway IPs for each vlan

 

 

Now, any traffic (allowed by the switch ACLs) from one vlan to another, will be directly routed by the switch, and not the gateway.  This will reduce potential congestion on the gateway links and perform much better and lower latency that if they remained routed on the gateway.

 

You can expand this across multiple switches, setting downstream switches who also host clients (acting as gateways) default route to the Transit vlan IP of the next upstream switch, making the transit vlan used for any and all internal routing, keeping those packets off the “real” networks

 

Here I have another switch, also with SVIs for each vlan which WiFi clients use as gateways, with its static route pointing to the upstream switch.