@Piviul, you need to set at least a Group ID and the Mode (Active/Passive) to enable LACP on a port:

 

 

 

 

Since the LAG is created dynamically with LACP, it will show up only after both switches did successfully negotiate the LAG:

 

 

 

 

 

Piviul wrote

the problem was that selecting options do nothing.

 

Probably you had been logged out automatically from the web UI, then AJAX calls have no more effect.

I suggest to increase the session time for HTTP/HTTPS sessions.

 

on the 2 port configured as LAG1 there is linux server with 2 NIC configured in bond as bond-mode 802.3ad; the same for the LAG2. If try to test the speed of the link between the two server (I use iperf) I can test a speedness of 941 Mbits/sec. If I remove both LAG from the switch and I configure the linux bond of both server to bond-mode balance-rr I can test a speed of 1.44 Gbits/sec... I think something is wrong in switch LAG configuration I've done. Do you know where I wrong?

 

I'm not familiar with Linux bonding, but AFAIK LACP requires to set Linux bonding mode 802.3ad (4), not balance-rr (0). What makes me wonder is that balance-rr does work at all. Are you really sure? AFAIK the switch does not support round-robin balancing, at least the predecessor model TL-SG2008 did not support this.

 

I think you have to use mode balance-xor (2) and static LAG on the switch to match the hash policies in Linux and the one set in the switch. In addition, you probably need to adjust the xmit_hash_policy setting on your Linux system accordingly.

 

See https://www.kernel.org/doc/Documentation/networking/bonding.txt

 

Note also that the switch's hash policy SRC MAC XOR DST MAC in T1500G-8T requires to measure total LAG throughput on several systems accessing the server in parallel, since the LAG port to be used is computed using the source and destination MAC of a frame.

 

 

Piviul wrote

I don't think so, I think the web interface is bugged: to make changes I have to apply changes, save, logout and login again (it is not necessary to  poweroff/on the switch) otherwise javascript stop to update the page. 

 

Can't second that. The web UI works fine for me, even with old Safari/Chrome browsers as well as with new ones. Maybe you can try another browser?

 

I have tried balance-xor (2) and static LAG but nothing to do: the speedness is ever 941 Mbits/sec, the same speed reached when only one interface is used, the same speed should be when bonding is not configured.

 

Sure, if you set static LAG in the switch (balance-xor in Linux), it will select the port according to the source and destination MAC address in your global settings. And since both MAC addresses don't change, it will always use the same port when measuring throughput between the same two devices. That's how LAG is supposed to work, it just does link aggregation in this mode, which means that if one link goes down, the backup link is used. You could set source/destination IP to achieve load balancing for different devices resp. packets with different source/destination IPs.

 

If I use bond-mode 802.3ad the default xmit_hash_policy kernel setting is layer2 that should be 802.3ad compliant. If do you think that is not supported by the T1500G-8T switch do you know the xmit_hash_policy values supported by?

 

I never tried LAG with a Linux server, only between two switches and only with source/destination IP hash policy.With those settings it does kind of load balancing for different systems (but it still does not do bandwidth aggregation, you never get more throughput per system compared to using only one switch port).

 

I can't understand... what do you mean saying "requires to measure total LAG throughput on several systems accessing the server in parallel"?!?

 

LAG throughput measured from system A (e.g. a laptop) to system B (the server) will always use only one (always the same) switch port with a source/destination MAC hash policy. See above – the MAC addresses are always the same for both systems A and B. LAG can do load balancing only for more than one system (A1, A2, A3) communicating with system B (or systems B1, B2, B3).

Hi Piviul,

 

Piviul wrote

But I have selected "SRC MAC + DST MAC" as hash algorithm! So I expect that if A open in the same instant two connection to B, A should use one NIC for the first connection and the other for the second one and the switch should use one B NIC for the first connection and the second one for the other connection... didn't you?

 

if I understand correctly, A1 is a laptop/PC connected to a simple access port of the switch when running two iperf instances, right?

Or do you run two iperf instances on two clients connected to the switch?

 

When sending frames from laptop/PC A1 to server B, their MAC addresses are fixed (even if B has two NICs with two different MAC addresses, only one will be used for a given connection. The switch learns A1's (SRC) and B's (DEST) MAC addresses due to adaptive bridging and this one is what goes into the SRC XOR DEST hash when computing which port of the LAG the switch should use.

 

If you would interconnect two Linux systems, they probably would use two SRC/DEST MACs alternately in balance-rr mode, but not if the frames arrive from a switch.

 

At least, this is how I understand switch LAGs from what I read in the switch manual. So, yes, IMO this behavior to just use one port of a LAG at the same time is compliant to the LACP protocol standard.

 

OTOH, when using a second laptop/PC A2 and its hash result determines a different (second, third, fourth) port of a switch LAG, load will be balanced and you should see a higher throughput of the LAG in total with SRC XOR MAC hash_policy in 802.3ad bonding mode in the Linux kernel (»in total« here means the sum of the first and second port's throughput). But if the hash result determines the same port A1 is using, no load balancing happens.

 

As I wrote this is how I understand LAGs, but I might be wrong. I only tested LAG between two TP-Link switches. As for a Linux server I have a multi-homed server serving two networks and Linux bonding would not be of much use for in my network.

 

Thank you very much for the patience and competence you showned me.

 

 

You're welcome. We both learn in this discussion, so thank you for any insight in Linux bonding from your tests. :-) 

 

Piviul wrote

On my switch there are only two PC, so to test multiple connection I have opened on the same client, iperf twice: two connection from the same client to the same server.

 

I know iperf very well, I just didn't know your test setup.

 

Now, if your PC A1 (or A2) resolves the MAC address of the Linux server B using ARP, how many MACs does it get for the server? One MAC. This DEST MAC is used in the distribution algorithm together with your PC's SRC MAC. The XOR result of both MACs determines which link of a LAG to use.

 

802.3ad (which BTW has been transferred to the IEEE 802.1 workgroup in 2008 and hence has been renamed 802.1AX) does neither define nor dictate how the frame distributor of an 802.1AX LAG should behave. This is from the IEEE 802.1AX standard, chapter 5.2.4 Frame Distribution:

 

"This standard does not mandate any particular distribution algorithm(s); however, any distribution algorithm shall ensure that, when frames are received by a Frame Collector as specified in 5.2.3, the algorithm shall not cause

a) Misordering of frames that are part of any given conversation, or
b) Duplication of frames.

The above requirement to maintain frame ordering is met by ensuring that all frames that compose a given conversation are transmitted on a single link in the order that they are generated by the MAC Client; hence, this requirement does not involve the addition (or modification) of any information to the MAC frame, nor any buffering or processing on the part of the corresponding Frame Collector in order to reorder frames."

 

IMO the switch does not need to negotiate a certain frame distribution and as far as I know 802.1AX does not require a LAG to implement load-balancing (in contrast to the Linux-specific balance-rr or even 802.3ad mode). It just needs to ensure that if one link of a LAG drops, another link of the same LAG is used. It does not require to use both links at the same time (and it can't do so, since the switch knows only one single MAC for the server).

 

That's why I did point out that there might be a difference between two Linux servers connected via LACP directly or over a dumb switch and one Linux server talking LACP directly to a managed switch. The switch does not need to implement Linux bonding features; it just has to fulfill the 802.1AX requirements to be standard-compliant.

 

But A1 have two NIC in bond 802.3ad and the switch have a LAG for these NIC so if A1 send two different packages should send the first package using one NIC and the other using the second NIC beacause AFAIK 802.3ad should balance the network traffic so the SRC XOR DEST hash should be different.

 

The switch doesn't care whether the Linux server (or a client A1) has two NICs and supports what Linux calls bonding with extended functionality of load balancing for traffic from the same client device, blinkenlights and additional background music. :-)

 

Of course, a distribution algorithm (SRC MAC XOR DEST MAC or SRC IP XOR DEST IP) can imply load balancing over LACP, but only for different sources (if MACs are used) or different sources/destinations (if IPs are used).

 

The switch just learns one MAC address for the server and this determines the link to use according to the distribution algorithm used by the switch. Linux can offer any other distribution algorithm – such as round-robin distribution –, but needs to supports at least the frame distribution requirements of 802.1AX in 802.3ad mode as a common denominator if connected to a non-Linux device: all frames used in the same communication are required to use the same link. The switch can not send one frame on one link and the next frame on the other link (except if the link determined initially by the algorithm drops: then it selects a backup link).

 

Linux can (and often does) implement additional functions on top of a standard which might be used between two Linux servers, but you can not expect such non-standard features from a switch. Compliance with a standard does not restrict you from implementing things differently, it just means that a device has to fulfill at least the standard to be interoperable. Therefore, Linux and the switch are both compliant to 802.1AX, even if Linux offers more functionality for devices who do so, too.

 

What's more, switch ports are not NICs, since for simple forwarding operations there is no need for routing. The forwarding in a switch is done solely in hardware and the switch's decision which link of a LAG is used for a specific communication depends only on the distribution algorithm, not on Linux having two NICs which could be used in other operation modes.

 

I think that's the reason why Linux calls its link aggregation »bonding« and the links »bonds«. A switch will never inform Linux which bond to use, thus IMO the warning in the log has just informative character about missing functionality not required by the standard. You can check this by connecting two Linux servers directly and by comparing the behavior in 802.3ad bonding mode with the behavior when the Linux server is connected to LAG of a switch.