Are LAN0 and LAN1 bridged in hardware?

[doron]

Folks,

I’ve just received my Espresso and am trying to quickly get up to speed.

One basic question re the hardware. I was curious about the hard separation between the WAN port and two LAN ports.
So my question: are the two LAN ports bridged in hardware – so that they are part of the same broadcast domain, regardless of software interface settings – or are they separate interfaces, so that e.g. net filters (such as iptables) apply to traffic between them, each one can be placed into promiscuous mode, etc.?

The design charts show a distinct separation between the soc and the switch; however that in and of itself does not answer the above.

Thanks for any clarification!

[barryf]

I would love to know this also, and to know if there is any way to change it so that the two lanX ports can operate as normal ethernet ports. For my use-case, I need to be able to see all traffic from both lan0 and lan1 for a monitoring application using a pcap style capture on these ports.

Thanks

[deleted]

When I contacted GlobalScale (several months ago) and asked if the WAN and LAN ports were bridged, they replied that they are.

BUT!

It seems it’s possible to configure the Topaz switch by software and that uboot can disable/enable the ports.
As far as I can see on the schematic, each Gbit port is connected individually to the Topaz switch and there’s no real difference on the ports; one may be labelled “WAN” but I’m pretty sure that you in theory could pick any of the ports for “WAN” (even two or all 3).

Thus in order to build a router/firewall, you definitely want to have the WAN and LAN separated (eg. by switching off the WAN in uboot until the router software turns it on).
-It’s not 100% secure, but at least the ports are separated quickly after a reboot and it’s not likely that much traffic will get through during that time.
However, should uboot fail before disabling the WAN, then there would be a serious security problem.

An alternative I’m much interested in, is to add a separate Gbit Ethernet port (for the WAN) via USB3.0. This particular device has been tested with Armbian (see the download page for details on builds it works with).
I’ve also seen a similar interface from ugreen, which includes a USB3.0 hub – just a few dollars extra and using the same IC, so I think it’s a fairly safe bet. I cannot promise that it would work, but my guess is that it’s likely it will.

[barryf]

Very interesting, thanks.

[doron]

Thanks, @pacman.

As far as I can see on the schematic, each Gbit port is connected individually to the Topaz switch and there’s no real difference on the ports; one may be labelled “WAN” but I’m pretty sure that you in theory could pick any of the ports for “WAN” (even two or all 3).

Very interesting. Sounds solid; but then when I boot e.g. Ubuntu on my Espresso, interfaces lan0 and lan1 actually look as lan0@eth0 and lan1@eth0, while interface wan seems to have its own h/w port. It does seem as if the two switch ports dubbed LAN are positioned differently into the CPU bus. Again I don’t know enough about the hardware yet to test, let alone confirm that.

Thus in order to build a router/firewall, you definitely want to have the WAN and LAN separated (eg. by switching off the WAN in uboot until the router software turns it on).

I’ll take that; does this require compiling my own u-boot or is this something that can be passed to u-boot via the preboot environment?

Thank you very much!

[deleted]

while interface wan seems to have its own h/w port

They’re all connected to the Topaz switch.
Eg. The CPU talks to the Topaz switch via a 2.5Gbit line, the Topaz switch talks to 3 x 1Gbit Ethernet ports.
(this means the maximum total speed on your 3 x 1Gbit ports will be 2.5Gbit/sec, not counting any protocol overhead).
“WAN” is really only a name; you should be able to connect all of them to the same switch in order to get a ‘combined’ 2.5Gbit port (eg. using link-aggregation).

I’m pretty sure you can change the name of lan0, lan1 and wan in configuration files. Try something like …
cd /etc; grep -RE ‘\Wwan\W’ * 2>/dev/null
… and see if something shows up (likely inside /etc/network).

I’ll take that; does this require compiling my own u-boot or is this something that can be passed to u-boot via the preboot environment?

As far as I understand what @steveb is saying, you’ll need to change a .dts file only.
Notice that steve links directly to the line that needs to be changed. (line 87, which says ‘port-mask = <0xf>;’)
-So I would not expect it to be necessary to re-compile anything.
I can’t test, because it’ll still be a while before my EspressoBIN boards (which I ordered two days ago), will arrive.

[doron]

@pacman,

Thanks. Again, extremely helpful and informative.

Re changing the dts, it seems like it’s a compiled device tree file (to make a dtb). Need to figure out how to get this into the unit’s NVRAM.

Re searching for the wan/lan port naming, I did just that (I’m not near my Espresso right now) and came up empty. Will look closer.

[deleted]

Thanks. Again, extremely helpful and informative.

No problem; I’m glad I could be of assistance.

Re changing the dts, it seems like it’s a compiled device tree file (to make a dtb).

Just made a search for ‘dts to dtb’ and found this.

Need to figure out how to get this into the unit’s NVRAM.

That I don’t know (yet), but I’m quite convinced you’ll get there fairly soon now. 😉

[barryf]

Hi

The dts gets compiled when you compile the kernel. The dts would be found in the kernel tree at arch/arm64/boot/dts/marvell/armada-3720-community.dts

This file also is where the interface names are set.

As for how to load it – I am pretty sure that the kernel loads the .dtb from /boot/ and it does not require loading into NVRAM specifically.

[deleted]

Thanks @barryf. -This is indeed very useful information for all of us. 🙂

If both the kernel and uboot loads the dtb file, then it should be possible to rebuild it, without rebuilding the kernel or uboot ?
(for instance, if not changing anything but the port mask).

[doron]

exactly – and, specifically, where can you place the modified DTB so that the uboot code sees and uses it? (easy to stash into rootfs for the booted OS – not sure re the uboot)

[deleted]

where can you place the modified DTB so that the uboot code sees and uses it?

This I don’t know, but if I were to try it out, I’d start by attempting to mount /boot and do a …
find /boot -iname “*.dtb”
… if /boot will mount and I get a result from the above, I’d mount /boot as read/write, then move the old .dtb file aside and move the new .dtb file in and rename it to match the original .dtb file’s name.

-But as uboot may be located in SPI-flash, I’m not sure how to go about this; it might be stored like a file system as well.
You’ll probably have to look in /dev, perhaps lsblk is able to list it.

I’m pretty sure someone else in here knows much more about this than I do. 😉

[barryf]

Hi,

As an experiment I edited arch/arm64/boot/dts/marvell/armada-3720-community.dts to change the name of wan to wan1. I then recompiled the kernel and copied the new armada-3720-community.dtb into /boot on the espressobin. Rebooted and the interface is called wan1 now, and works fine.

Just keep in mind that the armada-3720-community.dtb is kernel-specific, so you should follow the instructions on compiling your own kernel and use the armada-3720-community.dtb and Image.

I don’t know what the other settings in the dts do though!

[deleted]

@barryf Thank you for doing the hard work; in addition you’re informing us where the .dtb files should go. Excellent work. 🙂

the “port-mask” under “mvswitch” is a binary bitmap of ports that should be enabled; a one means it’s enabled, a zero means it’s disabled.
The Topaz-switch has a total of 4 ports. According to Peter’s suggestion of setting port-mask to 0x03 in order to enable LAN0 and LAN1, but disable WAN, I believe that the port named ‘WAN’ is bit 2, then LAN0 would likely be bit 0 and LAN1 would likely be bit 1.

-But I do not think that bit 3 is the connection between the Topaz-switch and the CPU. I do believe, though, that it’s the unconnected port on the Topaz-switch.
My guess why this port has not been connected, is that the Topaz-switch is connected via a 2.5Gbit SerDes; that means we’d need 3 ports in order to saturate that 2.5Gbits; adding an extra port would still only give us a maximum of 2.5Gbit.

I do find it a little strange that the flash-interface is defined at the bottom of the .dtb file. If this file is loaded by uboot, which resides in flash; how does uboot know how to load it, because the file system would need to know the flash interface before the file can be loaded, right ? 🙂

… Hmm … Maybe uboot does not reside in SPI-flash, maybe SPI-flash only contains a bootloader, which loads uboot from eMMC/NAND/SATA/USB/SD …

@barryf … Did you mount /boot from SPI-flash or from eMMC/NAND ?

Note: I would have wished for the second SerDes being routed to a 1GbE port or another Topaz switch (or some optional footprints on the PCB) – I don’t know the details why this wasn’t done; it could be due to a pin/mux conflict.

[barryf]

@pacman – I think you tell u-boot where to find this in the very initial setup described here: http://wiki.espressobin.net/tiki-index.php?page=Boot+from+removable+storage+-+Ubuntu#Setting_U-Boot_parameters_2

Specifically when you run:

Marvell>> setenv image_name boot/Image
Marvell>> setenv fdt_name boot/armada-3720-community.dtb

I am running off a microSD card.

[Author]

Posts