A few months back I wrote a bit about my unusual home network topology and, in particular, how I’d been planning to modernize it. Though it had worked pretty well for years already, the aim then was to improve it further by moving the firewall to newer, more power-efficient hardware and from pfSense to Vyatta, my favorite network operating system. Well, that’s essentially what happened, but with a slight detour.
In fact, I did migrate to a new Atom D525-based Supermicro X7SPA-HF + 4-port I350, and successfully ditched pfSense in favor of Vyatta 6.6R1. At least, for a short while. But after a couple of days, before I was even finished writing my new policies, I wound up abandoning Vyatta. You see, much like Fredo Corleone in The Godfather Part II, Vyatta broke my heart, and is now dead to me. As it turned out, the community builds of Vyatta hadn’t been updated at all since the company’s acquisition by Brocade. Thankfully, there was a light at the end of the tunnel.
VyOS is the new community fork of Vyatta, the open source routing and security platform based on Linux. Since there doesn’t seem to be any interest on the part of Brocade in maintaining Vyatta’s open source codebase or its community any longer, VyOS has stepped in to pick up the slack.
Some key points:
- The project seems quite active and there have been several releases already.
- New features have been added and longstanding bugs fixed since splitting from Vyatta.
- Features which were previously paid-only in Vyatta have been added back into VyOS
including the webUI. [Edit: apparently this is just a stub for possible future use.]
- There seems to be some direct collaboration from the EdgeOS folks at Ubiquiti, and possibly even some of the Vyatta folks who stayed at Brocade. Therefore VyOS may end up becoming the natural upstream for both projects, though only time will tell.
- Vyatta started out with Quagga for advanced routing, then moved to a proprietary solution (inspiring much controversy in the process). VyOS has reinstated Quagga.
- As of this writing, VyOS is backwards compatible with configs from Vyatta… though that may not be the case forever. Thus Vyatta users currently have a clean upgrade path to VyOS.
The Migration Process
In the future it’s possible that the addition of new features in VyOS will break backwards compatibility. But currently, upgrading from Vyatta to VyOS is dead simple. First you add the VyOS maintainers GPG key:
curl http://vyos.net/so3group_maintainers.key | sudo apt-key add -
Then you can pull down the new system image. For example, if you wanted to add VyOS 1.0.2:
add system image http://mirror.tuxhelp.org/vyos/iso/release/1.0.2/vyos-1.0.2-amd64.iso
Afterwards, you can reboot to your new VyOS system. Easy peasy.
Feature Parity with pfSense
Since I was originally coming to
Vyatta VyOS from the FreeBSD-based pfSense, it was important that some of the more advanced features I was using continued to function as expected. I’m happy to report that VyOS covered all the bases nicely.
I’m using WAN load balancing for all outbound connections across two distinct and asymmetric Internet providers. I never had any problems with WLB in pfSense, and it works just as well in VyOS. With this facility, I was easily able to achieve the following:
- Force wired network traffic out through the faster of the two Internet connections.
- Conversely, set the wireless network to prefer the ADSL link.
- If one of the two should fail, all traffic swings to the healthy link automatically.
There are of course other must-have features like traffic shaping, which keep latency tight even when the WAN pipe in question is totally saturated. And a decent QoS policy makes sure latency-sensitive apps get the bandwidth they need when they need it.
The new VyOS rig performs much better than my old pfSense box. Granted, this due in no small part to newer, faster hardware. But based on the performance bottlenecks inherent in the PF firewall stack, I’d still expect VyOS to outperform pfSense quite handily on the same hardware.
With six gigabit ethernet interfaces, I am able to achieve a total aggregate throughput of 12Gb/s — even with a thorough, zone-based firewall policy in place between each interface. Latency is exceptional too, as traversal of the firewall adds only ~0.08ms on average, even on a low-power Atom CPU that’s now several generations old.
VyOS (and Linux’s firewall connection tracking facility in general) is also much more efficient in terms of memory consumption. While PF consumes about 1k of non-swappable kernel RAM for every state in the table, Netfilter requires less than a third of that at a meager ~300 bytes. This means VyOS can handle roughly 3x the simultaneous firewall states in the same memory footprint as pfSense.
While I really didn’t have any problems per se with my transition, there were a few minor things in VyOS I felt could be better handled. For instance, I’ll acknowledge that the combination of WAN load-balancing and DHCP on the same interface might be an unusual scenario. But unfortunately, that’s exactly what I have to work with. The WLB function in VyOS lacks the capability of monitoring the dynamic gateway for health checks as a variable. Thus the monitor target for the mandatory gateway health check must be updated manually if it changes when the lease is renewed.
In my case both carriers give leases on the order of six months to a year, so the impact of this shortcoming is negligible. Still, I’ve filed a bug (well, more of a feature request really), so we’ll see what comes of it.
While pfSense did an admirable job of protecting and segmenting my network for several years, the migration to VyOS definitely felt like an upgrade. For the cost of an entry-level server (that’s less than the price of an ASA 5505 of yesteryear, which could only firewall 70mb/s at peak), I have wire-speed filtering across six gigabit ethernet interfaces. And being a bit of a network guy anyway, I actually prefer the straightforward CLI interface to the web interface of pfSense — though in fairness, I do find the pfSense web UI more appealing than many of its proprietary competitors.
If you’re interested in a similar solution but you’d prefer a nicely bundled turnkey product (and the accompanying commercial support), I’d strongly recommend looking into Ubituiti’s EdgeMax line of routers. The EdgeMax series runs EdgeOS, which is Ubiquiti’s fork of Vyatta. While it lacks some of the nice features which made it into Vyatta after the EdgeOS fork, such as global state policies, it does have a pleasant looking web UI which makes creating basic policies simple for novice users.
At the low end Ubiquiti offers the SOHO style wall-mount EdgeRouter Lite for under $100 USD, which can filter at wire speed across its three gigabit ethernet ports and handle an equally impressive 1 million packets per second. Or, if you’d rather go rackmount, for around $500 USD their high-end EdgeRouter Pro unit can pass several million PPS, filter 8Gb/s, and sports two gigabit RJ45/SFP combo ports should you require fiber.
Want to try VyOS for yourself? You can get the images here.