Hands-on with the new EdgeRouter Infinity

I was recently able to spend some time with Ubiquiti‘s new EdgeRouter Infinity (ER-8-XG). It’s a nice piece of hardware, especially for the money, with 8 SFP+ ports and an integrated gigabit copper port (ostensibly for management duty). So, I thought I’d write a little about it while it’s still fresh in my mind.

Specs

The ER-8-XG, while not overwhelming, is remarkably well-equipped for the price point (currently < $1,600 USD):

CPU Cavium Octeon III CN7360
Cores 16
Frequency 1800 MHz
RAM 16GB DDR4
Ports 8x SFP+ (1G/10G), 1x 10/100/1000 Ethernet
PSU 2x hot-swappable AC/DC 100W (DC/DC optional)
Console RS-232 (RJ45)
Throughput (1514 byte packets) 80Gbps
PPS (1514 byte packets) 6,500,000
Throughput (64 byte packets) 12Gbps
PPS (64 byte packets) 18,000,000

Launched in mid-2015, the CN7360 is a reasonably powerful 16-core MIPS64 CPU (clocked @1.8GHz in the ER-8-XG) with decent hardware fast-path/offloading features and an impressive TDP envelope.

As for the RAM, it should be more than sufficient for handling millions of simultaneous firewall states, storing large routing tables, etc. with plenty left over to accomodate the Debian-based EdgeOS platform that runs under the hood.

Perhaps the most interesting asset of the ER-8-XG are its eight SFP+ ports. While not mentioned in the official literature (that I could find, anyway), in addition to the archetypal 10G SFP+ optical tranceivers, it does also support less expensive fiber/copper SFP modules as well as 10G DAC. For short runs, such as router-to-router or router-to-switch in the same cabinet, DAC may be a more appealing option for you.

And, of course, redundant PSUs. The EdgeRouter Infinity ships with two AC-to-DC PSUs, though if you’re running a direct DC infrastructure for better efficiency, DC-to-DC PSUs are also available. Both models are fully hot-swappable, as one might expect, but it’s a pleasantly carrier-grade feature for a sub-$2k router nonetheless.

Fast-path support

In conjunction with the Cavium CN7360 that powers it, the ER-8-XG supports the following “offload” fast-path features to accelerate throughput:

General networking Forwarding (+NAT), DPI, VLAN, GRE, PPPOE
ESP encryption algorithms 3des, aes128, aes256
ESP hash algorithms md5, sha1
IKE encryption algorithms (all)
IKE hash algorithms (all)

PHY considerations

The PHY of a network system, short for the physical layer of the OSI model, does exactly what you’d think. It handles the physical aspects of network hardware negotiation, including the speed and duplex of its interfaces. The ER-8-XG is equipped with two PHYs for its 8 SFP+ ports, plus another for its built-in gigabit Ethernet port. The implications of this design are that you have some flexibility on how you use those ports, but with the caveat that you must put some thought into how you’ll be using them. I’ll explain.

If you’re planning to use all of the SFP+ ports for 10G connections, such as 10 Gbit/sec optical links, this won’t be a concern for you at all. They all run at 10000 Mbit/s by default. Likewise if you only want to use these links at 1000 Mbit/s; just hard-code the speed and duplex and you’re off to the races. If, on the other hand, you want to mix media types of different speeds, you’re going to need to do a bit of planning…

  • PHY 1 handles ports eth1-eth4.
  • PHY 2 manages ports eth5-eth8.

So, if you intend to have a handful of 10G connections and some slower, 1G ones (like, say, to handle legacy ISP-provided CPEs), it’s necessary that you group these ports accordingly. For example, your legacy modules might live on PHY 1 (eth1-4) and your faster 10G modules might live on PHY 2 (eth5-8). Should you intend to upgrade 1G modules later, remember that you’re going to need to upgrade all SFP(+) modules within a particular PHY at once, not piecemeal.

User experience

In the case that you’ve had encounters with Ubiquiti’s EdgeRouter products before, the Ininity holds no surprises for you. But if you haven’t, it probably still won’t be a completely alien experience. EdgeOS, the operating system which runs UBNT’s EdgeRouter family, is a fork of Vyatta’s Debian Linux-based platform. In addition to the GNU/Linux userland, this provides a JunOS-like command interpreter. Like JunOS, EdgeOS is tree-based, and offers command completion / prediction via the TAB key. Experienced network professionals will almost certainly prefer this approach to building configurations.

In addition, EdgeOS also features a user-friendly web-based UI with pretty graphs, system health indicators, etc.

Many users have come to expect a webUI, and UBNT’s is fairly nice. Unfortunately I can’t really give much perspective on this, as I typically don’t use it at all.

Possible use cases

I can easily envision the ER-8-XG in a handful of scenarios. Some of these are pretty obvious, such as performing routing duty for a small ISP, BGP aggregation for a mid-enterprise, etc.

Small and medium enterprise

The Infinity also makes a decent firewall, and could certainly be used to augment and securely segment core networking functionality, even for reasonably sized SME. It could be employed, for instance, to inexpensively aggregate and segregate multiple internal 10Gbit/s trunks into firewall zones without posing a bottleneck. A shoestring startup might even use one Infinity to handle both core and edge firewalling duties in one. And frankly, I don’t think it’d have any problems performing both, provided it’s correctly configured.

Home users

Enthusiasts with >1G links to their homes should certainly be giving the ER-8-XG a strong second glance. It’s relatively low power and cooling footprint, along with its modest 1U form factor, would lend itself well to your home rack or cabinet. The only real drawback for running this thing out of your home office or den is the noise factor…

Those three 40mm exhaust fans can be pretty loud, especially under load. In my time with the Infinity, I observed RPMs anywhere from ~3,000 to ~11,000. At the latter speed, they’d be no louder than most anything else you’d expect to find in a typical 19″ rack, but would definitely be disruptive in an otherwise quiet environment. This isn’t necessarily a detractor for the ER-8-XG, but it should definitely be a consideration.

The appeal

The EdgeRouter Infinity is an interesting product in an unusual market position. It’s competing for a slot typically held by the “800lb gorilla” vendors, and at a 10x lower price point.

But even feature or performance parity with the incumbents’ products at a drastically lower cost aren’t the biggest appeal for me, personally. It’s more about the openness of the platform. That is, open in several ways:

  • Open source. EdgeOS is Debian-based, and the source code is freely available. This essentially guarantees a very long functional lifespan.
  • Open architecture. You don’t need to “root” your system to add functionality or change behavior. As the owner, you already have root.
  • Open paradigm. Right down to the terminology, there is nothing particularly proprietary about UBNT products. Some features may be advanced, sure, but they’re still ultimately generalized networking concepts. There’s also no vendor lock-in on support contracts (e.g. “you must have an account to download firmware updates”), etc. Just buy the products and start using them.

Thus I lean heavily towards such open solutions whenever planning networks. In my experience, they’re simply better in the long term.

Questions? Comments? Feel free to leave them below.

One thought on “Hands-on with the new EdgeRouter Infinity

Leave a Reply

Your email address will not be published. Required fields are marked *