Recently I’ve had to research 10Gbps switches, not being an everyday network person I’d typically just ask Cisco and be done with it, after all as the saying goes no one ever gets fired for buying Cisco but wanted to look into what other vendors have for options. The purpose of the blog post is not to suggest this is the end all be all, but provide some basic research to others and to open conversation as to anything I might be missing, or considerations I need to look at that I’ve not (again I’m not a full time network person).
This particular use case is for a small data center to support new servers using 10Gbps Ethernet for shared storage and networking.
- Each physical server will have no more than two 10Gbps interfaces
- Future growth will likely not exceed 20 total physical servers for a total of 40 required ports.
- The 10Gbps switch will handle L2 10Gbps traffic
- Outbound VM network traffic will be routed to an existing core switch and firewall
- Provide enough ports for future physical host considerations
- Existing equipment with only 1Gbps links will stay connected to existing switches
To support redundancy I would like to have at least two physical switches to connect one each of the two 10Gbps server ports. The expected physical layout would be similar to the image below:
Raw technical specifications were just about equal across the board, most of the switches to support the size of the deployment needed all had roughly the same throughput, PPS and MAC table support. Cost was also fairly equal (note cost was pulled from public reseller sites, you should check with your reseller or VAR on pricing). Here is what I looked at (crowd sourced list via Twitter; thank you Michael Davis, Rob Nelson, Ed Swindelles, Robert Novak):
- Number of ports (and type)
- Available port speed
- Number of uplinks (and type)
- Available uplink speed
- Packets per second
- Port licensing
- Cable compatibility
- First year cost
- SFP cost
- Maintenance cost
A few things to note that aren’t listed here; all switches support LAG groups, the number of groups and group members vary generally between 16 and 32 ports per LAG group. The SuperMicro switch supported 8 per LAG.
Thanks to @bvdlatisit for suggesting the 3524 (and thus 3548 – I’ll check those out and update this post shortly)
|Manufacturer||Model||Size||Redundant PS||# of Ports||Port Type||Port Speed||Uplinks||Uplink Type||Uplink Speed||Throughput||PPS||Latency||Buffers||MAC Table||Warranty||Switch Cost||Twinax 5M Cost||Product Link|
|Cisco||3064X||1U||Yes||48||SFP+||Up to 10Gbps||4||QSFP+||4×10+40Gbps||1.28Tbps||950 mpps||?||9 MB shared||128000||1 Year||18500||280||http://goo.gl/7dUrQG|
|Cisco||3064T||1U||Yes||48||RJ-45||Up to 10Gbps||4||QSFP+||4×10+40Gbps||1.28Tbps||950 mpps||1.41 us||9 MB shared||128000||1 Year||18000||280||http://goo.gl/7dUrQG|
|Arista||7124SX||1U||Yes||24||SFP+||Up to 10Gbps||0||–||–||480Gbps||360 mpps||500 ns||?||?||1 Year||16500||135||http://goo.gl/B34DE7|
|Arista||7050S-64||1U||Yes||52||SFP+||Up to 10Gbps||4||QSFP+||4×10+40Gbps||1.28Tbps||960 mpps||1.2 us||9 MB shared||128000||1 Year||26000||135||http://goo.gl/Pu9U8C|
|Juniper||QFX3500-48SQ-ACRB||1U||Yes||48||SFP+ / SFP||Up to 10Gbps||4||QSFP+||4×10+40Gbps||1.28Tbps||960 mpps||< 1 us||9 MB shared||120000||1 Year||25000||170||http://goo.gl/pqH4Bc|
|Extreme||X670V-48x – VIM4-40G4X||1U||Yes||48||SFP+||Up to 10Gbps||4||QSFP+||4×10+40Gbps||1.28Tbps||952 mpps||< 1 us||9 MB shared||128000||1 Year||21000||?||http://goo.gl/0oSlEn|
|Extreme||X670V-48x||1U||Yes||48||SFP+||Up to 10Gbps||0||–||–||960Gbps||714 mpps||< 1 us||9 MB shared||128000||1 Year||18000||?||http://goo.gl/0oSlEn|
|Supermicro||X3348SR||1U||Yes||48||SFP+||Up to 10Gbps||4||QSFP+||4×10+40Gbps||1.28Tbps||?||?||?||?||1 Year||13365||156||http://goo.gl/bbme0B|
|Dell||N4064||1U||Yes||48||SFP+||Up to 10Gbps||4||QSFP+||4×10+40Gbps||1.28Tbps||952 mpps||?||9 MB shared||?||Lifetime||16825||156||http://goo.gl/XcEFuw|
|HP||5900AF||1U||Yes||52||SFP+||Up to 10Gbps||4||QSFP+||4×10+40Gbps||1.28Tbps||952 mpps||< 1.5 us||9 MB shared||128000||1 Year||24200||?||http://goo.gl/ptXgWQ|
At this point, I seem to have circled back to Cisco, specifically the 3064X which was also suggested by a re-seller I spoke to about pricing. While the SuperMicro was the least expensive, not being a network person I’m not comfortable taking on the support of it directly (if some vendor slapped their logo on it and supported it directly that’s fine) and while Arista seems to be the new hotness, I also don’t have an unlimited budget so comparing features and price, Cisco seems to be it. If you’ve deployed 10Gbps switches, what do you think of this information? What did I miss? And what have your experiences been?
SFP+: Enhanced Small Form-Factor Pluggable
QSFP+: Quad SFP
ns: nanosecond (http://goo.gl/M9LxEM)
us: microsecond (http://goo.gl/M9LxEM)