So as a bit of a cheapskate and with a like of recycling. It is possible to get good storage 2nd hand and cheap and fast…. oh yeah – Good, Cheap, Fast.
They normally say it can’t be done… but I think it can, and it can be done in software rather than hardware.
Here’s my thinking…
Loads of cheap desktops, rammed with largish spinning rust with 1 SSD for write cache and OS. Then clustered together to make one massive fault tolerant file system. No special controllers, just the mobo ones.
Here’s my goals…. I want to be able to saturate a 10Gb NIC with consistent read and writes. I want to use cheap commodity hardware, or used server kit that’s reached the 2nd life market.
You may want to set a static IP and there are two easy ways to do so.
The first way, you can use the GUI. go to the arrows in the top right, and right click or secondary click on this and then select ‘Wireless and Wired Network settings’. From there choose the interface, which is usually eth0 and fill in the IP address, router and DNS server you wish to use.
The second way is to edit a config file. Open a terminal from the bar and type in “nano /etc/dhcpcd.conf” then return. this will open the file in the nano editor within the terminal. Using the down arrow navigate to a commented out section with the example static IP. you’ll need to uncomment the interface line, static IP line, static routers line, and static domain name servers line. Then edit the address to what you need for your setup. If you’re only using IPv4 then remove the IPv6 DNS server. do a sudo reboot to apply.
Once back up you can verify in two ways.
The first way is to look in the GUI as before to see the details are now present.
The second way is to again open a terminal and type “ip address show” to list the IP details for each interface.
void setup() { // initialize serial communication at 9600 bits per second: pinMode(A3, INPUT_PULLUP); Serial.begin(9600); } // the loop routine runs over and over again forever: void loop() { // read the input on analog pin 0: int p = analogRead(A0); int t = analogRead(A1); int r = analogRead(A2); int b = analogRead(A3);
include "Keyboard.h" void setup() { pinMode(A3, INPUT_PULLUP); Keyboard.begin(); } void loop() { int p = analogRead(A0); int t = analogRead(A1); int r = analogRead(A2);
if (digitalRead(A3) == LOW) { //Send an ASCII 'h', Keyboard.write(104); }
if (p < 300) { //Send an ASCII 'j', Keyboard.write(106); } if (p >900) { //Send an ASCII 'l', Keyboard.write(108); }
if (t < 300) { //Send an ASCII 'i', Keyboard.write(105); } if (t >900) { //Send an ASCII 'k', Keyboard.write(107); }
if (r < 300) { //Send an ASCII 'q', Keyboard.write(113); } if (r >900) { //Send an ASCII 'e', Keyboard.write(101); }
I tested these with a switch but as I only had one 10Gb nic so I went ahead and brought 2 more. Once they arrived I put them in the old xw4600 based workstation and gave them a go with iPerf. The one in the top x16 slot got 8.84 gb/s and in the x8 slot below that got 4.33 gb/s.
So why is this? Well firstly lets look at the hardware – the x8 slot is PCIe Gen1…and wired only x4. And then x16 slots are Gen2. So the theory says that the x16 slot should be able to run a 10Gbit nic just fine, in x8 mode.
The 8x slot is actually only x4 electrically, and only Gen1, which means a max of 8Gb/s.
system topology from the manual
PCIe version
Year
Transfers
x1 speed
x4 speed
x8 speed
x16 speed
1
2004
2.5GT/s
2Gb/s
8Gb/s
16Gb/s
32Gb/s
2
2007
5GT/s
4Gb/s
16Gb/s
32Gb/s
64Gb/s
3
2010
8GT/s
7.88Gb/s
31.52Gb/s
63.04Gb/s
126.08Gb/s
4
2017
16GT/s
15.76Gb/s
63.04Gb/s
126.08Gb/s
252.16Gb/s
Here’s the output showing the PCI link status’. Top card is running x8 at 5GT/s so Gen2, and bottom card is running x4 at 2.5GT/s so Gen1.
I can also tell you that in an iPerf loopback the Dell 7010 gets 19.8Gb/s, and the xw4600 gets 14.8Gb/s.
So i’m not 100% sure where the bottleneck is. In subsequent testing I managed to get a 9.22Gb/s rate which I think is probably the limit of transfer given the TCP overheads – i’ll try with Jumbo packets.
With an MTU of 9000 I managed to get bursts of 9.90Gb/s. While running htop on both the dell and the xw4600 I noticed that the puney dual core E6550 at 2.33GHz in the xw4600 was pegged at 100%, While the i3-2120 at 3.3GHz in the dell didn’t go above 60%. I’m pretty sure the bottleneck is in the CPU/PCIe workings, after all using a dual core from 2007 probably wasn’t designed to handle 10Gb networking.