UDP Server Study Part 1

Download Repo Here

So a few weeks ago I secured a new job with a technology company here in Indy. Part of my new role will be re-architecting the backend solution to help improve performance, and eventually migrate the solution to AWS.

I started doing some digging into the best practices for some of the various tasks I’ll be required to complete, one of which will be a UDP server. This server will be accepting ~45million requests a day. That averages out to 520 requests per second, though, I’m assuming the load isnt that evenly spread out. To be safe I wanted to be able to handle 1000 requests / second with the server I develop.

I started tinkering around on my local machine with .NET / C# and quickly realized I needed to make this a little more real worldish. So off to AWS.

I already had a T2.Micro Ubuntu 14.04 Server running so I got to work using that.

Luckily what I had developed compiled and ran with the latest version of Mono

 For the record, I also tried to run with the latest release of .NET Core but it is still too incomplete.

 This site will guide you in getting an app compiling with either Mono or .Net core

Here is the MonoServer

Here is the UdpClient

Note, the client was ran on my Windows 10 machine on my own network.

How Does it Work?
Ok so the Udp client will open a Socket, generate 1000 20 byte messages (of random junk) and then send them one after the other until the message queue is depleted.

The server’s only job is to receive the messages and print out a number of messages it has received.

After you have followed the above tutorial for mono / .net core the following command will execute the server

 dnx -p project.json run

Here is a video of the results:

As you can see the results do vary each time I execute the client. But, overall they are pretty good. Further testing revealed ~95% success rate. Again this is using a T2.Micro instance which is said to have moderate network performance. Using a different AMI with better network performance may produce better results.

I did get a few 100% success rate tests, but unfortunately that was before I started recording :(

Another interesting thing to consider is cost. A T2.Micro instance averages $15.73 / month for me, running 24/7/365. Thats pretty low. Using AWS Route 53 DNS load balancing you could geographically route these requests as well as perform some type of Round Robin with them and still keep the solution on the cheap.

Part 2 of this investigation will be using Node Js

Ok since writing this just a couple days ago, I’ve been doing some research into how I can increase the performance even more. It seems that the packets that weren’t getting accounted for in the Mono app were simply never making it there. Mono just wasnt reading from the buffer fast enough and it was causing errors and packets to be dropped.

I made the following sysctl changes and have increased the efficiency of the app to 99.8% success rate

net.core.rmem_default = 8388608
net.core.rmem_max = 83886080
net.ipv4.tcp_rmem = 4096        87380   6291456
net.ipv4.udp_rmem_min = 4194304

root@ip:~# sysctl -w net.core.rmem_max=83886080
net.core.rmem_max = 83886080
root@ip:~# sysctl -w net.core.rmem_default=8388608
net.core.rmem_default = 8388608
root@ip:~# sysctl -w net.ipv4.udp_rmem_min=4194304



Now read this

Comcast Blocks VPN Traffic

So I’m sitting at a friend’s restaurant today, helping him out with some various business topics. He currently doesnt have an Internet connection, but, thankfully, another tenant in the building has Comcast’s XFinity service, and Comcast... Continue →