Bandwidth, Round Trip Trime (RTT) and Latency


In a world wide distributed computing environment, the large scale in terms of users and transactions spread over the whole planet becomes the main factor for performances.
Cheap hardware availability, strengthening of CPUs and growing sizes of RAM seem still have margins of growth, on another hand, network and disk spindles (we’ll talk about spindles in a future post)  are the worst nightmare for SAs.
You can scale web server front ends, create master and slave databases or federated them with the best hardware layer 7 load balancer, to cache data in a well organized LRU cache farm, but you can’t do too much on physical limitations of mechanics and electromagnetism.

Let’s start with some definition. The bandwidth is the amount of data that you can pump in a OSI layer 1 media (with other companion layers smart management) per unit of time. Latency is the time necessary to physically travel trough the media and depends (not only but essentially) on media length in term of distance that needs to be covered (keeping in mind that until someone doesn’t have better ideas than Mr. Einstein, the upper limit remains light’s speed).
Bandwidth and Latency calculation are left as exercise
I read a nice analogy on W. Richard Stevens‘ TCP/IP Illustrated, Vol. 3: imagine a garden hose. The bandwidth is the volume of water that comes out from the nozzle each second, and the latency is the amount of time it takes the water to get from the faucet to the nozzle. You can grow bandwidth buying a garden hose with a larger section (i.e. you can buy a Gigabit network) but if you want to shorten the latency the only thing you can do is to shorten the hose.
Both latency and bandwidth influence network Round Trip Time (RTT), that can be defined as the one-way time necessary to travel from a network source to a network destination plus the one-way time from the destination back to the source. The two one-way components of RTT should be quite the same in a well designed route.
Really RTT depends on packet size and many other factors also. In real life it is quite impossible that two network elements talk over a plan network. Switches and many level of routers have to be considered. Every network element causes delay especially if packets fragmentation is introduced by different MTUs on the path (in this case cut-through switching cannot be used) and network elements serialization of delay grows the RTT.
So, what we can do to improve the network performance of our application? Some advises follow:
Bandwidth:  It depends on two factors: your ISP offer and the size of your wallet. You can change ISP if he doesn’t have the necessary bandwidth. If you can change the size of your wallet, you are a lucky guy.
Latency: The only effective solutions is to get closer your customers. So, if you are offering services to the whole world, you need strategically distributed data centers housing your applications in different countries and keeping distributed databases aligned. This is a really big fight, and very often reserved to great companies, but it ensures data redundancy and a better uptime of your services also because if a data center fails, business traffic can be switched to nearest working one.
RTT: It can looks a bit strange, but the better that you can do to lower RTT is on the software side. If your services are offered over the Internet, you can’t control the routing path, hops and MTUs. The best you can do is to find a good compromise between database transactions complexity and number of times that client needs to ask information to server. You can have 200 small queries, everyone with its own RTT or just 20 bigger ones with ten times less equal value RTTs. In this case, you need to empower your computational ability and data access management, but you have shifted part of the performance problem from the freeway to your own backyard.
As always, there isn’t the grandma’s perfect recipe to manage the problem. Our business was born complex and now is still a very emotionally disturbed teen that we, as parents, need to manage using knowledge, experience and some madness (just to confuse the enemy).