In this blog post, we attempt to demystify the topic of network latency. We take a look at the relationship between latency, bandwidth and packet drops and the impact of all this on "the speed of the internet". Next, we examine factors that contribute to latency including propagation, serialization, queuing and switching delays. Finally, we consider the more advanced topic of the relationship between latency and common bandwidth limitation techniques.
While the concept of bandwidth is easily understood by most IT engineers, latency has always been something of a mystery. At first glance, it might seem that it doesn't matter how long it takes for a chunk of data to cross the Atlantic ocean, as long as we have enough bandwidth between the source and destination. What's the big deal if our 100GE link is one kilometer or one thousand kilometers long, it's still 100GE isn't it?
This would be correct if we never experienced any packet loss in the network. Latency impact is all about flow and congestion control (which these days is mainly part of TCP). When one end sends a chunk of data to the other, it usually needs an acknowledgment that the data has been received by the other end. If no acknowledgment is received, the source end must resend the data. The greater the latency, the longer it takes to detect a data loss.
In fact, packet drops are part of the very nature of IP networks. Packets may be dropped even in a perfectly-operating network that has no issues with congestion or anything else.
Latency and the Unknown
We can regard network latency as a buffer for the data "in fly" between the source and the destination, where packets could possibly disappear. The greater the latency, the larger this buffer is, and the more data is in fly. But being in fly, the data is like Schrodinger's cat: neither sender nor receiver know if it's safely traveling towards its destination or already dropped somewhere along the way. The bigger the amount of data in this unknown state, the longer it takes to detect a loss event and recover from it. Consequently, the lower the effective bandwidth between the source and destination.
The maximum theoretical bandwidth that can be achieved is determined as follows:
As the Round Trip Time (RTT) can not be changed for a given source/destination pair, if we want to increase the available bandwidth we have to increase the amount of data in fly. Technically speaking, to achieve a given bandwidth over a route with a given RTT (Delay), the TCP transmission window (amount of data in fly) must be equal to the so-called Bandwidth-Delay Product (BDP):
So the bandwidth in packet networks comes at the cost of entropy (the amount of data in the unknown state being transported between the source and the destination) and is a linear function of network latency.
As noted above, packet drops are normal on the Internet due to the nature of packet networks. Modern, sophisticated congestion control schemes can increase the amount of data in fly (TCP transmission window) to large values quickly, in order to reach maximum bandwidth. However, when a drop is experienced, the time needed to detect that loss is still a function of latency.
Finally, all this translates into "the speed of the Internet". We can see the difference by downloading a 1GB file from a test server in Paris and then the same file in a version with 90 ms of additional simulated latency:
(You will need a good internet connection to carry out this test)
Latency User Experience Guidelines
The following guidelines can be used to roughly estimate the impact of latency:
little or no impact on user experience
still OK but noticeable for certain applications (gaming etc.)
mostly acceptable, but users do start to feel it: websites a little slower, downloads not fast enough, etc
user feels typically that "the Internet is slow".
"it works", but is not acceptable for most commercial applications these days
These numbers are subjective and may vary for different applications and user groups. They also depend on the user's habitual location. For example, people living in New Zealand or North-East Russia are somewhat used to higher latency for the majority of internet resources. They can tolerate "slower Internet", while US West Coast users who are more used to "fast internet" are often unhappy with the RTT to the European resources.
Where Latency Comes From
In modern networks, the primary source of latency is distance. This factor is also called propagation delay. The speed of light in a fiber is roughly 200,000 km per second, which gives us 5 ms per 1000 km single-direction and the mnemonic rule of 1 ms of round-trip time per 100 km.
However, fibers rarely follow as-the-crow-flies lines on the map, so the true distance is not always easy to estimate. While the routes of submarine cables are more or less straightforward, metro fiber paths in highly urbanized areas are anything but. And so, real-world RTT values can be roughly determined by coupling the above rule with the following considerations:
- For multi-thousand kilometers long, transcontinental routes: as-the-crow-flies distance multiplied by a factor of 1.5
- For shorter, terrestrial urban routes: as-the-crow-flies distance multiplied by a factor of 2.
See the examples below.
5,800 km, 58×1.5 = 87 ms of RTT
PING 22.214.171.124 (126.96.36.199) 56(84) bytes of data.
64 bytes from 188.8.131.52: icmp_seq=1 ttl=52 time=84.0 ms
64 bytes from 184.108.40.206: icmp_seq=2 ttl=52 time=83.9 ms
64 bytes from 220.127.116.11: icmp_seq=3 ttl=52 time=84.0 ms
--- 18.104.22.168 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 83.992/84.015/84.040/0.237 ms
11,000 km, 110×1.5 = 165 ms
par1-instance$ ping 22.214.171.124 -c 3
PING 126.96.36.199 (188.8.131.52) 56(84) bytes of data.
64 bytes from 184.108.40.206: icmp_seq=1 ttl=49 time=158 ms
64 bytes from 220.127.116.11: icmp_seq=2 ttl=49 time=158 ms
64 bytes from 18.104.22.168: icmp_seq=3 ttl=49 time=158 ms
--- 22.214.171.124 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 158.845/158.912/158.991/0.330 ms
2,500 km, 25×2 = 50 ms of RTT
par1-instance$ ping www.nic.ru -c 3
PING www.nic.ru (126.96.36.199) 56(84) bytes of data.
64 bytes from www.nic.ru (188.8.131.52): icmp_seq=1 ttl=52 time=50.5 ms
64 bytes from www.nic.ru (184.108.40.206): icmp_seq=2 ttl=52 time=50.0 ms
64 bytes from www.nic.ru (220.127.116.11): icmp_seq=3 ttl=52 time=50.3 ms
--- www.nic.ru ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 50.041/50.314/50.522/0.328 ms
450 km, 4.5×2 = 9 ms of RTT
par1-instance$ ping 18.104.22.168 -c 3
PING 22.214.171.124 (126.96.36.199) 56(84) bytes of data.
64 bytes from 188.8.131.52: icmp_seq=1 ttl=52 time=9.58 ms
64 bytes from 184.108.40.206: icmp_seq=2 ttl=52 time=10.2 ms
64 bytes from 220.127.116.11: icmp_seq=3 ttl=52 time=9.71 ms
--- 18.104.22.168 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 9.589/9.852/10.250/0.308 ms
Other Sources of Latency
While distance is the main source of latency, other factors can also delay data propagation within modern networks.
The second most crucial latency factor is the so-called serialization delay. Some time ago it was actually competing with distance for first prize, but nowadays is becoming less significant.
Both end-point and intermediate devices (routers, switches) are ultimately just computers, which store data chunks in memory before sending them to the transmission media (e.g. optical fiber, copper wires, or radio). In order to send these packets to the network interfaces, computers need to serialize them, i.e. encode the data bits into a sequence of electromagnetic signals, suitable for transmission over the interface media at a constant rate. The time the data spends in a buffer before it gets sent down the wire is called serialization delay.
Let's imagine that we have a 1500 byte-long Ethernet frame which needs to be sent down a 1GE interface:
12 µs (microseconds) are needed to put a 1500-byte frame onto the wire using a 1GE interface. For 1GB of data the time will be around 8 seconds in total. This is the time the data spends in buffers in order to get signaled to the transmission media. The higher the interface bandwidth, the lower the serialization delay.
The opposite is true, as well. For lower-rate interfaces like xDSL, WiFi, and others, the serialization delay will be higher and so this factor becomes more significant.
The astute reader might notice that a 48-byte ICMP Echo Request packet (plus the Ethernet overhead) should take much less time to get serialized than a 1500-byte frame. This is correct, however, usually, each link is used for multiple data transfers "at the same time", so a ping via an "empty" ADSL or WiFi link will give very different RTT values than a ping in parallel with a large file download.
This phenomenon is known as queuing delay. At the time of low-speed interfaces, it would severely affect small packets used by latency-sensitive applications like VoIP, transmitted along with large packets carrying file transfers. When a large packet is being transferred, smaller packets behind are delayed in the queue.
Typically, today's home and mobile internet users experience somewhere from 5 to 10 milliseconds of latency added by their home WiFi and last-mile connection, depending on the particular technology.
Serialization and queuing delays can be compared with what people experience when getting onto an escalator. The escalator is moving at a constant speed, which can be compared with the interface bit-rate, and the crowd just behind it is the buffer. You can notice that even if there is nobody around, when a group of 3 or 4 people comes to the escalator they need to wait for each other to get onto it. A similar phenomenon can be observed on highway exits where vehicles must slow down. During busy hours this creates traffic jams.
Sticking with this analogy, we can imagine that you want to drive from your home to some place 1000 km away. The following factors will contribute to the total time it takes you to make this trip:
- Distance-related propagation delay is added because you can't drive 1000 km in much less than half a day, no matter how expensive your car is (you can't overcome the speed of light in a fiber).
- Serialization delay is added when you drive through urban areas, with intersections, traffic lights and one-way lanes (low bit-rate access interfaces).
- Queuing delay is added when you get stuck in traffic jams (interface buffers).
This one is something of a false latency factor in real-life computer networking. Once a router receives a packet, a forwarding decision must be taken in the router data-plane in order to choose the next interface to which the packet should be sent. The router extracts bits from the packet header and uses them as a key to perform the forwarding lookup. In fact, modern packet switching chipsets perform this lookup within sub-microsecond times. Most modern switches and routers operate at a near wire rate, which means that they can perform as many forwarding lookups per second as the number of packets they can receive on the interfaces. There might be special cases and exceptions, but generally when a router cannot perform enough lookups to make a forwarding decision for all arriving packets, those packets are dropped rather than delayed. So in practical applications, the switching delay is negligible as it is several orders of magnitude lower than the propagation delay.
Nevertheless, some vendors still manage to position their products as "low-latency", specifically highlighting switching delay.
Latency and Bandwidth Limitation Techniques
Traditionally in residential broadband (home internet) and B2B access networks, the overall bandwidth per subscriber was determined by the access interface bit-rate (ADSL, DOCSIS, etc). However, Ethernet is now becoming more and more popular. In some countries where the legacy last-mile infrastructure was less developed, and aerial links were not prohibited in communities, copper Ethernet became the primary access technology used by broadband ISPs. In datacenter networking we also use 1GE, 10GE, 25GE, and higher rate Ethernet as access interfaces for servers. This creates a problem of artificial bandwidth limitation, where bandwidth must be limited to that which is stated in the service agreement.
Two main techniques exist to achieve this goal.
Traffic shaping technique is based on the so-called leaky bucket algorithm, which delays packets in a buffer in order to force flow-control mechanisms to decrease the amount of data in fly and therefore reduce the effective bandwidth. When the buffer gets full, the packets are dropped, forcing TCP to slow down further.
As such, this technique effectively reduces bandwidth by increasing the queuing delay, which consequently increases end-to-end latency. Residential broadband Internet subscribers or B2B VPN customers can often observe an additional delay in the order of several milliseconds which is added by their ISP's broadband access router in order to enforce service agreement bandwidth rate.
This technique is considered more straightforward than the previous one, and is cheaper to implement in hardware as it doesn't require any additional buffer memory. Policing is based on the so-called token bucket algorithm and employs the notion of a burst. When the amount of transferred data bypasses a certain configured limit, packets get dropped, and the burst counter is incremented at constant rate, corresponding to the desired bandwidth.
Choosing the right value for the policer burst size has always been a mystery for network engineers, leading to many misconfigurations and mistakes. While many guides still recommend setting the burst size as the amount of data, transferred in 5 or 10 ms at the outgoing interface rate, this recommendation is not applicable for modern high-speed interfaces like 10/100GE and higher. This is because it is based on the hypothesis that the serialization delay is the most important factor of the overall end-to-end latency. As discussed above, this hypothesis doesn't hold nowadays, as the interface rates have risen dramatically during the last decade, meaning that the most important latency factor now is propagation delay.
So if we want a customer to be able to communicate with internet resources, let's say, 200 ms away, the policer burst size must be equal or greater than the amount of data, transferred at the desired rate in 200 ms. If we want to limit the customer's bandwidth to 1 Gbps we should calculate the burst-size as follows:
Setting the burst size at less than this amount will impact communication with Internet resources hosted far away from customers. This is a known recurrent problem in some regions which are rather remote from major internet hubs: if an ISP somewhere in Kamchatka or South America sets a burst size which doesn't accommodate those 200-250 milliseconds of traffic, their customers have trouble communicating with resources hosted in Frankfurt or Seattle. The same applies to hosting providers: lower policer burst sizes applied to these servers will make their services unusable for end-users on remote continents.
- Latency and bandwidth are closely connected in modern IP networks.
- End-to-end latency is determined by several factors, the most important of which is propagation delay, in turn determined by light speed.
- In the modern Internet, most source/destination pairs are within 200-250 ms.
- RTT of up to 100 ms is acceptable from the customer experience point of view, <60ms is considered "good" and <30 ms perfect.
Other articles about:
For example, when downloading data files, a 10% packet loss might add only one second to a ten second download. If packet loss rate is higher, or there is high latency, it can cause delays to be worse as some of those packets are not reaching their destinationation.How does network latency affect network speeds? ›
Latency can affect almost everything about your internet user experience. For example, with higher latency, even high speed internet site load times can be slower, web page loads take longer and you'll notice delays because of slower round trip times between your internet device and other servers.What does network latency affect? ›
Latency impact is all about flow and congestion control (which these days is mainly part of TCP). When one end sends a chunk of data to the other, it usually needs an acknowledgment that the data has been received by the other end. If no acknowledgment is received, the source end must resend the data.What is the relationship between latency and bandwidth? ›
Bandwidth is the amount of data and latency is how fast the data travels. So, no: bandwidth does not affect ping or download speed. It just may affect how fast things feel. High bandwidth + low latency = greater throughput.Does bandwidth affect packet loss? ›
Bandwidth congestion — when your network can't handle the high volume of current traffic — can lead to packet loss.How does packet loss affect network? ›
The effects of packet loss are felt as network performance drops and include the following: Lack of communication quality. Real-time applications, namely voice over IP (VoIP), experience jitter and frequent gaps in received speech. Packet losses less than 2% may be noticeable.What affects bandwidth and latency? ›
Latency is affected by the type of connection, distance between the user and the server and the width of bandwidth. The Internet connection is impacted by the type of service you use to access the Internet.What is the difference between latency and bandwidth? ›
Latency refers to the amount of time a data packet takes to travel from one point to another, AKA the delay between the time data is sent and received, measured in milliseconds (ms). Whereas bandwidth refers to the volume of data sent, latency refers to the speed at which it's transmitted.What affects network speed? ›
Multiple factors affect the speed and quality of your internet connection. Transfer technology, your location, the number of people you share the connection with and the device you use are only some of these factors. There are also differences between a fixed network and a mobile network.What happens when latency is high? ›
High latency decreases communication bandwidth, and can be temporary or permanent, depending on the source of the delays. Latency is measured in milliseconds, or during speed tests, it's referred to as a ping rate. The lower the ping rate the better the performance.
Distance is usually the main cause of latency—in this case, it refers to the distance between your computer and the servers your computer is requesting information from.How do I reduce network latency? ›
- Connect Devices to Ethernet Cables Where Possible. ...
- Check the Ethernet Cables. ...
- Move the Router to a Better Location. ...
- Declutter the Router's Immediate Area. ...
- Change the Router's Channel.
High bandwidth and low latency translate to the best speeds and the fastest response times—that's what you want for your internet connection. Low bandwidth and high latency mean slow downloads, choppy streams, and delayed responses.How can I increase my bandwidth and latency? ›
- Check Your Internet Speed and Bandwidth. ...
- Aim for Low Latency. ...
- Move Closer to Your Router. ...
- Close Any Background Websites and Programs. ...
- Connect Your Device to Your Router via an Ethernet Cable. ...
- Play on a Local Server. ...
- Restart Your Router. ...
- Replace Your Router.
Latency is the literal time it takes for a packet of data to go from its origination and reach its destination. The measurement of latency is measured in milliseconds. Just 50 milliseconds of latency — a time of less than one-tenth of a second — can result in poor network and application performance.What happens when bandwidth is low? ›
What is low bandwidth? Low bandwidth means that a limited amount of data can be transmitted over an Internet connection in a given period. It results in sluggish Internet speeds, making performing tasks such as streaming videos or downloading large files difficult.What is the cause of packet loss? ›
There are many potential causes of packet loss: A faulty ethernet port or cable. Issues with the network interface controller (NIC) in your PC. Outdated router firmware.Why does bandwidth drop? ›
If you find that your Wi-Fi speeds drop periodically or at certain times of day, then it's possible that software or other devices are eating away at your bandwidth. Large downloads, video streaming, or online gaming can be bandwidth hogs, so check for these first.What causes packet loss and high latency? ›
Causes of packet loss
Network Congestion: One of the primary reasons for packet loss and implies a situation where network traffic is at its peak. With excessive traffic, it becomes critical for each packet to wait until its delivery.
Network latency is a measurement of round trip time (RTT) for a packet of data, or the number of milliseconds it takes a packet to travel to a destination (server) and back again. When packets of data fail to reach their destination, your network is experiencing packet loss.
Retransmission and packet loss impact throughput on TCP.
Packet loss will have two effects on the speed of transmission of data: Packets have to be retransmitted (even if only the acknowledgment packet got lost and the packets got delivered) The TCP congestion window size will not permit an optimal throughput.
Low latency or lag in your home network is essential to gaming. Low latency means there is a strong, reliable network connection, which reduces the chance for a connection loss or delay.How much network latency is too much? ›
Generally, an acceptable latency (or ping) is anywhere around 40 – 60 milliseconds (ms) or lower, while a speed of over 100ms will usually mean a noticeable lag in gaming.What is the difference between speed and bandwidth? ›
Speed refers to the maximum rate you can transmit data, typically measured as megabits per second (Mbps). Bandwidth refers to the maximum amount of data your connection can handle at any moment, also measured as Mbps (and increasingly Gbps, for gigabyte connections).What is latency speed? ›
To be specific, latency (or ping) is the measure of how long it takes (in milliseconds) for one data packet to travel from your device to a destination and back. Latency is one of the primary indicators of network performance quality.What are the 3 factors that affect network? ›
The performance of a network can be affected by various factors: the number of devices on the network. the bandwidth of the transmission medium. the type of network traffic.What are 3 factors that can slow down a network? ›
- Network Latency. Network latency refers to the time needed to send a packet from the source to the destination. ...
- Network Congestion. ...
- Infrastructure Parameters (QoS, Filtering, Routing) ...
- Client or Server Health.
Your bandwidth is mostly determined by your device, your router, your Internet Service Provider (ISP), and the bandwidth they promised you.What are the 3 types of latency? ›
A computer system can experience many different latencies, such as disk latency, fiber-optic latency, and operational latency.What is the main cause of latency? ›
The main cause of latency is distance. The longer the distance between the browser making the request and the server responding to that request, the more time it'll take the requested data to travel there and back.
Examples of response latency include the time between a doctor giving a shot to a baby and the baby crying, the time between a mom ringing a bell and the family coming to dinner, and the time between an alarm clock ringing and a sleepy man hitting the snooze button.What is an example of a network with high bandwidth and high latency? ›
Data tapes sent by courier are one example of a high bandwidth and high latency method of transporting data.What is the best latency for network? ›
Latency is the amount of time a message takes to traverse a computer network. It is typically measured in milliseconds. Any latency below 100 milliseconds (ms) is considered good, and below 50 ms is very good.What is an example of bandwidth? ›
Bandwidth specifically refers to the capacity at which a network can transmit data. For example, if the bandwidth of a network is 40 Mbps, it implies that the network cannot transmit data faster than 40 Mbps in any given case.Does high bandwidth mean high speed? ›
Just as more water flows through a wide river than a small, narrow creek, a high bandwidth network generally can deliver more information than a low bandwidth network given the same amount of a time. Because this can make the network feel faster, high bandwidth networks and connections often are called "high-speed".How do I check my network latency? ›
Network latency can be measured in two ways: roundtrip time (RTT) or time to first byte (TTFB). RTT measures the amount of time it takes for a data packet to make a successful journey from source to destination and back again.Which is more important speed or latency? ›
Latency is most important for online gaming, while speed is most important for streaming. Both are useful for browsing and video chatting. To improve your latency (lower is better), use closer servers, get fiber-optic or cable internet, use a wired internet connection and get an up-to-date router.Does packet loss cause slow internet? ›
Like high ping, packet loss can cause lag spikes and in-game issues like rubberbanding. However, packet loss can occur even when you have a fast internet connection. Basic troubleshooting steps include power cycling your router, switching from Wi-Fi to ethernet, and changing ethernet ports.What is latency and packet loss? ›
Network latency is a measurement of round trip time (RTT) for a packet of data, or the number of milliseconds it takes a packet to travel to a destination (server) and back again. When packets of data fail to reach their destination, your network is experiencing packet loss.What causes high latency and packet loss? ›
In the case of “High Packet Loss and High Latency”, some of those packets are not successfully reaching their destination, which may be responsible for (or will at least contribute to) an increased latency. In typical packet loss scenarios, the cause of lost packets is due to network congestion.
In general, a packet loss rate of 1 to 2.5 percent is seen as acceptable. Packet loss rates are generally higher with WiFi networks than with wired systems.What happens during packet loss? ›
For users, packet loss manifests itself in the form of network disruption, slow service and even total loss of network connectivity. Any application can be disrupted by packet loss, but the most likely victims are applications that rely on real-time packet processing, such as video, audio and gaming programs.What is the packet loss rate of a network? ›
The reliability of a communication network path is expressed by the packet loss rate. This metric is equal to the number of packets not received divided by the total number of packets sent.What is typical packet loss on internet? ›
Acceptable packet loss
Losses between 5% and 10% of the total packet stream will affect the quality significantly." Another described less than 1% packet loss as "good" for streaming audio or video, and 1–2.5% as "acceptable".
The delay of a packet is calculated by adding the following four components: propagation delay, transmission delay, queuing delay, and processing delay.How do you test latency and packet loss? ›
- Open Command Prompt/Windows PowerShell (either will work) Right click the windows icon in the bottom-left corner. ...
- Enter the command as follows: ping -n 100 22.214.171.124. ping = The command to ping, leave unchanged. ...
- Check the summary when completed for percentage of packet loss.
One of the principal causes of network latency is distance, specifically the distance between client devices making requests and the servers responding to those requests.How do I fix high network latency? ›
- Don't download more than one item at a time. ...
- Avoid running too many applications simultaneously. ...
- Scan for viruses. ...
- Wire your network connection to support lower latency. ...
- Don't delete your cache.
Latency is measured in milliseconds, or during speed tests, it's referred to as a ping rate. The lower the ping rate the better the performance. A ping rate of less than 100ms is considered acceptable but for optimal performance, latency in the range of 30-40ms is desirable.