The ultimate speed test test 2020

TL;DR

The best free online speed tests in 2020 are:

1. Google Fiber (speed.googlefiber.net), best overall score
2. Xfinity xFi Speed Test (speedtest.xfinity.com), most accurate
3. Internet Speed at a Glance (speedtest.zomdir.com), quickest insight

Other fine speed tests are:

1. Fireprobe ®(www.fireprobe.net)
2. Measurement Lab (speed.measurementlab.net)
3. Ookla Speedtest (www.speedtest.net)

The advice above is based on a comprehensive test as described below.

Contents

1. Introduction
2. Declaration of Interest
3. What is the best?
4. Method of measurement
5. Speed tests to test
6. The results
7. Conclusions
8. Other findings

Introduction

An internet speed test is often used to check if there are any problems with the internet connection.

If your internet is to slow it is difficult to watch a movie or play a game. For example, Nexflix recommends a download speed of 5 Mbps for HD quality TV shows and movies. When Stadia existed, Google recommended a download speed of at least 10 Mbps to stream games on Stadia.

When you contact your Internet Service Provider (ISP) about your Internet issues, your ISP will likely ask you to test your Internet speed with their own speed test. This speed test is almost always based on Ookla's speedtest.

In addition to the Ookla Speedtest, another commonly used speed test is Measurementslab's speed test. When googling speedtest, Google gives you the option to immediately start a speed test based on the speed test from Measurementslab.

However, more independent speed tests are available to measure your internet speed in bits per second.

This ultimate speed test must make clear which speed test is the best speed test.

Declaration of Interest

In the past I (Hans van der Graaf) programmed my own speed test Internet Speed at a Glance. This because I was often not interested in the exact internet speed. Usually I want a quick indication if there are any problems with the internet connection.

In this speed test test I will treat Internet Speed at a Glance like all other speed tests being tested.

What is the best?

To find the best speed test, you need to know in advance, "What is the best?".

BroadbandSearch lists the following requirements for a speed test:

1. Ease of use
2. Accuracy
3. Commitment to real-world testing conditions
4. A simple design
5. A wide server range
6. Freedom from bias

Lifewire states that: The best internet speed test would be one between you and any given website you're using, but these should give a general idea of the kind of bandwidth you have available.

For this speed test test, it is assumed that the best speed test has the following characteristics:

Method of measurement

The following measurement method is used for this ultimate speed test:

Free to use

When the speed test is free to use and there are no ads, the speed test earns 1 point. If the speed test is free, but there are ads, the speed test does not earn points. If the speed test cannot be used for free, it will be excluded from this test.

Easy to use

The speed test must work in:

1. Edge on Windows
2. Chrome on Windows
3. Chrome on Android
4. Safari on iOS
5. Firefox on Linux

If not, the speed test is excluded from this test.

The speed test must be mobile-friendly according to Google (Gone) and Bing. If not, the speed test is excluded from this test.

The accesibility of the speed test is tested with AChecker (unfortunately this excellent tool is gone). If the speed test passes the automatic WCAG 2.0 (level A) test, the speed test earns 1 point.

When the speed test could be started with a single click (or no click at all), the speed test earns 1 point. If 3 or more clicks are required, the speed test is excluded from the list.

Accuracy and consistency

Each speed test is tested at least 5 times with a clean browser on a wired desktop.

Speedtest.net concluded in June 2020 that the global average download speed is 35 Mbps.

Browsers used to measure the actual download speed are Firefox and Chromium. The maximum limitation that Firefox supports is 30Mbps. Therefore, the speed during the test is limited to 30 Mbps.

The Internet service provider used during this speed test test indicates that it offers a download speed of 50 Mbps. It can safely be assumed that testing with 30 Mbps leaves enough space for internet use by other devices without disturbing the test results.

The average of the measured download speed and the standard deviation is calculated for all speed tests. If the download speed deviates more than 20 percent, the speed test is excluded from this test.

Based on these results, all speed tests with significant results are selected. The resulting speed tests are sorted by the average of the measured download speed and the standard deviation. Number 1 on the list earns 5 points, number 2: 4 points, number 3: 3 points, number 4: 2 points and number 5: 1 point.

Fast

It is manually timed how long it takes to take a speed test to display the final download speed. If the time it takes to display the final download speed exceeds 30 seconds (when testing at 30 Mbps), the speed test is excluded from this test. The speed tests are ranked (fast to slow) based on the average of the measured time. Points are awarded in the same way as described above.

Speed tests to test

Taking into account the basic requirements to be met, these speed tests are tested:

The results

Based on the test results of

1. Bandwidth Place
2. Comparitech
3. DSLReports
4. Fast
5. Fireprobe ®
6. Google Fiber
7. Internet Speed at a Glance
8. Measurement Lab
9. Ookla Speedtest
10. SamKnows Speed Test
11. SpeedOf.Me
12. Speedsmart
13. Xfinity xFi Speed Test

the following can be noted.

Free to use

9 of the 13 speed tests tested have no ads. These speed tests are:

1. Comparitech
2. Fast
3. Fireprobe ®
4. Google Fiber
5. Internet Speed at a Glance
6. Measurement Lab
7. SamKnows Speed Test
8. Speedsmart
9. Xfinity xFi Speed Test

Easy to use

4 of the 13 speed tests tested passed the WCAG 2.0 Level A test. These speed tests are:

1. Bandwidth Place
2. Internet Speed at a Glance
3. Ookla Speedtest
4. Xfinity xFi Speed Test

10 of the 13 speed tests tested could be started with a single click (or no click at all). These speed tests are:

1. Bandwidth Place
2. Comparitech
3. DSLReports
4. Fast
5. Fireprobe ®
6. Internet Speed at a Glance
7. Ookla Speedtest
8. Sam Knows
9. SpeedSmart
10. Xfinity xFi Speed Test

So far the scores are:

1. 3 points: Internet Speed at a Glance and Xfinity xFi Speed Test
2. 2 points: Bandwidth Place, Comparitech, Fast, Fireprobe ®, Ookla Speedtest, Sam Knows and Speedsmart
3. 1 point: DSLReports, Google Fiber and Measurement Lab
4. 0 points: SpeedOf.Me

Accuracy and consistency

The download speeds are shown in the graph below.

Remarkably, only two speed tests cut the 30Mbps line; Fast and Internet Speed at a Glance. It should come as no surprise that these two speed tests have the greatest variance or bandwidth.

Assumptions are the mother of all fuck ups!

Almost all speed tests gives an internet speed above the 30Mbps. This may come a surprise, but in this article regarding Firefox throttling it is clearly stated that you should not rely on this feature for exact performance measurements; it's intended to give an approximate idea of the user experience in different conditions.

So the first question should be: What speed is really delivered when using browser throttling?

Due to the fact that many measurements are taken, it is possible to make an informed estimate of the actual delivered speed. To do that the following test scenario should work:

1. Set the browser limit to 30Mbps
2. Test the speed with different speed tests
3. Calculate the average speed and corresponding variance

Based on this test the following graph is created:

The throttling speed is 31.9Mbps (with a variance of 1.53Mbps) instead of 30Mbps.

Probably a better way to get the throttle speed is to just look at speed test results from DSLReports, Fast, Fireprobe ®, Google Fiber, Ookla Speedtest, Sam Knows and Xfinity xFi Speed Test.

The results of these speed tests are remarkably close together.

Based on this speed test, the average speed is 31.5Mbps with a deviation of 0.38Mbps.

For the rest of the test, only these speed tests are taken into account.

Sorted on the difference between 31.5Mbps and the average speed measured by the speed test, the top 5 is:

1. Ookla Speedtest
2. Xfinity xFi Speed Test
3. DSLReports
4. Google Fiber
5. Fireprobe ®

Doubt is the origin of wisdom

The list above is a nice list, however there are some doubts. Measurement Lab does not work with a throttled browser. May be there are some other speed tests which have problems with a throttled browser. May be speed tests behave different at another speed. Therefore, a second test is done without throttling.

In two graphs, the results of these tests are:

DSLReports and SpeedSmart didn't work fine during this test. So these tests are for now out of scope.

If you look at the second graph, you will notice that there is more or less the same problem as before. It is not clear exactly what the speed to be measured should be.

Inquiries with the ISP used show that the ISP applies a safety margin on top of the download speed to be delivered. The ISP did not want to explicitly state the size of this safety margin. However, from what has been said, it can be concluded that this margin is 1 Mbps. For now the assumption is that the test is done with a download speed of 51Mbps.

Sorted on the difference between 51Mbps and the average speed measured by the speed test, the top 5 is:

1. Xfinity xFi Speed Test
2. Google Fiber
3. Measurement Lab
4. Fireprobe ®
5. Ookla Speedtest

Combining the list above with the list created for the throttled test is relative easy when you consider a speed test as accurate or not. Both lists contain the same speed tests (with one exception, DSLReports is replaced by Measurement Lab). So this will be the top 5 of acccurate speed tests.

Fast

Based on the graph below, it is easy to come up with a top 5 fast speed tests. Note: only the accurate speed tests are included.

Note that Measurement Lab isn't included in this test. Separate testing learns that Measurement Lab is slower than Fireprobe ® and faster as Ookla Speedtest. Hence the top 5 of fast speed tests is:

1. Internet Speed at a Glance
2. Google Fiber
3. Fireprobe ®
4. Measurement Lab
5. Ookla Speedtest

Conclusions

Based on the above research, these are the final scores:

Note that this list is very different from the curated top 5 based on different speed test comparisons.

Most accurate speed tests

The most accurate speed test is Xfinity xFi Speed Test.

Google Fiber is second best.

Fastest speed tests

The fastest speed test is Internet Speed at a Glance.

Google Fiber is again second best. It is much faster than other speed tests and accurate.

Other findings

Some other findings are:

1. In contrast to other speed tests Measurement Lab first measures the upload speed followed by the download speed;
2. The Ookla Speedtest is the base for almost every Internet Service Provider speed test;
3. Fast initialy shows only the download speed, but is also capable of showing the latency and upload speed;
4. The following speed test allows you to select a specific test server:
   1. Bandwidth Place
   2. DSLReports
   3. Fireprobe ®
   4. Ookla Speedtest
   5. SpeedSmart
5. It depends on the speed test used if the test server location is relevant. For the Ookla Speedtest the test server location does not influence the measuered download speed;
6. DSLReports has by far the most options;
7. Not all speed tests respect the browser throttling, these speed tests are:
   1. Bandwidth Place (Firefox)
   2. Internet Speed at a Glance (Firefox)
   3. M-Lab (Chromium and Firefox)
   4. Ookla Speedtest (Firefox)
8. The throttling speed of Firefox compared to the throttling speed of Chromium is not significant different;
9. The throttled speed of 30Mbps used in Chromium and Firefox is slightly more than promised (31.9Mbps instead of 30Mbps);