The most accurate speed test

TL;DR

The most accurate speed test depends on the speed you measure. M-Lab is the only speed test that appears in the top 3 of most accurate speed tests for every speed tested.

When is a speed test accurate?

At College Sidekick it is stated that:

Science is based on observation and experiment—that is, on measurements. Accuracy is how close a measurement is to the correct value for that measurement.

The picture below of Wikipedia Commons makes clear that there is a difference between accuracy and precision. The difference between accuracy and precision graphical explained

Preferably, a speed test is accurate and of high precision.

However, the starting point for determining an accurate speed test is that you need to know the correct value. So the question is, what is the correct value to measure?

The correct value to measure

Progressive insight shows that a mainstream speed test should measure 100.8% of the advertised internet speed.

Based on a meta-analysis of hundreds of test results, the following can be concluded:

It depends

The most accurate speed test depends on the speed to be measured

The best speed test at 50 Mbps

With an advertised speed of 50 Mbps, the following speed tests are most accurate:

  1. M-Lab, accuracy 0.0 Mbps, standard error 0.9 Mbps
  2. Fireprobe®, accuracy 0.0 Mbps, standard error 1.1 Mbps
  3. Ookla Speedtest, accuracy 0.1 Mbps, standard error 0.9 Mbps

The best speed test at 75 Mbps

With an advertised speed of 75 Mbps, the following speed tests are most accurate:

  1. DSLReports, accuracy 0.1 Mbps, standard error 1.0 Mbps
  2. M-Lab, accuracy 0.5 Mbps, standard error 0.2 Mbps
  3. Ookla Speedtest, accuracy 0.5 Mbps, standard error 1.1 Mbps

The best speed test at 100 Mbps

With an advertised speed of 100 Mbps, the following speed tests are most accurate:

  1. Cloudflare, accuracy 0.0 Mbps, standard error 0.5 Mbps
  2. M-Lab, accuracy 0.3 Mbps, standard error 1.2 Mbps
  3. TestMy.net, accuracy 0.3 Mbps, standard error 1.0 Mbps

The best speed test at 200 Mbps

With an advertised speed of 200 Mbps, the following speed tests are most accurate:

  1. Cloudflare, accuracy 0.0 Mbps, standard error 0.6 Mbps
  2. TestMy.net, accuracy 0.2 Mbps, standard error 1.0 Mbps
  3. M-Lab, accuracy 0.4 Mbps, standard error 1.6 Mbps

Earlier conclusions

Below you can read the earlier conclusions made regarding the accuracy of speed tests.

Findings of an earlier meta-analysis

Based on that knowledge all previous relevant test results have been reanalyzed.

Speed tests with an accuracy of max 1% are considered as accurate. Sorting this speed tests on the precision (or better said the standard error) gives the following top 5 of accurate speed tests:

  1. Google Fiber Accuracy: 1.0%, Standard error: 0.11 Mbps
  2. M-Lab Accuracy: 0.4%, Standard error: 0.18 Mbps
  3. Fireprobe® Accuracy: 0.7%, Standard error: 0.23 Mbps
  4. Ookla Speedtest Accuracy: 0.1%, Standard error: 0.25 Mbps
  5. DSLReports Accuracy: 0.4%, Standard error: 0.37 Mbps

The correct value to measure (again)

When you measure the internet speed with different speed tests, you get different results. The graph below shows this clearly. The measurements per speed test (median and standard deviation) This chart is based on the measurements taken as part of the ultimate speed test test 2022 (Advertised speed is 75 Mbps).

The graph above has the same pattern as the graph below which was created in 2020 (Advertised speed is 50 Mbps). Based on this graph it was concluded in 2020 that there are three categories of speed tests. Speed test results for a 50Mbps connection in a min avg max diagram

 

The tested speed tests for the ultimate speed test test are classified into the following categories:

  1. Realistic
  2. Mainstream
  3. Conservative
Realistic speed tests

A realistic speed test gives an indication of the actual download speed provided by your ISP.

This speed is higher than the advertised internet speed. You can imagine your ISP delivering more speed than agreed to minimize discussions about the service provided with customer service.

Mainstream speed tests

Since almost every ISP has a speed test based on Ookla's Speedtest, or recommends the Ookla Speedtest, it's not surprising that Ookla's results are considered the gold standard. You might be wondering if a competitive speed test won't sideline itself if it deviates too much from Ookla's Speedtest?

It is therefore not surprising that there are so many speed tests that all give more or less the same result as Ookla's Speedtest.

Speed tests in this category usually gives as result a speed test that is near the advertised speed.

 
Conservative speed tests

A conservative speed test gives you a low download speed. These speed tests are clearly not accurate.

For Fast, the speed test of Netflix, that is actually not at all wrong. If you achieve the minimum required speed for Netflix with the Fast speed test, you will almost certainly have no problems with watching Netflix. You can probably imagine without a doubt that this will prevent a lot of phone calls to the Netflix support center.

Two correct values to measure?

Based on the findings above there are values which might be considered as the correct value: the advertised speed and the actual speed.

The correct value and therefore also the accuracy depends on what you expect a speed test to measure: the advertised speed or the actual speed.

 
Advertised speed versus actual speed

The ultimate speed test test of 2020 concluded that 18 of the 23 speed tests tested gave a higher result than advertised.

Therefore, the ISP vendor was asked if the speed they provide is higher than advertised. The answer was: "Yes it is."

Although the support desk would not say what the real download speed is, the following tip was given: "Visit our internet forum and ask your question there."

On that internet forum I was told that the highest reading is probably closest to the actual download speed.

At that moment in time I have a contract for 50 Mbps. However when I was reseaching speed determining factors I once measured 60.7 Mbps.

A more robust conclusion regarding the actual speed was based on the graph below.

Unexpectedly high test results in a min avg max chart Based on this graph I concluded that the actual internet speed is 56 Mpbs in stead of 50 Mbps. So the actual internet speed delivered seems to be 12% higher than the advertised internet speed.

In 2021 I got a 75 MBps contract. Hence I tested all speed tests in 2021 with the assumption that the actual internet speed is now 84 Mbps (this because 75 Mbps + 12% = 84 Mbps).

The best internet speed test in the test of 2021 was SpeedOf.me. When you take a look at the max download speed measured, the median was 83.52 Mbps with a standard deviation of 0.03 Mbps.

83.5 Mbps is 11.3% more than 75 Mbps, or the other way around: 75 Mbps is 10.2% less than 83.5 Mbps.

Due to the fact that my ISP has an internet speed test based on Ookla's Speedtest like almost every other ISP, it is safe to state that your actual internet speed is also 12% higher than the advertised internet speed.

 

Crossing out inaccurate speed tests

A seemingly easy way to find an accurate speed test is to cross out inaccurate speed tests.

To do this, a speed test is considered inaccurate if:

  1. The speed test gives no result
  2. The median is low, that is the median is less than the adverstised speed (75 Mbps)
  3. The median is doubtful, that is the median is somewhere between the advertised and the actual speed (between 77 and 82 Mbps)
  4. The speed test has an unintended large standard deviation, that is the standard deviation is larger than 2 Mbps

Based on these criteria the following venn diagram has been made.

A venn diagram of inaccurate speed tests

A long list of accurate speed tests

By crossing out 16 of the 27 speed tests, the following 11 speed tests could be considered as accurate:

  1. Broadband Speed Checker
  2. Cloudflare
  3. DSLReports
  4. Fireprobe
  5. Google Fiber
  6. Internet Speed at a Glance
  7. M-Lab
  8. Ookla Speed Test
  9. SamKnows
  10. SpeedOf.me
  11. SpeedSmart

Most of these speed tests are mainstream speed tests. The exceptions are:

  1. Internet Speed at a Glance which is considered as realistic and conservative
  2. SpeedOf.me

The most accurate speed test to measure the advertised speed

In the speed test test of 2022, speed tests with a median between 72 and 78 Mbps were considered a mainstream speed test. These speed tests measure the advertised speed. In the graph below we zoom in on this category. The measurements of the mainstream speed tests Based on this graph it is hard to conclude which speed test is most accurate. Frankly, there are several possible considerations.

Consideration 1, Cloudflare is most accurate

Cloudflare's median is closest to the advertised speed (the 75 Mbps line in the graph above), hence Cloudflare is the most accurate speed test although the standard deviation is relative high.

Consideration 2, SpeedSmart is most accurate (although wrong), with a small standard deviation

Because SpeedSmart's range does not reach the 75 Mbps line, this speed test always indicates too high a value. However, SpeedSmart has such a small standard deviation that the final measurement result is always relatively close to but always above the actual value. Although it is always wrong, you could argue that SpeedSmart is the most accurate speed test.

Consideration 3, Google Fiber is the overall best (although wrong)

When you accept wrong results (consideration 2) and realise that Cloudflare has an overlap in scope with eleven other speed tests. Scientifically, this means that these speed tests are just as good as Cloudflare.

Accuracy alone is clearly not distinctive enough to rate these 12 speed tests. Therefore, the 2022 speed test test examined which mainstream speed test is the best. Various criteria were used for this study. The conclusion of this research is that Google Fiber has the best overall score.

The most accurate speed test to measure the actual speed

The speed test test of 2021 was a relative short test to find out which speed test has the lowest bias in relation to the actual delivered download speed (84 Mbps in 2021).

Consideration 4, SpeedOf.me (max) is most accurate

At the time, the conclusion was that the maximum speed as stated by SpeedOf.me most accurately measures the actual internet speed.

Based on the graph below, you can safely assume that this is still the case. The measurements of the realistic speed tests

Consideration 5, Internet Speed at a Glance is the best

Of the 2 speed tests crossing the 84 Mbps line only Internet Speed at a Glance is a recommended speed test. Hence Internet Speed at a Glance is the best speed test there is.

The top 5 of accurate speed tests

The considerations mentioned above results in the following top 5 of accurate speed tests:

The top 5 of accurate speed tests
  1. Cloudflare - A mainstream speed test. The median is slightly above the advertised speed, the standard deviation is relative large
  2. SpeedSmart - A mainstrean speed test. The result is always slightly higher than the advertised speed, the standard deviation is relative small
  3. Google Fiber - A mainstream speed test. Overall the best speed test of 2022 although the result is always higher than the advertised speed
  4. SpeedOf.me - A realistic speed test. This is the speed test to use if you want to measure your actual internet speed (which is 12% higher than the advertised internet speed)
  5. Internet Speed at a Glance - A realistic and conservative speed test. Although this speed test has a large standard deviation. It is the only "accurate" speed test that accurately reflects the precision of the measurement