What makes a speed test excellent?

If you want to rate speed tests, you need to know what criteria the rating is based on.

After performing several speed tests, the conclusion is that an excellent speed test has the following characteristics:

  1. Accurate and consistent
  2. Free to use
  3. Easy to use
  4. Privacy-friendly
  5. Informative
  6. Fast
  7. Secure
 

Accurate and consistent

An excellent speed test is accurate and consistent.

That's very shortsighted. An explanation of what exactly is expected is in order.

At the excellent Wikipedia page regarding Accuracy and precision the following image is used to explain that test results should be accurate and consistent (have a high precision). When that isn't the case the result is considered to have a low accuracy.

Two targets with four hits each, showing that a result is only good if the precision is high and the hits are on the bullseye

Research -as summarized on the most accurate speed test- shows that the actual download speed provided by an ISP is 12% higher than the advertised speed.

A speed test is accurate and consistent when the result reflects the:

  1. Advertised speed:
    1. the median is higher than the advertised speed and
    2. the median is lower than the advertised speed plus 3%
    3. the standard deviation is less than 3%
  2. Actual speed:
    1. the median is lower than the actual speed and
    2. the median is higher than the actual speed minus 3%
    3. the standard deviation is less than 3%

That is the median of the test results is in one of the green solid areas in the graph below.

A colored bar which indicates when a speed test is considered to be accurate

 

Free to use

An excellent speed test is free to use. That is:

  1. You don't have to pay money to use it
  2. There is no need to subscribe to a mailing list
  3. No ads will be shown

It is acceptable for the speed test to collect measurement results anonymously.

 

Easy to use

An excellent speed test is easy to use (has an excellent usability) when:

  1. the speed test works in any browser on any device (it is mobile friendly)
  2. the speed test is accessible and WCAG compliant
  3. the speed test can be started with a single action (mouse click or keystroke), or starts automatically
  4. a retest can be started with a single action (mouse click or keystroke)

The user is in control

  1. When the speed test offers multiple tests besides the download test, the speed test gives the user the option which test to start
  2. When a test takes to long the user is able to abort the test
  3. It is posssible to install the speed test as a progressive web application
  4. The user is able to select a dark mode color scheme

The feeling of being in control

At least an excellent speed test gives the user the feeling of being in control. That is the speed test should at least respond within 1 second to the user input.

 

Privacy-friendly

An excellent speed test is privacy-friendly. That is:

  1. You don't have to login to use the speed test
  2. There is no tracking mechanism (like cookies)
 

Informative

An excellent speed test is informative. For example the following results are presented:

  1. An indication in advance about the duration of a test
  2. An indication in advance about the number of the maximum number of bytes that will be downloaded and uploaded
  3. A busy indicator
  4. A total progress indicator
  5. A progress indicator per test when multiple tests are started at once
  6. A countdown timer for the (total) expected wait time
  7. The download speed
  8. The maximum download speed
  9. A graph of the download speed
  10. The upload speed
  11. The maximum upload speed
  12. A graph of the upload speed
  13. Ping
  14. Latency (network delay)
  15. Jitter (Packet delay variation)

All results are shown with respect to the error margin

An excellent speed test shows the results with respect to the error margin. A speed test therefore uses as many significant figures as are consistent with the estimated error of that test.

Error messages are helpful

The speed test should show a helpful error message when an unforeseen problem occurs. For example when JavaScript is required but not available.

According to Wix a helpful error message:

  1. explains what happened and why
  2. provides reassurance
  3. is empathetic
  4. helps the user fix the issue
  5. gives the user a way out
 

Fast

An excellent speed test is fast.

Speed test loads a lot of data to measure your download (and upload) speed. Due to the widespread use of speed tests, they contribute significantly to global CO2 emissions.

The longer a speed test is testing, the more data is transferred. It is therefore preferable to use a fast speed test if it is really necessary.

A fast speed test is a climate friendly speed test! Tools like the Website Carbon Calculator should praise you for the low CO2 emissions.

Besides that, the faster a speed test is, the better the user experience.

 

Secure

An excellent speed test is secure.

If you use a speed test, you can rightly assume that it is secure enough to run the speed test safely.