Differences in Ookla test servers measured with a non-optimal network

TL;DR

More than a quarter (this test) to a third (our previous test) of the Ookla test servers offer 1% too high internet speeds.

By 1% too high we mean that the measurement result is 1% above the maximum speed that can be measured. This indicates an 'flaw' or anomaly in the measurement method of the Ookla speed test.

 

Contents

  1. Introduction
  2. Method of measurement
  3. Test servers to test
  4. The measurements
  5. Conclusions
 

Introduction

Earlier we concluded that there are differences in Ookla test servers.

 

Method of measurement

For this test we use the original Ookla Speedtest, and we will run one speed test per test server (hosted in Amsterdam).

To avoid ambiguities about the maximum speed to be measured, we place the 100 Mbps RO001 Sweex router in our network. In theory, the maximum true speed may not be higher than 100 Mbps.

 

Test servers to test

We test all servers located in Amsterdam.

 

The measurements

  1. Rounded 79.9 Mbps
    1. Global Layer: 79.86 Mbps (retested 85.45, 87.77 and 85.00 Mbps)
  2. Rounded 84.4 Mbps
    1. Hack The Box Ltd: 84.39 Mbps (retested 94.12, 94.04 and 93.81 Mbps)
  3. Rounded 92.1 Mbps
    1. VDSina: 92.10 Mbps
  4. Rounded 92.3 Mbps
    1. Labixe Ltd: 92.32 Mbps
  5. Rounded 92.5 Mbps
    1. Eranium B.V.: 92.54 Mbps
  6. Rounded 92.7 Mbps
    1. Qonnected B.V.: 92.68 Mbps
    2. Odido: 92.69 Mbps
    3. Solcon Internetdiensten B.V.: 92.69 Mbps
    4. Hivelocity: 92.70 Mbps
    5. TNGNET B.V.: 92.70 Mbps
    6. Usenet.Farm: 92.70 Mbps
    7. I4 Networks: 92.71 Mbps
    8. ServerRoom.net: 92.71 Mbps
    9. Sharktech Inc.: 92.71 Mbps
    10. WD6.net: 92.71 Mbps
    11. GSL Networks: 92.72 Mbps
    12. Melbicom: 92.73 Mbps
    13. NewsXS B.V.: 92.73 Mbps
    14. XS News B.V.: 92.73 Mbps
  7. Rounded 92.8 Mbps
    1. Macarne: 92.75 Mbps
    2. BlackHOST Ltd.: 92.76 Mbps
    3. CLOUD LEASE: 92.76 Mbps
    4. dstny: 92.76 Mbps
    5. Exascale: 92.78 Mbps
    6. MIRHosting: 92.79 Mbps
  8. Rounded 92.9 Mbps
    1. Clouvider Ltd: 92.86 Mbps
  9. Rounded 93.1 Mbps
    1. WARIAN: 93.08 Mbps
    2. HyperFilter DDoS Protection Solutions: 93.11 Mbps
  10. Rounded 93.4 Mbps
    1. RETN: 93.42 Mbps
  11. Rounded 93.6 Mbps
    1. Astroman Dev: 93.59 Mbps
  12. Rounded 93.7 Mbps
    1. Voiped Telecom: 93.71 Mbps
  13. Rounded 93.9 Mbps
    1. Kamatera, Inc: 93.93 Mbps
  14. Rounded 94.0 Mbps
    1. 31173 Services AB: 94.01 Mbps
    2. Asimo Networks B.V.: 94.03 Mbps
    3. Nextpertise: 94.03 Mbps
    4. Fiber NL B.V.: 94.06 Mbps
  15. Rounded 94.6 Mbps
    1. AVUR AS: 94.64 Mbps
  16. Rounded 94.7 Mbps
    1. PhoenixNAP Global IT Services: 94.73 Mbps
  17. Rounded 101.4 Mbps
    1. NFOrce Entertainment B.V.: 101.40 Mbps (retested 92.75, 92.66 and 92.70 Mbps)

We re-measure our internet speed using the 10 most accurate speed tests:

  1. M-Lab 90.56 (89.01), 90.56 (89.98) and 90.59 (92.24) Mbps (NDT (MSAK))
  2. Ookla Speedtest 93.49, 96.38 and 92.68 Mbps (server: VodafoneZiggo, Schiphol)
  3. RealSpeed 101/92.7, 101/92.7 and 101/92.7 Mbps (Router/Device)
  4. Think Broadband (the standalone version) 94.6, 95.3 and 96.3 Mbps (IPv4)
  5. Down at a Glance 92, 92 and 92 Mbps (5 seconds)
  6. Waveform (Bufferbloat and Internet) speed test 94.5, 91.3 and 92.2 Mbps
  7. Fireprobe 94.29, 94.27 and 94.05 Mbps
  8. True Speed at a Glance 102, 102 and 102 Mbps (5 seconds) Realistic
  9. Google Fiber 94.6, 94.0 and 93.9 Mbps
  10. Meter.net 92.72, 92.84 and 92.47 Mbps

As bonus we also test SpeedOf.Me (because it is a realistic speed test) and Cloudflare (because they clearly state that they calculate the 90th percentile).

  1. SpeedOf.Me 102.01 (102.83), 103.02 (103.02) and 103.09 (103.09) Mbps (Download (Max.)) Realistic
  2. Cloudflare 91.7, 91.8 and 91.3 Mbps (server: VodafoneZiggo, Schiphol)
 

Conclusions

  1. The true internet speed is 102/103 Mbps
  2. The mainstream ("advertised") speed is 91.8/92.7 Mbps
  3. When we ignore the initial measurement of the two lowest and the single highest test server, and consider everthing below the 92.5 Mbps as to low, and everthing above the 93 Mbps as to high then:
    1. 2 test servers deliver speeds that are too low (6%)
    2. 22 test servers delivers speeds as expected (61%)
    3. 12 test servers delivers speeds that are too high (33%)

Most test servers deliver 92.7 Mbps.

When we compare these results with our previous Differences in Ookla test servers test, the following stands out:

  1. The list of test servers is different now. Now these servers were not selected:
    1. bouwhuis.network
    2. drie-host
    3. fdcservers.net
    4. CSL Networks
    5. Vancis
  2. Of the 11 test servers that gave too high a speed in the previous test, 8 now give too high a speed again

These results clearly show that the extremely high measured speeds are not the result of a higher true internet speed than expected by our ISP.

More than a quarter (this test) to a third (our previous test) of the Ookla test servers offer 1% too high internet speeds.

1% may not seem like much, but it is a 1% that is added on top of the maximum measurable value. This makes it inexplicable to us where this 1% too high value comes from.

In our opinion, this indicates a 'flaw' or anomaly in the Ookla speed test measurement methodology.