When you test your internet speed with different speed tests, you will get different results even if you have the same test conditions.
Unfortunately, in practice you cannot speak of "Same test conditions". A speed test measures the speed of an inherently unreliable network.
The Internet protocol (IP), on which all Internet traffic is based, is designed as an unreliable protocol. Data corruption, packet loss, and duplication might occur.
The Internet protocol is designed to choose "the best" available path for each packet, based on the destination IP address.
Note that "the best" is not tightly defined, because Internet traffic uses dynamic routing.
It is therefore normal that if you run multiple speed tests you will get different results even it you have the same test conditions.
Differences in speed tests
The standard explanation for this is that each speed test is programmed in a unique way. However that is not the main reason.
Of course, the operation of any speed test is basically the same: divide the number of downloaded megabits by the number of seconds and you have the download speed in megabits per second (Mbps).
The main reason for the large differences are the assumptions about the reliability of the intrinsically unreliable internet protocol.
It seems that ISPs assume that their customers will experience a 10% reduction in Internet speed as offered by the ISP due to things like data corruption, packet loss and duplication.
It turns out that speed tests can be divided into realistic, mainstream, and conservative categories. In addition to these main categories, two more categories (ambiguous and miscellaneous) were added later.
In short:
- A realistic speed test gives an indication of the actual (maximal) download speed provided by your ISP (This actual download speed is approx 12% higher than the advertised speed);
- An ambiguous speed test gives results between realistic and mainstream speed tests;
- A mainstream speed test gives gives an indication of the advertised download speed. Ookla's Speedtest -the gold standard-, adviced by many ISP's shows the advertised download speed;
- A conservative speed test gives an indication lower than the advertised download speed;
- Miscellaneous speed tests doesn't match criteria mentioned in which speed tests to test.
The way of programming does not seem to be the main reason for the observed differences. The objectives of the companies that do speed tests seem to explain the estimate made of the reliability of the internet protocol and thus the large differences that exist in the measurement results.
This is most evident by looking at the possible business objectives per main category.
Speed tests by category
The speed tests belong to a certain category as follows:
Realistic speed tests
Creators of realistic speed tests probably try to measure the actual speed test as good as possible.
Customer retention
Some of these realistic speed tests are sold to or used by ISP's. Examples are:
As ISP you want to keep your customers. One way to do that is by displaying the actual internet speed as it is 10-12% higher than the advertised speed.
Mainstream speed tests
The Ookla Speedtest is the gold standard. The majority of ISP's advice their customers to use an Ookla Speedtest or an Ookla based speed test. Hence it is logical that a lot of speed test vendors create a speed test similar to the Ookla Speedtest.
The business objectives range from "Inform the public" to "Selling speed tests".
- Inform the public (Bredbandskollen, Measurement Lab)
- Collecting and selling speed test data (Broadband Speed Checker)
- Showcase (Cloudflare, Google Fiber, SamKnows)
- Selling speed tests (Ookla, Fireprobe)
- Attract visitors interested in a new ISP broadband contract (DSLReports)
Conservative speed tests
Some conservative speed tests are owned by ISP resellers. A low speed test can be a trigger for a customer to switch to another ISP. Examples of ISP resellers are:
Fast
Fast is a selfservice tool for Nexflix users to detect if they have internet problems. When you as user conclude that your internet speed is low, you will not bother the Netflix helpdesk when you have problems watching Netflix.
Which speed test to use?
In general the advice is to use the interactive decision tree to answer the question which speed test to use.