Usability of speed tests

Written for ZOMDir by Hans van der Graaf, page generated at: 2024-09-23

TL;DR

Of the 8 speed tests that score well on usability, Cloudflare and Fast are the best known.

Theory
Initially we discuss all kinds of possible usability aspects (usability is broad)
Practice
Secondly we test the usability of speed tests in practice

Usability

Jakob Nielsen -a well known retired usability consultany- explains clearly in Usability 101: Introduction to Usability what usability is.

In short: (...) Usability is a quality attribute that assesses how easy user interfaces are to use (...)

(...) Usability is defined by 5 quality components:

Learnability: How easy is it for users to accomplish basic tasks the first time they encounter the design?
Efficiency: Once users have learned the design, how quickly can they perform tasks?
Memorability: When users return to the design after a period of not using it, how easily can they reestablish proficiency?
Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?
Satisfaction: How pleasant is it to use the design? (...)

Learnability

At the web page What are learnability principles for usability? it is stated that (...) the following principles fall under the category of learnability principles:

Predictability: (...) the user should be able to predict the future interaction based on their previous history with the interface (...)
Synthesizability: (...) refers to (...) the system state (...)
Familiarity: (...) allows users to utilize prior knowledge to understand the interface's features (...)
Generalizability: (...) users can apply their experience of specific interfaces to new, similar ones (...)
Consistency (...) the same input/output actions should perform the same functions with the same behavior (...)

Efficiency

At another web page (the page What Does Usability Mean: Looking Beyond ‘Ease of Use’) it is stated that (...) Efficiency can be described as the speed (with accuracy) in which users can complete the tasks for which they use the product. ISO 9241 defines efficiency as the total resources expended in a task. Efficiency metrics include the number of clicks or keystrokes required or the total ‘time on task’ (...)

Memorability

There are many websites that write about usability. In the article Usability 101: Memorability it is stated that (...) memorable things are usually due to one or two factors: (...)

System feedback
Visual cues

Errors

Regarding errors the earlier definition of Jakob Nielsen is clear: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?

One important aspect of an error is the error message. In general, most error messages have plenty of room for improvement (When life gives you lemons, write better error messages).

Satisfaction

Satisfaction seems to be a derived usability component. In User Satisfaction vs. Performance Metrics, Jakob Nielsen finds that in general (...) users prefer the design with the highest usability metrics (...). So you might be wondering if satisfaction is really a separate usability component.

In the article User Satisfaction, Aesthetics and Usability (PDF) it is concluded that (...) Satisfaction was repeatedly found to be a complex construct comprising 'emotion', 'likeability', and 'expectation' as well (...).

An attractive design seems to increase satisfaction in advance. It is noted that (...) Given the known importance of first impressions for subsequent judgments, our results suggest that user interface designers (...) would do well to design beautiful and usable sites (...)

Heuristics

The above definition of usability offers few practical tools for creating a usable system. Fortunately, Jakob Nielsen has also developed usability heuristics. These provide a slightly more concrete interpretation of the concept of usability.

Visibility of System Status (...) always keep users informed about what is going on (...)
Match Between the System and the Real World (...) Use words, phrases, and concepts familiar to the user (...)
User Control and Freedom (...) Users (...) need a clearly marked "emergency exit" to leave the unwanted action (...)
Consistency and Standards (...) Follow platform and industry conventions (...)
Error Prevention (...) prevent problems from occurring (...)
Recognition Rather than Recall (...) making elements, actions, and options visible (...)
Flexibility and Efficiency of Use (...) Allow users to tailor frequent actions (...)
Aesthetic and Minimalist Design (...) Interfaces should not contain information that is irrelevant or rarely needed (...)
Help Users Recognize, Diagnose, and Recover from Errors (...) Error messages should be expressed in plain language (...), precisely indicate the problem, and constructively suggest a solution (...)
Help and Documentation (...) It’s best if the system doesn’t need any additional explanation (...)

Usability testing of speed tests

In Usability Metrics Jakob Nielsen states that (...) It is easy to specify usability metrics, but hard to collect them (...).

We completely agree with that. It took some effort to come up with some -user-agnostic- usability tests for speed tests.

The status of speed tests

In terms of learnability, it seems that we can only test, without involving users, whether speed tests clearly indicate their state. The Yes! The speed test is running test makes clear that all speed tests clearly indicates almost immediately that the speed test is running or finished.

The efficiency of speed tests

We can easily measure how efficient a speed test is by counting the number of clicks needed to start a speed test. Based on the One click speed tests test, we conclude that, when using a secure web browser, most speed tests could be started with one click.

Note that the time to complete a task is also based on the time needed to complete the test. Because the speed of speed tests is in our opinion very important, we consider speed as a separate quality factor beside usability.

The memorability of speed tests

Testing the memorability is only possible with users. Because we do -user-agnostic- usability tests, we performed the Obviously a speed test test. In this test we tried to find out, how many clues there are to help the user to understand where she is at the world wide web. Based on this test we conclude that the following speed tests provide the most clarity about the purpose of the web page: Meter.net, Broadband Speed Checker, Internet Speed at a Glance, LibreSpeed and Open Speed Test.

Errors and speed tests

We have run speed tests many times but have never encountered an error. That's not proof that speed tests are error-free (although not every speed test is accurate).

The only mistake a user can make is inadvertently starting a speed test. Hence we performed the following tests:

Based on these tests we conclude that Bredbandskollen, Google Fiber and LibreSpeed offer the option to abort the speed test.

Cloudflare and Fast offer the option to pauze the test.

All other web-based speed tests can be stopped by using the back button, closing the tab, or closing the entire browser. If you stop a speed test, data usage also stops.

Satisfied with speed tests?

How satisfied users are with speed tests involves users. In terms of design, we can only think of one test, namely whether the speed test is ad-free.

It turns out that a small majority of speed tests tested show no ads at all. Some well-known ad-free speed tests are Cloudflare, Fast, Google Fiber and Xfinity xFi Speed Test.

The user in control

For the heuristic User Control and Freedom we have the following tests:

Based on these tests we conclude that almost all speed tests offer no options on what to start. Astound speedtest, the SpeedOf.me API Sample Page and TestMy.net do allow the user to choose which type of test should be started.

Meter.net, OpenSpeedTest™ and TestMy.net offer the option to manually set a dark mode theme.

The most usable speed tests

To come to a final conclusion, we count the number of times a speed test comes out positive from the above tests. This results in the following list:

Internet Speed at a Glance 6 times mentioned
LibreSpeed 6 times mentioned
Astound speedtest 5 times mentioned
Bredbandskollen 5 times mentioned
Cloudflare 5 times mentioned
Fast 5 times mentioned
SpeedOf.me API Sample Page 5 times mentioned
Speedtest4.PHP 5 times mentioned
Comparitech 4 times mentioned
Google Fiber 4 times mentioned
Open Speed Test 4 times mentioned
SamKnows 4 times mentioned
SpeedOf.me 4 times mentioned
SpeedSmart 4 times mentioned
Bandwidth Place 3 times mentioned
Broadband Speed Checker 3 times mentioned
M-Lab 3 times mentioned
Meter.net 3 times mentioned
Ookla Speedtest 3 times mentioned
Which Broadband Speed Test 3 times mentioned
Xfinity xFi Speed Test 3 times mentioned
Fireprobe 2 times mentioned
SpeedCheck 2 times mentioned
TestMy.net 2 times mentioned
Toast 2 times mentioned
DSLReports 1 time mentioned
N Perf 1 time mentioned

The speed tests with the highest usability scores are: Internet Speed at a Glance, LibreSpeed, Astound speedtest, Bredbandskollen, Cloudflare, Fast, SpeedOf.me API Sample Page and Speedtest4.PHP.