Speed determining factors

How do I run an internet speed test?

This study is designed to investigate which factors may affect your test results. A remarkable result was that an earlier assumption about the Internet speed of the ISP was incorrect. For this test almost all factors mentioned in "The 10 things that could be affecting your internet speed" (webpage gone) are tested.

All factors are compared with the original ideal test conditions.

No significant influence

  1. Browser used (and Browser (reprise))
  2. Other active programs
  3. Test server used
  4. DNS server used
  5. Video streaming

Significant influence

  1. Use of TOR (>60% slower)
  2. Large downloads (>70% slower)

It depends, there may be significant influence

For the following factors it is impossible to say how they affect your download speed. It depends on your specific configuration. Some configurations don't affect the internet speed test at all, other configurations do have significant impact.

With the configurations used for this test, it was concluded that:

  1. It is possible to select a VPN without a significant impact on the download speed
  2. The computer quality/operating system used does have significant influence
  3. The use of Wi-Fi is NOT necessary a significant factor
  4. The use of a powerline connection might have a significant influence

However you can't say that these factors have significant influence or NOT, because it can't be proven in general what the influence of these factors are.

For example, I don't see a significant drop in the download speed when I use Wi-Fi. However this might be the case when the test was done with a standard download speed of 500Mbps instead of the standard download speed of 50Mbps.

The same applies for the use of a powerline. It was concluded that there is a significant influence, however, when you have a standard download speed of 20Mbps instead of 50Mbps it might be concluded that there is no significant influence.

So in general the following factors might have significant influence on your download speed.

  1. Router quality, connection type (cable/powerline/Wi-Fi)
  2. Computer quality/Operating System
  3. Other devices or users

To hard to test

For the following factors it is hard to test if they have any influence on your download speed. There are too many options. The best is to assume that these factors have some influence on your download speed.

  1. Your ISP and the subscription taken out there
  2. Malware
  3. Router Location
  4. Time of the day

This study examines the influence of these factors. Based on this, it should be possible to indicate the ideal conditions for an internet speed test.

 

Ideal test conditions

A wired computer is believed to be best for an Internet speed test. This computer must be the only device connected to the router.

The following configuration is used:

  1. Dual boot desktop computer
    1. Intel® Core™ i5 CPU 661 @ 3.33Ghz x 4
    2. 3.6Gib memory
    3. 320.1GB disk capacity
  2. Ubuntu 20.04.1 LTS
  3. Firefox Browser 80.01 (64-bit) with a cleared cache and 1 open tab

In these circumstances the Xfinity xFi Speed Test is used. The following download speeds are measured:

  1. 51.0Mbps
  2. 50.9Mbps
  3. 50.9Mbps
  4. 51.0Mbps
  5. 50.9Mbps

The average speed is 50.9Mbps, with a standard deviation of 0.05Mbps.

Starting point

A test is significant different when the P-value is less than 0.05 (P<0.05), then the conclusion is that the two means are significantly different.

This Comparison of means calculator will be used to calculate the P-values.

 

Browser used

In ideal circumstances the internet speed is tested with other browsers. With Chromium these are the results:

  1. 50.9Mbps
  2. 50.9Mbps
  3. 50.9Mbps
  4. 51.0Mbps
  5. 50.9Mbps

The average speed is 50.9Mbps, with a standard deviation of 0.04Mbps.

With the Falkon browser these are the results:

  1. 50.9Mbps
  2. 50.9Mbps
  3. 50.9Mbps
  4. 51.0Mbps
  5. 51.0Mbps

The average speed is 50.9Mbps, with a standard deviation of 0.05Mbps.

With the Qute browser these are the results:

  1. 50.9Mbps
  2. 50.9Mbps
  3. 50.8Mbps
  4. 51.0Mbps
  5. 51.0Mbps

The average speed is 50.9Mbps, with a standard deviation of 0.08Mbps.

With the Gnome Web browser, the Midori browser and the Netsurf Web Browser the Xfinity xFi Speed Test doesn't work.

The used browser is NOT a significant factor

The conclusion so far is that the linux browser used has no significant influence on the internet speed test.

See also Browser (reprise).

 

Other active programs

In ideal circumstances the internet speed is tested with OpenSCAD rendering a 3D object (a 5 minute process). During this rendering, these are the results:

  1. 50.9Mbps
  2. 50.7Mbps
  3. 51.0Mbps
  4. 51.0Mbps
  5. 50.9Mbps

The average speed is 50.9Mbps, with a standard deviation of 0.12Mbps.

The next test is with 11 open programs (with OpenSCAD rendering) besides Firefox. Now the results are:

  1. 51.0Mbps
  2. Disastrous, the Linux computer hung up and must be restarted. This test is therefore aborted.

A retest with 6 open programs besides Firefox gives the following results:

  1. 50.9Mbps
  2. 50.9Mbps
  3. 50.9Mbps
  4. 51.0Mbps
  5. 51.0Mbps

The average speed is 50.9Mbps, with a standard deviation of 0.05Mbps.

Other active programs are NOT a significant factor

The conclusion so far is that, as long as the computer does not freeze, the other active programs do not affect the internet speed test.

 

Test server used

In the The ultimate speed test test 2020 five speed tests tested allows you to select a specific test server.

  1. Bandwidth Place
  2. DSLReports
  3. Fireprobe ®
  4. Ookla Speedtest
  5. Speedsmart

All these speed tests are tested with the suggested nearby test server and a test server at the other end of the world.

The test server used is (mostly) NOT a significant factor

Only for Bandwidth Place there was a significant different result. So, more often than not, the test server does not affect the download speed.

Besides that there isn't a reason at all why you should change the, by the used speedtest, suggested test server.

 

DNS server used

Does the DNS server used influence the download speed?

In the standard test environment there isn't a DNS given (it's automatic).

Techradar.pro suggests several free and public DNS servers. First we test with the Linux dig command what the query time is for speedtest.xfinity.com. These times are:

  1. Default, no DNS selected: 579ms
  2. OpenDNS 208.67.222.222: 2103ms
  3. Cloudfare 1.1.1.1: 1039ms
  4. Google Public DNS 8.8.8.8: 667ms
  5. Comodo Secure DNS 8.26.56.26: 1511ms
  6. Quad9 9.9.9.9: 855ms
  7. Verisign DNS 64.6.64.6: 1151ms

The same is done for xfinity.com. Now the results are:

  1. Default, no DNS selected: 11ms
  2. OpenDNS 208.67.222.222: 103ms
  3. Cloudfare 1.1.1.1: 11ms
  4. Google Public DNS 8.8.8.8: 11ms
  5. Comodo Secure DNS 8.26.56.26: 91ms
  6. Quad9 9.9.9.9: 91ms
  7. Verisign DNS 64.6.64.6: 143ms

Based on these results it is decided to test with Google Public DNS 8.8.8.8. The download speeds measured are:

  1. 50.9Mbps
  2. 51.0Mbps
  3. 51.0Mbps
  4. 51.0Mbps
  5. 50.9Mbps

The average speed is 51.0Mbps, with a standard deviation of 0.05Mbps.

The DNS server used is NOT a significant factor

Due to the fact that exactly the same download speeds are measured as in the ideal situation, the conclusion is that the DNS server used is not a significant factor.

 

Video streaming

While watching YouTube: Rowan Atkinson Dusts Off An Old Comedy Bit, these are the test results:

  1. 50.9Mbps
  2. 50.9Mbps
  3. 50.6Mbps
  4. 49.7Mbps
  5. 50.9Mbps

The average speed is 50.6Mbps, with a standard deviation of 0.52Mbps. It may be somewhat surprising. Watching YouTube does not significantly affect download speed. While watching Netflix: The Big Lebowski, these are the test results:

  1. 51.0Mbps
  2. 50.0Mbps
  3. 51.3Mbps
  4. 50.5Mbps
  5. 50.6Mbps

The average speed is 50.7Mbps, with a standard deviation of 0.50Mbps. Again. it may be somewhat surprising. Watching Netflix does not significantly affect download speed.

Video streaming is NOT a significant factor

Although video streaming services like YouTube and Netflix state that you need 5Mbps for their services, based on the tests above the conclusion is that video streaming is not a significant factor.

 

Use of TOR

In ideal circumstances the internet speed is tested with the TOR browser.

  1. 18.0Mbps
  2. 24.1Mbps
  3. 11.7Mbps
  4. 14.0Mbps
  5. 20.1Mbps

The average speed is 17.6Mbps, with a standard deviation of 4.91Mbps.

Use of TOR is a significant factor

Obviously, due to the use of the Onion services used by the TOR browser, your internet speed is significant slower when you use the TOR browser.

 

Large downloads

While downloading the 1GB (Very Large) file from this page with Download Test Files (Gone), these are the test results:

  1. 15.9Mbps
  2. 10.7Mbps
  3. 13.0Mbps
  4. 13.7Mbps
  5. 13.2Mbps

The average speed is 13.3Mbps, with a standard deviation of 1.86Mbps.

Large downloads are a significant factor

It is clear that the a large download has a negative impact on the download speed.

 

VPN

In ideal circumstances the internet speed is tested with the Browsec VPN add-on for Firefox.

  1. 17.1Mbps
  2. 41.2Mbps
  3. 46.7Mbps
  4. 37.4Mbps
  5. 45.6Mbps

The average speed is 37.6Mbps, with a standard deviation of 12.04Mbps.

The add-on Hola Free VPN Proxy Unblocker (Gone) give these results:

  1. 51.0Mbps
  2. 50.8Mbps
  3. 50.9Mbps
  4. 50.9Mbps
  5. 50.8Mbps

The average speed is 50.9Mbps, with a standard deviation of 0.08Mbps.

Your VPN service may be a significant factor

An unambiguous conclusion is not possible. It apparently depends on the VPN service being used whether or not the test results are affected.

The Browsec VPN add-on gives a significant slower result, while use of the Hola Free VPN Proxy Unblocker has no (significant) influence at all.

 

Computer quality/Operating System

In ideal circumstances the internet speed is tested the dual boot computer described before with the Windows 10 operating system.

  1. 52.0Mbps
  2. 51.8Mbps
  3. 60.7Mbps
  4. 53.1Mbps
  5. 49.5Mbps

The average speed is 53.4Mbps, with a standard deviation of 4.28Mbps. The maximum speed measured is now 60.7Mbps.

The download speed of 60.7Mbps is that strange that I decided to test multiple times with the Windows 10 computer in combination with Firefox. Now these are the results:

  1. 51.2Mbps
  2. 52.0Mbps
  3. 52.1Mbps
  4. 52.5Mbps
  5. 52.1Mbps

The average speed is 52.0Mbps, with a standard deviation of 0.48Mbps.

  1. 52.2Mbps
  2. 52.0Mbps
  3. 51.8Mbps
  4. 52.2Mbps
  5. 52.1Mbps

The average speed is 52.1Mbps, with a standard deviation of 0.17Mbps.

Based on these 15 measurements with Firefox, the average speed is 52.5Mbps, with a standard deviation of 2.40Mbps.

Another operating system is NOT a significant factor

With the same dual boot computer, the average speed at measured with the Windows 10 operating system is higher than the average speed measured while running Ubuntu. Nevertheless it can't be concluded that this is a significant factor (P = 0.16).

In ideal circumstances the internet speed is tested with a Windows 7 Thinkpad laptop.

  1. 52.5Mbps
  2. 53.4Mbps
  3. 53.1Mbps
  4. 53.2Mbps
  5. 55.5Mbps
  6. 57.2Mbps

The average speed is 54.2Mbps, with a standard deviation of 1.81Mbps. Due to the higher-as-expected results, I tested 6 times instead of 5 times. Wow, the maximum speed measured is now 57.2Mbps.

Computer quality/Operating System is a significant factor

Based on this measurements the conclusion is that Computer quality/Operating System is a significant factor.

Pity enough, based on these tests it is impossible to declare which "configuration" let you measure the highest download speed.

The third test is with a Windows 10 Lenovo laptop.

  1. 51.7Mbps
  2. 51.9Mbps
  3. 52.1Mbps
  4. 52.3Mbps
  5. 52.8Mbps

The average speed is 52.2Mbps, with a standard deviation of 0.42Mbps.

Computer quality/Operating System is a significant factor

Based on this measurements the conclusion is that Computer quality/Operating System is a significant factor.

There is no significant difference between the download speed at the Windows 10 computer and the Windows 10 laptop. That suggests that the computer quality isn't a significant factor. However, it is impossible to test all possible configurations. Hence this test doesn't proof that the computer quality has no influence on the download speed. To be safe, the conclusion is that the internet speed might depend on the quality of your computer.

Inconsequent test results

Based on the tests regarding the computer quality/operating system, the conclusions are:

  1. Operating system is NOT a significant factor
  2. Computer quality is NOT a significant factor
  3. Computer quality/Operating System is a significant factor

Besides these confusing conclusions, most shocking is that unusual high download speeds were measured: 60.7Mbps and 57.2Mbps. While the assumption was that the ISP delivers 51Mbps instead of 50Mbps.

 

Browser (reprise)

With the Windows 10 computer, the speed test is now done with Google Chrome (should theoretically give the same results as Chromium).

  1. 52.7Mbps
  2. 52.7Mbps
  3. 52.2Mbps
  4. 52.8Mbps
  5. 52.4Mbps

The average speed is 52.6Mbps, with a standard deviation of 0.25Mbps.

That isn't significant different from the download test done with Firefox.

The used browser is NOT a significant factor

It can be concluded that the windows browser used has no significant influence on the internet speed test.

 

Wi-Fi

With the Windows 7 laptop connected via 5Ghz Wi-Fi (TP-link Archer C6 router) the results are:

  1. 52.9Mbps
  2. 53.9Mbps
  3. 53.7Mbps
  4. 54.2Mbps
  5. 53.6Mbps

The average speed is 53.7Mbps, with a standard deviation of 0.48Mbps.

The use of Wi-Fi is NOT necessary a significant factor

Although slightly slower, there isn't a significant difference.

However it is not possible to say that the same applies to your router capabilities (in combination with your ISP contract). Hence the only conclusion so far is. The download speed might depend on your router.

 

Powerline

With the Windows 7 laptop connected via a cable to Devolo Magic 1 LAN the results are:

  1. 40.2Mbps
  2. 42.1Mbps
  3. 41.4Mbps
  4. 42.1Mbps
  5. 41.2Mbps

The average speed is 41.4Mbps, with a standard deviation of 0.78Mbps.

The use of a Powerline might be a significant factor

This is significant slower. However, the configuration used for this test will be different, than your configuration. Besides that you might have another contract with your ISP. Hence, it can only be concluded that use of a Powerline might be a significant factor.

Powerline Wi-Fi

With the Windows 7 laptop connected via a Wi-Fi to Devolo Magic 1 LAN the results are:

  1. 40.5Mbps
  2. 42.3Mbps
  3. 41.2Mbps
  4. 41.5Mbps
  5. 40.4Mbps

The average speed is 41.2Mbps, with a standard deviation of 0.78Mbps. This is significant slower.

There is no significant difference between a cable or Wi-Fi Powerline connection.

However it is not possible to say that the same applies to your router capabilities (in combination with your ISP contract).

 

Assumptions are ...

A very important assumption made was that the ISP delivers 51Mbps instead of 50Mbps. However these tests with other computers/operating systems give these results:

  1. 52.5Mbps, standard deviation 2.40Mbps
  2. 54.2Mbps, standard deviation 1.81Mbps
  3. 52.2Mbps, standard deviation 0.42Mbps
  4. 52.6Mbps, standard deviation 0.25Mbps
  5. 53.7Mbps, standard deviation 0.48Mbps

Unexpectedly high test results in a min avg max chart

Note that in this graph you do not see that the maximum speed measured was 60.7Mbps with the Windows 10 desktop and 57.2Mbps with the Windows 7 laptop.

So shouldn't the assumption be that the ISP delivers 56Mbps? That is the max shown in the graph above.

When that's the case the ideal test conditions should be reconsiderd and a rerun of all tests is required ...