Final TrueNet report, January 2018

Thats all folks!

As of 31st January TrueNet will no longer be measuring the performance of our ISPs for you, our loyal and awesome readers! So now we would like to take the time to thank you all for your support over the last 7 years! Without you, our volunteers, subscribers and readers we wouldn’t have been able to nudge the ISPs into providing a better service for consumers here in NZ. TrueNet entered the market with a stated mission “ to improve Internet experiences for NZ businesses and consumers ” - and we’ve done a good job of it!

Thank you and goodnight From the team at Jonette Consulting

We whipped up a quick chart, and as you can see there is rise in performance from 2012 till 2018 ( taking into account the incredible “Netflix effect” in 2015!)

Holding pattern continues

This months results are generally looking good, in a holding pattern again.

However Trustpower and Vodafone fibre show significantly lower speeds than in the past when downloading from Australia. Downloading at less than half the speed of competitors in Fibre and much less in the other technologies. The majority of panelists with these ISPs experienced a significant drop from the 14th of December onward. 


Summary of Performance Measures

Webpage Average Speed

  • NZ
  • International

NZ File Peak Speed by Time of Day (ToD)

  • Technology comparison 
  • Fibre & Cable
  • Copper (ADSL and VDSL)
  • Fixed Wireless

International File Peak Speed

  • Fibre & Cable
  • Copper (ADSL and VDSL)
  • Fixed Wireless

Upload Median Decile Speed

Video test results


Summary of Performance Measures

Our interactive charts enable comparisons according to your preferences. The default sort of both Table 1: ISPs, and Table 2: Technologies, prioritises Surfing on NZ websites, because this is the most likely activity that users will experience. Our test sites  include Trademe, ISPs, and Banks amongst almost 30 sites.

TrueNet's Table 1 compares broadband performance with unlimited, naked, long-term price by ISP, for each of the technologies.  

To explore Table 1, first type into the search bar to filter the table  (eg AD will get only ADSL or Fi will compare Fixed Wireless and Fibre, sp will get all Spark services and so on.) You can check technology performance with price at this point.

Click on any header in the blue bar to sort by each column, (e.g. prices will show ascending or descending price).  

HINT for researchers: Right click on the table to select "Open frame in new tab" then select 25 lines to see the full table (Note - this does not work in Chrome).

Table 1: ISPs

Notable features of Table 2 when comparing technologies: 

The fastest technology for everything is Fibre, with those lucky enough to live in the right suburbs of Wellington and Christchurch, Cable (Vodafone sell Cable as FibreX) is the next option.  It is a significant drop to the other technologies.

The new 4G based product we call Fixed Wireless, sold by Mobile carriers, has a faster peak speed than ADSL but is about the same for surfing speed (Note, this product is delivered using a SIM card, just like your mobile phone and has a variety of names with each carrier).  Higher levels of Latency are expected to be an issue for Fixed Wireless - see our page showing Latency by technology here.

Peak speeds at 9pm are very close to advertised peak speeds for most technologies, but peak speed (raw data speed) has little impact on the overall speed experienced when surfing.
Fibre and Copper (ADSL & VDSL) are all achieving better than 95% minimum/maximum peak speeds.

Table 2: Technology

Note: Webpage average speed is the webpage size divided by the time to download.  Webpages are often sent from the owners site in many small files, which means the speed is not as fast as that achieved during a single file download (Peak Speed).  

A simple analogy to explain Peak vs Average speed, assumes you have a very fast car like a Ferrari.

Peak speed is the speed reached during a time trial on a track - say 100's of km/hr
Average Speed (webpage surfing speed) is the average speed you would achieve in Featherston street, Wellington, after navigating lights, say 30km/hr

Peak Speed Ratio compares the slowest hour to the fastest hour (as fast as your line will allow) in the File download Peak Speed test.  

The Price column is the price of Unlimited, Naked services for the given ISP and technology. A full list of ISP Prices can also be seen here.

Webpage Surfing (Webpage Average Speed)


Responsive website surfing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.

TrueNet tests Internet surfing by downloading a selection of Live Webpages from NZ, Australia, China, UK and USA; measuring the size of each webpage, and the time to fully download all files on the page.  From this we calculate the speed.  These pages are changed from time-to-time so the actual average webpage speeds cannot be compared between months.  This month there are 15 NZ pages, 3 Australian, 1 UK, 2 China and 10 USA pages.  DO NOT COMPARE SPEEDS BETWEEN COUNTRIES because average webpage speed is dependent on the actual webpages downloaded, not the ISPs.  The list of test webpages is in the Glossary.

We download webpages from each connection, and compare the average speed achieved for each download.  This test replicates daily activity for many people, and we then group the webpages into NZ and International, so that readers can compare ISPs based on their own preferences.

Fibre webpage average speeds show no material difference based on Fibre speeds sold, so we show them inclusive of 100Mb/s, 200Mb/s, and Full Speed (sometimes called GigE) services.

NZ Webpage Surfing

Technology Comparison - Webpage Average Speed of Selected NZ Webpages by Time of Day.

Please click on the different time periods in the interactive chart (Peak, Business, Off-Peak) to see the performance changes during each period.

Once again there is minimal difference shown between Fibre 100Mb/s and 200Mb/s this month, 200Mb/s cable (FibreX) remains behind Fibre for all three TOD results. Good results for Fixed wireless this month, sitting equal to ADSL for Business and peak hours, and just ahead for Off-peak hours.

Chart 1a: Technology Comparison - Webpage Average Speed of Selected NZ Webpages by Time of Day

Fibre and Cable (FibreX)

A tighter grouping of ISPs this month with Spark improving on last months rankings. Vodafone is no longer showing the evening dip in both its cable and fibre services.

Chart 1b: Fibre & Cable - Webpage Average Speed of Selected NZ Webpages by Time of Day

Copper webpage surfing speed is limited by the speed of internet over copper (a standard telephone line connection). VDSL has an average peak speed of 40Mb/s and ADSL about 10Mb/s, but there is a wide range of speeds about these averages.  The actual average speed of each ISP on these technologies is dependent on the location of our panelists, not the ISPs network.  The speed at any time of day, relative to the maximum speed possible, is however within the control of the ISP on copper.

Copper (VDSL)

Two distinct groupings of ISPs again this month, a very tight cluster of ISPs, with the positions changing slightly with Orcon dropping down the ranks in comparison to last month.

Chart 1c: VDSL over copper - Webpage Average Speed of Selected NZ Webpages by Time of Day

Fixed Wireless (4G Mobile based services)

Fixed wireless performance is still in a holding pattern, all ISPs continue with the steady decline in speed throughout the day, and Vodafone is still in the lead for average surfing speed, and as shown in charts above no longer has the distinct evening dip.

Chart 1d: Fixed Wireless - Webpage Average Speed of Selected NZ Webpages by Time of Day

International Webpage Surfing

Popular Webpages from Australia, China, US, and the UK are downloaded, and we compare the average speed to download all webpages.  This test replicates the daily activity of many people accessing webpages from various countries, and includes the impact of network design at an ISP, where NZ caching, international links and other enhancements improve international website performance.

Fibre and Cable (FibreX)

Staus quo again this month with the ISPs sitting within the same groupings as last month. Orcon and Vodafone have swapped places in the ranks and Bigpipe no longer shows the quirky rise in speed as shown in last months findings.

Chart 2a: Fibre & Cable - Webpage Average Speed of Selected International Webpages by Time of Day

Copper (VDSL)

All ISPs, apart from Orcon show an improvement when compared to last months results

 Chart 2b: Copper - Webpage Average Speed of Selected International Webpages by Time of Day

Fixed Wireless (4G Mobile based services)

Vodafone still ahead of Spark an Skinny this month for all but the early morning hours, where Spark claims the top spot.

Chart 2c: Fixed Wireless - Webpage Average Speed of Selected International Webpages by Time of Day

Peak Speed

For TrueNet's peak speed tests each panelist's probe regularly downloads a 2MB and/or 5MB file from Auckland, Wellington, Dallas and Sydney.  We identify the fastest quartile/decile as the Peak Speed.  The faster the connection, the larger the file we need to download to ensure that the maximum speed can be reached during our test for ISPs. 100Mb/s connections easily reach full speed before 2MB of data is downloaded from NZ or Australia.  Slower connections can test accurately with much smaller files. 

Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the Internet during the evening hours of 8pm to 10pm.  A poor result typically shows the line drop below 90%, which usually occurs in the busy period between 7pm and 10pm, i.e. if this is true, the average user for that ISP is getting less than 90% of their line capability.  

We set the "Zero" line at 70% because panelists with less than 70% report difficulty using the internet.

New Zealand Peak Speed

NZ peak speed (the best quartile speed of a file being downloaded) can be influenced by how well ISPs peer at the internet exchanges, where they connect to TrueNet's server provider for our file test. Without effective peering a file can travel from a TrueNet test probe located in Wellington, through to Auckland, on to Sydney then back to the Wellington exchange, creating a problem with slowdown in service delivery during busy hours. 

TrueNet however, recommends you compare ISPs based on webpage surfing speeds rather than peak speed alone.  To make the surfing comparison fairer, webpages are from popular sites that all ISPs have equal access to.

Technology Comparison

Chart 3 compares Peak Speed of technologies in Off-peak, Business, and Peak hours between 6pm to midnight. Fixed Wireless results had the biggest relative speed swings between Peak Hours and Off-Peak periods. Results are otherwise approximately the same as November.

Chart 3: Fibre, Cable and Copper (DSL) Peak Speed by Time of Day from NZ

Fast Broadband: - Peak Speed by Time of Day

The peak speeds amongst the Fibre ISPs remains about the same as in recent months. Most ISPs average above 100Mb/s in this test, with TrustPower and Vodafone averaging just under advertised level. There is no sign of contention in the Fibre speed tst results.

Chart 4a: 100Mb/s Fibre & Cable File Peak Speed by Time of Day from NZ

TrueNet uses an index for copper (DSL) reporting because ADSL and VDSL are sold using the phrase "as fast as your line will allow", so we find the maximum speed, and compare each result with that maximum.  Hence every line reaches 100% at some hour.  

The Copper network is becoming "deloaded" as more users upgrade to Fibre.  This generally means that performance is excellent.  Where once we hoped that ISPs could achieve 90%, we now see most doing better than 95%.

High Speed Copper (VDSL) Peak Speed by Time of Day from NZ

VDSL Time of Day performance was excellent again in December, staying above 95% of best hour speed, with the odd exception for Voyager. In general, there is more variability than in recent months.

Chart 4b: VDSL File Peak Speed by Time of Day from NZ

Low Speed Copper (ADSL) File Peak Speed by Time of Day from NZ

ADSL results were good in December, though the variability seen in VDSL is a bit more for ADSL. Each ISP is above 95% of best hour speed most of the time, but there are times where speed is between 90% and 95% of best hour speed.

Chart 4c: ADSL File Peak Speed by Time of Day from NZ

Fixed Wireless Peak Speed by Time of Day

Spark speeds early morning are now equal or better to Vodafone results, but during the day when the network is busy, Spark results closely resemble those of Skinny.

Chart 4d: Fixed Wireless File Peak Speed by Time of Day from NZ

Australian Peak Speed

To compare international performance, TrueNet downloads files every hour to measure Peak Speeds from Sydney.  

We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance.   

The connection between the ISP and the Sydney peering exchange is critical to many services based in Australia, and the rest of the world.  Many services are "cached" in Sydney on servers that are supplied by third parties to improve download speeds in remote places.  Two examples are Apple, or Microsoft software updates, which are cached by Akamai in NZ.  

We test by downloading our file from a server connected to the Sydney exchange. This file is random and has checks on it to ensure it is not cached in NZ, which means it is an honest measure from Sydney.

Fast Broadband - Australian Peak Speed by Time of Day

Download speeds from Australia are similar for the ISPs at the top of the chart. There was a big shift in TrustPower and Vodafone during the month, with the majority of panelists experiencing a significant drop in this metric from the 14th of December onward.

Chart 5a: Australian File Peak Speed for High Speed Connections (2MB file size only) by Time of Day

Copper Connections - Australian Peak Speeds by Time of Day

Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem, and the exchange equipment which means that ISPs have little influence on the peak speed of each connection.

Australian Peak speed needs to be compared to the best NZ Peak speed by ISP due to the influence of distance from the exchange, which has a direct affect on Peak Speed.  We show the Australian Peak speed as a percentage of the NZ Peak speed (Chart 5b). 

VDSL (High Speed Copper) Australian Peak Speeds by Time of Day

Spark achieved similar speeds in download tests from NZ and Australian test servers, and along with 2Degrees perform well above the other ISPs. As mentioned in the Fibre results above, the Vodafone result fell to about 30% of NZ peak speed, implying the cause for each are related.

Chart 5b:  VDSL Peak Speed from Sydney, Australia (2MB file size only) by Time of Day

ADSL (Low Speed Copper) Australian Peak Speeds by Time of Day

2Degrees peak speeds are very similar from NZ and Australian servers, with Orcon and Spark averaging about 90% of their best NZ speed in the Australia download test. Slingshot lags behind at 70% of NZ speed, which is an established pattern over several months.

Chart 5c: ADSL- Australia Peak speeds

Fixed Wireless: Australian Peak Speeds by Time of Day

Special note that this too compares Australian with NZ file speeds, so the chart equalises for probes further away from the cellsite, which shows Skinny with better results, partly becasue their speed in NZ is slower. Vodafone has had lower relative speed to Australia in these results for several months, and appears unrelated to the change in Fibre & VDSL results since November.

Chart 5d: Fixed Wireless FIle Download performance from Sydney, Australia (2MB file size only) by Time of Day 

Upload Speed - Performance

Upload speed is important to users sending large amounts of data through the Internet, or uploading files to the Cloud.  TrueNet's upload test sends a 1MB file to our Wellington server, and records the results as the median of the 10 deciles measured for each download.

By Technology / Service Upload Speed

Upload service speeds by Technology / Advertised speed stayed roughly the same or slightly up on last month.

Chart 6: Upload Speed by Technology

Upload Speed - by ISP

Spark 20M Upload service improved from last month's irregul;ar result (16 Mb/s) to 23 Mb/s, in line with other ISP results. As seen in this chart over successive months, the main ISPs meet or exceed advertised Upload service speeds.

Chart 7: Fibre & Cable Upload Speed by ISP

Upload performance of Fixed Wireless as a percentage of best hour speed is shown in Chart 8. Skinny and Vodafone have similar results, and both maintained upload speed better during business and evening periods than Spark.

Chart 8: Fixed Wireless Upload Speed by Time of Day

Video Test Results

TrueNet's video test chooses specific videos (Two videos, a HD one and an identical content 4K video)  The video will be cached (stored locally) by ISPs in NZ for all viewers, simply because we are downloading the video seven times a minute from more than 400 different NZ locations.

Each test downloads 10 seconds of content at a time, to fill any buffer to last 40 seconds of viewing.   So on startup, the test will immediately download (at the connection's full speed) 40 seconds worth of content, after a time gap (to allow for playback) it will then download a second 40 seconds of content, simulating keeping the 40 second playback buffer full.  This appears to be an international standard for consumer video testing.

TrueNet records buffer events by counting "Stops", where the simulated buffer would become empty.   We also record Resume and Finished events.   Chart 1 shows a count of the number of "Stops" (sometimes more than one stop in a single download attempt).

We also measure the start delay, ie the time taken for a video to start downloading so it can play.  The typical period for this is so small, usually less than 1 second, that we have ceased publishing this statistic.

Buffering Events during Youtube watching

Buffering on all internet lines in NZ is usually very rare on any of our 480 test sites. If you have buffering on your connection, check your wifi.  We do not test over wifi and that is often the culprit when it comes to buffering.  This scale is used to keep the results in perspective, ie there is no issue.

The rate of buffering for all technologies is very low, typically less than 1% to 3% depending on technology. This month there were a few isolated events causing greater buffering in ADSL and VDSL results, not present in the very low buffering rates in November.

Chart 9a: Buffering Events by Day

Buffering for different times of day is shown in Chart 9b. Click on the different periods (Peak, Business, Off-Peak) in the chart to see changes in performance. This means that even during the most challenging period, 97% or more of videos are free from buffering.  The change in Fixed Wireless is so significant, that we have changed many of our earlier comments, buffering has all but vanished on this technology after being a lot higher when our measurement of the technology started.

Chart 9b: Buffering Events by Time of Day

Note: All Fixed Wireless data collection is supported by a contract with Chorus. All other technologies are supported by a contract with the Commerce Commission. Analysis of the data and the production of reports are the work of TrueNet under the Commerce Commission contract.

© Jonette 2011