OpenSignal to reveal the “actual” network speed of Airtel and Jio

OpenSignal today said it will reveal an analysis of the mobile networks in India and settle the debate — between Bharti Airtel and Reliance Jio — on network speeds.

OpenSignal competes with Ookla. Ookla has stated that Bharti Airtel is the best telecom network in India in 2016 analyzing mobile network speed in Q3 and Q4.

According to TRAI reports, several telecom operators have taken top spot in 3G and 4G speed tests for both download and upload speeds.

Speed test reports from TRAI and Ookla, which have revealed significantly different result, resulted in a public spat between Airtel and Reliance Jio over Airtel’s claim of fastest network based on Ookla test results.

“We are sharing our thoughts to help India’s burgeoning 1.13 billion mobile consumers make sense of these differing results, based on our experience analyzing mobile connections globally,” said OpenSignal in a statement.

The below statistics is from Ooka test result.
How two testing regimes can produce vastly different results when measuring speeds on the same networks? The answer lies in the methodology used to collect the data and analyze the results.

Depending on whether an equal weight is placed on individual speed tests, or on individual users, the analysis can yield vastly different results. Because any crowdsourced testing relies on a diverse set of volunteer users to run speed tests — either actively or in the background — on their downloaded apps.

The below charts are from TRAI.
There will always be a group of users that run a lot more speed tests than others. Those users are constantly firing up their apps in order to keep close tabs on their operators’ performance or to be active contributors to their crowdsourced communities.

The end result is these extremely active users contribute a much higher percentage of tests than ordinary users. If the methodology gives equal weight to all tests, then the results would be drastically skewed to the experience of that handful of very active testers.
“We believe the best way to accurately examine the mobile network experience in India is to put each user on equal footing – that is, to first take an average of each individual user’s tests during the sample period and then weight those averages equally in the overall analysis,” OpenSignal said.

If the goal is to assess what the typical consumer experiences then you don’t want the analysis to be skewed by what a subset of extremely active user’s experience.

At the same time it is still useful for users to continue running multiple tests, since it is better for one to have more data to analyze. “We recommend against weighting each individual speed test equally when it’s possible for individuals to contribute very different amounts of tests,” OpenSignal said.

OpenSignal factors the use of dual-SIM phones to ensure right measurements are assigned to the right network. “We also weight our analysis to individual user experience rather than individual speed tests to ensure we are best reflecting what typical users are experiencing rather than those users who run more frequent tests than others,” OpenSignal said.