Network Performance Score

Source from Rohde-schwarz.com

The single score for network QoE

The QoE-centric Network Performance Score (NPS) is a single metric that characterizes the overall network performance. The NPS compares the quality of mobile networks and visualizes the quality of experience (QoE) that the end users perceive when using common applications. This benchmarking approach covers all mobile networks in the tested area.

Mobile network operators can use the NPS

as a management tool: “I want my team to improve the network quality by x points over the next y month.”
to create marketing claims: “Best network”; “Fastest network”; “Best for streaming services”; etc.

Infrastructure vendors and service providers can use the NPS

to visualize a competitive advantage: “Networks using our infrastructure perform better than xy.”
to generate more business: “We can help you to win next years’ public benchmark.”

The NPS is integrated into the Smart platform and is the ideal point of reference for facilitating benchmarking and reducing the complexity of targeted improvements in network quality and performance.

In August 2019 the global telecoms standards organization ETSI released the technical report TR 103 559 which defines best practices for network QoS benchmark testing.

The scoring challenge – which score should be used?

Many different proprietary scores have swept the market over the last few years. The market is fragmented.
The methods for scoring and the weighting and aggregation of the results are not transparent.
Scoring the same network with different scoring methods might provide different results.
Optimizations in the network might affect every scoring methodology differently.

The ETSI STQ group completed the harmonization process and released the technical report TR 103 559 “Best practices for robust network QoS benchmark testing and ranking” in August 2019.

The NPS methodology according to ETSI

Weighting and aggregating regional scores to a countrywide network score

Scoring mobile networks

Level 1: The basis of the NPS scoring methodology is the QoE measurement of common applications. What drives QoE of a service? What are its dimensions? Most services have three dimensions affecting end-user quality of experience:

Service availability: Can the user access the service?
Waiting time: How long is the acceptable duration and patience for downloading a web page, connecting a call or starting a video?
Quality of the media: Does the quality meet the user’s expectation?

In a first step, these dimensions and technical KPIs are transformed into a common, comparable metric to aggregate them to a “per service” performance. This scaling also considers saturation in perception, meaning that it makes no sense to report the differences in a technical performance where the difference is not perceived by the users.

Level 2: These “per service” performance figures are then further aggregated by certain weights and aggregated to a score describing the network’s performance, for example in a specific region.

Level 3: The regional performance scores can be weighted again, for example by population density over all regions, and aggregated to an overall score for the whole network.

Maximized QoE via an optimized network

This scoring methodology together with the details from the multi-dimensional QoE evaluation, including the perceptual impact, provide a guideline for MNOs to optimize their networks effectively. Knowing the actual vs. the maximum achievable points per test category/service/region/technology etc. provides insight into the potential and the concrete area for network optimization.

This is in particular important because resources and investments for network optimization are often limited. Knowing exactly what to optimize and how to achieve the most significant improvements gives mobile operators a competitive edge. In essence, the standardized Network Performance Score (NPS) methodology is the glue between benchmarking campaigns and targeted network optimization activities.

Network Performance Score