WISP QoE Benchmarks
The Preseem network telemetry platform processes over 400 million metrics per day from WISP networks. This page presents metrics extracted from this huge data set that we hope are useful to anyone interested in the fixed wireless industry.
This benchmarks data will be updated at least once per month. Please subscribe to our WISP-only newsletter to be notified when this is updated and receive industry news, networking articles and more.
Subscriber Peak Hour Metrics
The peak or busy hour of the network represents the worst case for most Quality of Experience (QoE) metrics. These subscriber metrics are calculated across all Preseem WISP customers and are designed to give an indication of the experience for a typical WISP subscriber.
The typical (as defined by the 50th percentile) active WISP subscriber achieves 1.3Mb/s average download rate during the peak hour. Note that active means that they meet a minimum usage threshold and are not simply connected to the AP. Unsurprisingly, the achieved upload rates are significantly lower.
What is a Percentile and Why Do You Use Them?
A percentile is an alternative to simpler summaries like the average which can be misleading. For example, if two people have salaries of $0 and $100,000, the average salary is $50,000. If two people both have salaries of $50,000, the average salary is still $50,000.
The simplest explanation of a percentile, eg the 80th percentile, is that 80% of samples (subscribers, AP, etc) are below the 80th percentile value and 20% are above it.
Latency is a key component of the subscriber experience. Often, decreasing latency can have a much more profound positive impact on the subscriber experience than adding bandwidth. Preseem’s latency calculation covers the path from the point where Preseem is deployed, down to the subscriber and back again.
These Latency Numbers Seem High?
Preseem’s latency metrics are calculated by extracting the timing between packets in the subscriber’s traffic. This is fundamentally different than ICMP based ping methods that traditional network management solutions provide.
For more information, take a look at the video embedded next which goes into detail about how QoE based networking differs from traditional network monitoring.
AP Peak Hour Metrics
The access point (AP) is the key infrastructure element for WISPs. Therefore understanding key AP metrics such as the number of customers per-AP and typical throughput enables high-level business and network planning activities.
Active Subscribers per AP
The number of subscribers per AP is a key planning metric for many WISPs. Obviously, the higher the number of subscribers per AP that can be achieved while still providing a good QoE, the more profitable the AP becomes.
It is interesting to compare this aggregate number with the per AP model metrics below.
What’s with the Peak Hour?
It’s easy to calculate a metric over a long or fixed interval of time but often the value isn’t meaningful. For example, we could provide the metrics for a fixed period of 9-10PM every day but for some networks, this isn’t the busiest time so the value wouldn’t reflect the true load on the network.
Instead, we slice the day into 96-hour long intervals, a new interval starts every fifteen minutes. We then find the ‘busiest/worst-case’ hour and use that for analysis. In this way, we can automatically adapt to different usage patterns across WISPs and across APs in the same WISP.
Throughput per AP
Does it surprise you that the typical WISP AP only delivers 5.6Mbps during peak? Remember, this is the actual delivered rate not a theoretical rate and it includes APs that have few subscribers.
Latency per AP
The latency per AP numbers look quite similar to the top-level latency per subscriber numbers presented earlier. However, they do differ because this metric groups by the AP first and then orders the AP by latency to calculate the percentiles.
When looking at latency metrics, keep in mind that a round trip latency above 150ms results in a poor VoIP call. Gamers are even more sensitive to latency.
AP Make/Model Peak Hour Metrics
Since the AP is the key infrastructure element for WISPs, it is interesting to compare the performance of the various makes and models. The metrics presented here break down the per-AP metrics above, by the AP model.
Peak Hour Active Subscribers by AP Model
Probably the biggest takeaway from this chart is the large variation between the various models that Preseem customers have deployed (see sidebar about missing make/models).
Does This Mean AP Model A is better than Model B??
No, yes, well, ugh, maybe? The numbers represent the real world metrics across Preseem’s customer base. However, it’s possible that it is biased in some way such as everyone who uses a particular model deploys it on the end of a slow backhaul or perhaps a particular model is most often used in a geographically challenging environment. Of course, there are also a lot of other things to take into account when choosing an AP or SM such as the price, management features etc.
Peak Hour Throughput by AP Model
Again, the most striking aspect of the throughput data is the variation between the various models. The Cambium PMP platform pulls quite far ahead of other models when it comes to the raw throughput during peak usage.
There are a couple interesting anomalies in the throughput metrics. Notably the throughput for the ePMP 2000 is lower than the ePMP 1000 and the Rocket Prism AC5 Gen2 is lower than Gen1. We suspect that this is the result of either a lower sample size or deployment differences across WISPs. Watch future updates to this page for more insight into this.
Peak Hour Latency by AP Model
Unlike the number of active subscribers and throughout, the latency variation between models isn’t nearly as dramatic. There appears to be a general trend that more modern models have lower latency characteristics.
Where is My Vendor/Model?
Throughout this analysis, we have excluded any AP model for which we don’t have data for at least three live APs. As the Preseem customer base adopts additional AP models, look for these AP models to appear in these charts.
Network telemetry platform, developed for WISP network owners and administrators to monitor & optimize QoE, lower support costs and improve profitability.
© 2017 Aterlo Networks Inc.