Recently, Consumer Reports released new findings pointing out how the Surface line of products fared against its competitors when it comes to reliability and satisfaction. It found that the line of high-end devices from Microsoft suffered some of the highest return rates, deprecating it from the list of recommended devices by the research firm.
However, Microsoft responded to these findings after engaging with Consumer Reports, and found that its data was heavily skewed towards the Surface Pro 4 and Surface Book. The Pro 4 device, in particular, was marred by driver issues relating to the Skylake processor from Intel, on which Microsoft blamed the high return rate at that time, and subsequently fixed. Furthermore, the Surface Book had several issues, including one with the electronic connection which holds the display and dock together.
In the leaked memo acquired by Paul Thurrott; Panos Panay, Corporate Vice President at Microsoft, decided to give clarity on these reports and what the company did to rectify the reported issues. According to Panay, since the launch of these products, Microsoft has "worked tirelessly" to improve its subsequent devices, including the Surface Laptop, Surface Studio, and sequels to the Surface Pro.
He went on to say that the term "failure" which was used by Consumer Reports was "too broad" and was incorrectly attributed to any issue that occurred with the device in question including minor issues that could easily be corrected by the user. On top of this, he pointed to the company"s quality control:
“We take quality seriously, conducting rigorous reliability testing during development to forecast failure and return rates, which are then continually viewed against [real world data] post-launch. We also regularly review other metrics to understand the experience we are providing to our customers and our findings show our products are in a much healthier place than noted by Consumer Reports.”
Relating to the "other metrics" pointed out by Panay, the company uses the Net Promoter Score (NPS), something OEMs like HP use to determine how to approach future products and revisions. This metric however, does not address reliability, instead focusing on whether the consumer would be willing to recommend the product in question to someone else. According to Paul Thurrott, this is an unreliable metric to base quality on, since the consumer might feel obligated to justify the purchase of said product, especially when it comes to premium devices like the Surface.
Panay pointed out that the company admittedly had a high Incidence-per-Unit (IUP) with the Surface Pro 4 and Surface Book, each at 17% and 16% respectively. However, other devices in the range remain low at less than 1%, which has consistently stayed at this rate for 12 consecutive months. Panay also stressed its commitment to excellence when it comes to its range of hardware products in a recent blog post.
Source and image: Thurrott.com
Update: Added reference links to relevant articles. Corrected an incorrect referral to the blog post by Panos Panay, to the leaked memo.