Qualcomm has been accused of cheating on its Snapdragon X Elite/Pro benchmarks, casting doubt on its ability to rival Apple’s custom silicon.

According to SemiAccurate, OEMs working with Qualcomm’s Snapdragon SoC are seeing a major difference in performance, compared to what Qualcomm says the chip is capable of.

After OEMs got initial samples and made something close to the final designs, SemiAccurate got reports of poor performance. By poor we mean far sub-50% of the numbers Qualcomm was telling them in the technical docs and presentations.

When asked about the discrepancy, Qualcomm engineers blamed poor cooling and using a non-Arm native benchmark build as the likely culprits. Unfortunately, even when taking that feedback into account, OEMs saw little improvement.

Later, with more Snapdragon X Elite samples in the wild and many more revisions of WART (Windows on Arm), we got similar reports from OEMs and another Tier 1. Both reported numbers that were nowhere close to what Qualcomm promised. How not close? Above 50% this time but one used the term ‘Celeron’ to describe performance. The claims of better than Apple’s Rosetta 2 x86 emulation are clearly not real on what is probably the release hardware and software. Actually the silicon emulation may be better but everything else is unquestionably not.

SemiAccurate acknowledges the reports from the OEMs is not conclusive and goes on to cite a source. In the interest of protecting their source, the outlet says it has to blur things more than they’d like.

Two major OEMs with serious engineering capabilities are strong evidence but not proof. Here is where we will blur things out more than we like, sorry about that. A while back we were digging on performance or lack thereof in preparation for the promised briefings. A deep source at Qualcomm told us that the benchmarks were cheats, told us how they were cooked, and told us that Qualcomm was well aware of it. This same technique meant the numbers looked far better than they could be on a non-trivial set of tests. Ironically some other benchmarks could have looked much better than those presented if Qualcomm adequately disclosed their testing details. Lose some, lose some, but no winning here.

The outlet then goes on to make the point that Qualcomm is engaged in a coordinated effort to make the “cheating values” accepted.

In the end the problem is simple. You have a product with promised performance that is not achievable with the claimed settings. Because the claims can’t be independently verified, the influencers will parrot back how ‘good’ the product is based on something we know is not true. Between the initial publicity blitz and the time things go on sale, the cheating values will become ‘truth’ and few if any will question them months down the road. Cynical hardly begins to describe this mechanism. The most interesting part is whether OEMs will parrot back numbers they know they can’t replicate or release their own. Time will tell.

What more is there to say? Qualcomm has been making promises about disclosure and testing for about six months now. They have broken every one of those and at least to SemiAccurate, repeatedly lied.

If the outlet’s investigation is correct, Qualcomm could find itself in a precarious situation. At the very least, the firm could lose the trust of its partners, especially at a critical juncture. With many companies hoping the new Snapdragon chip could dethrone Apple’s M-series chips, a disappointing showing by Qualcomm could further delay a true Windows alternative to Apple’s MacBooks, and potentially cause companies to look at Qualcomm’s competitors for future solutions.