IQRush says AI visibility rankings need a stability test

IQRush released research Tuesday that argues AI visibility scores should not be trusted until they prove the ranking has stabilized and the gaps between brands are larger than the margin of error. The Seattle company says the test can help marketers separate real movement in AI search from statistical noise across systems like ChatGPT, Perplexity, Gemini and SearchGPT.

Why it matters: - AI visibility dashboards are becoming a marketing decision tool, but the underlying systems can change their answers from one query to the next. - IQRush says brands risk acting on noise unless rankings are tested for stability and statistical separation first. - Search Engine Journal covered the research and cited SparkToro founder Rand Fishkin’s advice that buyers should make sure a provider “shows their math.”

What happened: - IQRush released new research today titled From Stochastic to Stable: Rank Stability and Structural Sufficiency in AI Visibility Measurement. - The paper was written by IQRush co-founder and Chief AI and Data Officer Ron Sielinski. - The research is available at the paper. - Sielinski also has related work scheduled for presentation this month at the IAB Measurement Leadership Summit in New York.

The details: - The research focuses on AI answer engines including ChatGPT, Perplexity, Gemini and SearchGPT. - The paper argues that a citation score on a dashboard is only one sample, not a fixed fact. - Two brands that appear first and second on a leaderboard may actually be statistically tied. - Sielinski tested 30 combinations of platforms and topics across Gemini, SearchGPT and Perplexity. - The number of citation-bearing answers needed before a ranking could be trusted ranged from 33 to 94. - Three of the 30 tests never reached a stable ranking even after 125 questions. - Those three tests were all on SearchGPT, where the top sources stayed too close together to separate reliably. - The paper says a ranking is ready only when two conditions are met: the order stops changing as more data comes in, and the gaps between brands are bigger than the measurement’s margin of error. - IQRush says the method makes AI visibility measurement repeatable for teams that measure in-house or buy from a vendor. - IQRush says its platform is built to answer four questions on every metric it reports: how many times each query ran, the margin of error on each score, whether the ranking would hold next week, and which positions are truly separate versus tied.

Between the lines: - The research challenges a common analytics habit: treating probabilistic AI systems as if they were deterministic search engines. - That matters because data quality has become a top CMO concern, and AI visibility may be one of the clearest places where bad measurement can look polished on a dashboard. - The paper also reframes “decision-grade” reporting as a measurement problem, not a branding claim. - Sielinski said the industry moved quickly to produce AI visibility scores and more slowly to test whether those scores can support the decisions people make with them. - Sielinski said providers should show their math and prove that a ranking has settled and that the gaps between brands are real before anyone acts on the number.

What's next: - Brands and agencies can book a 20-minute walkthrough on the IQRush.ai site to see how their numbers hold up under the same test. - IQRush says the goal is to help marketers decide when an AI visibility ranking is ready to trust and when it is still statistical noise. - The company says the research gives the market a repeatable standard that can be used whether the measurement is built internally or purchased from a vendor.

The bottom line: - IQRush is pushing AI visibility vendors toward a higher bar: prove the ranking is stable, prove the gaps are real, and then call it a number worth using.