Aaru and EY: What a 0.90 Median Spearman Correlation Covers
On this page
A synthetic-research vendor publishes a correlation against a Big Four partner's survey, and the number circulates as proof the category works. Before that number changes where a research budget goes, a buyer needs to know which statistic it is, which questions it covers and where the two sources diverged.
The claim: a partnership-published correlation
Aaru, a multi-agent behavior simulation vendor, and EY compared Aaru's simulated answers with a global EY survey. EY reports a median Spearman correlation of 0.90 across 53 single-select questions, against a survey of 3,600 respondents worldwide, in a page dated October 7, 2025 describing an August 2025 comparison (EY, "How AI simulation accelerates growth in wealth and asset management").
The same page reports two error measures and a miss. Root mean squared error averaged 7.1 percentage points, and the Euclidean distance across multidimensional response patterns averaged 12.9. Inheritance planning showed a correlation of -37.82%. EY describes the divergences as differences between how people behave and how they answer hypothetical questions.
This is a figure the two firms published together. This article does not test it, and it is not a peer-reviewed result.
What does a 0.90 median Spearman correlation measure?
Spearman correlation compares rankings. When the human results rank answer options in a certain order, the simulated results rank them in a similar order. It does not establish that the two match in level. A simulated series that is shifted or compressed relative to the human series can keep the same rank order, which is why EY's mean 7.1-percentage-point RMSE matters beside the correlation.
A median across 53 questions also says that half the questions scored at least 0.90. It does not say how the other half scored. The inheritance-planning result shows that a single question can reverse the pattern while the median stays high.
What does the figure not establish?
- It does not establish causation. Reproducing the pattern of answers to a survey question is not the same as estimating what an intervention would change.
- It does not transfer to a different question type. Single-select survey questions are evidence about that task. They do not rate every use of synthetic research.
- It does not describe a decision you face. The published summary does not establish validation for your population and decision; request the question-level design and results before transferring the finding.
How does it compare with Subconscious's own published figures?
The statistics are different, so the numbers cannot be ranked against each other. The July 2026 causal fidelity working paper, which is not peer reviewed, reports Spearman rank correlations on estimated choice parameters. It finds a mean of 0.73 across the 43 studies that passed its design filters, and 0.55 across roughly 300 replications.
Those figures compare the order of parameters estimated from simulated choices with the order estimated from human choices in past studies. They are not an accuracy percentage, they are not agreement on effect sizes, and they do not predict a buyer's next study. The paper's limitations section also reports misses, and validation sets can overlap with model training data.
Both vendors' figures answer a narrower question than the one a buyer has. The buyer needs to know whether the simulated result would hold for the decision at hand.
A short checklist before trusting a validation claim
| Question to ask | Why it matters |
|---|---|
| Which statistic is it, and over what? | A median Spearman correlation over questions, a mean rank correlation over parameters and an error in percentage points answer different questions. |
| What task and population does it cover? | A result on stated-preference or concept-reaction questions may not transfer to a launch, pricing or market-entry decision. |
| Where did it miss? | A report that names its worst question lets you judge the downside. A single headline number does not. |
| Is there a human comparison for this decision? | Whether a study adds a human comparison is a scoping choice, and a simulated label does not replace one. |
| Can you inspect the study? | A published study that reports its design, scorer and limits can be checked. A summary figure cannot. |
What's the practical takeaway?
A 0.90 median Spearman correlation is a reasonable signal that simulation can reproduce the ordering of survey answers on the question type tested. It is the start of an evaluation. Before moving budget on any vendor's headline number, run it through the checklist above. The research page lists Subconscious's published evidence with its limits, and the case studies show applied buyer decisions.
Teams comparing methods at this stage often also want to see how a causal experiment is structured and run before deciding which validation path fits their decision.