# Subconscious.ai — full article library > Full text of every published article at https://subconscious.ai/blog. Index and site map: https://subconscious.ai/llms.txt --- # Message Testing for Sensitive Corporate Communications URL: https://subconscious.ai/blog/use-cases/message-testing-sensitive-corporate-communications Published: 2026-08-07 A reorganization memo, an earnings guidance revision, an M&A announcement, or a policy change almost never gets tested before it goes out: testing feels riskier than publishing it untested. Showing an internal layoff memo to outside readers before it is announced creates legal and HR exposure, and running a guidance revision past institutional investors before the call is not something most investor relations teams can do. So the message ships untested, and the organization finds out whether it worked by watching the reaction. That is the wrong tradeoff. A team can compare draft variants of a sensitive communication against a defined target-audience segment and measure the causal effect of framing choices on stated reaction, without showing the draft to the real people the release is about. This is the same discrete choice experiment logic long used to test message variants in communication research (Frontiers in Communication, 2025). [figure: A branching diagram: a single sensitive-communication draft splits into framing variants, each tested against a different audience segment (employee, investor, analyst), then converges on one released version.] ## Why do sensitive communications resist ordinary testing? Three properties make reorg memos, guidance revisions, and M&A announcements harder to test than routine marketing copy. Multiple audiences read the same document for different reasons. A reorganization memo has to be honest about the business rationale while staying sensitive to employee anxiety. A guidance revision has to satisfy institutional investors who want financial detail while remaining legible to retail shareholders who want a plain-language explanation. Framing changes the reaction more than the facts do. Call a price increase an investment in product quality and readers respond one way; call the identical increase a response to rising costs and they respond another, even though nothing about the underlying numbers changed. Once published, the communication is permanent. Earnings-call transcripts sit in SEC filings indefinitely. An internal memo that leaks can circulate long after the news it described has changed. ## Testing message framing before a sensitive release The useful question is not whether a draft is well written. It is which of several draft variants moves a defined audience segment's stated reaction in the direction the team needs, and by how much. For a major communication, that usually means identifying the three to five stakeholder groups affected and testing draft variants against each one. For an internal audience, that means testing draft variants of a reorganization or policy memo against a segment defined by the organization's own employee population, measuring whether one version scores better on the reactions that matter: does it acknowledge the change's impact, is the business rationale legible, does it answer the question the reader will ask next. For a board or investor-facing communication, that means testing draft variants of an earnings statement or strategic update against a segment defined by institutional-investor criteria, measuring where each version is more or less credible on the specific financial claims it makes, before the call happens. For a release aimed at press and industry coverage, that means testing draft variants of a release or FAQ against a segment defined by industry-analyst criteria, measuring which framing produces the interpretation the team intends. In each case, the deliverable is a causal comparison between draft variants for a defined audience segment, with confidence intervals where the study design supports them, not a verdict on what a group of individuals "thinks" of the message. ## Practical applications ### M&A communications Get the first 24 hours of an announcement wrong and the integration can spend the rest of the deal recovering from it. A team can treat each outward-facing document as its own draft variant against its own audience segment before the announcement: the press release against press and analyst readers, the internal memo against the workforce, the customer-facing FAQ against the customer base. Run separately, they show where the narratives conflict across audiences before that becomes a public problem. ### Earnings guidance revisions The difference between a guidance revision the market accepts as prudent and one that draws a shareholder claim is often a matter of framing and timing, not the underlying number. Testing draft variants of the revision against an investor-defined segment before the call gives the investor relations team a causal read on which framing is least likely to trigger an adverse reaction. ### Internal policy changes Compensation adjustments and restructuring announcements are usually checked through manager feedback and legal review, neither of which simulates how an individual employee reads the memo. Testing draft variants against an employee-defined segment surfaces the failures those internal loops miss: ambiguity about what the change means for one person, corporate language substituting for plain language, and a tone that skips the emotional weight of the change. ## How this compares to the review a sensitive release already gets The table above puts the causal framing test's gaps next to legal and manager review's own gaps, so a reader can check where each one stops. The causal framing test does not replace the first two rows. It runs in addition to them, on the question neither one is built to answer. ## What are the limitations of a framing test? A result published without its limits is marketing, and this section states the limits of a framing test. A framing test does not replace legal, HR, or compliance review of a sensitive disclosure. It cannot simulate insider information leaking ahead of a release, regulatory reaction, or the full emotional response of the employees, investors, or analysts who will read the real thing, and it is not a recruited panel of real executives or institutional investors. The result is a directional signal about which framing performs better for a defined segment, not a guarantee of how the market or workforce will respond. Where the decision depends on it, a team can move from a simulated framing study to a study with real human participants without changing the underlying causal question. That step matters when the stakes of a release justify it, not for every sensitive communication. Teams building this into their communications process can start by reading how Subconscious structures a causal experiment, or look at the research behind the method. For a specific release under deadline, the fastest path is to talk to the team about the audience segment and comparison. General background on the company is on the about page. --- # A Faster MMM Pipeline Doesn't Answer Whether the Numbers Are Causal URL: https://subconscious.ai/blog/methods-and-validation/faster-mmm-pipeline-doesnt-answer-whether-numbers-causal Published: 2026-08-07 Marketing Mix Modeling teams spend most of their time wrangling data: pulling spend, impressions, and conversions from a dozen ad platforms into one schema before a model ever sees them. A newer connector pattern removes that friction. Standardized data pipelines can hand a Bayesian MMM tool clean, wide-format tables in a few lines of code instead of weeks of custom ETL. That's a real gain. But it answers a different question than the one a budget decision needs answered. ## What does this pipeline actually do? The pattern, as implemented in the open-source PyMC-Marketing package, has five phases: - Connect to the ad platforms. - Standardize the schema with dbt. - Load the result into PyMC-Marketing. - Run inference through a Bayesian MMM. - Act on what it produces. A dbt-based ad-reporting package harmonizes spend, impressions, clicks, and conversions from platforms like Google, Meta, LinkedIn, TikTok, and Amazon into one unified schema, and supports unioning multiple accounts or markets while preserving a lineage field back to each source. PyMC-Marketing then supplies loader functions, process_fivetran_ad_reporting and process_fivetran_shopify_unique_orders in the reference implementation, that turn those standardized tables directly into the wide, date-indexed design matrix and target series a Bayesian MMM expects: ### What happens once the loaders feed the model? Once the data is loaded, the MMM itself estimates channel effects with posterior uncertainty, models carryover (adstock) and saturation, and supports budget-allocation scenarios, all standard Bayesian MMM capabilities documented in the PyMC-Marketing data module. The underlying dbt package for ad-reporting standardization is documented at fivetran/dbt_ad_reporting. ## What step does this pipeline leave out? Every stage in that chain (connect, standardize, load, infer) is about getting from raw platform exports to a fitted model faster. None of it changes what the model is fitted on: historical spend and historical outcomes, correlated in time. A Bayesian MMM targets a causal media effect, but only under assumptions the pipeline doesn't verify: no unmeasured confounding, correct adstock and saturation form, and spend that isn't set in anticipation of demand. Historical spend is typically set exactly that way, and channel spends move together, so attribution stays weakly identified and prior-driven. That distinction matters most at the exact moment the pipeline hands off to "act." A finance or growth leader looking at a cleaner, faster MMM read is still looking at an estimate whose causal identification hasn't been verified. The pipeline doesn't supply causal identification or confirm that reallocating budget based on the model's channel attribution will produce the outcome the model implies. Acting on that read at pipeline speed doesn't reduce the risk of a wrong reallocation. It just lets a team reach the wrong number faster. ## Where a causal check belongs in this workflow The fix isn't to distrust MMM outputs wholesale. It's to treat "infer" as a hypothesis about which budget move should work, and to test that specific move before it drives spend, the same discipline PyMC-Marketing itself supports through experiment calibration, which feeds randomized incrementality results back into the model as priors on channel coefficients, addressing that identification gap directly; testing the specific move first is what supplies that calibration input. Subconscious runs controlled experiments on the customer-facing attributes behind the proposed move: given a specific reallocation the MMM suggests (shift spend from Channel A to Channel B, for example), a discrete choice experiment on the message, offer, or price driving that channel's performance estimates the causal effect of those attributes with a confidence interval, rather than reading it off historical correlation. When the decision is high-stakes enough to justify it, that same causal question can also be tested with real human participants; transportability from a simulated population to human respondents is a separate empirical question, and human stated-choice results run high relative to actual behavior (hypothetical bias). ## What this means for an MMM-driven budget decision A faster pipeline is a legitimate improvement to data plumbing, not a substitute for testing the action its output implies. Treat the model's channel attribution as a candidate hypothesis, not a verdict, and validate the specific reallocation before it moves budget, particularly for calls a finance leader will need to defend after the fact. Subconscious's case studies cover how that validation step works in practice; a demo walks through applying it to a specific MMM-driven budget call. [figure: Five steps: connect, standardize, fit MMM, then a branch. Going straight from model to "act on budget" is marked risky; a test-the-reallocation step sits between the model and the decision.] --- # Why a Flat Choice Model Gets Cannibalization Wrong URL: https://subconscious.ai/blog/methods-and-validation/why-flat-choice-model-gets-cannibalization-wrong Published: 2026-08-07 A consumer-goods pricing or revenue-growth leader planning a new product launch needs to know one thing before committing trade spend: will this product mostly take share from close substitutes in its own segment, or will it pull incremental volume from across the whole category? Get that wrong and the launch budget goes to the wrong shelf. ## The proportional-substitution trap A basic discrete choice model treats sales as the outcome of many individual purchase decisions. Each product has a utility, driven by attributes like price and shelf availability (All Commodity Volume, or ACV). Utilities convert to purchase probabilities through a softmax function that includes an outside (no-purchase) option, and the model estimates each attribute's association with sales volume. This structure supports a counterfactual simulation, not a causal estimate: price and ACV are set by the firm and correlate with the unobserved demand shocks in the error term, so running the model forward with a product's availability set to zero and comparing that counterfactual to the observed outcome estimates the associated, not causal, sales impact of introducing (or removing) the product. The trap is in what the flat version of this model assumes about the error term, the catch-all for unobserved reasons a customer picks one product over another (a coupon, a craving). A flat multinomial logit model assumes those unobserved preferences are independent across products. The consequence is mechanical: when a new product enters, the model predicts every existing product loses market share in exact proportion to its starting share. That is rarely how real substitution behaves. A new kids' toothpaste is far more likely to pull volume from other kids' toothpaste than from an adult whitening line. A flat model that assumes proportional substitution understates cannibalization within a product's own segment and overstates the incremental volume it draws from unrelated competitors, misdirecting launch investment and trade spend. ## How does nesting products by segment change the substitution prediction? A nested (hierarchical) logit model addresses this by grouping products into nests, typically by segment or brand, that are meant to capture the unobserved factors driving substitution. The nested logit model computes the overall choice probability in two steps: the probability of selecting a nest, then the conditional probability of selecting a specific product within that nest. Within-nest choice probabilities are governed by a shared scale parameter that determines how strongly substitution concentrates among nest members relative to the market as a whole. Structured this way, and depending on the estimated within-nest scale parameter, a counterfactual that removes one product can predict that its nest-mates absorb most of the resulting share shift, while products outside the nest are less affected; as that parameter approaches the flat-logit limit, the nested model reproduces the same proportional substitution it was meant to correct. That pattern, close substitutes cannibalized first, is the one launch teams actually see, and the one a flat model cannot produce. ## Why does the nest structure have to be chosen, not assumed? The nested model's realism depends on getting the nest structure right. An analyst has to decide which grouping (segment, brand, sub-brand, or some hierarchy of these) reflects how consumers actually substitute; the wrong choice still produces a model that misattributes cannibalization, just along a different axis. Building this kind of hierarchical structure to arbitrary depth while keeping estimation numerically stable is itself a nontrivial part of the modeling work, detailed in PyMC Marketing's documentation on nested logit and non-proportional substitution patterns (PyMC Marketing, "Nested Logit and Non-Proportional Patterns of Substitution"). Nest choice is a judgment call that depends on category knowledge, which is also why an automated, always-on cannibalization matrix across an entire portfolio is not a reasonable standing output: a model that silently assumes one nest structure can mislead as confidently as the flat model it replaces. The literature on new-product cannibalization versus market stealing bears this out empirically: evidence from shopping-bot data shows that the two effects behave differently and neither is safe to assume away (Sinha, "New Model Introductions, Cannibalization and Market Stealing: Evidence from Shopbot Data," The Manchester School). ## Where does this fit in a launch decision? Subconscious's causal experimentation can compare launch alternatives: test a proposed product's positioning, price, or segment against holdout conditions and estimate directional causal effects for a defined population before committing capital. A segment-aware substitution matrix answers a narrower question than a general launch comparison, so it should be scoped as its own study design, with the nest structure proposed and checked against category knowledge, rather than treated as a standing report a platform generates automatically. When the question calls for real-world confirmation, Subconscious can also test or validate studies with real human participants, without changing the underlying causal question. See how this fits into a study design. ## Limitations and failure conditions - A flat discrete choice model works when segments genuinely don't compete for the same occasion, but it silently forces uniform substitution any time they do. - A nested logit model only fixes this if the nest structure (which grouping, and in what order) matches real consumer substitution behavior; a mis-specified hierarchy still misattributes cannibalization, just less obviously. - Neither model replaces observed, consumer-level purchase data where it's available. Both are built for the common case in consumer goods: manufacturers who see aggregated product-level sales, not individual purchase records. - Treat a specific nest structure and its cannibalization estimates as the output of one well-scoped study, not a self-updating matrix across the product line. [figure: Two columns: a segment-first nest groups kids' products together; a brand-first nest groups one brand's products together. Same launch, different predicted cannibalization pattern per hierarchy.] ## Next step Compare a proposed launch's segment fit and pricing against a defined population using Subconscious's leaderboard of causal effects, or read more on how a study moves from simulated comparison to real-human validation. [figure: Two columns. Left, flat logit: every product's bar shrinks in proportion to its starting share when a new one enters. Right, nested logit by segment: bars in the new product's nest shrink sharply, other nests barely move.] --- # AI Social Listening Tells You What Happened. It Can't Tell You What to Do Next. URL: https://subconscious.ai/blog/use-cases/ai-social-listening-tells-what-happened-cant-tell-what-next Published: 2026-08-07 A sentiment dashboard shows a spike, a cluster of negative posts, or a rival gaining share of voice. It cannot tell a brand, insights, or communications lead which of three possible responses their broader target audience would actually accept. That gap is the real decision: treat the listening data itself as enough to act on, or run a controlled test of the candidate response before committing budget or making a public statement. ## What does AI actually add to social listening? Modern listening tools (the category includes vendors such as Brandwatch, Talkwalker, Sprout Social, and Meltwater) use natural language processing to read sentiment in context, unsupervised clustering to group thousands of posts into themes, and anomaly detection to flag when mention volume breaks from a historical baseline. Large language models now summarize that volume into an executive brief instead of a raw spreadsheet. All of this is detection. It tells a team what has already been said, by whom, and how much. None of it tells a team what a specific audience would do if shown a message, product change, or crisis statement that has not been published yet. ## Why the data itself can't answer the "what next" question Social listening only sees people who chose to post publicly. Research on stance detection finds platforms carry a small, vocal share of users who generate most visible content, while a much larger, structurally passive population stays silent on any given topic (Zhu et al., 2024, EPJ Data Science). A dashboard built from public posts inherits that skew, reporting the vocal minority's reaction, not the reaction of the audience a brand actually needs to reach with its response. A second, structural limit sits underneath the sampling one: the people in a listening feed never agreed to be asked anything. A team can observe what they already wrote, but it cannot put a new claim, price, or statement in front of them and record a reaction, because that requires consent to participate in research, not just a public post to scrape. ## Turning a detected signal into a tested response The fix is not a better dashboard. It's pairing detection with a controlled test of the response before it ships. Subconscious runs randomized experiments: a specific claim, message, or product concept is placed in front of a defined audience, and the test estimates which version changes the outcome, a different question from tallying what people already said. ## What does this workflow require, and what doesn't it replace? This tool publishes its own scope limits so a buyer can check them before relying on the result. Running a test like this assumes the team already has candidate responses: it is not a tool for discovering that a conversation exists, only for deciding what to do once one has been found. And the confidence-interval strength of any single result depends on how that specific study is designed; it is not automatic on every output. When the decision is high-stakes enough that the team needs more than a directional read, Subconscious can test or validate the same study with real human participants without changing the underlying causal question, moving from a simulated first pass to human evidence on the same claim, not a different one. That path matters most for a public statement or a claim the team is not willing to walk back. Stating exactly what a result proves keeps the claim honest for anyone checking it against the leaderboard. Real-human validation answers "does this hold up with real people," not "did this happen in a clinical trial or a market-representative sample." A causal test of a message is still a test of that message; it does not become a usability study, a regulatory submission, or proof of market performance just because a human panel ran it. ## Where does this fit? Social listening stays the detection layer, the way a team first learns something changed. Subconscious sits downstream of it, as the layer that turns "something changed" into "here is the response that works" before that response goes public. See how Subconscious runs a study, the current research and leaderboard results, or talk to the team about a specific response you're weighing. [figure: Four-row table comparing listening (detects spikes, surfaces words, can't test unpublished material) against Subconscious (estimates which framing moves an audience, compares statements before publishing).] --- # Data Twin, Self-Serve Persona, or Causal Test: Which Synthetic-Audience Method Fits the Decision URL: https://subconscious.ai/blog/comparisons/data-twin-self-serve-persona-causal-test-which-synthetic-audience-method Published: 2026-08-07 Synthetic-audience methods fall into three categories that answer different questions: a data-grounded digital twin built from an organization's own audience data, a generative persona tool built from descriptions, and a causal behavioral platform that tests which action changes an outcome. For a consequential pricing, messaging, launch, or positioning decision, category fit matters more than feature count. The wrong method can produce a credible-looking answer without resolving the decision. ## Start with the decision, not the interface The useful comparison is not enterprise versus self-serve but known-audience fidelity versus exploratory conversation versus causal evidence. Each has a legitimate job, but proof from one category does not answer the question posed to another. [figure: List of three methods: a data twin needing first-party data shows what a known audience would say; a persona tool shows what a described persona might say; a causal experiment shows which action changes an outcome.] ## When do you need known-audience fidelity? Electric Twin presents its product as synthetic audiences for enterprise research. A data-grounded twin makes an existing audience more queryable: the organization supplies deep first-party subscriber or customer data, and the platform builds and calibrates a replica of that known group. That category fits a publisher, brand, or research team asking how its own defined audience is likely to react. Its value depends on the quality and coverage of the supplied dataset. Without it, the team pays for onboarding and calibration it cannot complete while the decision window closes. A queryable replica does not become a controlled action test because its answers are grounded in first-party data. If the decision is between two prices or claims, the buyer should ask whether the method estimates the effect of changing the action, not only whether the replica resembles the known audience. ## When a persona conversation is enough A generative persona begins with a description rather than an ingested audience dataset. For example, a brief might specify a 45-year-old German procurement manager. That detail can focus a conversation and surface objections, language, or hypotheses worth investigating. The output remains dependent on the description and prompt. It does not establish how a defined population would respond or identify the causal effect of one action over another. For exploratory work, that may be sufficient. For a launch-critical pricing or messaging call, a plausible opinion can look like validation while leaving the commercial decision untested. ## Which action actually changes behavior? Subconscious is a causal behavioral platform. It runs controlled experiments on simulated markets to estimate which price, message, feature, or claim changes a defined outcome. It is built for buyers asking which action is most likely to move behavior, not what the audience looks like or what a persona might say. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market (the causal fidelity paper). A number without its limits is marketing. This is the boundary on this one: study design determines what uncertainty, segment variation, and decision outputs can be supported, and those are not identical across every engagement. Subconscious also does not offer a packaged workflow that ingests first-party subscriber data and turns it into a queryable replica of that specific audience. A data-grounded twin remains the better category when fidelity to a known audience is the requirement. ## Do not transfer proof across categories Electric Twin publishes a first-party account of its approach to measuring synthetic-audience accuracy. That proof should be judged against its intended use: reproducing a particular audience from supplied data. Persona plausibility should be judged against the usefulness and stability of the generated hypotheses. Causal replication should be judged against whether a simulated experiment reproduces the direction and outcome of a human study. ## Make the procurement call Name the question before comparing contracts or interfaces: - Choose a data-grounded digital twin when the organization has deep first-party data and needs a queryable model of that specific audience. - Choose a generative persona tool when the team needs exploratory conversations and hypotheses from described persona types. - Choose a causal behavioral experiment when the team must compare defined actions for a defined market outcome. For a consequential pricing, messaging, launch, or positioning choice, review the public replication results, see how a causal study is structured, or bring the decision to a demo. --- # What Is a Research Panel? Panels vs. Controlled Experiments URL: https://subconscious.ai/blog/comparisons/what-research-panel-panels-vs-controlled-experiments Published: 2026-08-07 A research panel is a group of people, recruited ahead of time, who agree to answer research questions over a defined period. Unlike a one-off survey, a panel lets a research team reach the same people repeatedly, track how their answers change, and build a demographic and behavioral profile before the first study starts. Panels are usually managed by a research agency, panel company, or internal team that maintains membership and incentives. ## The panel types buyers commission most often - Consumer panels. General-population members recruited to represent a target market, used for brand tracking and product testing. Major providers such as Nielsen, Ipsos, and Kantar run panels numbering in the hundreds of thousands across markets. - B2B panels. Professionals screened by industry, job function, company size, and seniority. Harder to maintain than consumer panels: qualifying professionals in a niche industry are scarcer and less willing to join. - Expert panels. Domain specialists such as doctors, engineers, financial advisors, and other subject matter experts, recruited when the research question needs professional judgment to answer meaningfully. Usually small, hundreds of members rather than thousands, and expensive to maintain and access. - Online access panels. Large, pre-recruited pools who complete surveys online for incentives. With millions of members under management, providers can put a study in front of a representative sample in short order. That speed can cost data quality: respondents who join repeatedly to collect incentives skew results over time, a risk documented in AAPOR's review of online panel methodology. - Longitudinal panels. The same participants tracked over months or years, used to measure attitude change or the long-run effect of a campaign. - Internal customer panels. A company's own customers, recruited from its user base for recurring product feedback and satisfaction tracking. These panelists are actual product users, which raises response quality for that specific use. ## Why panel research costs what it costs A number without its limits is marketing. Panels are valuable: real people with known, verified characteristics. That value comes with structural costs: - Cost. Large-scale consumer studies can run into tens of thousands of dollars, and access to well-screened niche or expert populations adds cost quickly. - Speed. Members are already recruited, but the study still has to run: fielding, waiting on responses, and analysis take days or weeks, slower than pricing, launch, and messaging decisions typically allow. - Sample quality. Professional respondents who game incentive systems degrade data quality over time and require constant panel management to correct. - Depth. Most panel research runs on surveys, which cap how much a single answer reveals about the reasoning behind it. - Fixed population. A panel can only represent people who exist today and can be recruited. It cannot speak for a hypothetical customer segment or a market that has not formed yet. ## Should I recruit a panel now or run a controlled experiment first? Not panel versus no panel. The choice is whether the next question needs statistically validated, real-respondent data immediately, or whether a controlled experiment can narrow the option set first, with panel-grade validation reserved for the decision that survives. Naming the failure mode here is what lets a buyer check for it before committing budget. Picking the wrong instrument has a real cost. Paying panel-level time and money for a question that only needed a directional read wastes weeks and budget. Shipping a launch or pricing decision on an unvalidated read, when the decision actually warranted real-human confirmation, risks a costlier mistake downstream. [figure: Decision path from a research question: regulatory or representative questions go to a recruited panel; directional reads go to a controlled experiment first; both converge on real-human validation before shipping.] ## Where does a controlled experiment fit before a panel? Subconscious runs controlled discrete-choice experiments on a simulated market to estimate which action moves an outcome, before a team commissions a full panel study or in place of one for questions that do not need statistical validation. The misses sit on the same leaderboard as the hits, so a buyer can check both. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, documented in the causal fidelity paper. It is a validation result, not a guarantee for a new market. For decisions big enough to justify it, Subconscious can test or validate studies with real human participants, moving from simulation to real-human validation without changing the causal question under test. It is not a recruitable panel, not a synthetic-respondent replacement for regulatory or compliance-grade panel research, and not a claim that every population Subconscious can simulate is available to recruit. ## How do I choose the right tool for my next study? Use a recruited panel when the study needs statistical validation, tracks change over a defined population, or falls under a compliance requirement that mandates real participants. Use a controlled experiment first when speed matters, the population is hard or expensive to recruit, or the goal is narrowing options before a panel study confirms the winner. See how the two connect in a live experiment or review the leaderboard evidence behind the replication figure above. --- # Persona Chat vs. Controlled Discrete-Choice Experiments: Choosing the Right Research Method URL: https://subconscious.ai/blog/comparisons/persona-chat-vs-controlled-discrete-choice-experiments-choosing-right Published: 2026-08-07 Two things get lumped together as "AI market research" that measure completely different things: an open-ended conversation with a chatbot persona, and a controlled discrete-choice experiment run against a defined population. The chat gives you a plausible-sounding impression. The experiment gives you a measured effect with a confidence interval. Confusing the two is the actual risk, not which vendor you pick. ## The decision this comparison is actually about A research, product, or marketing leader has been running ad hoc persona conversations in a general-purpose chatbot to get a read on a customer segment before a positioning, messaging, or product decision. The question is not "which chatbot persona is better." It is which category of tool the decision needs: an open-ended interview that produces one plausible narrative, or a controlled experiment that measures which alternative changes stated choice for a defined population, with a confidence number attached to the read. Getting the category wrong is the expensive mistake. Treating a single persona conversation, or a vendor's self-reported accuracy figure, as a measured result can send budget toward a message, price, or feature direction that a rigorous test would have shown does not move the target population. ## What an open-ended persona chat produces Prompting a general-purpose chatbot to "act as" a customer segment produces a conversational impression. It is fast and useful for early brainstorming, not a measurement: - The persona is a prompt, not a population. Two people writing the same brief into the same chatbot will get two different personas, because the persona's behavior depends on how the prompt was written, not on a defined sampling frame. - The output is a narrative, not a comparison. A chat produces an opinion in response to a question. It does not compare two or more concrete alternatives (a price, a message, a feature) and report which one changed the audience's stated choice, or by how much. - There is no confidence interval. A chat response reads as confident regardless of how reliable it actually is. Nothing in the output tells you whether the "reaction" would hold up against a second independent sample. Chat-based exploration is a reasonable way to think out loud before the decision is defined. It is a poor substitute for evidence once real budget is on the line. ## What a controlled discrete-choice experiment measures A discrete-choice experiment starts from a different premise: define the population, define the alternatives, and measure which alternative changes what people say they would choose. Subconscious runs these as structured causal experiments (McFadden discrete choice and mixed logit models) over synthetic populations, and returns the measured effect of each alternative together with a confidence interval, not a single narrative reaction. - A defined population, not an improvised persona. The population a test targets is specified up front and held constant across the alternatives being compared. - A controlled comparison, not an open-ended conversation. The experiment tests two or more concrete alternatives against each other and reports which one wins and by how much. - A measured effect with a confidence interval, not a plausible-sounding answer. The result states how much a given alternative changes the outcome, and how much uncertainty surrounds that number. ## Comparing the three ways teams actually make this call The first two methods are useful for exploration. Only the third produces a number a team can defend when the decision is expensive to get wrong. ## Where each method actually fits Open-ended persona chat is the right tool when a team is thinking out loud, drafting early messaging options, or exploring a segment before the decision itself has been defined. Its maker positions it that way, for broad, ambitious knowledge work rather than as a research-measurement tool (OpenAI, "ChatGPT is now a partner for your most ambitious work"). That framing fits drafting and brainstorming, not measuring which alternative changes stated choice. A controlled discrete-choice experiment is the right tool once the team has narrowed to concrete alternatives (two or more prices, messages, or feature framings) and needs to know which one actually changes the target population's stated choice, with enough confidence to commit budget behind the answer. Research and the leaderboard show how these experiments are structured and validated across studies. [figure: Two-column comparison scored on population, output, and confidence signal. Chat: improvised prompt, narrative reaction, no confidence. Experiment: defined population, measured effect per alternative, confidence interval.] ## From a simulated experiment to real-human confirmation Naming the next verification step is part of publishing a result honestly. A team can move from a synthetic discrete-choice experiment to a recruited real-human study on the same causal question without changing what is being tested; only the respondent source changes. That step matters when a decision is large enough to justify a second, independent check before commitment. ## Limitations A controlled discrete-choice experiment on a synthetic population is a decision-support signal, not a substitute for direct human research ahead of a high-stakes launch or a major repositioning call. It narrows which alternatives are worth testing further and quantifies the uncertainty in that narrowing; it does not replace judgment, and it is not a clinical trial or a usability study. ## Frequently asked questions ### Is a chatbot persona conversation ever good enough? For early exploration and brainstorming, yes. Once the decision comes down to two or more concrete alternatives and the cost of choosing wrong is real, a narrative reaction from one improvised persona is not enough evidence to act on. ### What makes a discrete-choice experiment different from a persona chat? A discrete-choice experiment defines the population and the alternatives up front, then measures which alternative changes stated choice and by how much, with a confidence interval attached. A persona chat produces one narrative reaction to one prompt, with no defined population and no confidence signal. ### Do teams need to choose only one method? No. Open-ended exploration is often how a team narrows down which alternatives are worth testing. The controlled experiment comes after that narrowing, once the decision is concrete enough to measure. See how we work for the method, or book time to scope a specific comparison. --- # What Market Researchers Should Stop Doing Manually URL: https://subconscious.ai/blog/buyers/what-market-researchers-should-stop-doing-manually Published: 2026-08-07 A stakeholder wants a clear answer. An AI-generated draft lands on the desk before the researcher has finished going through the raw data. Somewhere in there, a manager floats the idea of letting AI handle the first pass entirely. None of that means research is disappearing: the U.S. Bureau of Labor Statistics projects employment of market research analysts to grow 7 percent from 2024 to 2034. It means the mechanical layer of the job (formatting, cleanup, first-pass synthesis) is shifting to AI assistance, while the decisions built on top of that layer still have to hold up. The real question for a research leader is narrower than "how much AI should we use." It is: which tasks move to an AI-assisted or synthetic exploratory stage this quarter, and which decisions still require real respondent data or fielded validation before a claim leaves the building. ## The mechanical layer is not the valuable layer The old advantage in research was partly access: knowing how to field a study, clean the responses, and package a finding. AI narrows that advantage. More people can now draft a survey, summarize a transcript, or generate a first-pass read on a concept. Once anyone can generate a plausible-looking answer, judgment shows up in a different place: deciding which of those answers can be trusted, and defending that call. For a research team, that means framing the question before any AI tool touches it, then attaching the caveats once the tool has produced something: what decision is being made, what evidence would change it, and where an unvalidated read could mislead the business if it reaches a deck unlabeled. ## What decides whether a task should be automated? Every research task can be sorted by one question: what does it cost if this specific output turns out to be wrong? Formatting a summary incorrectly creates rework. Shipping a pricing or launch decision on an unvalidated read can damage the decision and the team's credibility with leadership. That cost, not the subject matter, is what should decide whether a task stays in exploration or gets escalated to validation. A rough workflow: - Exploration: draft hypotheses, alternative framings, and objections as material for review, with AI assistance where it's useful. - Directional testing: compare concepts, messages, or positioning options to determine what a fielded study must validate. - Human review: check the audience definition, the neutrality of the prompt or stimulus, and whether the read is grounded in something real. - Validation: bring in real respondent data, behavioral data, or fielded research before a decision that is expensive or goes external. Skipping straight from exploration to an external claim is the failure mode, not the use of AI itself. ## Where does controlled synthetic testing fit in market research? A number without its limits is marketing. This is where a platform like Subconscious is useful, and where it is not. Subconscious runs controlled discrete-choice experiments against a simulated population and returns causal effects with confidence intervals, a way to test how a specific audience segment responds to a concept, message, or pricing change as directional evidence before fielded validation. It is a directional-testing layer inside the workflow above, not a replacement for the validation step: when a decision is expensive or public, Subconscious can test or validate the same causal question with real human participants; the causal question does not change between the simulated run and that validation step. The causal fidelity paper reports that the best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. The misses sit on the same leaderboard as the hits. It is a validation result, not a guarantee for every question or audience: a directional read from a simulated population is not the same evidence as a result from real respondents. Subconscious does not automate report writing, first-pass coding, or desk research summaries; those stay human-reviewed work. ## What mistake erases the AI advantage in research? The failure mode is automating the visible deliverable and leaving the messy middle untouched: the team gets a fluent answer, the deck needs a conclusion, and an unlabeled directional finding reaches a stakeholder as if it were validated evidence. AI can produce useful output. It cannot decide on its own whether that output is valid for the decision in front of a team. The fix is not more caution about using AI. It is stating the limit as part of the deliverable: name what the AI-assisted or synthetic work covered, flag what it left untouched, and spell out the next validation step. Teams that do this consistently don't sound less confident; they sound more precise, because they can explain where their confidence ends. [figure: Diagram: one causal question held constant across two paths. One is a simulated run giving directional confidence; the other is real-respondent validation giving decision-grade confidence.] ## A one-month planning exercise Pick one recurring task with a live decision behind it, not a hypothetical one. Write the business decision the task feeds in one sentence, then define the audience and the cost of getting it wrong. Route the task through exploration, directional testing if useful, human review, and validation, in that order, and label the output at each step before it goes further. For one month, track that workflow against a plain before-and-after log of the decision and outcome. The useful artifact is not a list of tools adopted. It is a working system that shows where AI-assisted exploration and human judgment each did their job, and where a claim still needed real evidence before it left the room. Teams weighing where synthetic testing fits into that system can see current replicated study results or read how the validation step works before scoping a first study. [figure: Four stages: Exploration drafts hypotheses with AI; Directional testing compares concepts via synthetic experiments; Human review checks audience and neutrality; Validation uses real respondent data.] --- # Should Your Agency Build a Research Practice, or Stay Execution-Only? URL: https://subconscious.ai/blog/buyers/should-agency-build-research-practice-stay-execution-only Published: 2026-08-07 Build the practice only if the agency can define a client action, compare alternatives under controlled conditions, and defend the limits of the result. Otherwise, outsource the work or remain execution-only. A weak research offer does more damage than no offer because it turns an uncertain recommendation into a claim the client expects the agency to prove. ## Retainers reward continuing value, not a new label A research practice can strengthen a retainer when it gives the client a repeatable way to decide what to do next. A number without its limits is marketing, so this limit gets stated directly. The retainer itself does not prove that research caused better retention. A 2026 report lists annual churn at 18% for retainer-based agencies and 42% for project-based agencies (Focus Digital, "Average Marketing Agency Churn: 2026 Report"). Treat those figures as industry benchmarks, not as a forecast for an agency that adds a research line. A historical planning example put added contract value at $10K-$50K per engagement depending on scope. The misses sit next to the hits so a buyer can weigh both. That range is not a current Subconscious price, a market average, or an expected result. An agency still has to model its own labor, method review, account economics, and demand. ## The offer should answer which action changes behavior The unit of value is not a report format. It is a tested decision. The agency should be able to name: - the action the client may take, such as changing a price, claim, or launch message; - the alternatives being compared; - the buyer group whose behavior matters; - the outcome that would change the client's decision; - the uncertainty and assumptions that limit the recommendation. A generated-response survey can summarize plausible reactions. Naming the failure mode here lets a buyer check the claim against it. It cannot, by itself, establish that changing one action caused a behavioral outcome to move. A causal experiment is designed around that intervention and comparison. ## Use a four-stage plan, with a gate at each stage The windows and staffing quantities below are historical planning examples, not current Subconscious timelines, staffing requirements, or delivery commitments. The gates matter more than the dates. An agency should scale because the method produces a defensible record across more than one client decision, not because a calendar window ended. [figure: A four-stage horizontal path: Foundation, Pilot, Scale, Differentiate. Each stage box has a gate beneath it that must pass before the arrow continues to the next stage.] ## Proof has to match the promise The first proof should show a real decision, the tested alternatives, the result, and the boundary around that result. One documented pricing case identified $60 as the price for maintaining market share and reported approximately $65,000 in traditional research costs avoided (Subconscious case studies). Publishing this limit is what makes the number usable instead of marketing. These are case-specific outcomes that do not guarantee the same result for another client or agency. Subconscious's research approach is relevant when the client needs to test which product, pricing, messaging, or go-to-market action is more likely to move a defined outcome. Where the decision warrants it, Subconscious can test or validate studies with real human participants without changing the causal question. A modeled audience definition and recruited human participants are separate capabilities and should not be presented as one. ## Three questions determine whether to build or outsource ### Do clients bring repeated intervention decisions? Build when several clients repeatedly choose among consequential actions and the agency already owns the surrounding strategy. Outsource when demand is sporadic or when an independent method partner would carry more credibility. ### Can one person refuse an unsupported claim? Assign a method owner before selling the offer. That person must be able to reject a study that lacks a defined comparison, uses an outcome the method cannot support, or turns directional evidence into certainty. If no one has that authority, the practice is not ready. ### Can the agency state where the offer stops? The agency remains responsible for pricing, staffing, quality control, and client interpretation. Subconscious does not currently confirm a packaged agency-reseller program, named pricing tiers, an hour-by-hour delivery workflow, or a specific revenue projection. A causal study also does not automatically prove real-market performance. High-stakes decisions may require real-human validation or evidence from the market after the client acts. ## Scope the first offer around one live decision Start with one client action that has clear alternatives and a costly downside if chosen poorly. Write the decision brief before writing the sales page. Review the study workflow, then scope a pilot only after the agency can state the action, comparison, outcome, proof standard, and stopping rule. --- # AI Purchase Intent Research: Testing Trade-Offs Before You Launch URL: https://subconscious.ai/blog/use-cases/ai-purchase-intent-research-testing-trade-offs-before-launch Published: 2026-08-07 A product marketing or GTM leader greenlighting a launch, pricing tier, or competitive claim needs to know how target buyers actually trade the new offer against what they use today, not a stated purchase-intent score. Launch spend, sales messaging, and pricing commitments built on inflated stated-intent numbers rarely survive contact with real budget constraints and switching costs. [figure: Two-column comparison: left, a stated-intent survey giving one unconstrained score; right, a discrete choice experiment forcing the offer against a named incumbent under a fixed budget, yielding a causal effect.] ## Why stated purchase intent overstates real demand Asking someone to rate purchase likelihood on a 5-point survey scale captures a fleeting sentiment, isolated from budget, existing alternatives, and the complexity of the evaluation itself. The gap between what people say and what they do is where most launches fail (Journal of Economic Behavior & Organization). In a survey, respondents are optimistic: the concept sounds appealing and they want to be helpful. Actual purchases get made more cautiously, constrained by a fixed budget, existing habits, and a pull toward whatever the buyer already uses. A stated-intent question misses this friction because it never forces a trade-off: it asks for an opinion, not a decision. ## Testing the decision instead of the opinion Buyers don't evaluate a product in isolation; they weigh it against a specific incumbent alternative and a fixed budget. A controlled discrete choice experiment puts that same trade-off in front of a target audience segment: the new offer against the named alternative, with price, features, and switching cost all forced into the decision. The output is a causal effect on choice, reported with a confidence interval, not a single intent score. That structure supports the trade-offs a purchase decision turns on: - The evaluation itself. The experiment can hold constant a target buyer's stated pain points, budget, and current tool stack, then measure how the new offer changes their choice relative to what they already use. - A named competitive set. Testing against an actual incumbent, not a hypothetical, surfaces which specific attribute (price, a feature, a support model) moves the choice in either direction. - Price sensitivity. Instead of asking whether a price point is acceptable, the experiment varies price across the discrete choice design and measures the point at which demand for the offer drops, holding the rest of the decision fixed. ## Where this changes a launch decision New product launches. Before finalizing a roadmap, test the proposed product against the incumbent it would need to displace, and measure the causal effect on choice, not simply whether the audience is favorable to the concept. Pricing and packaging. Compare pricing tiers and packaging models, such as usage-based against flat subscription, and measure how the causal effect on choice differs by segment. Competitive positioning. Rather than waiting on a quarter of closed-lost reasons, test the specific trade-off between the offer and the competitor it's losing to, and see which attribute is driving the result. Go-to-market prioritization. Identify which features or messaging points produce the largest causal effect on choice across market segments, so marketing spend follows what actually moves demand. [figure: Four launch decisions each paired with the trade-off a discrete choice experiment forces: offer vs. incumbent, tier vs. tier, offer vs. named competitor, feature vs. feature by effect on choice.] ## Limitations Publishing the causal effect without its limits reads as marketing copy. A discrete choice experiment measures the causal effect of the trade-offs built into its design. It does not replace watching an actual purchase happen in the market or tracking revealed sales behavior over time. Treat it as evidence for the decision in front of you, whether to greenlight a launch, a price, or a positioning claim, not as a forecast of total market performance. When a launch decision is big enough to warrant it, the same causal question can move from a simulated audience to real-human validation without changing what's being tested, so the comparison between simulated and human-baseline results stays apples to apples. The fastest way to see the difference from a stated-intent survey is to run one comparison directly: test a specific pricing or positioning trade-off against the incumbent it needs to beat, and look at the confidence interval instead of a single score. --- # Recruited-Incentive Research vs. a Causal Simulated Experiment First URL: https://subconscious.ai/blog/comparisons/recruited-incentive-research-vs-causal-simulated-experiment-first Published: 2026-08-07 A product or research-ops lead with a recruiting budget faces the same question before every round of paid, incentive-based sessions: spend that budget now on real participants, or run a causal experiment on a simulated market first to find out which questions are worth paying people to answer. [figure: Flowchart from a raised question. If a real human is required, it goes to a paid recruited session. Otherwise it passes a flat-cost simulated screen; only passing questions reach recruiting.] ## The two things being compared are not substitutes Recruited research pays real people, screened and scheduled, to sit through a session. The incentive is the cost of their time; the recruiting workflow is the cost of finding and confirming them. This is the right tool when a regulator, a stakeholder, or the method itself requires a real human: usability testing, ethnographic fieldwork, or any study where the record has to show an actual person responded. A causal simulated experiment runs before any of that spend happens. It tests actions, comparisons, and audience definitions on a simulated market to estimate which ones are worth carrying into a recruited round. It answers a narrower question than a full study: is this concept, message, or price point differentiated enough to justify the incentive and scheduling cost, or should it be cut or reworked first. The real decision is sequencing: which questions get answered with a simulated pass first, and which go straight to recruited participants because only a real session will do. ## Where does the incentive-recruiting workflow actually spend the budget? Per-session incentive research has two structural cost drivers that scale with volume. The first is the payment itself, sized to session length and audience difficulty. The second is the operational overhead of screening, scheduling, and covering no-shows, which a current industry guide to research incentives documents as a standard part of budgeting a recruited study rather than an edge case (The Ultimate Guide to User Research Incentives, User Interviews). Because both costs are per-session, the total scales roughly linearly with the number of questions a team wants answered, and published per-session and per-seat pricing tiers confirm that recruiting cost tracks volume rather than flattening out (User Interviews pricing, User Interviews). ## What does Subconscious do in this sequence? Subconscious runs controlled causal experiments on a simulated market first, then can test or validate the same question with real human participants when the decision calls for it. That lets a team spend its recruiting budget on the questions that survived the first pass, not on every question it started with. The proof behind the first pass: our best configuration reaches 87% of the measured human ceiling on one study, 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. A number published without its limits is marketing copy. It is a validation result, not a guarantee for a new market (the causal fidelity paper). ## Does Subconscious replace the recruited session it's meant to precede? Subconscious does not recruit, screen, or pay participants, and it does not replace real-respondent research where the method itself or a regulatory mandate requires an actual human session. It is an upstream causal experimentation layer, not a recruiting or incentive-payment platform. When the decision depends on validation, a team can move from the simulated experiment to a real-human test of the same question without changing what was being asked. ## Cost of getting the order wrong The misses go on the page next to the hits so a buyer can check the sequencing themselves. Run recruiting first on every question, and incentive spend plus multi-week scheduling burns on questions a first pass would have screened out or sharpened. Skip the recruited round on a question a regulator or stakeholder requires a real human to answer, and a directional simulated read gets mistaken for confirmation it was never designed to provide. Both failures are sequencing errors, not tool-selection errors. ## Next step Start with the questions on the research list and sort them: which require a real, recruited human session no matter what, and which are candidates for a causal simulated pass first. For the second group, see how the method works, check the research library for study examples, or book time to walk through a live experiment. --- # When Your MMM Gets the Channel Ranking Backwards URL: https://subconscious.ai/blog/buyers/when-mmm-gets-channel-ranking-backwards Published: 2026-08-07 A marketing mix model can rank two channels by ROAS and get the order exactly wrong, not close, inverted. Before a growth or marketing leader shifts budget on the strength of that ranking: has anything calibrated it against a real intervention, or is it running on spend and sales data alone? [figure: Five-step chain: a hidden demand signal drives one channel's spend and sales, inflating its estimated effect and reversing the ranking, until a lift test corrects it.] ## The failure mode: a hidden variable inflates one channel An MMM estimates each channel's contribution from historical spend and outcome data. That works cleanly when nothing else is pulling on both the channel and the result at the same time; it breaks when something is. In a published simulation, one channel's spend was also driven by a demand signal the model never saw, something correlated with when people were already inclined to buy. Because that hidden driver pushed up both the channel's spend and the outcome, the model credited the channel with an effect that wasn't fully its own. The result: the model ranked that channel as more effective than a second channel, when the underlying process that generated the data showed the opposite ordering (Media Mix Model Calibration With Bayesian Priors, Google Research). This is not a data-quality problem you fix by collecting more of the same kind of data. An unobserved confounder (a variable influencing both the channel and the outcome but absent from the model) stays invisible no matter how many additional weeks of spend and sales history get added. ## Why is the fix an intervention, not a better regression? The correction in the published example didn't come from re-specifying the model with cleverer priors. It came from small controlled interventions: deliberately changing one channel's spend for a period and measuring the resulting change in sales directly, independent of the historical spend pattern. Two such tests per channel, applied as constraints on the model rather than as another input to regress on, moved both channels' estimates back toward the values implied by the true data-generating process and reversed the ranking to the correct order (worked example replicated from an experiment-calibrated simulation study of media mix models, Juan Orduz). The general principle holds independent of any specific software implementation: an observational model answers "what happened," and only a deliberate intervention answers "what would happen if we changed this." ## What does this mean before the next budget reallocation? The decision a leader actually faces is not "trust the model or don't." It's narrower: does this specific ranking, the one about to move budget, rest on anything besides spend and sales history? If the answer is no, the honest move is to treat the ranking as provisional and test the channel the model says to cut before cutting it. This is the same logic behind running a controlled experiment rather than reading correlation off a dashboard: the question "what would sales do if we changed this channel's spend" only gets a trustworthy answer from something that changes the input and observes the result. Subconscious runs that kind of controlled test directly against the causal question at stake, and the same underlying test can move from a simulated read to real-human validation without changing the question being asked. [figure: A decision path from a model's ranking flagging a channel to cut. It branches on whether the ranking was checked against a real intervention: no leads to testing the channel first; yes leads to acting on the ranking.] ## Where does this example stop being evidence? The worked example above is a third-party technical illustration of a calibration principle, not a Subconscious case result. No accuracy figure or customer outcome from Subconscious applies to it. What it is useful for is the pattern: a plausible-looking ranking, an invisible variable, and a specific fix that required an actual intervention. See the current leaderboard for how that kind of testing gets evaluated in practice. --- # How to Vet a Creator Partnership Before Signing URL: https://subconscious.ai/blog/use-cases/how-vet-creator-partnership-before-signing Published: 2026-08-07 A creator partnership should be chosen on the creator-content concept, not reach alone. Compare finalists under the same buyer definition, product framing, and choice task. Then validate the preferred final creative with real people before committing the partnership fee and production budget. The cost of choosing poorly compounds. The creator fee is sunk. Production spend follows. The campaign can still read as forced or off-brand to the buyer it was meant to move. ## Why does choosing a creator partner require a causal test? Follower count, engagement rate, audience demographics, and past partnerships describe a creator's history. They do not establish what will happen when that creator presents this product with this message to this buyer. In a 2026 survey of more than 600 marketing professionals, 72.2% of respondents who answered expected influencer budgets to increase by 50% or more (Influencer Marketing Hub). The report also cautions that intent does not guarantee execution. The evidence available before signing answers different questions: The decision is not “Who is the best creator?” It is “Which creator-content concept causes the strongest relative buyer response for this campaign?” ## How do you build the comparison around one contract? A clean comparison changes the creator-content alternative while holding the buyer definition, product, offer, choice task, and outcome constant. - Define the target buyer. Use the traits that matter to the purchase decision. For a clean-beauty campaign, a planning example could focus on women ages 25-35 with relevant category interests. That range is an example, not a Subconscious requirement. - Represent each finalist fairly. As a planning example, assemble 3-5 representative posts per creator across the formats the campaign may use. Do not compare one creator's strongest work with another creator's weakest work. - Specify the campaign concepts. Pair each creator with the proposed framing, product placement, benefit, and call to action. General creator appeal is not the decision being funded. - Use the same choice task. Ask the defined buyer group to choose between alternatives under the same conditions. Record the outcome that would change the signing decision. This structure turns a vague preference exercise into an auditable, decision-specific experiment: a reviewer can see what changed, what stayed fixed, and which outcome determined the recommendation. ## Read the result as a directional decision Subconscious helps teams test messaging and go-to-market actions before committing capital. For creator vetting, the supported use is a causal comparison across creator-content alternatives for a defined buyer segment. The result should identify the stronger relative choice and the conditions under which it was observed. Confidence or uncertainty language belongs only when the specific study design supports it. The result is not a universal creator score. It answers the contract question that was tested. The Subconscious research approach is useful here because it keeps the decision tied to controlled alternatives rather than treating historical engagement as proof of future effect. [figure: Five-step path: define target buyer, represent finalists fairly, pair each with its concept, run the same choice task and outcome, validate the winning concept with real people before signing.] ## Why should fit, fraud, and performance be kept separate? The comparison supports one part of creator due diligence. It should not absorb questions that require different evidence. - Creator-content fit: The experiment compares buyer response to the message and creator combination. - Audience authenticity: Follower or audience fraud requires separate verification. - Campaign performance: The comparison does not predict an individual creator's views, clicks, or conversions. - Final creative: A preferred concept is not automatic proof of market performance. Subconscious can test or validate studies with real human participants. A team can move from the simulated experiment to real-human validation without changing the causal question. These boundaries prevent a relative concept result from being stretched into a claim about audience quality or guaranteed campaign outcomes. ## Put the signing logic in one decision memo A useful signing memo records: - the creator-content alternatives; - the target buyer and behavioral outcome; - the variables held constant; - the directional result and supported uncertainty; - separate audience-authenticity checks; - the condition that triggers real-human validation of the final creative. Bring the shortlist, representative content, proposed campaign framing, and the exact contract decision to the Subconscious study-scoping process. If the alternatives cannot be expressed as controlled choices, the partnership is not ready for this method. --- # Video-Avatar Production vs. Causal Message Testing: Where Should Pre-Launch Budget Go? URL: https://subconscious.ai/blog/comparisons/video-avatar-production-vs-causal-message-testing-where-should-pre-launch Published: 2026-08-07 A pre-launch team should fund causal message testing when the claim, positioning, or audience is still uncertain, and video-avatar production when the script is settled and the remaining job is to produce the asset. ## Start With the Open Question Video-avatar production answers, "How should we render this approved script?" Causal message testing answers, "Which message, claim, or action changes behavior, and for whom?" If the second question is unresolved, producing the asset first commits budget before the decision has evidence behind it. ## When Is Video-Avatar Production the Right Spend? Video-avatar tools are appropriate when a team needs a scripted video asset for training, localized product demos, spokesperson content, or marketing. The input is a written script. The output is a voiced video. Current vendor materials describe script-to-video creation and related production capabilities (product overview). Recent product updates add more production options, but those options do not provide evidence that the script's central claim will change audience behavior (May 2026 product update). A team producing dozens of localized versions may still need this category after its message is validated. ## When the Message Is Still Unproven Subconscious.ai is the causal behavioral platform. It runs randomized experiments on a simulation of the target market, validated against real human behavior, to show why people choose and which action moves the outcome. For a campaign, the action under test can be a message, claim, or positioning alternative. The output is evidence for the decision, not a finished creative asset. This matters when the team has several plausible scripts but no causal basis for choosing one. The experiment compares controlled alternatives before the creative format is fixed; the production team then executes against the selected message. ## One Budget, Two Different Outputs ## What Is the Expensive Sequencing Error? Producing a polished asset around an untested claim spends the execution budget while leaving the central market question unanswered. If the claim fails after launch, a new render cannot repair it; the team must revisit the decision and produce the asset again. A historical planning example used a 12-variant message test. The useful lesson is structural: those plausible messages create a choice problem before they create a production problem. The example is not a current platform benchmark, vendor claim, or delivery promise. The opposite error is also possible: if the message is already supported and the only open question is how to produce the video, the production tool is the appropriate purchase. ## How Do You Keep the Causal Question Intact? A team can move from a simulated experiment to real-human validation without changing the causal question. This does not turn the study into an observed usability session or guarantee market performance; it adds a human validation step before the message becomes a finished asset. ## Fund the Unresolved Decision If the script is approved and the team needs video, buy production tooling. If the team still needs to know which claim will change buyer behavior, test that action first. Teams with both needs may use both categories in sequence, but Subconscious does not generate video, voice, avatars, or lip-synced clips. Review the research and validation approach, then see how a causal study moves from question to action. [figure: Branching diagram: unproven message routes to causal message testing, producing a comparison that feeds the decision; approved script routes to video-avatar production, producing a rendered video.] --- # Real Panel or Causal Experiment First: Deciding Before You Field a Survey URL: https://subconscious.ai/blog/comparisons/real-panel-causal-experiment-first-deciding-before-field-survey Published: 2026-08-07 A consumer insights or growth marketing lead facing a concept, message, or feature question has two starting moves: recruit a real-consumer panel and field a survey, or run a controlled experiment first to see which comparisons are worth fielding. The second move exists because the first is expensive to get wrong. ## What a consumer panel answers Pollfish is a consumer survey panel platform. A buyer defines a target audience by demographic and behavioral criteria, and Pollfish's network of mobile apps reaches those respondents. That workflow returns real human answers to the exact question fielded, recruited from an existing operational pipeline. The strength of a real panel is also its constraint. Once a survey is written and fielded, changing the question means re-fielding it. Each round trip carries real recruitment cost and real calendar time. That is fine when the question is already scoped. It is expensive when the question is still being worked out. ## What does it cost to guess wrong before you field a survey? There are two ways to lose money on this decision. Field a full panel survey on a poorly scoped question, and the budget produced an answer to the wrong question. Or skip human validation on a decision that needs verified respondent behavior, and the business ships an unconfirmed claim. Neither failure is about the panel vendor's quality. Both are about sequencing: what gets tested before a team commits real recruitment budget. ## How do you narrow the question before you recruit anyone? Subconscious runs randomized experiments on a simulation of your market, validated against real human behavior, to show why people choose and which action drives the outcome. Run ahead of a fielded panel study, that experiment does one job: compare candidate messages, concepts, or features against each other and estimate which move the outcome, before any real respondent is recruited. That narrows; it does not substitute for the panel. Subconscious can test or validate studies with real human participants, so a team can move from the simulated comparison to a real-human check on the same question without redesigning the study. The practical advantage: panel budget goes toward the two or three comparisons that survived the first pass, not the full list of things someone in the room thought might matter. ## Comparing the two starting points ## Where the two combine A team can run the causal experiment first to narrow ten candidate messages or concepts down to the two or three that show a real effect, then field only those on a real panel like Pollfish for the confirmation the decision requires. That sequencing keeps panel spend attached to comparisons that already showed a signal, not spread across every idea in the room. ## Where does this approach break down? Subconscious does not recruit or field real consumer respondents, and it does not replace a real-panel survey when the decision requires verified human sample data, not just a directional estimate. The research library documents where simulated results have and have not replicated human outcomes; that scope, not a general claim of interchangeability, should guide the call. ## Making the call Before recruiting anyone, ask which constraint binds: an unscoped question that needs narrowing, or a scoped question that needs a verified human answer. The first calls for an experiment first. The second calls for fielding directly. Most research programs need both moves, in that order, more often than they need to pick one tool and stop. [figure: A branching path from a research question. One branch runs a narrowing experiment, feeding survivors into a real panel. The other, already scoped, goes straight to a real panel.] Check current leaderboard replication results before weighting a simulated comparison, or book time to talk through where a given question falls. --- # SparkToro vs. Causal Testing: How to Divide a Pre-Launch Research Budget URL: https://subconscious.ai/blog/comparisons/sparktoro-vs-causal-testing-how-divide-pre-launch-research-budget Published: 2026-08-07 A pre-launch research budget should cover distinct decisions. Use SparkToro to map where an audience already pays attention. Use simulated conversations to develop hypotheses about possible reactions. Use a causal experiment to estimate which message, concept, or positioning action changes the target behavior. None of these outputs proves the others. That separation matters before media, creative, and launch spend are committed. [figure: Four-step path: audience discovery feeds hypothesis generation, which feeds a randomized causal experiment comparing messages, which can be checked with real human participants.] ## Match the method to the launch question This is a budget-allocation decision, not a vendor scorecard. A CMO can fund more than one method while refusing to let one method answer a question it was not designed to answer. ## What does SparkToro's audience research actually measure? SparkToro says its audience research uses anonymized clickstream data, Google search results, and public social profiles to describe behaviors, demographics, and affinities across websites, podcasts, YouTube channels, social networks, and search terms. SparkToro product overview That output is useful when the open question is where to place a campaign, which communities matter, or which publishers and creators may reach the intended buyer. A channel map without its limits is a sales pitch, so the boundary is published here before spend is committed. It cannot establish that a headline, claim, or positioning choice caused consideration or preference to change. ## What are simulated conversations useful for? Simulated conversations can help a team surface possible objections, vocabulary, and message directions. Their value is generative: they give researchers candidates to examine more carefully. Agreement without its failure modes is marketing copy, so the misses are named here before a buyer trusts the reaction. Their limit is evidentiary. Language-model reactions can show variance collapse, demographic flattening, over-rationality, and prompt sensitivity. Agreement can therefore look more conclusive than the underlying method warrants. The safe use is to turn a reaction into a testable hypothesis, not a launch verdict. ## How does causal testing work? Subconscious is the causal AI company. It runs randomized experiments on a simulation of a market to compare defined actions against a behavioral outcome. For a launch, the action might be message A versus message B. The outcome might be consideration, preference, or another decision named in the study. The practical difference is control. A causal design holds the question steady, changes the action, and estimates what moved. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That describes audience-graph coverage for study design. It is not a recruitable human sample. Separately, Subconscious can test or validate studies with real human participants. A team can keep the causal question intact as it moves from a simulated experiment to real-human validation. ## Build the sequence around the risk A defensible pre-launch sequence has three gates: - Define the buyer and map where that buyer already pays attention. - Generate candidate messages and objections, then write the behavioral decision each candidate is supposed to change. - Randomize the viable alternatives against that outcome. Use real-human validation when the consequence of being wrong warrants it. ## Know what remains unproven A causal result without its scope is marketing, so the boundary is published next to the finding. A causal experiment needs a defined audience, explicit alternatives, and a measurable decision. It cannot repair a vague launch question. It does not replace channel discovery, creative judgment, or observation of actual market performance. A validation step without its limits reads like a guarantee, so the boundary sits on the record next to the result. Real-human validation is a separate evidence step. It does not turn a study into automatic proof of future sales or campaign performance. Confidence intervals, segment heterogeneity, willingness-to-pay, and scenario rankings depend on the configured study. They are not guaranteed standard outputs. Before committing the launch budget, write down the action, alternative, audience, and outcome. Then assign each research method only the part it can prove. Read the research approach for the validation boundary and how studies are designed for the operating sequence. --- # Survey Response Rates Are Falling. What Replaces the Survey for a Decision That Needs a Causal Answer? URL: https://subconscious.ai/blog/methods-and-validation/survey-response-rates-falling-what-replaces-survey-decision-needs-causal Published: 2026-08-07 A research or insights leader facing a declining-response survey program has five common replacements: behavioral analytics, continuous in-product feedback, social and community listening, passive telemetry, and AI-assisted qualitative interviews. All five describe what customers already did or said. None answers what a defined population would choose under a specific alternative that has not shipped yet without first building and exposing that alternative to real customers, the question a launch, price, or messaging decision needs. ## Why is the survey program shrinking? Phone and email survey response rates have been declining for decades as people screen calls, ignore inbox requests, and get asked for feedback after every support ticket and purchase (Pew Research Center). Lower response rates raise a concern beyond sample size: whether the respondents who remain differ from the customer base a decision is meant to serve. Pew's own analysis found that after weighting, low response rates produced little bias on most measures, with bias concentrated in a few specific measures such as civic engagement (Pew Research Center). The mechanical fix is a bigger sample. The structural problem: a shrinking, self-selected pool doesn't become representative by asking more of it. ## What the common replacements actually measure Research teams are not switching to a single new tool. They are assembling a portfolio, and each piece answers a narrower question than the survey it replaces. Each is a genuine improvement over a low-response survey. None answers a different kind of question: what happens if we change the price, the message, or the offer before we commit budget to finding out. ## The question a decision needs, not a description Behavioral analytics and listening data describe a world that already exists; they are observational, not experimental. Observing that customers who saw feature X converted more than customers who didn't is not the same as knowing that shipping feature X would raise conversion, because the two groups may have differed for other reasons before either saw the feature. Answering "what would this specific population choose among these specific alternatives" requires a controlled experiment: present a defined population with structured, stated-preference trade-offs and measure the causal effect of each one within that population, with a stated confidence interval. ## Where does a controlled experiment fit? Subconscious runs controlled discrete-choice experiments on simulated populations to estimate the causal effect of a specific action within the simulated population: a price point, a feature, a message, before it ships. Details on the experimental design and validation process are at /research and /how-we-work. [figure: Four boxes (analytics, feedback and listening, telemetry and AI interviews) point to "describes the past." A separate box, controlled experiment, points to "estimates effect of an unlaunched action."] This doesn't replace the other four methods in the table above. A controlled experiment doesn't track NPS or CSAT over time, monitor product usage on an ongoing basis, or substitute for community listening or passive telemetry as continuous-monitoring tools. It answers one narrower question, at the point a team needs to decide whether to take a specific action. The population size a simulated experiment can run against is not the same claim as how many real people have been recruited into a study. When a decision needs confirmation beyond the simulated result, the same causal question can be tested again with real-human participants. ## What does this mean for a research team? Stop expanding survey programs that fight declining response rates with more volume. Keep the survey where it still works: internal employee research, simple binary questions, and standardized cross-time benchmarks where a common yardstick matters more than precision. For decisions that hinge on a specific, unlaunched action, add a controlled experiment rather than making behavioral analytics or listening data answer a question they weren't built to answer. Teams making this call on a live pricing, packaging, or messaging decision can book a walkthrough of the experiment structure described above. --- # Does a simulated vaccine-preference study match a published one? A Kreps et al. replication check URL: https://subconscious.ai/blog/case-studies/simulated-vaccine-preference-study-match-published-one-kreps-et-al Published: 2026-08-07 A head of insights weighing a simulated discrete choice experiment against a full human-recruited study needs one answer: does the simulated ranking of preferences track how real people order the same attributes? Greenlighting a launch, price, or messaging call on a result that doesn't track wastes the study budget and the credibility of whoever recommended it. ## What was the published benchmark study? Kreps, Prasad, Brownstein, and colleagues ran a discrete choice experiment on US adults weighing COVID-19 vaccine attributes: efficacy, duration of protection, side effects, FDA approval status, and political endorsement, among others (Kreps et al., JAMA Network Open, 2020). The published paper reports which attributes respondents weighted most and least heavily: a ranking, not a single number. ## How closely did the simulated ranking match the published study? Subconscious ran a simulated version of the same choice task and compared the resulting attribute ranking to the one reported in the published paper. The two rankings correlated closely: a Spearman rank correlation of rs ≈ .87, computed by Subconscious against the published ranking (Kreps et al., JAMA Network Open, 2020). That correlation describes one documented replication of one published study's attribute ordering, not a general accuracy percentage. ## What this does and doesn't establish - It establishes that, for this vaccine-attribute preference task, a simulated ranking tracked a peer-reviewed human study's ranking closely enough to be directionally useful before a full field study. - It does not establish that the correlation holds for every domain. Health and vaccine-attribute preference research is one case; a different category of decision needs its own check. - It does not mean Subconscious recruited or fielded the original human sample. That sample belongs to the Kreps et al. study; Subconscious's role was running the comparable simulated task and comparing the resulting order. - It does not substitute for real-human validation when a decision's stakes call for it. A simulated ranking pressure-tests attribute priorities early, before a team decides whether a full human-recruited study is worth the spend. ## Where this fits in a research plan Run the simulated discrete choice experiment first, compare its output against any relevant published or prior study, and use that comparison to decide whether the priority order is stable enough to act on or needs a human-recruited study first. Subconscious supports moving from a simulated experiment to real-human testing without changing the underlying causal question: see how we work for that path, and research or case studies for other documented results. Teams that want to see current comparative performance can also check the leaderboard. [figure: Decision path: run the simulated experiment, compare its ranking to a relevant published study, then branch, if it holds up, act on the order; if not, run a human-recruited study first.] ## What are the limitations of this replication? This is a single replication against one published study's attribute rankings. It is not a benchmark across studies, domains, or populations, and it is not a guarantee that any other simulated task will correlate this closely with a comparable human study. [figure: Five of the vaccine attributes listed side by side, each showing two bars: the simulated ranking and the published Kreps et al. ranking, with the bars closely aligned across the five attributes shown.] --- # Why Churned Customers Leave, and Which Fix to Test First URL: https://subconscious.ai/blog/use-cases/ai-churn-interviews-explore-why-customers-leave Published: 2026-08-07 Customers leave for reasons usage data alone cannot name, so the fix to test first is the one a causal test confirms actually changes the outcome for that specific at-risk segment. A retention lead staring at a churn dashboard already has the number. What's missing is the reason, and which fix to try on the next at-risk cohort before it cancels too. A dropdown reason on a cancellation form offers a guess, not a story, so a team ends up choosing a discount or an onboarding tweak without knowing whether it targets the actual problem. ## The gap between a churn score and a churn reason Most retention stacks carry two kinds of data. Product usage signals, declining logins, unused features, a rising support-ticket count, a missed payment, say who is likely to leave and roughly when. They stop short of explaining why. A cancellation form tries to close that gap with a short list of preset reasons: too expensive, didn't use it enough, found something else. "Too expensive" could mean the sticker price is out of budget, the product never proved its value, or a competitor simply undercut on price. Getting that detail from someone who has already canceled is hard by design. Once a customer is gone, they have little reason to explain themselves, and response rates on post-cancellation and lapsed-customer surveys run low (Improve Survey Response Rate: 25 Proven Ways That Work; The low survey response rate crisis: 2025 guide for CX & insights leaders). The few who do respond tend to be the most frustrated or the most agreeable, neither of whom speaks for the larger group that just quietly stopped logging in. ## Turn a guess into a test Guessing at the reason behind a cancellation and guessing at the fix are two separate failure points, and most retention programs only try to close the first one. The next step treats retention like a pricing or messaging decision: name the specific action under consideration, a discount, a rebuilt onboarding step, a rewritten cancellation-flow message, and test it against the segment it's meant for before it ships to every account. Subconscious runs that step as a controlled experiment rather than an open-ended conversation. A team defines an at-risk segment from its own usage and support data, then puts two or three candidate retention actions in front of that segment in a causal test, with a confidence interval attached instead of a single plausible-sounding guess. The output names the action that moves the outcome, not a sentiment read on how the group feels about canceling. [figure: Four-step path: usage decline flags an at-risk segment, a cancellation survey returns a vague category, a causal test compares candidate retention actions for that segment, and the winner is confirmed with real people.] ## Will one offer fit every segment? The reason behind a cancellation rarely stays constant across a customer base. An enterprise account is more likely to leave over a missing integration; a self-serve account, over price; a casual account, because it never became a habit. A single retention offer tested once against the whole customer base hides that split and usually gets tuned to whichever segment is loudest, not whichever segment is largest or most valuable. The fix is to test segment by segment instead of company-wide. Start with the highest-risk segment's own candidate action, repeat the same test for the next segment, and expect the winning action to differ each time. A discount that saves a price-sensitive self-serve account will not do much for an enterprise account walking away over a missing feature. ## Why confirm the winner before it becomes the default? A retention action that tests well in a simulated study still carries risk once it reaches every renewal on a segment. Subconscious can move a study from a simulated test to validation with real people without changing the underlying causal question, so a team can compare the simulated read against a real one before committing budget broadly. How that comparison has played out on other decisions, in prior studies, helps calibrate how much weight to put on a simulated result alone. ## What does this not replace? This isn't an open-ended, roleplayed conversation standing in for a canceled customer, and it doesn't hand back an automatic, segment-by-segment substitution or cannibalization breakdown; that output needs a study designed for it specifically, with uncertainty bounds attached only where the design supports them. This is a decision-specific causal test aimed at one retention action and one segment, not a churn-prediction engine or a packaged interview product. The score that flagged the segment still comes from your own usage and support data; the causal test only judges which candidate action changes the outcome once that segment is defined. ## Where to start Pick one segment your team already tracks and one candidate action, a discount, an onboarding fix, or new cancellation-flow copy, and test that action instead of guessing at it. Book a walkthrough or read how a study gets built before the next churn cohort repeats the same unanswered question. --- # When a Directional Message Read Is Enough, and When It Isn't URL: https://subconscious.ai/blog/use-cases/when-directional-message-read-enough-when-isnt Published: 2026-08-07 A brand strategist rarely loses the argument over which message is best. The argument gets lost earlier, when the loudest internal opinion picks a line before anyone tests it against the target customer. The decision that actually needs protecting is narrower than "which message wins": it is which message variant advances to creative production, and whether a fast directional read is enough evidence or the choice is expensive enough to need real-human validation first. [figure: Flow diagram: message variants move through a controlled experiment on a defined segment, then a decision point on stakes. One branch goes to production; the other adds real-human validation first.] ## The Decision Brand Strategists Actually Own The question is not whether AI can generate a message. Drafting is now cheap. The question is whether the team can tell, before the message reaches a stakeholder deck or a production budget, which variant is more likely to change how the target customer responds. Getting that wrong has a specific cost: production spend and campaign budget committed to a message chosen on internal preference rather than audience response, plus the credibility cost of presenting an untested read as proof once a stakeholder asks how it was validated. Market research work is not disappearing under this pressure. Employment for market research analysts and marketing specialists is projected to grow from 2024 to 2034, according to the U.S. Bureau of Labor Statistics outlook for the role. What is changing is where the strategist's judgment adds value: less in producing the first draft of an analysis, more in defining the decision, choosing the right evidence, and knowing when a fast read is sufficient and when it is not. ## Two Ways This Goes Wrong Putting a failure mode on the record is what lets a team catch it before it repeats. The first failure mode is the one AI makes worse before it makes anything better: the team ranks message options by internal preference and calls it done. A fluent draft or a quick reaction round can feel like evidence when it is really just another opinion, generated faster. The second failure mode is newer and easier to miss. A team runs a quick directional comparison, gets a clear-sounding preference between two messages, and presents that preference to stakeholders as proof rather than as a hypothesis. Both failures share the same root cause: nothing in the workflow forces a distinction between a directional signal and a result built to isolate cause from noise. ## How Do You Run a Controlled Experiment Instead of a Preference Poll? A controlled experiment starts from the decision, not from the message. Before touching any wording, write down what changes in the plan if the test points one way instead of another, then define the target-customer segment the message has to move: their current behavior, their alternatives, and the outcome the message is supposed to shift. From there, the workflow compares message variants against that defined segment under conditions that hold everything else constant, so a difference in response can be attributed to the wording rather than to who happened to see which version. The output is a comparison of which variant is more likely to move the intended outcome, with confidence or uncertainty language attached where the study design supports it. That is a different claim than a preference ranking, which says people liked one message more. ## Where Each Method Fits ## What Can and Can't This Method Settle? A controlled experiment on a simulated market tells a brand strategist which message variant is more likely to move a defined audience, before the message reaches creative polish or a media budget. It does not replace the strategist's read of stakeholder dynamics, or human judgment about which decisions are consequential enough to require a further step. That further step is real-human validation: testing or validating the same question with real people once the decision is expensive or public enough to warrant it. The practical advantage is that a team can move from the simulated comparison to real-human validation without changing the underlying causal question, so the two steps build on each other rather than starting over. Naming this distinction is what lets a buyer check the result against what it actually covers. The simulated population used for the first comparison is a controlled-study population, not a recruitable real-world panel, and treating it as one would misstate what the test can prove. ## A Workflow to Start This Week The change does not require rebuilding the whole research process at once. It requires one visible discipline, applied to one real decision: - Write the business decision in one sentence: which message advances, and what happens next if it's wrong. - Define the target-customer segment and how expensive a wrong answer would be. - Run a controlled comparison across the candidate message variants against that segment. - Read the result for direction and uncertainty, not just for which option "won." - Decide, explicitly, whether this decision needs real-human validation before it moves to creative production. - Present the result labeled honestly: what was tested, what the result supports, and what still needs validation. As a planning example, one workable cadence is testing five message options and rewriting the top two based on the audience's objections, then repeating that once a week for a month. Treat those numbers as a starting point to adapt, not a fixed rule. Repeated across a few decisions, this produces something more durable than a faster first draft: a record of which messages moved which segments, and why, that the next decision can build on instead of starting from opinion again. ## Next Step Teams that want to see how a controlled message experiment is structured can review the research methodology behind the approach, read how the process fits into an existing workflow, or look at case evidence from other consequential decisions. To test a specific set of message variants against a defined audience, start with a demo. --- # Trade Show Booth Messaging: Test the Headline Before the Signage Prints URL: https://subconscious.ai/blog/use-cases/trade-show-booth-messaging-test-headline-before-signage-prints Published: 2026-08-07 ## The decision: which headline goes on the backdrop before it prints A field marketing lead has to lock a booth headline, a session title, or a swag line before a fixed production deadline, days rather than weeks. Once the signage ships, the message cannot be revised until the next event. There is rarely time for a traditional focus group between the creative draft and the print run, so the decision usually gets made by committee opinion instead of evidence. Trade show messaging is not untestable. The standard research timeline just does not fit the print calendar. ## Why does booth messaging fail without anyone noticing? Booth creative rarely fails visibly. It fails as a lower number: fewer people stopped, fewer badges scanned, fewer qualified conversations. Nobody traces that back to the headline, because there is no diagnostic step between "we printed this" and "traffic was soft." Common, quiet failure modes: - Insider language. The headline leans on category jargon before the target buyer has learned why it should matter to them. - Vague benefit claims. Generic phrasing about smarter workflows shows up on multiple booths at the same show. Nobody stops to read the third version of the same sentence. - Mixed-audience confusion. One line tries to reach buyers, users, champions, and procurement simultaneously, and lands clearly with none of them. - Weak category framing. At a show with hundreds of exhibitors, a visitor has a few seconds to place a booth in a category they already recognize. Assuming that context exists is a bet, not a fact. - No specific hook. The booth explains what the company does but gives no reason to stop walking. Each of these is a testable claim about how a specific buyer reacts to specific wording. None of them get tested in practice. ## What does a fast pre-print comparison test look like? The fix compresses testing into a controlled comparison that fits inside the production window: - Define the target attendee precisely. Not the general ideal customer profile, the version of that buyer who is actually walking a trade show floor: title, seniority, company size, industry, and event-attendance behavior. Build that definition once and reuse it for every show that season. - Draft three to five distinct headline variants. A safe version, a more pointed version, a category-reframe version, and something the team is nervous about. Testing one variant with no comparison is a guess with extra steps. - Ask a direct stop-or-skip question. Put the variants in front of the defined target buyer and ask which one would make them stop at a booth, given limited time between sessions. - Run a second round on the leading variant. Ask what would make someone walk past it instead, and what it implies about the product behind it. The second round often surfaces what to cut, not just what to keep. - Extend the same test to the rest of the messaging bundle. The backdrop headline is one asset among several: the sub-head, the demo prompt, the swag line, the sales team's opening question, the pre-show invite subject line, the session title, and the post-show follow-up. Each is a separate, testable claim. This is a controlled discrete-choice comparison of message variants against a defined target buyer, the same causal-effect method Subconscious runs for other early-stage messaging decisions, applied to a deadline-constrained use case. It is not a packaged trade-show pre-test product, a live floor-attention simulator, or a same-day workflow guarantee. Read more on the research page. ## What tends to change after one comparison round? Across messaging tests, headlines that survive a comparison round move in the same few directions: shorter, more specific, and more clearly differentiated from what a target buyer expects to see elsewhere in the category. A vague claim nobody on the team questioned often turns out to be the exact phrase a buyer skims past without registering. ## Pricing the decision A mid-sized B2B trade show carries an all-in bill, booth build, travel, staff, sponsorship, plus outbound before and after, that typically lands between $75,000 and $250,000, and can go higher depending on scope (exOptions, Trade Show Budgeting Guide). Exhibitor cost pressure has been rising industry-wide, which is why an untested message riding on that spend is a bigger bet than it looks (Trade Show Executive, CEIR Releases Exhibitor Spending Report). The messaging bundle that determines whether that budget converts into pipeline costs comparatively little to test before it ships. Skipping the test is a habit, not a decision. ## Where this fits and where it doesn't Subconscious runs controlled experiments that compare how a defined target buyer responds to different message variants, and reports the causal effect with a confidence interval. That is a directional decision input, not a guarantee of floor traffic or lead volume. It is not a live simulation of a show floor, and it does not replace watching an actual booth perform. When the stakes on a specific claim are high enough, or the result contradicts what an internal comparison test suggested, the same causal question can move from a controlled experiment to a recruited-human validation study without changing what is being asked. See how we work. ## Next step If a show is inside the next production window, define one target-attendee profile, draft three to five variants of the headline plus one other asset (an outbound subject line, a session title, or the sales team's opening question), and run the comparison before the print deadline instead of after the show. Document the result and measure booth engagement or lead conversion afterward against it. See a worked example in case studies, or book time to scope a comparison test against an upcoming show date. [figure: Five-step path: define the target attendee, draft several headline variants, run a stop-or-skip comparison, refine the leading variant, then extend the test to the full messaging bundle.] --- # How to Sequence Target-Group Research Before You Field a Study URL: https://subconscious.ai/blog/implementation-and-procurement/how-sequence-target-group-research-before-field-study Published: 2026-08-07 Target-group research for a launch should follow four gates: define the decision, establish the behavioral baseline, screen the live hypotheses with a controlled causal experiment, then validate the surviving question with recruited human participants when the evidence standard requires it. This sequence protects fielding budget from an unrefined concept without treating a simulated experiment as a substitute for human evidence. [figure: Four-step path: define the action and evidence threshold, establish the behavioral baseline, screen hypotheses with a controlled causal experiment, then carry the surviving question into human validation.] ## Put the launch decision before the audience profile "Understand the customer" is not a research decision. It can produce a broad profile without resolving what the launch team should do. Start with the action that will change if the evidence changes. The decision might be choosing between two product concepts, selecting a message for a defined buyer segment, or deciding whether a pricing hypothesis is ready for a higher-stakes study. Name the alternatives, the target segment, the behavior that matters, and the evidence threshold for proceeding. Sending an unrefined concept directly into a full recruited study can burn weeks of recruitment work and the study budget. The result may look like launch evidence while answering a question that should have been screened first. ## Use four gates before full fielding ### 1. Define the action and evidence threshold Write the decision in a form that can be tested: "Which of these two concepts should advance for this segment?" is stronger than "What does this audience want?" Set the evidence threshold at the same time. A directional screening decision, a representative population estimate, a regulatory claim, and a final pricing decision with real financial exposure do not require the same proof. The study plan should state which one it must support. ### 2. How do you establish the behavioral baseline? Use existing market, customer, product, and behavioral data to define the starting conditions. This step can reveal who is already engaging, which segments matter to the decision, and where behavior differs. Baseline analysis cannot establish why one intervention will change the outcome. It helps define the population and alternatives for the experiment, not replace it. For planning, a team might reserve one day for baseline mapping or budget an initial fielded pass around one or two variants. Those are historical planning examples, not current Subconscious delivery commitments or universal method limits. Use the actual decision, recruitment conditions, and evidence standard to set the plan. ### 3. Screen the hypotheses with controlled choices Turn the remaining concepts, messages, or positioning options into structured alternatives. Discrete-choice modeling studies preferences by asking people to choose among alternatives described by different attributes (Displayr, "Discrete Choice Modeling: A Market Researcher's Guide"). Subconscious runs controlled causal experiments against defined target segments to compare those alternatives before a team commits to a full recruited fielding pass. The experiment can screen concepts, messages, and positioning. It can report uncertainty when the study design supports it. The result is evidence for deciding what deserves the next research investment, not an automatic recommendation or a final population estimate. ### 4. How do you carry the surviving question into human validation? Keep the alternatives, target segment, and causal question stable as the evidence moves into a recruited-human study. Subconscious can test or validate studies with real human participants. That continuity lets the team validate a screened question instead of rebuilding the study around a new premise. Human validation does not turn a causal action test into a usability session, clinical trial, or automatic forecast of market performance. It adds the evidence required for the decision at hand. ## Budget fielding around evidence, not activity The useful procurement comparison is not software against recruiters. It is an unrefined fielding commitment against a gated evidence plan. This plan does not make recruitment optional. It gives each research stage a distinct job and makes the handoff criteria inspectable. ## How do you match the method to the claim you need to make? A controlled experiment is appropriate for comparing defined actions and screening which alternative should advance. It should not be presented as regulatory-grade evidence, representative population sizing, or final proof for a pricing decision with real financial risk. A scoping review has examined how discrete choice experiments are used to derive preferences for health-screening programs (NCBI/PMC, "Methodology to derive preference for health screening programmes using discrete choice experiments: a scoping review"). That is evidence that the method is used to study structured preferences. It is not proof that every target-group study has external validity or predicts real-world uptake. "A total of 27 studies that used a DCE to elicit preferences for cancer ( n = 26) and cardiovascular disease screening ( n = 1) programmes were included in the final analysis." Brain, Jadambaa, and Kularatna, BMC Health Services Research (source) Do not turn uncertainty into a universal confidence-interval promise. Do not treat an audience-reach figure as a recruitable panel. Do not treat a ranked result as an automated decision. The research owner remains responsible for matching the evidence to the launch decision. ## Bring one fieldable question to the research plan Choose one upcoming launch decision. Write down the alternatives, the target segment, the behavior that matters, and the proof required to act. Then separate the work into baseline mapping, causal screening, and any recruited-human validation the decision requires. Research explains the experimental method. How We Work shows how the screening-to-validation sequence runs end to end. Bring the decision and evidence threshold to a demo to scope the first experiment. --- # Adding Causal Testing to an Existing Client Scope: A Decision Guide URL: https://subconscious.ai/blog/implementation-and-procurement/adding-causal-testing-existing-client-scope-decision-guide Published: 2026-08-07 An agency or in-house marketing team should add a research step to an existing client scope only when a specific decision carries real budget risk and the current process resolves it with opinion instead of evidence. The test is not whether a client would pay for another line item. It is whether the decision on the table needs a causal answer, not a directional one. ## The revenue argument is real, but it points at the wrong question Demand for outside marketing-research and analysis services keeps expanding as brands look for outside partners to answer audience questions they cannot resolve internally (Marketing Research and Analysis Services Global Market Report 2026, The Business Research Company). That growth is a reason agencies keep getting asked for research, not a reason to sell more research into every account. The failure mode is not skipping research. It is adding a research line item that produces a directional read for a decision with real money behind it, so the client makes the same wrong call with a research cost attached. A one-page summary that "feels informative" is not evidence that a specific action changes the outcome, and confusing the two costs the agency credibility with the client paying for it every month. ## What's the difference between a directional read and a causal answer? A directional read tells a team what an audience says or seems to prefer. It is useful for early-stage ideation and for questions where the cost of being wrong is low. A causal answer tells a team which of two or more specific actions is more likely to change a defined outcome, with the alternatives and audience held explicit. Selling a budget-committing decision with stated-preference research is the core risk here. The client still makes the expensive call on incomplete evidence, and the agency now owns the research recommendation as well as the creative one. ## Audit the account before pitching anything Skip the general pitch. Look at each client's near-term calendar and ask, for every candidate decision: - Is there real budget or reputational cost attached to the outcome (media spend, a launch date, a repositioning campaign)? - Are there two or more concrete alternatives already defined, rather than an open-ended "what do you think"? - Would the client change what they do if the evidence pointed a different direction than their current plan? A "yes" to all three means the decision needs a causal comparison between the specific alternatives on the table. A "no" to any of them usually means a lighter, exploratory pass is enough, or the client has no decision yet, only a question. This audit is the entire pitch: it replaces "let us sell you research" with "let us test the two options you are already choosing between." [figure: Decision path: a client decision checked against three yes/no questions - cost at stake, alternatives defined, willingness to change course. All yes routes to a causal comparison; any no routes to a lighter pass.] ## Building the case with one client first Do not pitch a new service in the abstract. Pick the strongest client relationship, name one budget-committing decision on their calendar, and run a controlled comparison against the alternatives they are choosing between. Share the result in the next status meeting as a decision input, not as a new line of business. A team that has seen a causal answer change or confirm a real call is a better audience for a broader conversation than one hearing about a new capability cold. Once the client has seen one comparison change how they acted, the second and third studies do not need the same introduction. That pattern, not a fixed package menu, is what folds a causal test into an existing scope: a campaign concept choice gets a comparison built into its pre-launch checklist, a repositioning effort gets one built into its brief. ## Where does a causal experiment fit, and where does it not? Subconscious is a causal behavioral platform: controlled experiments on a simulation of the market, validated against real human behavior, that estimate which action is more likely to change a defined outcome. For an agency or marketing team, that makes it a fit for the budget-committing decisions identified above, not a general-purpose replacement for every research task in a client relationship. When a decision calls for extra confidence before the client commits, the same experimental question can move from a simulated study to a recruited human validation without changing the underlying comparison. See how Subconscious works for the mechanics, and the current case studies for examples of causal comparisons before a launch decision. A causal answer is only useful to a buyer who knows exactly where it stops applying. It does not replace the agency's client relationship, strategic recommendation, or judgment about brand voice, politics, or client-specific context. It also does not replace recruited human validation for decisions where regulatory, safety, or reputational stakes require testing with real participants. A causal comparison is only as useful as the alternatives it is given: if the client has not yet defined concrete options, the immediate need is a strategy conversation, not a study. ## Common objections, answered by scope rather than by discount "We already do research." The relevant follow-up is what kind of question that research answers. Existing stated-preference or qualitative work can coexist with a causal test aimed at budget-committing decisions. "Can't we do this ourselves?" Running a comparison is one step. Defining the right alternatives, framing the outcome that matters, and translating the result into a recommendation the client will act on is the work an agency is already being paid for. "We don't have budget for another line item." That framing is the mistake to avoid. The pitch is not a new line item; it is a specific decision the client is already about to make with or without evidence behind it. "Is this reliable enough to act on?" An honest answer to that question names the limit instead of hiding it. Treat a simulated comparison as a directional-but-causal screen appropriate to the decision's stakes, and route regulated or high-reputational-risk decisions to recruited human validation before the client commits. To apply this to a live account, start with the research method or talk through a specific decision that still needs a clear evidence path before it ships. --- # Should You Run a Fake Door Test, or Test the Concept First? URL: https://subconscious.ai/blog/use-cases/should-run-fake-door-test-test-concept-first Published: 2026-08-07 A fake door test tells a product manager whether people click a button for a feature that doesn't exist yet. It doesn't tell them why those people clicked, what they expected to happen next, or whether they'd actually adopt the feature once it shipped. That's the real cost of building the fake door before testing the underlying decision: design and engineering time goes into an experiment whose single aggregate click rate can't say which audience segment reacted, what expectation drove the click, or whether the signal would hold up against real behavior. ## What does a fake door click rate actually measure? Fake door testing puts a button, landing page, or menu item in front of users for a feature that hasn't been built, then counts clicks as a proxy for demand (Amplitude). It's cheap relative to shipping the real feature, but it still requires copy, design, tracking instrumentation, and enough live traffic to produce a signal worth trusting (Chameleon). A click is also a single number. It doesn't separate the segment that clicked because the framing matched their workflow from the segment that clicked out of curiosity, and it can't be re-run against a different value proposition without re-instrumenting the whole test. ## Running a controlled experiment before you build anything A causal experiment on the feature concept asks a narrower, more useful question: does a specific framing or variant of this feature change stated adoption intent, and by how much, with a confidence interval attached. That analysis exists before a single line of tracking code goes into the product: no button, no landing page, no live-traffic requirement. This limit sits in the open, next to what the method does well, so it can be weighed the same way. Neither method replaces the other. The controlled experiment tells a team whether a concept is worth the cost of instrumenting a live test at all; the fake door tells them whether real visitors, in the wild, behave the way the concept study predicted. ## What do the results of a concept experiment tell you? Three outcomes come out of a concept experiment, and each points to a different next step: - A clear, consistent effect across the framing tested. The team has a defensible case for spending the design and engineering time on a live fake door, and the study already indicates which framing to lead with. - A mixed or inconsistent effect. The concept needs a sharper problem statement or a different framing before it's worth instrumenting anything live. - No meaningful effect, or a negative one. The team can retire the idea before spending a sprint on it. This is the same build/kill logic a fake door test produces from click data. The difference is where the signal arrives, and that a causal estimate says which specific variant moved the number rather than reporting one blended rate. ## Moving from a concept study to a live test The study population is named as simulated here so a buyer can check what stands behind the number before acting on it. When a concept clears a controlled experiment, the same causal question can be checked with real human participants rather than only a simulated study population. Subconscious can test or validate studies with real human participants. That step doesn't change what's being measured; it changes who is answering. ## What this doesn't replace A method's limits are listed here because a specification without them is only a pitch. A controlled concept experiment does not build, host, or instrument a live fake door, landing page, or in-product click funnel, and it does not report actual website click-through analytics. Its output is a causal effect estimate from a controlled study, a different signal than observed click behavior. Treat it as a step that precedes or complements a live fake-door test, not a substitute for one. Review methodology detail on the research page, or see how this fits into a broader evaluation workflow in case studies. If a feature concept is close to a build decision, the fastest way to see whether a causal concept test changes the call is to book a session and run it against a real feature on the roadmap. [figure: Side-by-side comparison of a fake door click test and a controlled concept experiment across what each measures, segment detail, build cost before running, and traffic requirement.] --- # What to Test in Simulation Before You Burn Your First Ten Customer Calls URL: https://subconscious.ai/blog/use-cases/what-test-simulation-before-burn-first-ten-customer-calls Published: 2026-08-07 A pre-seed founder with no research budget has to decide, before the first customer call, which questions belong in that call and which ones can be resolved first. Get the split wrong and the cost shows up two ways: burn the first ten conversations on things a live human never needed to answer, or treat simulated enthusiasm for a pitch as proof that a real buyer will pay for it. ## Why founders skip research instead of triaging it Traditional qualitative research is built for a company that can wait. Independent industry estimates put a proper qualitative study at roughly €10,000-€30,000 and 6-8 weeks of turnaround (Merren, "How Much Does Qualitative Research Cost in 2026"). A pre-seed team doesn't have either. By the time a study like that would land, the product has usually shipped three iterations and pivoted twice. So most founders skip the formal process and run on instinct, founder-network conversations, and whatever user calls they can schedule. That works to a point: instinct is overfit to the founder's own experience, network conversations skew toward people already inclined to agree, and user calls are slow to book with people who don't know the founder yet. The failure mode isn't skipping research. It's not triaging it. Some questions (does this positioning angle land better than that one, what's the first objection a skeptical buyer raises) don't need a live human to surface. Others (will this person pay, will they switch, will they refer it) can only be answered by one. ## What can a controlled comparison settle before the first call? A causal behavioral platform like Subconscious runs controlled experiments that compare one action against another, such as one positioning angle or one ICP definition against another, and estimates which is more likely to move a stated outcome. That's a narrower job than full customer discovery, but it's the triage this decision needs: a way to rule out the weakest hypotheses before spending a real conversation on them. Before talking to a single customer, a founder can use this kind of controlled comparison to: - Run a value proposition against a defined customer profile and see which framing gets less resistance. - Map the objections a skeptical buyer would likely raise first. - Compare several positioning angles against each other rather than guessing which one to lead with. - Run a competitive comparison conversation to see where a pitch holds up and where it doesn't. After a few real conversations have happened, the same comparison can be re-run grounded in what surfaced: scaling the same questions asked of three real customers out to fifty simulated ones, testing whether an objection generalizes, or checking an adjacent customer type nobody has had time to reach yet. ## A fit worth being honest about Subconscious's stated ideal customer is a team with a consequential, repeated decision and enough market context to define a testable population, not a pre-revenue founder with one untested hypothesis and no budget. That's a real mismatch: this approach sharpens the questions a founder brings into real interviews, it does not substitute for a team built or priced for a single founder's stage. The sprint looks like this: name the customer profile and the hypothesis, ask the comparison five questions (the biggest frustration with current solutions, the first reaction to the pitch, the first skeptical objection, how it compares to the main competitor, and what would have to be true to try it in the next 30 days), then take whatever ruled itself out into the first real customer conversations instead of testing it there. ## What can't a controlled comparison tell a founder? A controlled comparison of hypothetical responses cannot tell a founder whether a real person will change behavior, pay money, or refer the product to someone else. Enthusiasm for a simulated pitch is not conversion evidence; treating it as such is the second way founders waste runway on this problem. The method compresses the hypothesis-testing cycle before real validation; it doesn't replace it. When a team does need to move past hypothesis testing, Subconscious supports testing or validating a study with real human participants without changing the underlying causal question: the same comparison, run against people instead of a model. For an early triage pass, that step usually isn't necessary yet. ## Where does this fit for a founder before product-market fit? Before product-market fit, the most useful thing this kind of comparison can do is sharpen an ICP definition rather than confirm demand. Testing several distinct customer types against the same problem statement and watching which one responds most specifically and emotionally produces that signal faster than recruiting five customer types and interviewing all of them in person before knowing which one matters. That signal still has to be confirmed: it tells a founder which type of customer to prioritize for the ten real conversations that decide the go-to-market call, not what to decide. [figure: Two columns: left lists questions a controlled comparison can resolve before any call (positioning angle, first objection, competitive framing); right lists questions only a live call can resolve (pay, switch, refer).] [figure: Four-step path: name a customer profile and hypothesis, run five fixed comparison questions in simulation, see which hypotheses rule themselves out, carry only the unresolved ones into real calls.] ## Next step A founder deciding what belongs in the next ten customer conversations can see how Subconscious frames this in how we work, or look at case studies and the research library for examples of the same triage applied to a bigger decision. A demo walks through what this looks like end to end. --- # The Consumer Analyst's Staged Path From Concept Screen to Validation URL: https://subconscious.ai/blog/buyers/consumer-analysts-staged-path-concept-screen-validation Published: 2026-08-07 A consumer analyst rarely gets to choose whether AI enters the concept-testing process. The choice that remains is which stage of that process gets a fast, exploratory read and which stage still needs a controlled experiment before fieldwork budget and stakeholder trust are on the line. Get that staging wrong in one direction and a weak concept moves into full validation on the strength of a fluent but uncontrolled reaction. Get it wrong in the other direction and a concept never gets a causal test at all, so the team only discovers the mistake after the study is fielded and the recommendation has shipped. ## The pressure is real, the job is not disappearing What began as a novelty add-on has settled into everyday research tasks: drafting surveys, summarizing transcripts, and producing first-pass reads on a concept. That does not remove the need for the role. Federal labor projections back this up: the U.S. Bureau of Labor Statistics expects the market research analyst and marketing specialist workforce to keep expanding between 2024 and 2034. What's at risk is something short of replacement. When the mechanical parts of the job get faster and cheaper, the analyst has to move closer to the decision itself: better questions, better evidence choices, sharper caveats, and more influence over what gets tested next. The people who only produce output become easy to substitute. The people who can say which output deserves trust do not. ## Draw the line between a directional read and a causal test An open-ended AI-assisted first pass is useful for generating hypotheses, objections, and alternative framings of a concept. It is not a substitute for a controlled experiment, and treating it as one is the mistake that makes this stage dangerous: asking a fast method to pick a winner without exposing the trade-off behind the pick. The distinction to hold onto is between an exploratory hypothesis and a measured, comparative result. One tells you what might be true. The other tells you, within a stated population and a stated set of alternatives, which option produced the stronger effect and how confident that estimate is. ## Where does a controlled experiment fit between exploration and fieldwork? This is the stage where Subconscious is built to sit. Once an analyst has defined the concept alternatives and the population they matter to, Subconscious runs a controlled discrete-choice experiment comparing those alternatives and returns causal effects with confidence intervals, rather than a single fluent reaction to one concept at a time. That gives the analyst a disciplined middle step: still fast enough to run before committing to full fieldwork, but structured enough to produce a comparative, causal result. When the decision is expensive or public enough to warrant it, the same causal question can move to real-human validation rather than switching to an unrelated method. See /research for how these experiments are structured and /case-studies for how that comparison has played out on specific decisions. ## What does a controlled experiment not replace? A controlled discrete-choice experiment is not a sensory test, a physical prototype evaluation, or a qualitative refinement conversation, and it does not turn into one by adding more questions. It answers a comparative causal question about a defined population; it does not tell you whether a product tastes right, feels right in hand, or survives a regulator's read of a claim. Those still need their own method. It is also not open-ended persona chat. The output is a measured comparison across defined alternatives, not a simulated conversation with a synthetic customer. Keeping that boundary explicit in a report is what makes the caveat credible instead of decorative. ## Build the staging into the deliverable, not just the workflow The habit worth building is not "use AI," it is "label the stage." State what the exploratory pass was used for, state what the controlled experiment measured and at what confidence, and state what still requires real-human validation before anyone treats the finding as external-facing. An analyst who can explain the boundary of their own confidence reads as more rigorous, not less. A workable first move: take three concepts a team is already debating, define the population each one has to win with, and run one as a controlled comparison instead of an open-ended exploratory pass. Compare what the causal result adds versus what the exploratory pass already told you. That single comparison does more to build the evidence system than a long list of AI tools ever will. [figure: Three steps. Exploratory pass surfaces objections, not a finding. Controlled experiment compares alternatives, returns a causal effect. Real-human validation confirms it with recruited participants.] If a concept decision is close enough to warrant this staging, the next step is to see a working example of how a controlled experiment gets defined before it runs. --- # UX Survey Methods for Product Teams URL: https://subconscious.ai/blog/methods-and-validation/ux-survey-methods-product-teams Published: 2026-08-07 A product team about to ship a redesign usually reaches for a survey. The harder question is which instrument, because "how usable does this feel" and "which version will more people actually adopt" are different questions that need different tests. Pick the wrong one and the team either fields a full usability study to answer a simple wording question, or walks away with a satisfaction score that says nothing about which design change will move behavior. ## Match the instrument to the decision, not the other way around Before drafting a single question, name the decision the result has to support. A UX measurement instrument answers a narrow slice of that decision: how usable a design feels to the people who try it. It does not tell you which of two designs will get more people to complete a purchase, upgrade a plan, or return next week. ## Perceived-usability instruments These methods ask people to report their experience with a design. They are stated-preference tools: straightforward to field and useful for catching friction before launch. ### 1. What is the System Usability Scale (SUS)? A ten-item, standardized questionnaire that produces a single usability score for a design or flow. It is well suited to tracking usability over time or comparing versions of the same product, and its scoring method is documented and widely replicated (Nielsen Norman Group). A usability score only means something next to the limit on what it scores. It measures perceived ease of use, not purchase intent or adoption likelihood. "The SUS is a well-established 10-question survey administered at the end of a user test; it gives you a measure of the perceived usability of your product and enables you to compare it with others." Raluca Budiu, Nielsen Norman Group (source) ### 2. What is a task survey? Respondents attempt a specific task inside a prototype or live product, then answer structured questions about difficulty, confidence, and completion. This surfaces where a flow breaks down at the step level, which a single aggregate score cannot show. ### 3. What is a semantic differential scale? Respondents rate a design or concept along paired adjectives (confusing–clear, basic–premium, unresponsive–smooth). This is useful for capturing perception and brand association, but the result is a stated impression, not an observed choice. ### 4. Diary prompt Respondents log short, repeated entries about a product over days or weeks. This captures usage context and friction that a single-session study misses, at the cost of longer fielding time and smaller samples. ## When the real question is behavioral, not perceptual Teams often start with "let's run a usability survey" when the real decision is which of several design, feature, or message alternatives is more likely to move adoption or conversion. No amount of SUS scoring answers that directly. For that decision, Subconscious runs a randomized discrete choice experiment comparing the alternatives on a simulated population and reports the estimated effect on choice among the tested alternatives. Randomized attribute assignment identifies which design change moves choice; an aggregate usability score cannot. Simulated choice has replicated human study outcomes at 87% of the measured human ceiling (0.832 over 0.959; mean 0.73 across the 43 studies passing design filters), a training-data benchmark, not validation of this specific comparison (methodology). [figure: A decision path from "define target group and stimulus" splitting into perceived usability (SUS, task survey, diary) versus a behavior-change simulated experiment, both converging on real-respondent validation.] ## Configuring any of these instruments The setup work is the same across instruments: - Define the target group: who should respond, and what they already know about the product or category. - Define the stimulus: a prototype, task, concept, landing page, or feature list. - Define the decision the result has to support before writing questions, not after seeing the data. - Pressure-test question wording for leading phrasing, double-barreled questions, and missing answer options before the instrument reaches real respondents. ## Limitations and when to validate with real respondents Naming a method's limit here is what lets a buyer check it before relying on the result. None of these methods, simulated or fielded, should be the final source for representative statistics, regulatory claims, or formal market sizing on their own. The higher the financial or compliance stakes, the more a team needs real-respondent data behind the conclusion. Subconscious does not draft usability survey wording, administer SUS or diary studies, or replace fielded human usability testing where discovery, emotional nuance, or regulatory evidence is required. Simulated experiments are a first pass on which alternative is worth testing further. Subconscious can also test or validate studies with real human participants. [figure: A decision path: a simulated result branches on stakes. Low-stakes results are used directly; high-stakes results route through real-respondent validation. Both converge on a claim ready to ship.] ## Next step If the question is "how usable does this feel," a SUS or diary study is the right tool. If the question is "which version will more people actually choose," that is a randomized choice experiment, not a usability survey. See how Subconscious runs these comparisons or set up a study. --- # Synthetic Responses vs. Causal Experiments: Choosing a Market Research Tool URL: https://subconscious.ai/blog/comparisons/synthetic-responses-vs-causal-experiments-choosing-market-research-tool Published: 2026-08-07 Choose a market research tool by the evidence it produces for the decision at hand. A synthetic response helps a team explore what people might say; a controlled experiment fits better when deciding whether a price, position, or launch message will change an outcome. The procurement question is which result was checked against human behavior, not which report sounds most convincing. ## Fluency is not behavioral evidence A simulated respondent can give a coherent explanation of a preference, but that output reflects a model's prediction, not what people will do when they face a real price, tradeoff, or switching cost. Research on large language models as virtual survey respondents finds generated answers can resemble aggregate human patterns for some questions and diverge for others (Large Language Models as Virtual Survey Respondents). A review of model limitations documents the gap between fluent output and reliable grounding (LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Models). "Our evaluation reveals consistent performance trends across model families, highlights failure modes in structured output generation, and demonstrates how context and prompt design affect simulation fidelity." Zhao and colleagues, "Large Language Models as Virtual Survey Respondents," arXiv:2509.06337 (source) That gap matters before a launch: a plausible panel can support the wrong price or message just as confidently as the right one, and the cost appears only after budget or roadmap capacity is committed. ## How does each approach's evidence chain compare? The useful comparison is how each approach connects a proposed action to a measurable outcome. [figure: Two-column table. Left: synthetic response, a plausible reaction validated by a human check, useful for exploration. Right: causal experiment, a measured outcome change validated against a human study.] ## What replication benchmark should you ask for? Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, drawn from roughly 300 replicated human studies across 9 domains. Naming the failure mode lets a buyer check it before committing budget. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. Buyers can inspect the research behind the benchmark and the individual results on the replication leaderboard. A number without its limits is marketing. This benchmark supports a specific claim about reproducing studied human outcomes, not a guarantee for every market, segment, or decision, and it remains separate from audience reach: an audience graph is not a recruitable participant panel. When more certainty is needed, a simulated experiment can move to real-human testing or validation as a distinct, buyer-configured step that keeps the same alternatives and outcome rather than substituting a different study. ## How should you compare cost and scope together? Subconscious does not publish a self-service price list. Compare total cost for the specific decision rather than assuming a public price. A historical planning example for agency-led research used three to four weeks for the brief, participant recruitment, and final report, treated as historical calendar context, not a current Subconscious estimate or a promise from any vendor. The misses sit on the same leaderboard as the hits. Not every study includes confidence intervals or segment-level heterogeneity by default. Real-human validation is configured separately, not automatic. A validation result does not become an observed usability session, clinical trial, or automatic proof of market performance. ## Bring one decision into the vendor conversation Use one upcoming pricing, positioning, or launch decision to make this concrete: - Define the alternatives that could actually ship. - Name the behavioral outcome that would change the decision. - Ask whether the method estimates a causal difference or generates a plausible reaction. - Request the human baseline, replication definition, study count, and domain coverage behind any accuracy claim. - Confirm whether real-human testing preserves the same alternatives and outcome. - Ask which uncertainty measures and segment analyses are included for this study. - Price the complete decision path, including any separate human validation. The causal workflow shows how alternatives, outcomes, and validation fit together. To evaluate the method against a live decision, book a working session with those specifics ready. --- # Why a Bayesian Marketing Mix Model Still Needs Calibration Before You Reallocate Budget URL: https://subconscious.ai/blog/methods-and-validation/why-bayesian-marketing-mix-model-still-needs-calibration-before-reallocate Published: 2026-08-07 A Bayesian Marketing Mix Model (MMM) can tell you which channel looks most effective. It cannot tell you, on its own, whether that estimate is trustworthy enough to move tens of millions of dollars. That gap is where reallocation decisions go wrong. ## The decision a CMO actually faces An MMM produces a channel-effectiveness estimate and a recommended budget split. The question in front of a CMO or VP of Marketing Analytics isn't whether to trust the model. It's whether to act on the MMM's output as-is, or require calibration against an incrementality experiment (a holdout or geo-experiment) before shifting spend. Getting this wrong is expensive in a specific way. If the model's estimated effect for a channel is inflated because spend was allocated in anticipation of demand, or by adstock and saturation assumptions that don't hold in practice, the team reallocates spend toward that channel expecting a lift that isn't there. The shortfall only shows up after the budget has already moved. ## What is the structure of a Media Mix Model? A Bayesian MMM regresses an outcome, typically revenue or conversions, on media spend across channels, plus controls for seasonality, pricing, and other demand drivers. Two structural choices matter most for the estimate: - Adstock: media spend rarely converts in the week it runs. Adstock functions model the carryover effect of an impression over following weeks. - Saturation: each channel's return diminishes as spend increases. Saturation functions capture that diminishing return so the model doesn't extrapolate a straight line past the point where it breaks down. The Bayesian treatment adds priors over these functions and produces a distribution over the channel-effectiveness estimate rather than a single point value. The model's correlation problem is published here so a buyer can weigh the channel-effectiveness estimate correctly. It does not, by itself, resolve the deeper problem: media spend across channels is often correlated, and a correlational model can attribute an effect to the wrong channel when two channels move together. ## Why is calibration the step that closes the gap? Google Research published a methodology for calibrating Bayesian MMM priors against experimental results, using incrementality experiments to constrain what the model is otherwise unable to distinguish from spend patterns alone. The logic is straightforward: a well-designed incrementality experiment (a geo holdout, for example) produces an estimate of a channel's effect at the tested spend level, with its own uncertainty. Feeding that estimate into the MMM as an informed prior narrows the range of channel-effectiveness values the model will accept, instead of leaving it to infer the answer purely from historical correlation. Marketing measurement practitioners describe the same requirement from the buyer's side: an MMM without calibration is a starting hypothesis, not a validated budget plan. ## What calibration does not fix Naming this limitation here is what lets a buyer check the model before spend moves. MMM remains a correlational, aggregate-level model even after calibration. Calibration reduces the bias that comes from endogenous spend allocation and unconstrained adstock or saturation curves. It does not eliminate it, and it does not turn the model into a causal method on its own. A calibrated MMM is still an estimate built from historical spend and outcome data, refined by whatever experimental evidence was available to anchor it. It does not test a new, specific reallocation before you commit to it. ## Where does causal testing fit before the budget moves? Subconscious runs controlled discrete choice experiments to test how a specific action, such as a candidate budget shift or channel reallocation, changes the choice-relevant behavior it's designed to measure, and compares scenarios with quantified uncertainty rather than a single point estimate. That complements MMM calibration rather than replacing it: calibration improves the model's estimate of what already happened; a configured causal study tests respondent behavior under the specific reallocation being considered, not the resulting aggregate media response. Where the decision depends on validating a result against real behavior, a team can move from a simulated causal study to a study with real human participants while keeping the same causal question, This limitation sits next to the method's strengths so a buyer can check it before acting. Though internal validity in the simulated study is not evidence of external validity to in-market media response. [figure: Five-step chain: correlated spend leads to the model failing to isolate each channel's effect, then misattribution, then a budget shift toward the inflated estimate, then a late-visible shortfall.] ## Practical checklist before reallocating - Confirm the MMM's adstock and saturation assumptions were fit against your own channel mix, not a template setup. - Check whether any channel-effectiveness estimate has been calibrated against an incrementality experiment. If not, treat it as a hypothesis. - Before moving spend on the strength of an uncalibrated estimate, test the specific reallocation as its own causal question, ideally as a holdout or geo-experiment; a configured discrete choice study can test the underlying behavioral assumptions but does not substitute for market-level incrementality evidence, rather than reading the MMM output as a finished answer. [figure: Five-step path from a raw Bayesian MMM estimate through calibration against an incrementality experiment, noting the model stays correlational, to a causal test of the specific reallocation, ending in the budget move.] Testing a specific budget reallocation as a controlled causal question, rather than reading it directly off an MMM, is one way to see whether the estimate holds before spend moves. Learn how Subconscious runs causal studies. --- # 9 AI Brand Awareness Tracking Tools Compared (2026) URL: https://subconscious.ai/blog/use-cases/ai-brand-awareness-tracking-tools-compared-2026 Published: 2026-08-07 A CMO who owns the brand-health budget faces one decision every cycle: which signal to buy, and how often to buy it. Social listening, a recruited real-human tracker, and a controlled experiment answer different questions. Picking the wrong one is the expensive mistake, not the sticker price of any single tool. ## Nine tools, three question types They split into three groups by what they can actually measure. [figure: Three columns: social listening reads public mentions, a human tracker reads survey answers, a synthetic panel reads simulated responses on any cadence but needs a human baseline to trust the number.] ### 1. Self-serve synthetic panels A calibrated panel of simulated respondents built to stand in for a target audience, queried on whatever cadence a team chooses rather than a fixed seasonal wave. These platforms are priced to be self-serve, so frequent, low-stakes checks are affordable in a way an annual tracker budget cannot support. ### What Is Brandwatch? A long-established social listening platform. It scores sentiment and share of voice from public mentions rather than panel answers, so it describes what people say, not what they privately believe. ### 3. NetBase Quid Another social listening platform built for enterprise brand and communications teams, in the same category as Brandwatch: real-world mentions, sentiment, and share of voice at scale. ### 4. Talkwalker A third social listening entrant, strongest for category share-of-voice and competitive monitoring drawn from public digital conversation. ### 5. Latana A recruited real-human brand tracker with automated analysis layered on top. It is a traditional survey panel with better tooling, not a simulated one, and it reports directly on awareness, consideration, and preference. ### 6. Quantilope An automated insights platform that runs real-human surveys at scale, with brand tracking as one workflow among several. It fits large programs that need recruited responses fielded on a repeatable schedule. ### What Is YouGov BrandIndex? The most established syndicated brand tracker, built on daily recruited human responses across thousands of brands. It is the closest thing the category has to a shared benchmark, and it is why any serious brand-tracking review should check its published methodology before comparing a synthetic read against it. ### 8. Evidenza An audience-simulation platform where brand tracking is one of several supported workflows, run against a synthetic panel stable enough to query on a set schedule. ### What Is Aaru? A synthetic-panel platform built for large, statistically rigorous population-scale brand studies, priced and scoped for enterprise programs rather than self-serve use. ## Comparison criteria No single row of this table settles the decision by itself. ## Why the categories are not interchangeable Social listening reads public conversation. It is strong at flagging a sentiment shift or a share-of-voice change the same day it happens, and weak at anything the audience thinks but never posts: unaided awareness, consideration, and stated preference all have to be inferred rather than observed. Recruited human trackers ask the standard brand-health question set directly to real people. That makes them the strongest source for an absolute number a board can cite. The tradeoff is cadence: a quarterly wave is a practical ceiling for most budgets, and a wave already in the field cannot bend to capture an unplanned event. Synthetic panels answer the same question set from simulated respondents, on any schedule a team wants, at a materially lower marginal cost per wave. That speed is best used for relative comparisons, such as this quarter against last quarter or one message against another, and for directional reads. Reporting an exact percentage of a real population from a synthetic read alone goes further than the method supports without a human check on that number. ## Reading a vendor's own benchmark claim Any platform in this category may publish an internal accuracy claim against historical tracking data. Before treating one as decision-grade evidence, ask: - Was the comparison run at the aggregate level or the individual level? - Did it test stated preference, an observed choice, or open-ended text? - Was the benchmark audience close to the audience the team actually needs? - Were the prompts, calibration method, and failure cases disclosed? - Does the platform report a range or uncertainty, or only a single number? A claim that cannot answer most of these is a marketing figure, not a validation result. ## Running the stack instead of one tool Teams that get the most out of this category rarely pick just one row of the table. A repeatable pattern: - Keep one fixed set of core measures (unaided awareness, aided awareness, consideration, preference, brand attributes) worded identically every cycle, so a shift is comparable across waves. - Run a same-week check against the current decision when a competitor launches or category news breaks, instead of rebuilding the whole tracker. - Capture a pre-campaign baseline and a post-campaign read so a controlled test can separate exposure from non-exposure before results are read. - Compare narrow buyer segments cheaply through a synthetic read, then confirm any segment that matters commercially with a human check. - Run a recruited human wave on a fixed schedule and compare its pattern against the synthetic signal; investigate the audience definition and stimulus before trusting the synthetic trend if the two diverge. ## Where a controlled causal test fits None of the nine platforms above is a fit for every brand question. A distinct fourth option is a controlled experiment: define the audience, change one variable such as a message, a category frame, or a positioning statement, and measure which version moves stated preference. This is not a standing tracker and does not run on a continuous cadence; it answers one comparison at a time. Subconscious supports this fourth path rather than competing with the tracking categories above. It tests which specific action causes a measurable move in audience preference, and can validate that result with real human participants without changing the underlying question being asked. That is a different job from an always-on brand-health dashboard: it answers "would this change help," not "what is our current awareness score." A team already running one of the nine tools above can add this kind of test before committing budget to a specific message or position, then confirm the market outcome with continuous tracking after launch. See a worked example of how a causal comparison is scored, or book a walkthrough of a specific decision. --- # Delphi vs. a Causal Experiment Platform for Pricing and Messaging Decisions URL: https://subconscious.ai/blog/comparisons/delphi-vs-causal-experiment-platform-pricing-messaging-decisions Published: 2026-08-07 A pricing, packaging, or messaging change is on the table, and the shortlist includes a tool built to put a creator's voice in front of an audience. Name the actual decision first: is the team trying to scale one person's expertise, or find out which pricing or message alternative a buyer segment will respond to? Those are different jobs, and picking the wrong category produces an answer that does not transfer to the real decision. ## What Delphi is built for Delphi lets a creator, coach, or expert build a conversational clone trained on their own material, then deploys that clone to their audience so it can answer questions and hold conversations at a scale the person alone could not sustain (delphi.ai). The people using the finished product are the creator's own followers, not a target customer or prospect segment the creator is trying to understand (Delphi's discover page, which lists creator clones built on the platform). There is no control group and no randomized comparison across alternatives. It is built to scale one voice. ## What is a causal experiment platform built for? Subconscious is the causal AI company: it runs randomized, controlled experiments on a simulation of a target market and returns causal effects with confidence intervals, showing why people choose and which action moves the outcome. Instead of one voice talking to many people, a team names two or more pricing, packaging, or message alternatives and tests them against a modeled buyer or customer segment under the same experimental design. These two categories can look similar on a comparison page because both produce conversational or persona-driven output, but a pricing or messaging decision needs the second one. ## Where the two jobs diverge ## When is Delphi the right fit? Choose a creator-clone tool when the underlying job is scaling one person's expertise to their own audience, with no comparison across business alternatives involved. ## When a causal experiment platform is the right fit Choose a causal experiment platform when the decision is which pricing, packaging, or message alternative to back for a target segment, and the team needs a randomized comparison instead of a single conversational answer. When the decision warrants it, the team can move from simulation to real-human validation without changing the causal question. ## What does a causal experiment platform not do? Subconscious does not provide clone-hosting, follower administration, or content-delivery tooling for a creator's own community. Buyers evaluating a tool for that specific job should keep evaluating that category, including Delphi. Publishing what a method cannot claim is what lets a buyer check it against the study design before relying on it. A controlled study estimates effects under its study design. It does not guarantee the exact result of a market launch, and testing or validating a study with real participants adds external evidence rather than automatic proof of commercial performance. ## The core difference Delphi helps one person's knowledge reach more people. A causal experiment platform helps a team find out which action changes buyer behavior before the change ships. Scoping a pricing or messaging decision around the wrong category means shipping on a chat transcript instead of an experiment. The gap only shows up after the change is live and the market does not respond the way the transcript implied. [figure: A branching path from "name the decision" into two routes: scaling expertise leads to a creator clone with no control group; testing alternatives leads to a randomized experiment against a modeled buyer segment.] Before scoping a pilot, name the decision first. If it is "which action moves this outcome," start with how Subconscious runs a study, review replicated study results or the public leaderboard, and book a working session to define the alternatives worth testing. --- # Synthetic Research: When to Trust It, and When to Validate With Real People URL: https://subconscious.ai/blog/use-cases/synthetic-research-when-trust-when-validate-real-people Published: 2026-08-07 [figure: Four-row list matching situations to methods: early tests to synthetic alone, capital-committing claims to recruited alone, narrowing to finalists to both in sequence, hard-to-recruit audiences to synthetic alone.] A VP of Insights deciding whether to add synthetic research to the toolkit is really deciding which of this quarter's research questions simulation can answer, and which one still needs a recruited human study before capital moves. Get that split wrong: a recruited-research budget gets spent validating a question simulation could have answered in an afternoon, or a pricing, go-to-market, or public claim ships on a directional read that cannot produce a defensible confidence interval, and unravels under scrutiny. ## What is synthetic research? Synthetic research uses AI-generated personas, conditioned on demographic, psychographic, and behavioral data, to simulate how a target population responds to a stimulus: a concept, a message, a poll question. Researchers assemble personas into panels, run a stimulus against the panel, and get back a distribution of responses with natural-language reasoning for each one. Three terms often get conflated: - Synthetic respondent: the individual AI agent that answers a single study, conditioned to hold a specific set of beliefs and background. - Synthetic persona: the reusable profile behind a respondent, covering demographics, psychographic traits, and decision-making frameworks. Saved once, queried across projects. - Synthetic panel: a structured group of personas, typically ranging from 8 to 100 or more, assembled to represent a market segment. A panel study might show that 60 percent of personas accepted a feature concept, 30 percent raised a specific objection, and 10 percent asked about pricing: a distribution, not a single verdict. The premise traces to a 2023 finding: conditioning a language model on the detailed background of a real poll respondent produced opinion distributions that tracked actual human responses in benchmark national polls (Argyle et al., a study on prompting language models with individual respondent profiles to reproduce human sample distributions, published in Political Analysis by Cambridge University Press). That result moved the method from academic benchmarking into product, marketing, and insight teams. ## Where the method stops being enough Simulation is directionally useful and limited in predictable ways: - It doesn't produce statistical validation. Simulation cannot generate a population estimate with a defended confidence interval. If a regulator, auditor, or public claim needs a stated figure, such as 34 percent of a population holding a given view, that number has to come from recruited research. - It lags on novel behavior. Personas are built on historical patterns. A category with no real-world analog, or a sudden macroeconomic shift, can outpace it. - It inherits training-data skew. Models trained heavily on English-language, Western text default to generalized assumptions for audiences underrepresented in that data. A community outside the training distribution needs real members validating the finding. - It doesn't touch the physical world. A simulated persona doesn't pull out a credit card, sit through a shipping delay, or churn after a bad support call. Longitudinal behavioral tracking still needs real behavioral data. ## The decision: synthetic alone, recruited alone, or both in sequence The third row is the pattern most teams underuse. Run synthetic research first to explore the landscape, test variations, and refine the research instrument. Then field a smaller, targeted study with recruited participants against only the finalists. That sequencing lowers recruitment cost because the human study only tests survivors, and raises confidence in the final number because the questions were already stripped of obvious flaws before a real person answered them. ## Where does Subconscious sit in that sequence? Subconscious runs controlled discrete choice experiments, using causal DCE, Mixed Logit, and ICLV methodology, that return causal effects with confidence intervals rather than a directional read. That's the rigor layer synthetic-alone methods can't supply: a defensible population estimate, with error bars, for the moment a finding needs to survive audit or public scrutiny. The practical advantage: a team can move from a simulated experiment to a real-human study without changing the underlying causal question, so the narrow-with-simulation, confirm-with-people sequence runs on the same experimental design, rather than handing the finalists to a separate agency running a different methodology. The fit is bounded, too: Subconscious's positioning is the validation and causal-inference layer for decisions that carry capital risk, not a claim to replace every recruited-human study with simulation. A regulatory submission, a legal filing, or a public percentage claim still needs the sourced, audited number a controlled study produces. ## Running the sequence without wasting the human study - Define the decision, not just the topic. Name the population, the alternatives being compared, and the outcome that decides the action, not just "test the messaging." - Run the simulated pass first. Test the full set of variants, including the ones expected to lose, so the elimination is evidence-based rather than assumed. - Narrow to the finalists. Carry forward only the options that survived the simulated round, typically one to three. - Design the human study around the narrowed set. A smaller recruited sample against fewer options costs less than fielding every original variant. - Run the causal experiment. A controlled discrete choice study against the finalists returns the effect size and confidence interval the earlier simulated round couldn't produce. - Match the claim to the evidence. Directional findings stay internal; only the confidence-interval-backed result goes into a pricing decision, a public number, or a regulatory filing. Skipping step 5 for a capital-committing decision is the mistake this framework exists to prevent: treating an unvalidated directional read as a statistically defended one. Review Subconscious's published replication results and leaderboard, read how the causal experiments are built, or book time to scope a study before committing a budget line to either method. --- # MaxDiff vs. Conjoint vs. NPS: Which Instrument Matches the Decision URL: https://subconscious.ai/blog/methods-and-validation/maxdiff-vs-conjoint-vs-nps-which-instrument-matches-decision Published: 2026-08-07 An insights lead scoping a study has three common instruments to choose from, each answering a different question. Picking the wrong one does not just weaken the results: it fields a study that cannot answer the business question, after the programming and recruiting budget is already spent. ## What each instrument actually measures - MaxDiff (best-worst scaling) asks respondents to pick the best and worst item from small, rotating subsets of a list, producing a forced ranking of items (features, claims, messages) on a shared scale without the halo effect of plain rating scales. - Conjoint analysis (including discrete choice) shows respondents full product or offer profiles built from varying attributes and prices, then asks them to choose between profiles. It estimates each attribute level's utility, letting a team simulate trade-offs such as how much a price increase needs to be offset by an added feature. - Net Promoter Score (NPS) asks a single relationship question, likelihood to recommend, tracked over time, diagnosing loyalty and satisfaction trends rather than feature or price trade-offs. quantilope's comparison frames the MaxDiff-versus-conjoint choice around whether the goal is prioritizing items or estimating trade-offs among interacting attributes; aytm's guide makes the same distinction from a survey-design angle. Neither method substitutes for NPS. ## A decision table, not a preference Treat this table as a first filter, not a substitute for the two guides above, which go deeper into sample-size and attribute-count tradeoffs than a summary table can. ## Where Does the Cost of Choosing the Wrong Instrument Show Up? The failure mode is rarely a survey bug. It's fielding the wrong instrument: a full discrete-choice conjoint when a simple MaxDiff ranking would answer the question, or an NPS trend stretched to justify a feature trade-off it was never designed to measure. Both mistakes surface only after the study is in field, when there's no cheap way back. ## Where Does a Pre-Fieldwork Check Fit In? Before committing budget to any of the three, a team can run controlled comparisons of the actions (features, messages, prices) on a simulated population to see directionally which alternative moves the outcome. Subconscious runs this kind of experiment and can validate the result with real participants, so a team moves from a simulated first pass to fielded confirmation without changing the underlying causal question. This is a scoping step, not a replacement for the instruments above. Subconscious does not package MaxDiff scoring, formal conjoint utility estimation, or longitudinal NPS tracking as ready-made outputs. A team that needs a defensible utility estimate, a representative statistical ranking, or a tracked loyalty metric still needs the dedicated method and real respondents. ## Choosing between the three, in practice - Write down the decision in one sentence: ranking, trade-off, or relationship tracking. - Match the sentence to the instrument above. Resist the urge to pick the method the team already has a template for. - If the decision and stimulus are still fuzzy, sharpen the target group and stimulus definition before writing a single survey question, not add a fourth method to the study. - Where the stimulus set or action space is still broad and the fieldwork cost is high, run a directional simulated comparison to narrow the candidates, then field the already-chosen instrument on the comparison's shortlist. [figure: Three columns. MaxDiff ranks items but misses price sensitivity. Conjoint prices trade-offs but misses a stable ranking outside the tested set. NPS tracks loyalty but misses which change moves the number.] ## Next step See how a causal experiment frames a trade-off question before fieldwork, or how prior studies used real-human validation to confirm a simulated result. For hands-on scoping, book time to walk through a specific instrument decision, or read how the method works. --- # Should You Expand to a New Market? A Causal Framework for Founders URL: https://subconscious.ai/blog/buyers/should-expand-new-market-causal-framework-founders Published: 2026-08-07 A founder should prioritize the candidate market where a controlled discrete-choice experiment measures the strongest causal effect for a clearly defined buyer segment, rather than the market that feels most familiar. A founder with traction in one market is often asked to bet on a second one: a new geography, a new vertical, or a new customer segment. The decision that matters is not whether expansion sounds attractive, but which candidate market to prioritize, and which positioning, price, or offer to test with that market's buyers before committing engineering, compliance, or go-to-market budget. ## Why is this decision hard to get right? Runway and team focus are a startup's least replaceable resources. Spending months chasing a market that never responds is usually discovered only after the GTM spend is gone and the home market has lost attention. CB Insights' 2024 analysis of startup failure found that 43% of failed startups cite poor product-market fit as a primary cause. A wrong expansion bet is the same failure mode, run a second time with less runway to recover from it. Traditional options for reducing that risk are limited: guess based on instinct, ask an existing network biased toward what already works, or commission outside research that takes weeks to return a report of general market-size estimates rather than a testable answer. ## The five questions any expansion decision has to answer Before spending on market entry, a founder needs answers to five questions: - Does the new market have the same pain point the product solves? - Are buying behaviors different enough to require a new sales playbook? - What local competitors or substitutes already exist? - What regulatory or cultural factors could block entry? - Is the timing right, or is the team too early or too late? Each question traditionally requires its own research stream, expert conversation, or competitive scan. ## What causes market-entry bets to fail? Founders usually do not fail on execution. They fail on selection: choosing a market where the pain point is weaker than assumed, where the buying process runs through a committee instead of a single decision-maker, or where a well-funded local incumbent already owns the category. None of those conditions are visible from a TAM estimate. They only show up once real buyers compare specific offers against specific alternatives. ## How does a causal approach test expansion candidates? Subconscious runs a controlled discrete-choice experiment comparing defined positioning, pricing, or message alternatives across a precisely defined buyer segment in the candidate market. The output is a measured causal effect with a confidence interval: which alternative changes buyer behavior, for which segment, and by how much. That is different from an open-ended interview or panel discussion, where a founder collects opinions with no defined alternative to compare against and no measured effect size. "Does this market have the pain point we solve?" and "which message resonates?" are not the same kind of question. The first is discovery. The second is a comparison between defined alternatives, which a controlled experiment is built to answer. ## Running the comparison - Narrow the decision to two or three candidate markets or verticals. - For each, define the buyer segment and the specific alternatives worth testing: a positioning statement, a price point, or a messaging angle. - Run the controlled experiment against each candidate segment and compare the measured effects side by side. - Prioritize the market where the effect is strongest and the buyer segment is best defined, rather than the market that feels most familiar. This does not replace judgment about which markets are worth considering. It replaces guessing about which offer will land once candidates are on the table. [figure: A four-step path: narrow to two or three candidate markets, define the buyer segment and offer per market, run the identical experiment on each, then prioritize the market with the strongest measured effect.] Ranking candidate markets on the same measured effect, rather than on separate conversations, is what turns the comparison into evidence. ## Limitations A controlled experiment does not replace regulatory or compliance review, local competitive intelligence, sales relationships in the new market, or go-to-market execution. It answers one question well: given a defined set of alternatives and a defined buyer segment, which alternative is more likely to change behavior. Audience reach describes the scale of the simulated experiment; it is not the same as recruiting real people to participate. When a decision is consequential enough, a team can move from a simulated experiment to a study with real-human participants without changing the underlying causal question. See /research for how Subconscious structures and validates these experiments, and /case-studies for outcome examples. ## Practical next step A founder does not need a full market-entry study before making a first move. A directional, causally measured comparison between two or three candidate markets is enough to decide where to spend the next quarter of engineering and go-to-market effort. Book a walkthrough or read how Subconscious structures a market-entry experiment before committing budget to a guess. --- # Testing a Podcast Host-Read Script Before It Airs Once URL: https://subconscious.ai/blog/use-cases/testing-podcast-host-read-script-before-airs-once Published: 2026-08-07 [figure: Three script elements shown as variant pairs to test separately: tone (direct language vs. website copy), proof point (specific example vs. generic claim), call to action (short offer vs. long URL or code).] Compare a host-read ad script's tone, proof point, and call to action against a defined listener segment before the host records it. Published limits let a buyer check the number instead of just trusting it. The result estimates how each version changes stated trust and intent to act. It does not predict what the host will actually say on air. ## The decision: which script goes to the host A brand or agency media buyer books a podcast sponsorship slot, often a mid-five-figure to six-figure spend, and owns the ad script brief before it reaches the host. Podcast ad spend is projected to surpass $3 billion in 2025 (Adweek, IAB podcast upfront coverage). Unlike paid social or display, a host-read sponsorship airs once, in the host's own voice, inside an episode the buyer never previews. A script that reads awkwardly, borrows generic website language, or ends on an unspeakable call to action burns the full spend and can cost host goodwill on the next renewal. ## Why doesn't internal review answer the question? A brief reviewed inside the marketing team checks whether the copy is clear and on-brand. It does not check whether the words survive being read aloud to a listener who trusts the host and not the brand. ## How do you compare script variants before they reach the host? Subconscious can run a pre-launch experiment comparing draft variants of the script against a defined target-audience segment, before the buyer commits budget to the slot. Hold the offer and audience constant while changing one script element at a time. Changing tone, proof point, and call to action together makes it impossible to tell which change moved the result. ## Where does the experiment stop and the host's read begin? Naming what a method cannot do is what lets a buyer check it before relying on it. A pre-air experiment tests how a defined audience segment reacts to a script's text. It does not simulate a specific named host's vocal delivery, ad-libbing, or the trust that host has built with their own listeners. A host may still rewrite, cut, or riff on the tested script, and the buyer should expect that divergence. A claim earns trust when its source and its limits are both on the record. Third-party research on podcast host-read effectiveness, including recall and return-on-spend studies, is directional and produced outside Subconscious. It is useful context for a media plan, not a substitute for testing the specific script against the audience segment. ## Moving from a simulated read to real-human validation Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. A number without its limits attached is marketing. That describes audience reach, not a recruitable panel of participants, and it is separate from the number of people in any one study. For a slot large enough to warrant it, Subconscious can also test or validate studies with real human participants, without changing the underlying causal question. Neither route substitutes for the host's own judgment or guarantees the audience's response once the episode is live. Reviewing how a study like this gets built is a reasonable step before choosing between the two. ## Start before the next slot is booked Most teams that buy podcast sponsorships book somewhere between three and ten slots a quarter, enough for one pre-air comparison per slot to compound into fewer wasted reads. Freeze the candidate script language before it goes to the host, define the show's listener segment, and test tone, proof point, and call to action one at a time. Reviewing a comparable pre-launch study before the next slot is booked shows what the output looks like. Set up a study once the candidate scripts are ready. --- # How Independent Strategists Back a Recommendation With Evidence URL: https://subconscious.ai/blog/use-cases/how-independent-strategists-back-recommendation-evidence Published: 2026-08-07 A solo strategist or fractional CMO is judged against agencies with full research departments, without the headcount to match them. The gap shows up at the exact moment a client asks "how do you know?" [figure: Two columns: experience-based claims backed by pattern-matching, open to "how do you know for our customers?"; tested claims backed by a controlled comparison, open only to a test-design challenge.] ## The credibility problem, not the staffing problem A freelancer without a research budget usually has two options: skip primary evidence and lean on secondary sources and experience, or find a way to test the specific recommendation before it ships. The first option is common because traditional qualitative research is priced for agency retainers, not solo project fees. A market breakdown of research costs puts custom qualitative work into four- and five-figure territory per engagement, well past what a single freelance scope can absorb (User Intuition, market research cost breakdown). That math is also why this buyer segment keeps growing: independent and fractional operators are a larger share of how companies buy marketing leadership today than a few years ago, so more solo practitioners are pitching against agency teams for the same engagements (GTM8020, fractional CMO market statistics). ## What's missing from a recommendation based on experience? A recommendation built on secondary research and pattern-matching from past clients can still be correct. The problem is that it reads the same whether it is correct or not: nothing in the deliverable distinguishes a well-informed guess from a tested claim. An agency that includes research in its retainer can point to a specific finding tied to a specific audience. A solo strategist working from judgment alone cannot make that distinction visible. ## Where does a causal test fit into a solo practice? Subconscious runs controlled experiments that estimate which action (a positioning line, a message, a price) moves which outcome, for which segment, with confidence intervals where the design supports them. For a solo operator, that turns one specific recommendation from "my read of the market" into a claim with a stated method behind it, without requiring an in-house research team. How Subconscious runs a study covers the mechanics of setting one up. The stakes-dependent version of this matters too: when a client's decision is big enough to justify it, the same causal question a strategist tested first can move to real-human validation, testing or validating with real participants without changing the underlying question. That option is worth raising only when the size of the decision calls for it; most positioning and messaging checks inside a project fee don't need it. ## Building the habit into engagements A workflow pattern makes this durable across projects rather than a one-off for a single deliverable: - Reuse a small set of question structures across engagement types (positioning, messaging, content direction) instead of designing a new study from scratch every time. - Attach the test result to the deliverable directly: a strategy document that names what was tested and what the result was, not just a conclusion. - Run one test early in a prospective engagement, before the contract is signed, so the proposal itself shows a tested claim rather than a promise to test later. Case studies show what a finished write-up of this kind looks like. ## What doesn't this approach solve? Publishing what a method can't do is what keeps the claims next to it checkable. This approach doesn't remove the cost question. Subconscious has no published freelancer- or solopreneur-priced subscription tier, so a strategist cannot promise a client a specific price comparison against a project fee, and none is claimed here. It doesn't deliver same-day or rapid turnaround; a causal study still requires a properly designed experiment, not an instant lookup. It also isn't a recruitable pool of ready-made audience profiles a strategist can query on demand; the study measures behavior under controlled conditions, and calling it an on-demand roster would misstate how the method works. The advantage for a solo practitioner is narrower and more durable than a shortcut on cost or speed: one tested claim, cited correctly, is something a client can't get from judgment alone, from anyone, agency or freelancer. Book time to scope a study if a specific client decision is the reason to start. --- # Pre-Launch Causal Testing vs. Retail-Panel Measurement URL: https://subconscious.ai/blog/comparisons/pre-launch-causal-testing-vs-retail-panel-measurement Published: 2026-08-07 A causal experiment estimates how a proposed price, claim, or assortment change may move buyer behavior before launch. Point-of-sale and retail-panel data show what happened after the change reached the market. [figure: Two columns: pre-launch causal experiment estimates a proposed change's effect before it ships; retail-panel measurement shows what happened after.] ## Choose the method by the decision in front of you Retail measurement is the right instrument to monitor sales and market performance after launch. One current first-party product page describes its offering as a market-insights data platform for manufacturers (product overview). A controlled causal experiment compares specific actions before launch and estimates which action changes behavior. That distinction matters when the team must commit shelf space, retail support, and marketing spend before observed sales exist. ## Why does the expensive commitment come before the scorecard? A pricing, claims, or assortment decision can trigger slotting fees, marketing spend, and retail commitments. If the choice is based on internal judgment alone, the first strong signal may arrive only after those costs are sunk and the decision window has closed. A pre-launch experiment is not a replacement forecast for every market condition. It answers a narrower, consequential question: among the actions the team can actually take, which one has the strongest measured effect on buyer choice, and how uncertain is that estimate? ## How do you test the action the buyer will encounter? Subconscious runs controlled experiments on simulated populations to compare defined actions such as a price point, a product claim, or an assortment change. Where the study supports it, results include confidence intervals so decision-makers can see the estimate and its uncertainty. Studies can be run against a person-level audience graph covering 800 million real people. That figure describes audience reach, not the number of people recruited into a study. When a team needs another layer of evidence, it can validate the finding with real human participants without changing the causal question. This sequence gives a CPG team decision evidence before it commits retail capital: the experiment narrows which action is worth taking, and post-launch measurement tracks how that action performs in the market. ## Keep the market scorecard Pre-launch causal evidence does not provide point-of-sale tracking, continuous retail-panel measurement, syndicated buyer-survey infrastructure, or supply-chain and distribution analytics. Market conditions can also differ from a controlled study: distribution, competitor actions, inventory, media, and execution may affect the observed result. The strongest evidence sequence uses each method for its own job: - Define the price, claim, or assortment decision before launch. - Run a controlled experiment on the specific alternatives. - Use real-human validation when the decision requires it. - Track the shipped outcome with point-of-sale and retail-panel data. If the open question is which action deserves the retail commitment, discuss the decision and study design. If the open question is how the market performed after launch, keep the retail measurement platform at the center of the answer. --- # Gaussian Process Geospatial Modeling: Beyond Hierarchical Models URL: https://subconscious.ai/blog/methods-and-validation/gaussian-process-geospatial-modeling-beyond-hierarchical-models Published: 2026-08-07 A data science or research lead evaluating a causal experimentation vendor needs to know whether the vendor's model treats geography or segments as related, or as unrelated categorical buckets. The distinction matters the moment a decision extrapolates past the regions or segments with direct data: a model that assumes no relationship between neighboring groups gives the same flat, wide-uncertainty prediction for every untested region and fails to narrow that uncertainty where nearby evidence exists, a direct risk for pricing, messaging, or launch calls that extend beyond tested markets. ## Why categorical grouping breaks down at the edges A common modeling instinct treats geographic region, or any segment, as a categorical variable: observations from the same group share a group-level feature, and that feature explains within-group similarity. The flaw shows up at the boundary. A hierarchical model that groups by county, or by customer segment, shrinks every group toward the same overall mean, but treats a location or segment just across the line from a tested one no differently from one far away. Physical, social, and economic processes that generate this kind of data do not respect administrative or segment boundaries; they vary continuously. A Gaussian process (GP) encodes a different prior: nearby observations should be similar, and that similarity should decay with distance rather than vanish at a category line. ## What does the radon case study demonstrate? The clearest public demonstration of this behavior comes from a Minnesota radon dataset (Gelman & Hill, 2006): 919 household radon measurements across 85 of Minnesota's 87 counties. A standard hierarchical approach, used in PyMC's own multilevel modeling case study, treats county as a grouping variable and shrinks each county's intercept toward a common state mean. That approach has no way to relate a county's expected value to its neighbors'. A geospatial GP model replaces the county random effect with a spatially correlated GP draw, using a Matérn-3/2 kernel built on chordal distance (the straight-line distance between two points through the Earth's interior, computed by converting longitude/latitude to 3D Cartesian coordinates) rather than plain Euclidean distance, since raw latitude and longitude coordinates are not Euclidean on a sphere. PyMC supports this by subclassing its Stationary covariance class and overriding the distance calculation; the PyMC covariance kernel API documents the built-in kernels this extends. The kernel takes the form: where r is the chordal distance between two points, scaled by a fitted length-scale parameter. In this case study, the length scale carries a prior centered at 200 km with a standard deviation of 50 km, reflecting an expectation that counties within a few hundred kilometers share similar radon levels. Fit on the observed counties, the model recovered a southwest-to-northeast radon gradient across Minnesota and used it to generate predictions for 39 untested locations in total: the 2 Minnesota counties with no direct measurements, plus 37 bordering counties across state lines, none of them arbitrary extrapolations to a flat state average. ## Where does prediction uncertainty come from? The result that matters for a buyer evaluating a vendor's modeling discipline is not the point prediction, but what happens to the uncertainty around it. In this case study, posterior predictive standard deviation stayed small within Minnesota, where observations are dense. The two unobserved Minnesota counties also kept tight uncertainty, because they sit surrounded by measured neighbors. Uncertainty widened progressively for counties farther into neighboring states, tracking distance from the training data rather than jumping to a flat, uninformative value outside the observed set. A naive hierarchical model cannot make that distinction. Its posterior predictive uncertainty stays the same width for any unobserved group, no matter whether that group sits next to dense evidence or far away from it. ## The underlying discipline applies past geography The same discipline generalizes past latitude and longitude. Any vendor's causal experiments face the same question for segments or markets with thinner direct coverage: does a result for one tested segment inform a related, untested segment, or does the model treat every untested group as a blank slate? Subconscious's causal experiments estimate effects with quantified uncertainty for the segments actually tested, rather than resetting to an uninformative default outside direct coverage. See research and the leaderboard for how that uncertainty is reported across studies, and how we work for the process from a tested segment to a broader read. ## Limitations This is one open tutorial built on one public dataset and one specific kernel choice; it is not a benchmarked or replicated production capability claim, and it says nothing about the internal architecture, accuracy, or deployment specifics of any particular vendor's model, including Subconscious's. The model itself also omits the radon dataset's basement/floor indicator for simplicity, which its own documentation flags as a natural extension rather than a finished result. [figure: Comparison: hierarchical model falls back to a flat state mean with constant uncertainty for any unobserved county; GP model shows a spatial gradient, uncertainty narrow near measured counties and widening with distance.] ## What to check before trusting a vendor's regional or segment claims Before trusting any vendor's regional or segment-level effect estimate, ask what happens to its confidence interval outside the tested set: does it stay artificially tight and constant everywhere, or does it widen as coverage thins out? A model whose uncertainty does not respond to distance from evidence is making the same error a naive hierarchical model makes with county boundaries, just with a market map instead of a state map. --- # 10 Things to Check Before You Trust an AI Audience Simulator (2026) URL: https://subconscious.ai/blog/use-cases/ai-audience-simulator-platforms-compared-2026 Published: 2026-08-07 A marketing or brand leader testing a campaign concept, headline, or price against a target audience has a real menu of AI audience simulators to pick from. The dangerous mistake is trusting a synthetic panel that was never calibrated or validated, greenlighting a launch decision on that basis, and finding out after the campaign underperforms, or after a comprehension or fairness problem surfaces with real customers, that the read was wrong. Vendor rankings do not protect against that outcome, because a demo can make almost any panel look convincing. What protects against it is knowing which properties of a simulation approach predict whether its output can be trusted, and checking each one before the budget commitment. [figure: A checklist diagram showing five of the ten calibration, structure, and iteration properties a team should verify about an AI audience simulator before trusting its read for a real spending decision.] ## The ten properties worth checking These ten checks apply to any audience-simulation approach, whatever it is called and whoever built it. Each is a question to put to a vendor, or your own team, before a simulated read decides a real spending decision. ### 1. How is the audience calibrated? A tool that assigns demographic labels to generated personas is not the same as a tool that grounds each simulated respondent in verifiable evidence about how that kind of person actually behaves. Ask what data anchors each respondent, and whether the vendor can show the grounding, not just describe it. Research on generating synthetic survey responses with large language models finds that response quality depends heavily on how the underlying population is represented and conditioned, not just on model size (arXiv, 2026). ### 2. Whether the response format is structured or conversational A useful audience simulator returns structured comparisons: intent, comprehension, sentiment by segment, and where responses disagree. A tool that mainly produces chat-style commentary is closer to a brainstorming aid than a decision input. Before a study runs, confirm the output format will answer the comparison question, not just narrate around it. ### 3. Whether segment detail survives the rollup An aggregate top-line number hides the segment where a message fails. Ask whether the platform reports cross-tabs by the segments that matter, or whether detail gets averaged away before it reaches a dashboard. ### 4. What re-running a study actually costs Iteration is the point. A team that can only afford to test one version of a headline learns nothing about the eight variants it didn't run. Ask how a second, refined pass is scoped and priced, and confirm it preserves comparability with the first pass. ### 5. How does the method distinguish a controlled comparison from a single reaction? Asking a language model to react to one stimulus, and asking it to compare two or more alternatives under a controlled design, answer different questions. Research examining LLM-simulated experiments has found that a single-shot reaction to a prompt behaves like an observational read of the model's training data, not like a randomized intervention, unless the study is explicitly structured to compare defined alternatives (arXiv, 2026). Ask which kind of study the platform runs. ### 6. Are population-scale claims separated from panel-scale reality? Some platforms describe simulating an entire market; others run against a bounded, purpose-built panel. A vendor should be able to state how many simulated respondents inform a given result and what population that panel represents, rather than leaving "scale" as a marketing adjective. ### 7. How validation against real people is reported, and where the limits are disclosed Ask what the vendor's evidence shows about how closely a simulated result tracks a real-human result, on what kind of question, and where the two diverge. A study of how well large language models can reproduce individual survey respondents using socio-economic microdata found agreement varies by question type and demographic subgroup, rather than holding at one constant accuracy figure (arXiv, 2026). Treat a single unqualified accuracy percentage as a claim to interrogate. ### 8. Whether a simulated result can move to a real-human check without changing the question For consequential decisions, the simulated study should hand off to a real-human study that tests the same causal question, not a different one. Subconscious can test or validate a study with real human participants, letting a team move from a simulated read to a human-baseline check without redesign. ### 9. What audience the platform can actually reach A simulator's usefulness for a specific brief depends on whether it has real evidence about the audience in question, not just a plausible-sounding persona. Subconscious runs controlled studies against a person-level audience graph covering 800 million real people, a source of grounding evidence rather than a recruitable panel of respondents. Ask any vendor to be equally specific about what "audience" means in their system. ### 10. Whether the operating model matches how often you will actually run studies A self-serve workflow, an analyst-mediated service, and a full enterprise implementation carry different levels of analyst involvement, study constraints, and minimum commitments. A team running ten concept checks a month needs a different operating model than a team running one high-stakes pricing study a quarter. Match the model to the cadence of decisions you need to make, not the cadence a sales conversation assumes. ## A quick-scan reference ## Where causal action-testing fits among these ten Subconscious answers most directly to properties 5, 8, and 9: it runs controlled, discrete-choice-style experiments that compare defined actions rather than reading a single reaction, it supports moving a validated result to real human participants without changing the underlying question, and it grounds studies in a specific, disclosed audience graph rather than an unspecified panel. That's not a substitute for checking the other seven properties against any vendor, including Subconscious. ## Before the budget commits Score any AI audience simulator against the ten properties above, using the vendor's own disclosed evidence rather than a demo. For decisions where a wrong read is expensive, plan a step where a simulated finding gets checked against real people before it becomes a media buy or a launch decision. Research walks through how that causal design and validation work in practice, and the leaderboard tracks how simulated results compare against real-human benchmarks across study types. Teams that want a walkthrough of the operating model can see how the work happens or book time to scope a first study. --- # Where to Put the Human-Review Gate When AI Drafts Market Research URL: https://subconscious.ai/blog/implementation-and-procurement/where-put-human-review-gate-when-ai-drafts-market-research Published: 2026-08-07 An insights leader who lets an AI draft move straight into a pricing deck or a launch brief has made a quiet decision: that draft is now evidence, not a hypothesis. The decision that matters is where the human-review and validation gates sit in the workflow, and which evidence tier each AI-assisted output is allowed to support before it reaches a business decision. ## The Failure Mode Is Not the AI Draft Itself The risk in an AI-assisted research workflow is rarely a bad answer; it is an ungated output treated as a finished finding. A generic or ungrounded read ships into a pricing, launch, or positioning decision, the budget commits on its authority, and the mistake surfaces after the spend is gone. A researcher who forwards every AI draft without a review step loses the credibility that made the research function decision-relevant. Before adding an AI step to a research pipeline, ESOMAR's buyer guidance recommends pressing any AI-based service on how its outputs are grounded, validated, and disclosed to the people who will act on them (ESOMAR, 20 Questions to Help Buyers of AI-Based Services for Market Research and Insights). The same questions work internally: which layer is this output allowed to support, who reviewed it, and what would it take to move it up a tier. ## What Are the Four Layers of the Evidence System? A workable version separates AI involvement into four layers, each with a different evidence tier and a different owner: The value in this chain is not the AI output by itself. It is the recorded, repeatable path from a question to a decision the business can defend later. ## Where Does a Simulated Experiment Belong in the Chain? This is the layer where Subconscious fits: turning an AI-generated hypothesis into a controlled, directional read by running randomized experiments against a simulation of the market, compared against alternatives, before anyone commits budget. That is a distinct evidence tier from an AI-drafted summary or an unstructured chat answer. The distinction matters because a simulated read and a validated read answer different questions. A simulation compares how options perform against each other under controlled conditions; validation confirms the answer holds when real people respond. ## The Validation Gate: When to Move to Real Humans Not every decision needs the same evidence tier. A concept test that only informs an internal brainstorm can stop at directional testing. A pricing change, a launch claim, or anything stated publicly needs the next gate: real-human validation on the same causal question, not a different one improvised for the fielding stage. Subconscious can test or validate studies with real human participants, which lets a team move from a simulated experiment to fielded validation without rewriting the question the business is trying to answer. That handoff has a boundary. A simulated experiment run this way is a pre-commitment read, not a replacement for fieldwork, and it is not a recruitable respondent pool. Any AI-assisted step in the chain, whether it runs on Subconscious or a general-purpose model, needs a named human owner who checks grounding, neutrality, and fit to the decision before the output leaves the research team. ## What to Change This Week - Pick one live project with a real decision behind it. - Write the business decision the research needs to answer in one sentence. - Assign each AI-assisted step to one of the four layers above, in writing. - Name the human owner who checks grounding and neutrality before any output moves up a tier. - Decide in advance which findings require real-human validation before they can be stated externally. Repeat the exercise on the next study. The output is not a shorter report. It is a documented evidence chain a stakeholder can question and a researcher can defend. ## Where Does This Fit in a Broader Research Program? Directional simulated testing is one stage in a larger research program, not a standalone shortcut. Teams evaluating where it fits can review worked examples in case studies or map their own evidence layers onto a live decision in a working session. A walkthrough of the process end to end shows how the four layers connect in practice. [figure: Four-stage chain: Exploration, Directional testing, Human review, Validation. Each has a gate; a finding cannot skip ahead to a business decision.] --- # AI-Simulated Focus Groups vs. Causal Testing: What Each One Proves URL: https://subconscious.ai/blog/comparisons/ai-simulated-focus-groups-vs-causal-testing-what-each-one-proves Published: 2026-08-07 A research or marketing team evaluating an AI-moderated synthetic focus-group tool is usually asking one question: does a panel of AI participants describing their reaction give the team enough evidence to ship, or does the decision need a test that measures which action actually moves behavior? Those are different kinds of evidence, and treating one as a substitute for the other is the actual risk. A synthetic focus group collects stated opinions. It does not, on its own, measure what a market of buyers would do if that concept, price, or message shipped. ## What does a synthetic focus-group tool measure? Tools built around AI-simulated focus groups run structured sessions: an AI moderator guides AI panelists through a discussion of a research topic. OpinioAI is one example of this category, offering self-serve, moderator-led synthetic focus-group sessions aimed at researchers and product teams who already run focus groups and want a faster or lower-cost version of that same format. That format has a real, bounded use: exploring reactions to a concept, surfacing language a team hadn't considered, and rehearsing how a group discussion might unfold before recruiting real participants. It is a qualitative, exploratory read, not a measurement of causal effect. ## Where stated opinion runs out A focus-group format, human or AI-moderated, collects what participants say about an idea. It does not isolate which specific version of a message, price, or concept changes an outcome, because no session compares outcomes across randomized alternatives. This is the same say-do gap that traditional focus groups have always carried: what a participant says they would do and what a market actually does can diverge. Shipping a launch, price, or message because it played well in a synthetic panel discussion means acting on stated opinion where the study never tested the causal question: would this specific action outperform the alternative, and by how much. ## What does a controlled experiment add? Subconscious is a causal behavioral platform, not a focus-group tool. It runs randomized controlled experiments against a specific action, such as a message, a price, or a concept, on a simulation of the market, and reports which version changes the outcome with a quantified confidence interval, rather than a transcript of stated reactions. That distinction matters most at the decision point a focus group can't reach: choosing between two or more concrete alternatives before spending on a launch. A study built this way can also move into real-human validation without changing the underlying causal question, testing the same action against the same design with recruited human participants instead of the simulated panel. A published number needs its limits attached or it becomes marketing. Subconscious's replication accuracy against real human outcomes is measured and published on the leaderboard: our best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959). Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. It is a validation result, not a guarantee for a new market. ## Comparing the two approaches ## Using both without confusing what each proves The two are not competing for the same research budget line. A synthetic focus-group session is a reasonable way to explore a concept early and generate language. It is a weaker basis for the decision that follows: which version of that concept, message, or price should actually ship. For a team already running AI-moderated sessions, the practical addition is not replacing that exploratory step. It's adding a controlled test before the launch decision itself, so the choice rests on a measured effect instead of how a concept played in one simulated discussion. See how Subconscious runs a study or book a walkthrough before committing budget to an untested message, price, or concept. OpinioAI's own pricing is structured around self-serve access to that focus-group format (Pricing | OpinioAI), which fits a team whose evidence need stops at the exploratory stage. ## Limitations The misses belong on the same public leaderboard as the hits. Subconscious does not run AI-moderated group discussions or produce session transcripts; that is a distinct, exploratory research format Subconscious does not claim to replace. Subconscious's person-level audience graph is a simulation infrastructure for controlled experiments, not a recruitable panel of respondents; real-human validation draws from a separate, dedicated participant pool. [figure: Two columns. Focus group: stated opinions, answers what the group says, fits early exploration. Experiment: randomized comparison of alternatives, answers which action changes the outcome, with a confidence interval.] --- # Why a Media Mix Model Number Needs a Controlled Test Before It Moves Budget URL: https://subconscious.ai/blog/methods-and-validation/why-media-mix-model-number-needs-controlled-test-before-moves-budget Published: 2026-08-07 A media mix model can tell a VP of Marketing Analytics that paid social drove 18% of last quarter's revenue. It cannot tell them, on its own, whether cutting paid social next quarter would actually cost that revenue. That gap between a regression estimate and a causal one is the decision this article is about: should a team shift budget on the model's number alone, or hold the number until a controlled test confirms it? ## What does a media mix model actually estimate? A Bayesian media mix model (MMM) fits a regression across historical channel spend and outcomes such as revenue or conversions. It accounts for carryover effects (adstock, the fading influence of past spend) and diminishing returns (saturation, where extra spend on a channel earns less each additional dollar). The output is a contribution estimate per channel and, from that, a suggested reallocation of the next period's budget. Publishing the assumption behind a contribution number is what lets a buyer check it before spend moves. The method's causal claim rests on an unconfoundedness assumption: that spend is not correlated with unobserved drivers of the outcome. That assumption is fragile because media spend is chosen by marketers, not randomized, and the model fits the pattern in historical spend and outcome data without running an experiment. When two channels moved together historically, for example a paid search campaign that always launched alongside a TV flight, the model can struggle to separate their individual contributions. Google's own applied research frames MMM as a causal inference method that is valid only under assumptions such as no unobserved confounding of the spend-outcome path, and treats controlled experimentation as the source of informative priors that make those assumptions more defensible (Google for Developers, "About MMM as a causal inference methodology"). ## What calibration methods close the gap between an MMM estimate and a measured effect? Two experimental designs are the standard way analytics teams check an MMM estimate against real behavior: - Geo-experiments. Split matched geographic markets into a treatment group (spend changed) and a holdout (spend unchanged), then compare outcomes. The designed geo-experiment literature describes this as a way to recover a channel's incremental effect directly from a designed intervention, rather than inferring it from historical variation (Vaver & Koehler, "Estimating Ad Effectiveness Using Geo Experiments," 2011). - Difference-in-differences. Compare the change in outcomes for a group exposed to a spend change against the change in a comparable unexposed group over the same period, isolating the effect from other trends moving both groups together. The reason a team runs a geo-experiment or difference-in-differences test is that a miss should show up on the record before budget moves. Both designs exist because the regression alone cannot rule out confounding. A team that skips this step and reallocates budget purely on MMM output is trusting a fit, not a measured effect. ## Where this decision sits before spend, not just after it The same principle, that a number needs a controlled comparison before it earns budget, applies earlier than the media plan. Before a launch, a price change, or new messaging goes live, a team can test the action itself: put alternatives in front of respondents in a controlled discrete choice experiment and estimate the causal effect of the tested attributes on stated choice within the experiment, with confidence intervals attached over the simulated population, though stated preference tends to run high relative to real-world behavior (hypothetical bias). Subconscious runs this kind of controlled experiment for pre-launch product, pricing, and messaging decisions, and a team can move from a simulated version of that test to real-human validation to confirm the same effect holds outside the simulation. ## Where this does not apply Naming what a method does not cover is what lets a buyer pick the right tool for the decision in front of them. Subconscious does not fit historical media-spend time series, produce a media mix model, or replace adstock and saturation regression on past spend data. It is not a geo-experiment platform for allocating an existing budget. For a team recalibrating what already happened across channels, MMM and geo-experiments remain the right tools. Subconscious's fit is upstream of that: testing an action or an alternative before it is committed to, not reallocating spend that has already run. ## The practical rule Treat an MMM contribution number the way a geo-experiment team already treats it: as a hypothesis worth testing, not a budget decision on its own. If the model says a channel drove 18% of revenue, the fastest way to find out whether that number is real is to hold spend back in a matched sample and watch what changes. Book time to see how the same test applies to a decision that hasn't been made yet. [figure: Five-stage path: an MMM estimate, treating it as an unconfirmed hypothesis, a geo-experiment or difference-in-differences test, a fork on whether it confirms the effect, and budget moving only if confirmed.] --- # Validating Agentic Research Output: A Five-Layer Eval Framework URL: https://subconscious.ai/blog/use-cases/validating-agentic-research-output-five-layer-eval-framework Published: 2026-08-07 Every automated research pipeline runs into the same question: how do you know the output is real before you act on it? The cost of running a study has dropped so far that unvalidated output can now outpace anyone's ability to check it by eye. Trusting a result means picking a validation layer that matches how much is riding on the decision, not applying the same check to every study. ## What "accurate" needs to mean before you use it "The panel is 87 percent accurate" means nothing until you say against what. Three different things get measured under that label: - Stated-preference fidelity: does the output match what a matched group of real people would say to the same question? This is the most common benchmark and the easiest one to run. - Behavioral prediction: does the output correctly predict what people actually do (click, buy, churn)? Harder to measure, and where automated methods are structurally weakest. - Decision-quality outcome: did using the research lead to a better business decision than the alternative? This is what actually matters, and it is rarely measured because it needs a year of follow-through on decisions that were already made. Most published accuracy numbers measure the first one. The second and third are where the real validation work sits. ## Five layers, from routine-and-frequent to rare-and-decisive Treat validation as a stack, not a single gate. Each layer catches a different failure mode, at a different cost. ### Layer 1: automated sanity checks, on every response Run in the loop, at close to zero added cost: internal consistency across personas in the same segment, whether the response actually answers the question asked, and whether the language and reasoning fit the persona it claims to represent. Failures here usually mean the brief was malformed, not that the underlying method is broken. ### Layer 2: cross-run triangulation, per study Within one run, look at where personas agree and disagree. Responses inside a tight segment should cluster; if a study designed to surface disagreement between segments instead produces convergence everywhere, the run is flattening real variation. The two or three most extreme responses are usually either the most useful signal or the most broken output; read them by hand rather than averaging them away. ### How do you benchmark against historical human data? Keep a small set of questions (5 to 10, spanning the categories you actually study) where you already know the real-human answer, its sample size, and its date. Re-run the same questions on a cadence and track the delta over time. Small drift is normal; sudden drift usually means the model behind the method changed underneath you without an announcement. ### When should you pair a decision with a real-human check? When a study is about to inform an actual decision (a launch, a price, a campaign), pair it with a smaller real-human check on the same causal question. This is the highest-value layer because it is where the money actually moves: the team that pairs its top few decisions a quarter with real-human validation learns more about its own method's reliability than any number of generic benchmarks produce. Subconscious fits this layer directly: it can test or validate studies with real human participants, running the same causal experiment against a matched human sample without changing the underlying question. ### How often should you backtest against reality? On a yearly cadence, revisit the major calls a team made and grade each study against what actually happened. Decision quality is what this layer measures, and it's the layer teams skip most, since it means holding a researcher to a forecast made twelve months back. Everything above it correlates with the answer; this layer is closer to causal. ## Failure modes that slip past the first three layers A few patterns show up repeatedly in production, independent of platform: - Persona over-fitting. The output describes the persona instead of answering as the persona, using consultant-slide phrasing instead of a plain answer. The fix is a tighter brief, not a bigger model. - Agreement collapse. Every segment converges on the same answer. Usually a symptom of an underlying model update; Layer 2's between-segment check catches it. - Recency blindness. Output lags a market shift the model hasn't seen. Symptom: it doesn't know about something that shipped in the last quarter. - Sycophancy. Output agrees with whatever framing the question implies. Catch it by running the same study with the framing negated and comparing. - Feedback-loop drift. A method trained partly on its own earlier outputs drifts away from real-human ground truth across generations. Only Layer 3, benchmarked against fresh human data, catches this one. A Stanford study supports this risk: generative agents completed the American Voices Project interview, then predicted those same 1,052 participants' answers on the General Social Survey and personality batteries. The agents reproduced individual response patterns with meaningfully more fidelity than demographic-only personas, though predictions still diverged from the people they modeled (Stanford HAI). That gap between "closer than a demographic baseline" and "confirmed by the person" is what Layer 4 exists to close. ## What to ask a platform before you trust its number Before treating any platform's accuracy claim as usable, ask three questions: What does "accuracy" mean in the benchmark, precisely? How does the platform handle model updates that shift response patterns, and does it tell you when that happens? Can you re-run the benchmark yourself, on demand, against the current model version? A number without a stated method and a checkable date is marketing, not evidence. Subconscious does not publish one blanket accuracy percentage in place of this, and it does not replace a team's own outcome-backtesting discipline or the executive accountability for the decision itself; Layer 5 stays a human responsibility regardless of which method fed Layer 1 through 4. [figure: Five failure modes, each with a symptom: consultant phrasing, segments converging, missing recent shifts, agreeing with framing, drifting from human ground truth.] ## Why the volume changes the stakes When research ran at human pace, a bad study took weeks to produce, the cost was visible, and someone usually noticed if the output felt off. At automated volume, with hundreds of runs a week per team, that friction disappears, and unvalidated output compounds without anyone noticing. Teams that get this right run Layers 1 through 3 by default, pair Layer 4 with every decision that matters, and backtest once a year. Teams that skip straight to "the model ran a study, here's the recommendation" find out later that the recommendation was confidently wrong. The question is not whether to validate. It is at what depth, matched to what is riding on the call. For the decisions where that call is expensive to get wrong, see how Subconscious pairs a causal study with a real-human check. [figure: Five steps from frequent, low-cost checks to rare, high-stakes ones: sanity checks per response, triangulation per study, benchmarking on a cadence, human pairing before a decision, annual backtest.] --- # What Hotjar Can't Tell You About Why Users Leave URL: https://subconscious.ai/blog/comparisons/what-hotjar-cant-tell-about-why-users-leave Published: 2026-08-07 Say your dashboard shows a 67% drop-off on the pricing page. Hotjar can show exactly where visitors stall and where they leave. It cannot tell you whether they left because the enterprise plan looked expensive, because the feature list confused them, or because a competitor's page made a clearer case. That gap is where a product or growth lead who already runs Hotjar has to decide: keep guessing and ship a redesign on faith, or test the candidate fix before spending engineering time on it. ## What behavioral analytics actually observes Hotjar captures real visitor behavior on a live site: heatmaps, session recordings, and funnel reports showing where a flow breaks down (What Is Hotjar? Key Features & Why You Should Use It). Recordings track clicks, scrolling, and mouse movement, showing what a visitor did rather than what they say they did (Watch Users Scrolling, Moving, And Clicking With Recordings). A capability is only useful to a buyer once its edges are stated plainly. That data comes straight from live traffic, but it is strictly retrospective. It tells a team what happened after the fact, on the page as currently built. It cannot run a page that doesn't exist yet, and it cannot ask a visitor why they hesitated on the way out. ## The guess that follows the drop-off A 67% drop-off is a fact. The reason for it is a hypothesis. Was the price too high? Was the enterprise tier unclear? Was the visitor just comparison shopping? Behavioral analytics narrows down where to look; it does not settle which of several plausible causes is the real one. Teams that only have behavioral data tend to pick the most plausible-looking cause, redesign around it, ship, and wait weeks to find out whether the guess was right. When it wasn't, the engineering cycle is gone, and the team is back to guessing. ## How do you test the cause before committing engineering time? This is the decision a causal experiment is built to shorten: test the candidate explanations and candidate fixes before shipping a guess. Subconscious runs controlled experiments on simulated audiences to test whether a specific change, a different price point, a rewritten claim, a reframed feature, actually changes the decision, with confidence intervals where the study design supports them. This does not replace what Hotjar does. It sits upstream of it. Hotjar finds where the drop-off happens and confirms whether a shipped change moved the number. A causal experiment tests which of the candidate explanations actually holds, and which fix addresses it, before anything ships. ## Where each approach stops Listing what a method can't do next to what it can is what makes the claim checkable. A causal experiment run on a simulated audience does not replace continuous behavioral monitoring, and it does not observe real clicks or scroll behavior on a live page; only an instrumented site with real visitors can do that. It also isn't a usability study or a clinical trial: it stays a test of which action changes a decision. A limit stated in plain terms is what turns a claim into something a buyer can verify. The reverse limit holds too. Behavioral analytics cannot test a page that hasn't been built, cannot ask a visitor to explain a choice, and cannot isolate one variable, price, wording, framing, from every other thing that changed between two live sessions. ## When should you confirm a result with real people? When a decision is significant enough to warrant it, a team can move from a simulated experiment to real-human validation, carrying the same causal question into a study with recruited participants. Reach for it when a wrong call would be expensive; for smaller fixes, the simulated result and the next round of live analytics are usually enough. ## How do you put behavioral analytics and causal testing together? A workable loop looks like this: let behavioral analytics surface where the problem is, test the candidate causes and fixes before writing code, ship the version that held up under testing, then use behavioral analytics again to confirm it worked on real traffic. Teams already running Hotjar don't need to replace it to close the observation gap; they need a way to test the fix before it ships. Subconscious's research method is built for that step, and how Subconscious works covers what a study looks like end to end. Examples of tested changes across industries are in the case studies, and a demo is the fastest way to see a study design against a real pricing or messaging question. [figure: Five-stage chain: drop-off found, cause untested, candidate fixes tested before shipping, tested version ships, analytics confirms it worked, looping to the next drop-off.] --- # Checking a simulated immigration-attitudes study against Hainmueller & Hopkins 2015 URL: https://subconscious.ai/blog/case-studies/checking-simulated-immigration-attitudes-study-against-hainmueller-hopkins Published: 2026-08-07 A research or policy team weighing whether to trust a simulated discrete-choice result on a politically sensitive topic needs a prior human benchmark, not a general accuracy claim. This entry compares a Subconscious-simulated discrete-choice run to a published, peer-reviewed study on U.S. immigration attitudes. ## The published study behind the comparison Hainmueller and Hopkins ran a conjoint experiment asking 1,407 U.S. adults to evaluate hypothetical immigrant profiles varying by education, English-language skills, country of origin, job experience, and other attributes, then rank which profiles they preferred (The Hidden American Immigration Consensus: A Conjoint Analysis of Attitudes toward Immigrants, American Journal of Political Science; also available via the Immigration Policy Lab). Subconscious ran a simulated version of the same discrete-choice question, testing preferences over immigrant-profile attributes including education, English skills, country of origin, job experience, work plans, reason for migration, and prior U.S. trips, and compared the ordering of preferences against the published human result. ## What does the correlation show? The simulated and human orderings correlated at r_s = .5406. That is a Spearman rank correlation between how the simulation ranked immigrant-profile attribute levels and how the real study's respondents ranked them, a measure of relative ordering agreement, not a percentage-accuracy score. No sample size is reported for the simulation, so no significance claim is made here. A correlation in this range is moderate, not a claim of equivalence. It says the two rankings tend to agree more than they disagree; it does not say which specific attribute rankings matched and which diverged. Only one summary statistic is available for this comparison; there is no reported sample size, per-attribute breakdown, or confidence interval on the Subconscious side. ## What does this comparison not establish? One moderate correlation on one topic, U.S. immigration attitudes as measured in 2015, does not establish accuracy for a different topic, population, or decision. Do not round r_s = .5406 up. This entry also compares a simulation to a prior published study, not to a freshly recruited human panel. When a decision turns on live respondent behavior rather than a decade-old benchmark, Subconscious can validate the same causal question with real human participants. ## Reading this comparison before a fielding decision Before treating this single comparison as grounds for a policy-messaging or attitudes-research decision, check three things: whether the topic and attribute set match, whether a moderate correlation suits the stakes, and whether a 2015 benchmark still reflects current attitudes on a topic that shifts with policy and news cycles. See the full leaderboard for how this entry sits alongside other published comparisons, and research for the underlying replication method. [figure: Decision path with three checks: topic match, correlation strength versus stakes, and benchmark currency. Passing all three means reuse the comparison; failing one means run a new human-validated study.] A team deciding whether to trust a simulated result for its own attitudes or policy-preference question should look for the closest topic match on case studies and confirm the comparison method fits the decision at hand. Teams that need current-population confidence can move to how Subconscious works with real human participants on the same causal question. --- # When a Synthetic-Customer Read Is Enough, and When You Need a Causal Experiment URL: https://subconscious.ai/blog/methods-and-validation/ai-product-validation-synthetic-customers Published: 2026-08-07 A product or research leader has to decide, before engineering capacity or launch spend is committed, whether a synthetic-customer read is sufficient evidence for a feature, positioning, or pricing call, or whether it needs a controlled experiment first. The answer depends on the kind of question being asked, not on how confident the panel output sounds. ## Two different questions get asked of the same panel Synthetic-customer panels answer stated-preference questions: what a persona says it thinks, prefers, would choose, or would pay when shown a description. They do not answer observed-behavior questions: what a real customer does when a purchase, a workaround switch, or a real price is on the table. Prior work on whether simulated respondents built from language models can stand in for survey panels has found the match to human answers holds up better for some question types than others (Argyle et al., "Out of One, Many," on generating [synthetic respondent samples with language models](https://static1.squarespace.com/static/56bf7bbc5559867615ddc81a/t/6155e3a5b1b71173080762f6/1633018790791/Out_of_Many_One_Argyle_etal_Combined2.pdf)). Treating a directional read from a synthetic panel as a causal answer is the failure mode: a team ships a feature, a message, or a price that never changed real buyer behavior, and only finds out after the engineering time and launch spend are gone. The fix is not to distrust synthetic reads; it's to match the method to the decision's stakes. ## Where a directional read is the right first pass Four situations are well suited to an open-ended synthetic-customer screen. ### What is pre-launch feature screening? Before committing engineering capacity to a feature, running it past a synthetic-customer panel surfaces whether the persona understands what it is, why it would matter, and how it compares to whatever workaround it already uses. It gives a directional read on whether the feature makes sense and which scoping choices matter, not a go/no-go verdict. ### What is positioning screening? Different positioning variants shown to the same persona set reveal which framings read as confident versus defensive, plainspoken versus jargon-heavy, on-brand versus off. This is useful triage before a positioning decision gets locked into launch collateral. ### What is pricing-tier screening? Asking a persona which tier feels right, which feels too cheap, and which feels too expensive produces a categorical signal, useful for choosing among tier structures or feature distributions. It is not precise enough to set a price point. ### Segment-level reaction mapping Running the same launch communication past personas built for each priority segment surfaces which segments respond well and which need different messaging, feeding sales-enablement and customer-success planning. ## Running a screen that produces a usable signal A screening read is only as good as its inputs; four choices determine whether the output is signal or noise. Build the persona set around actual segments. Generic personas produce generic output. A persona library built from the team's real ICP segments, typically three to seven personas covering the priority segments, with a demographic profile, role context, and relevant attitudes, makes later panels comparable to each other. Frame the stimulus as a task, not a preference check. A prompt like "do you like this feature" produces low-information output. Prompts that ask a persona to explain the feature in its own words, then name one workflow where it would use it and one where it would not, produce reasoning the team can act on. So does asking which of two options a persona would pick and why, or asking a persona to list its biggest objections before trying a product. Read the panel as a distribution, not a single vote. One synthetic respondent is one data point. A panel of several personas is a distribution: where reactions cluster, where they diverge, and which segment breaks from the pattern is the useful output. Decide to ship, kill, or refine, not just record a reaction. Most screening rounds should end in refinement and a second pass, not a binary verdict on the first read. ## What a directional read can't tell you Publishing this limit lets a buyer weigh the read against the decision it will inform. It doesn't extend past what a language model has seen. A genuinely novel product category with no analog in the training data produces extrapolation, not measurement. Naming this boundary here is what lets a compliance team check it before relying on the read. It isn't evidence for a regulatory or compliance filing. A claim filed with a regulator needs real human respondents on record, not a simulated read. This gap is listed here so a buyer targeting a narrow segment can see it before running a panel. It thins out for niche audiences with little public signal. Mainstream consumer and common B2B roles are well represented in a language model's training; narrow roles in small industries are not. The failure mode is stated plainly so a team can check it against a real purchase decision before betting on the read. It doesn't capture behavior under real stakes. A persona answering a hypothetical question behaves differently than a real customer facing a real purchase decision, a real time constraint, or a real switching cost. ## When the decision needs a controlled experiment For the launches that move revenue, the pricing changes that touch material spend, and the positioning that defines the brand, a directional screen is not the same evidence as a controlled test. Subconscious runs controlled discrete-choice experiments that compare defined alternatives against a defined population and a defined outcome, and return causal effects with confidence intervals scoped to the simulated population, rather than one aggregated reaction. Current results are tracked on the leaderboard. Where warranted, Subconscious can test or validate studies with real human participants, so a team can move from a simulated experiment to real-human validation without changing the underlying causal question. Audience reach in a simulated experiment and a recruited real-human validation study are distinct claims and should not be blended into one number. [figure: Four-item list: build personas from real ICP segments, frame the stimulus as a task not a preference check, read the panel as a distribution of reactions, end with a decision to ship, kill, or refine.] ## Limitations that carry over either way A causal experiment answers which alternative changes the outcome and by how much. It does not replace customer discovery interviews, usability observation, or watching how a feature performs once it's in market. Real-human validation, when it's used, tests a model's structure against real stated choices; it does not turn a causal-choice test into an observed usability session, a clinical trial, or automatic proof of market performance. [figure: A branching path: the question splits into a stated-preference route to a synthetic-customer screen, and a randomized-design route to a controlled experiment. Both converge on ship, kill, or refine.] Method choice should follow the decision's stakes, not the other way around (a Wiley review of generating consumer-research samples with large language models, covering the challenges, opportunities, and guidelines involved). --- # AI Personas or Causal Experiments: Where Research Agencies Should Use Each URL: https://subconscious.ai/blog/buyers/ai-personas-causal-experiments-where-research-agencies-should-use-each Published: 2026-08-07 Research agency and consultancy leaders face one recurring decision: which engagement stage can run on an AI-driven method, and which requires a controlled experiment or real-participant research before a client acts on it. Get that sequencing wrong and a directional AI read gets presented as a defensible finding, the failure mode that costs agencies client trust. [figure: Four-step routing: hypothesis generation, instrument pre-testing, and interim updates route to AI persona sessions; the final recommendation routes to a controlled experiment or real-participant research.] ## Why this decision matters Clients now know some research work can move faster than a traditional agency timeline. Agencies that show exactly where that speed comes from, without diluting rigor, win the engagement; agencies that overpromise what a fast method can prove, or under-deliver on turnaround, don't. The risk sits at the handoff point. A hypothesis from an open-ended AI persona conversation narrows what to test. It is not a finding a client can act on with confidence: a conversation with a simulated respondent does not tell you which of two alternatives would actually change behavior. Presenting the first as the second damages credibility under scrutiny. ## What causes the failure? The failure is not "using AI in research." It is skipping the step that turns a directional impression into a causal answer. Traditional qualitative work already has this two-step structure: a handful of interviews surface themes, and a larger validated study confirms which theme actually predicts the behavior. Open-ended AI persona sessions suit the first step. They were designed to generate plausible responses in conversation, not to compare defined alternatives under controlled conditions and report which one moved the outcome. ## Where each method fits in an engagement The pattern: anything that shapes what to test can run on fast, exploratory methods; anything the client will use to commit budget or defend a strategy decision needs a method built to compare alternatives and report a result. ## Evidence: two different questions, two different tools An open-ended AI persona session answers "what does this simulated respondent say when asked?" That is a generative, conversational output, useful for hypothesis discovery in the same way early qualitative work is. A controlled discrete-choice experiment answers a different question: "when a defined population is exposed to alternative A versus alternative B, which one they choose, and with what confidence." Subconscious runs this second kind of study: a controlled experiment comparing defined alternatives across a defined population, returning causal effects with confidence intervals, for the stages of an engagement that need a defensible, client-ready result rather than a directional read. The distinction is not about which tool sounds more sophisticated. It is about which question was actually asked. ## Recommended decision process - Name the client decision the deliverable supports, and what it costs the agency if the finding turns out wrong under scrutiny. - If the deliverable narrows hypotheses or pre-tests an instrument, an open-ended AI persona exploration is appropriate. - If the deliverable is the finding a client acts on, budget for a controlled experiment or recruited real-participant validation before it ships. - State explicitly, in the deliverable, which category it falls into: a client who knows a finding is directional will not mistake it for validated. - When a directional read and a validated study disagree, trust the validated study and say so. ## Where does Subconscious fit? For the stages of an engagement that need a defensible, client-ready result, Subconscious runs controlled discrete-choice experiments that compare defined alternatives across a defined population and return causal effects with confidence intervals. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market, and it is documented in the causal fidelity paper. That fits engagements where a recommendation has to survive a client's own stakeholders asking "how do you know." An agency can also move from a simulated experiment to real-human validation without changing the underlying causal question, which matters when a client wants the same comparison confirmed with recruited participants before a launch decision. See how Subconscious structures a study and examples of engagements that used a validated result. ## What are the limitations and failure conditions? Publishing where a method stops is what makes the result above it worth trusting. A controlled causal experiment does not replace moderated qualitative interviews, recruited real-participant fieldwork, or the interpretive and strategic expertise an agency brings to a client relationship. It answers which alternative moves a defined outcome; it does not run a discovery conversation, do the strategic synthesis a client is paying for, or offer open-ended persona chat sessions. A buyer checking this claim needs the boundary stated as plainly as the number. Audience reach and recruited human validation are also not the same claim. The scale of a simulated population an experiment can run against is a different fact from how many real participants were recruited into a study, and an agency reporting either should be precise about which one it is describing. Naming the wrong-tool case is part of stating what the tool is for. If the underlying research question is really about generating early-stage hypotheses, cheaply and quickly, a controlled experiment is the wrong tool for that job too. Matching the method to the actual decision, in both directions, is the discipline this whole question depends on. ## What clients ask, and how to answer honestly Clients increasingly ask an agency directly whether a deliverable is "real research or AI." The honest answer names which method produced the finding and what that method can and cannot support. A client who hears "this stage used a fast exploratory method to shape the questions, and this stage used a controlled comparison to validate the answer" trusts the agency more than one who hears a blended answer that avoids the distinction. That value, interpretation, judgment about which questions matter, accountability for the recommendation, does not disappear when part of the work runs faster. It is what keeps a client paying for the engagement instead of running the tool themselves. --- # Prototype Feedback or Market Proof: Choosing the Right AI Persona Tool URL: https://subconscious.ai/blog/comparisons/prototype-feedback-market-proof-choosing-right-ai-persona-tool Published: 2026-08-07 A product or growth leader picking an AI-persona tool is really asking one question: is this decision about a screen, or is it about the market? Those are different questions, and no single AI-persona tool answers both well. ## Two questions get bundled into one buying decision "Will people get stuck on this screen" and "will people actually buy, switch, or respond to this message" sound related; both get answered with AI personas talking through a scenario. They are not the same decision, and the cost of using the wrong evidence for each is different. A confusing checkout flow costs conversion and support tickets. A launch, pricing, or positioning bet built on a persona that sounds plausible but was never checked against real human choices can burn a budget cycle, a sales team's time, and credibility with a board that asked for proof, not a transcript. ## What tools fit prototype-walkthrough feedback? Tools built around dropping in a Figma file or a live URL and watching an AI persona click through it answer the first question directly. Uxia is built for exactly that loop: upload a prototype, define a task, and get a walkthrough that reports where the flow breaks, from inside a design workflow. If the research question is "is this UI confusing," a tool shaped around screen-level friction answers it faster than a broader research platform will. Publishing where a tool stops is what lets a buyer check it against their actual question. What a prototype walkthrough does not do is tell a team whether a market will choose the product, respond to a message, or pay a given price. ## What tool fits a causal behavioral test? Subconscious is the causal AI company. Randomized experiments on a simulation of a market, validated against real human behavior, tell a team why people choose and which action drives the outcome. Instead of asking whether a screen is confusing, it compares alternatives, such as a price, a message, an audience, or a launch claim, and estimates which one moves the outcome. Subconscious is not a prototype usability tool. It does not do Figma or URL walkthroughs, click-path friction detection, or screen-level UX heatmaps. What it answers instead sits upstream of the screen: which product, pricing, messaging, or launch action is more likely to change buyer behavior before a team commits budget or roadmap capacity. ## Why isn't a plausible persona the same as a validated one? Recent research on using large language models for choice-style tasks found that model outputs can recover plausible attribute signs and rough tradeoffs, but remain sensitive to prompting strategy and struggle with segmentation and individual-level heterogeneity (Can large language models assist choice modelling? Insights into prompting strategies and current models capabilities, arXiv, checked 2026-07-27). That matters for a buying decision: a persona that answers fluently has not necessarily been checked against how people actually choose. "Our findings reveal that proprietary LLMs can generate valid and behaviourally sound utility specifications, particularly when guided by structured prompts (CoT)." Sfeir and colleagues, Journal of Choice Modelling (source) ## A short decision framework Use this table to route the decision, not to rank the tools. A team often needs both: designers running prototype walkthroughs before a screen ships, and a growth or insights team running a causal test before a price, message, or launch claim ships. ## Moving from a simulated read to a validated one When a decision is consequential enough, the test should not stop at a simulated read. Subconscious can validate studies with real human participants, moving a team from a simulated experiment to real-human validation without changing the underlying causal question. That step matters most for decisions with real budget or reputational exposure, such as a pricing change, a launch claim, or a positioning shift. It matters less for a quick internal read on whether a screen is confusing, where a prototype walkthrough already gives a fast, sufficient answer. ## Limitations and where this comparison stops This is a framework for routing a decision to the right kind of evidence, not a head-to-head product scorecard. A number without its limits reads as marketing. Vendor pricing, published accuracy benchmarks, and compliance claims change quickly and cannot be independently verified here, so none are repeated in this comparison. Subconscious's own methodology centers on controlled experiments and validation against human behavior, not a published accuracy number against any single competitor. Naming where a tool is the wrong fit is what lets a buyer match it to the right question. Neither tool category replaces the other. A prototype walkthrough tool answers "is this screen confusing" and does not answer "will the market respond." A causal behavioral test answers the market question and is the wrong tool if the only open question is a click-path problem inside a design file. ## Where to go next A team weighing this decision can see the underlying research behind causal behavioral testing, review case evidence from decisions tested this way, or read how the platform works. Teams comparing methods can also check the leaderboard for how simulated results are tracked against real outcomes. [figure: A decision path: screen confusion routes to a prototype walkthrough; price, message, or launch-claim questions route to a causal experiment; consequential bets route onward to real-human validation.] --- # What Comes After a Persona Document Like Make My Persona URL: https://subconscious.ai/blog/comparisons/what-comes-after-persona-document-like-make-my-persona Published: 2026-08-07 A team that has already built a persona document is not asking "what is our customer like." It is asking a sharper question: which message, price, or feature framing will that customer actually choose. A one-page document cannot answer that. Neither can a chat window that lets you type questions at a simulated customer. ## What does a persona document do? HubSpot's free persona generator walks a user through a guided form, then styles the answers into a single page carrying a name, demographics, goals, challenges, and a stock photo. It ships in about 15 minutes at no cost, and marketing teams have used it to produce a large volume of first-draft personas (HubSpot's Free AI Persona Generator). It is a reference document for sprint planning, onboarding, or aligning a team on who they are building for. It was never built to test a decision, and it cannot say whether price A or price B wins, or why one message beats another. ## Three answers to "who is our customer," and what each one leaves open The middle row is where most teams land once they outgrow a static document, and it is also where the risk hides. A transcript from a simulated persona can sound specific and confident with no way to check whether it reflects real buyer behavior. Building a launch, price, or pitch on that transcript carries the same risk as building it on a guess. ## What does a controlled experiment add? Subconscious runs controlled discrete choice experiments (McFadden DCE, Mixed Logit, ICLV) against a defined set of choices, then reports which option moves the outcome and by how much, with a confidence interval attached: a measured comparison, not one plausible response. /research documents the experiment design, and the leaderboard is the running record of how simulated results compare against real human studies, including where they miss. A score with no boundary attached is marketing copy. The leaderboard publishes where simulated results miss right next to where they hit, so a buyer can check the method before trusting the number. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. When a decision is big enough to justify it, a team can move from a simulated experiment to a real-human validation study without changing the underlying causal question: the same experiment design carries forward, so a team does not have to treat "fast and simulated" and "slow and real" as two unrelated projects. ## Where this does not fit Naming the case where a method does not apply is what lets a buyer check it before spending budget on it. A defined choice set and a decision worth testing are prerequisites. A team still doing open-ended, exploratory persona work, before anyone has settled on the message or price to test, has nothing yet to compare. That is still the job for early ideation and a lightweight, shareable persona document. Stating the exact shape of the output keeps a buyer from judging it against the wrong yardstick. This is not an open-ended interview with a single simulated persona. The output is a measured effect across a defined set of options, not a conversation transcript. ## Deciding which one to use - Need a shareable reference document for internal alignment: a free persona generator is the right, fast tool. - Need to explore a customer segment before knowing what to test: stay in open-ended research. - Have a specific message, price, or feature decision and need to know which option wins, and why, before committing budget: that calls for a controlled discrete choice experiment. Teams in the third case can see how the process runs or book a walkthrough. [figure: Three columns comparing a static persona document, a conversational persona interface, and a controlled discrete choice experiment, showing what each produces and what each cannot tell you.] --- # Should You Trust a Simulated Read on an Ad-Hoc Consumer Question? URL: https://subconscious.ai/blog/buyers/should-trust-simulated-read-ad-hoc-consumer-question Published: 2026-08-07 You have a backlog of ad-hoc requests, no budget for more fieldwork, and a stakeholder who wants an answer by Friday. The question is not whether a fast simulated read is possible. It is whether you can trust it enough to act on it, or you present it as settled fact and get burned when the market disagrees. A confident directional answer, delivered under time pressure, gets treated as ground truth. If the method was never checked against real respondents for that exact question, a wrong launch, claim, or budget decision follows it back to the analyst who signed off. ## What is a simulated consumer study? A simulated consumer study runs a defined research question (a concept test, a messaging comparison, a positioning question) as a controlled experiment against a modeled population, instead of fielding human respondents for every version you want to test. You still define the target segment, the stimulus, and the comparison; only the response source changes, so you can test more variations before committing recruitment budget to the ones worth validating with real people. It is not a shortcut around the causal question you are trying to answer. It changes the fielding step, not the analysis. [figure: Flowchart from a simulated read through three checks: confidence interval needed, novel behavior, audience coverage gap. Any yes routes to human validation before acting; all no allows acting on the read.] ## What the validation evidence actually supports Every vendor in this space will show you a correlation number. Treat it as a claim about that vendor's setup, on that question, not a general property of "AI research" that transfers to yours. A benchmark on brand-tracker questions in one country does not tell you how a method performs on a niche B2B segment or a regulated product claim. The academic grounding for conditioning a model on individual-level background to reproduce human response patterns comes from Argyle and colleagues, writing in the journal Political Analysis in 2023 (Cambridge University Press). Conditioning a model on a real respondent's detailed background produced opinion distributions that tracked benchmark national survey data. That result belongs to the method generally, established by its authors, not to any particular platform's accuracy claim. To know whether a simulated read matches real behavior for your question, run the same causal question through a simulated study and a small real-human validation study, and compare. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, then test or validate the same study with real human participants, without changing the causal question. That audience graph describes the population the experiment can address, not a recruitable panel of respondents. Be honest with stakeholders about where a simulated read stops being useful: - No statistical validation. A simulated study is not built to output a population figure with a confidence interval. Proving to an auditor or regulator that exactly 34 percent hold a view still requires traditional recruited research. - Unreliable for genuinely novel behavior. A modeled population is built on historical behavioral patterns. Launch into a category with no real-world analog, and a simulated read will lag the actual shift. - Cultural and language coverage gaps. Public-web training data skews English-language and Western. An audience underrepresented in that data gets a more generalized, less specific read. - No physical-world proof. A simulated respondent does not pull out a credit card, hit a shipping delay, or churn after a bad support call. Real-world behavioral data remains the standard for longitudinal tracking of real customer cohorts. ## Where does a simulated-first pass fit in your existing workload? Your trackers and ad-hoc human panels can stay as they are; a simulated read earns its place alongside them, not instead of them. Fold it in as an early, low-risk pass, and reserve human recruitment budget for decisions that carry real cost if you get them wrong: Treat the timing and effort figures as illustrative planning examples, not a current Subconscious delivery guarantee. ## A step-by-step framework for your first study - Define the target segment. Specify age range, geography, core challenges, and behavioral traits as precisely as you can. - Frame the causal question. State the decision the stakeholder actually needs to make, and the specific comparison (concept A vs. B, message X vs. Y) that would resolve it. - Design the research instrument. Write the questions, prompts, or stimuli you want tested, including any visual concepts, ad creative, or product mocks. - Run the simulated study. Submit the instrument against the modeled audience, and review the resulting distributions and qualitative reasoning together. - Analyze for reasons, not just rankings. Look at why a concept won or lost: the language, tradeoffs, and objections, before you report a result upward. - Validate high-stakes findings with real people. If the study informs a high-cost, final decision, use what you learned to design a smaller, targeted real-human validation study against the same causal question. ## Staying credible with your stakeholders The failure mode to avoid is presenting a simulated read as a stand-in for human feedback. The credible framing is narrower: a simulated pass lets you explore a wider space of questions and catch weak concepts and broken instruments early. It narrows what you need to validate, and tells you where to spend the budget you have. Case studies walk through this same causal-question-first process, and a live walkthrough can run it against your own comparison. ## Frequently asked questions What is a simulated consumer study? A controlled experiment that tests a defined causal question (a concept, a message, a positioning claim) against a modeled population instead of, or before, a fully recruited human panel. How do I know if a simulated read is accurate enough to act on? You do not know in the abstract. Accuracy is specific to the vendor, the question, and the population. Run the same causal question through a real-human validation study once and compare, rather than trust a general correlation figure from someone else's benchmark. Can a simulated study replace my existing tracker studies? No. Its best use is as a complement to trackers: a quick early pass to screen hypotheses, pretest a questionnaire, and dig into an unexpected tracker-wave shift, before you commit budget to a full human fieldwork study. Does a simulated study require processing real people's personal data? A modeled population does not require recruiting or storing individual respondent data, since there is no respondent to recruit. Confirm the data-handling practices of whatever platform you use before relying on this for a compliance decision. --- # How Agencies Shortlist Creative Concepts with Audience Evidence URL: https://subconscious.ai/blog/buyers/agency-creative-concept-shortlisting-buyer-panels Published: 2026-08-08 Test every concept against a defined target audience before the shortlist meeting, on separate measures rather than one blended score. The meeting then argues craft and narrative, not which of 12 to 20 concepts deserves a client's attention. The room still decides, but against evidence. ## The decision: which three concepts leave the building A creative team arrives with 12 to 20 concepts. A strategy lead has sixty minutes to cut them to 3. Advocacy decides the outcome: a few people argue hard for their favorites, and the directions that survive are the ones with the loudest defender, not the most evidence. Two failures follow from that. Unfiltered opinion picks get killed at client review, costing re-briefs and creative hours spent on directions nobody outside the agency wanted. The opposite failure is quieter: a single composite rank drops the distinctive, high-risk concept a creative director would have fought for on craft grounds. Structured concept testing is established practice for this stage of the work, defined as comparing alternatives against a described target audience rather than asking which one the team likes (Qualtrics, "Concept Testing: Definition, Methodology & Examples"). ## Four measures and one flag Creative evaluation is subjective in the parts that matter for craft. The parts that matter for shortlisting are not. [figure: A funnel: 12 to 20 concepts enter, pass through five checks (comprehension, relevance, distinctiveness, purchase intent, risk flag). Concepts exit at whichever check they fail.] ## A worked pre-shortlist sequence The durations below are inherited planning examples from an agency workflow: a way to budget effort, not a delivery commitment or a claim about study duration. ### How often should you define the target audience? Write the target audience once for each retainer client and reuse it for every concept review. Include age range, geography, role or company size, purchase context, recent category behavior, and the attitudes the brief turns on. A worked audience definition for a direct-to-consumer skincare client might hold 40 buyer profiles: women 28 to 45 in NYC, LA, SF, Chicago, ATL, and Miami, currently using mid-tier brands, following at least 5 beauty creators, and spending $80 to $250 per month on skincare. ### Give every concept the same evidence: 30 min An unfair comparison is worse than no comparison. Each concept gets a headline of 1 to 2 lines, a visual description or scamp, body copy of 2 to 4 lines where the format needs it, the call to action, and the channel context: print, social, TVC, or OOH. Finished art is not required: early-stage testing reads the idea, not the production, which is why rough executions are the normal input (Cubery, "A Guide to Early-Stage Creative Testing"). One inherited planning assumption holds that a scamped concept and a finished concept return 95 percent of the same audience response. That figure is a historical benchmark from an agency workflow, not a Subconscious result. Check it against your own work. ### How do you compare concepts in batches? Present concepts in batches of 4 to 6 and ask the same questions of each: what is the brand saying, how does this land, does it feel made for me, how does it differ from typical advertising in this category, would it change what I consider, and what feels wrong. Identical questions are what make the comparison readable. ### How do you sort concepts into kill, hold, and shortlist? Tag each concept once the comparison is in front of you: a bottom 6 to kill on low comprehension, low relevance, or a live risk flag; a middle 5 to hold as sound directions needing craft work; and a top 3 to 5 to shortlist because they perform on the measures tied to the brief. The room then opens with 5 to 8 scored concepts instead of an undifferentiated stack. ## Why one composite score kills the wrong concept The concept that scores low on stated intent while scoring high on attention and distinctiveness is the breakthrough archetype. A single blended rank buries it under safe, bland work that scores acceptably everywhere. Report the measures separately and let the creative director make the call. Evidence that a concept is polarizing argues for pushing it, not against it, and the discriminating numbers let an account lead defend that choice to a client. ## Taking the evidence into the client room The method slide should describe the work rather than assert rigor. Name the audience definition, the number of directions compared, the measures that separated the survivors, and the uncertainty around them: 18 directions tested against a defined target audience, the 3 that led on comprehension, relevance, and intent, and what distinguished them from the other 15. Clients tend to accept 3 well-examined directions more readily than 12 unfiltered ones, provided the screening is presented as what it is: pre-shortlist triage that removes weak concepts, not final validation of the winner. ## Where causal testing does the work Subconscious runs randomized experiments that compare a defined set of concepts or actions against a defined target audience and estimate the effect of each one, with confidence intervals where the study design supports them. That maps onto the pre-shortlist problem directly: the concepts are the intervention, the audience definition is the population, and the measures above are the outcomes. Not descriptive. Not predictive. Prescriptive. The method is described on our research page. The advantage at shortlist stage: the same causal question carries further without being rewritten. Subconscious can test or validate studies with real human participants, so a shortlist heading into a large production budget can be re-checked against real people while the question, the audience definition, and the measures stay fixed. Our engagement model covers how that sequencing works on a retainer. ## What this evidence cannot settle Be plain with the client and your team about the boundary. - It does not predict award recognition. It compares audience response, not craft judged by a jury. - It does not measure twenty-four-month brand health. It reads response to a concept now. - It does not see the news cycle in the week the work runs, so cultural timing stays a human call. - It does not decide production quality. Music, voice-over, casting direction, and finish remain craft decisions. - Recurring failure patterns are worth watching but are hypotheses, not rules: category-convention concepts often lose on distinctiveness, clever wordplay often fails when the audience cannot restate the message in 1 sentence, a bold visual can carry weak copy, and casting that contradicts the implied customer damages relevance before production starts. Test each in your own category rather than assuming it. ## Run it on one client batch Pick the largest retainer client, write the audience definition, and run the next batch of concepts through comparison before the internal meeting. Track which concepts the client rejects and why across 2 cycles, then compare that against your record from opinion-led shortlists. The point is not to remove creative judgment from the shortlist. It is to give that judgment something to argue with. If you want to see the experiment design applied to a live brief, book a working session. --- # Setting the Validation Line for Synthetic Panels URL: https://subconscious.ai/blog/buyers/setting-validation-line-synthetic-panels Published: 2026-08-08 An insights leader who lets every stakeholder invent synthetic-research rules loses control of the team's evidence standard. The fix is not banning early synthetic reads. It is drawing a governance line in advance: which questions can move on a simulated read, and which must wait for a real-human check before a pricing, launch, or positioning decision spends budget or reputation on evidence that was never validated. [figure: Flow diagram: a question enters, the leader assigns a tier. Exploratory runs a simulated read, labeled directional. Decision-grade runs the same question with real participants. Only a cleared question ships.] ## The pressure is real, and it points to a decision problem Research teams face daily pressure to move faster: a stakeholder wants an answer before the researcher has finished reading the data, or a manager asks whether the first pass can run on AI. Industry analysis of synthetic data in marketing studies documents this shift toward AI-assisted exploration, alongside continued demand for research judgment. The risk for an insights leader is not that every research task changes. It is narrower: letting a synthetic panel become either a toy nobody trusts or an ungoverned replacement for the validation a decision needs. ## What the leader's job becomes The old advantage in insights leadership was partly access: knowing how to field a study, clean the responses, and package a finding. Faster tooling weakens that advantage. More people can now draft a survey, summarize a transcript, or ask a synthetic audience for a first reaction. That does not make the leader's judgment less valuable. It makes it more testable. Once answers are cheap to generate, the scarce skill shifts to judging which one merits trust and which one needs a real-human check before it reaches an external claim. ## Draw the line before a decision needs it A governance line has to exist before the pressure arrives. A workable version has two tiers: Subconscious runs randomized experiments on a simulation of the market and can test or validate the same study with real human participants, so the leader decides which tier a question belongs to, not whichever stakeholder is in a hurry. ## What stays the leader's call? Three things still require human review regardless of which tier a question sits in: - Whether the decision is expensive or public enough to require human validation before it ships. - Whether the audience definition, prompt neutrality, and source grounding behind a simulated read are sound. - Whether the causal question stayed the same when a study moved from a simulated read to a human check. Moving to real human participants does not turn a causal action test into a usability session, a clinical trial, or automatic proof of market performance. It answers the same question with a different population. ## What does Subconscious not replace? Subconscious does not make that call, and it does not replace human review of audience definition, prompt neutrality, or source grounding. Those checks happen before and after every study, not instead of it. ## Put one decision through the line this week Pick one live research decision. Write down, in one sentence, which tier it belongs to and why. If it is exploratory, run the simulated read and label it as directional. If it is decision-grade, define what the human-validated study needs to confirm before the team commits to a claim. Repeat that once for every open project for a month. The output is not a faster team. It is a documented line between what a synthetic read can support and what a decision requires. See how we work for the underlying method and case studies for examples of studies carried through to validation, or start at research. --- # UXPressia vs. Subconscious: Journey Maps vs. Behavioral Experiments URL: https://subconscious.ai/blog/comparisons/uxpressia-vs-subconscious-journey-maps-vs-behavioral-experiments Published: 2026-08-08 UXPressia and Subconscious answer different parts of a product decision. Choose UXPressia when the team needs persona documents, journey maps, or impact maps. Choose Subconscious when the team needs to estimate which product, message, or journey action is likely to change a defined behavioral outcome. A map organizes what the team believes; an experiment tests whether that belief survives a choice. [figure: Four-step path: journey map flags an assumption, team restates it as a causal question, an experiment compares alternative actions, team commits to the tested action, not the mapped guess.] The distinction matters when a team must commit roadmap capacity, research budget, or brand equity. A journey map can locate an assumption but cannot establish causation. ## Start with the deliverable UXPressia presents itself as a customer experience management and journey mapping platform (UXPressia). Its value is the shared artifact: a persona, journey map, or impact map used in workshops or stakeholder reviews. Subconscious is a causal behavioral platform that runs controlled experiments on simulated markets to estimate which action is likely to move a specific outcome, and reports uncertainty where the study design supports it. ## The expensive mistake is treating a map as evidence A persona might say first-time buyers value simplicity, and a journey map might identify onboarding as where that belief matters. Neither artifact tells the team whether a shorter, guided, or current onboarding flow will change completion behavior. If the team treats the map as the answer, it can ship a redesign built on the author's best judgment and discover after launch that behavior diverges from the documented profile. The team still has no experimental record showing which action was tested. Turn the mapped assumption into a decision: which action should change, compared with what alternative, for which buyers, against which behavioral outcome? ## How do you design a useful causal test? For a journey decision, keep the audience, outcome, and context fixed, then compare alternative actions. The result should answer the commitment question directly, such as which onboarding flow is more likely to increase completion or which message is more likely to change product choice. Subconscious does not build or export visual persona documents, customer journey maps, or impact maps, and is not a design-collaboration workspace. Confidence intervals, segment heterogeneity, and decision memos depend on the study; none are standard packaged outputs. ## How accurate is Subconscious's causal fidelity? Subconscious's causal fidelity paper shows its best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. This is evidence about Subconscious's validation, not a score assigned to UXPressia or a promise that any single launch will match the estimate. When a decision needs further confirmation, a team can move from a simulated experiment to real-human testing without changing the causal question. That step validates the experiment against recruited participants; it does not become an observed usability session or automatically prove market performance. ## Can you use UXPressia and Subconscious together? Use UXPressia to make the persona and journey legible to the team; identify the assumption that carries the investment decision. Use Subconscious to compare the actions encoded in that assumption. This division also clarifies procurement: buy a mapping workspace for artifacts and collaboration, and use a causal behavioral platform when the cost of choosing wrong justifies a controlled test. Review how Subconscious structures studies and its case studies before deciding whether the method fits. If the open question is which product, message, or journey action to commit to, book a demo with the alternatives and behavioral outcome already defined. --- # When an AI-Accelerated Marketing Mix Model Is Trustworthy URL: https://subconscious.ai/blog/methods-and-validation/when-ai-accelerated-marketing-mix-model-trustworthy Published: 2026-08-08 A marketing mix model can now be configured in hours instead of months. That speed changes how often a team can rebuild the model. It does not, by itself, change whether the model's channel-lift estimate is correct. Bayesian marketing mix modeling (MMM) estimates how much each channel (paid search, TV, social, email) contributed to sales, and reports that estimate with uncertainty rather than a single number. Automation can remove most of the manual work in getting to that estimate: cleaning spend and sales data, choosing an adstock decay window, picking a saturation curve, and diagnosing a model that will not converge. None of that automation answers the harder question a VP of marketing analytics has to answer before moving budget: is this specific estimate grounded in anything other than the model's own assumptions? [figure: Chain showing adstock/saturation assumptions and confounding controls feeding lift-test calibration, producing a channel-lift estimate with uncertainty that feeds a budget reallocation decision.] ## Where automation actually helps Configuring a Bayesian MMM involves a series of judgment calls that used to require a data science team: - Merging siloed spend, sales, and promotion data and reconciling discrepancies between sources. - Choosing adstock decay length and a saturation function (a Hill curve, an exponential curve, or another shape) for each channel. - Diagnosing a model whose Markov chain Monte Carlo (MCMC) sampling will not converge, and deciding whether the problem is the priors or the input data. An automated configuration is worth publishing only alongside the check it still needs. Automating this configuration step is a legitimate use of AI for proposing a structurally reasonable starting point, but adstock decay length and saturation shape are themselves empirical claims about carryover dynamics and diminishing returns that drive marginal-ROI estimates, and they still need outside validation like everything else in the model. Open-source Bayesian modeling libraries expose high-level APIs for exactly these components (carryover, saturation, seasonality), so an automated agent can propose a structurally reasonable model without hand-coding it. ## What automation cannot do: prove the estimate is correct A channel-lift number earns trust only when its structural risks are stated alongside it. Two structural risks apply to any MMM, whether it took a data science team three months or an agent three hours to build: - Confounding. Observational spend and sales data can move together for reasons that have nothing to do with channel effectiveness: seasonality, competitor actions, or macroeconomic trends. A model that omits the right control variables will misattribute that shared movement to a channel. - Uncalibrated priors. A Bayesian model's output is only as good as its assumptions about plausible effect sizes. Without an outside check, a mis-specified model can produce a confident-looking posterior that is confidently wrong. A calibration against a real experiment, a lift test, a geo holdout, or another randomized intervention with a known effect, constrains the response curve for the tested channel at the tested spend level and time window. Naming where calibration stops is what lets a buyer check an estimate before moving budget outside the tested range. It does not remove omitted-variable bias for untested channels, untested spend regimes, or later periods. Common Bayesian MMM tooling supports adding lift-test measurements to a model before fitting, which lets the model's priors get pulled toward evidence instead of resting on assumption alone. This is one mechanism among several, alongside holdout validation and sensitivity checks on priors and specification, for making a channel-lift estimate more trustworthy. (See Measured's guide to marketing mix modeling, a vendor overview, and a hierarchical framework for uncertainty-aware channel attribution, an unrefereed preprint.) ## The decision this changes Before reallocating budget off any MMM output, the question worth asking is not "how fast was this produced," but: - Was this model calibrated against at least one real experiment, or does it rest entirely on observational data and priors? - Does the model include the control variables needed to isolate a channel's effect from correlated market movement, and does it account for spend being set in anticipation of demand rather than assigned independently of it? - Is the reported estimate accompanied by uncertainty, or presented as a single point number? Getting this wrong has a real cost: shifting spend off a channel that was actually working, or scaling up one that was not, based on an estimate the model never checked against reality. ## Where this connects to causal experimentation more broadly The same validity question underlies any modeled recommendation, not just MMM: does the estimate come from a controlled comparison, or from pattern-matching on observational data? Subconscious is not an MMM vendor, but the discipline of testing an action's causal effect before it ships budget, product, or go-to-market decisions is the same discipline an MMM team needs when deciding whether to trust a channel-lift number. Teams evaluating how a modeled estimate should inform a real decision can read more about how Subconscious approaches causal experimentation and the research behind it. [figure: Three sequential yes/no checks: calibrated against a real experiment, confounding controlled, uncertainty reported. All three pass leads to budget reallocation within the tested range with appropriate caution; any failure leads to do not act yet.] ## Limitations This scope is stated so a reader can separate the methods claim from any product claim. This is a methods explainer about MMM calibration, not a Subconscious product claim or case study. No Subconscious accuracy, speed, or pricing figure applies here, and no named customer result should be inferred from it. The validity questions above apply regardless of which tool produced the model. --- # Replicating a published rural job-preference study with a causal discrete choice experiment URL: https://subconscious.ai/blog/case-studies/replicating-published-rural-job-preference-study-causal-discrete-choice Published: 2026-08-08 ## The decision: trust a new method, or field a traditional study first A research or insights leader weighing a workforce or job-preference study faces a real choice: commission a traditional fielded study, or trust a causal discrete choice experiment to answer the same question faster and at lower cost. Committing budget to an unvalidated method wastes the study. Rejecting a method that already reproduces known findings delays a real workforce or incentive-design decision a team needs to make. One way to test a causal discrete-choice method before committing to it: check whether it reproduces a discrete choice experiment already published and peer-reviewed. ## What published study did Subconscious replicate? Rao et al. ran a discrete choice experiment to assess rural physician and nurse job acceptance and scarcity in India, published as Rural Clinician Scarcity and Job Preferences of Doctors and Nurses in India: A Discrete Choice Experiment in PLOS ONE. The paper measured how job attributes such as location and incentives shaped the stated job preferences of doctors and nurses choosing between rural and urban postings. "At five times current salary levels, 13% (31%) of medical students (doctors) were willing to accept rural jobs. At half this level, 61% (52%) of nursing students (nurses) accepted a rural job." Rao and colleagues, PLOS ONE (source) [figure: Five steps: published study exists, run same-question experiment, compare rankings, get r_s = .7286, decide whether to trust the method.] ## What did Subconscious compare in this study? Subconscious ran its own discrete-choice-style causal experiment, on a synthetic population, on the same rural clinician job-preference question and compared the resulting preference ordering against the published Rao et al. findings. A correlation by itself is marketing copy. Publishing it next to its limits is what lets a buyer check it: the comparison produced a Spearman rank correlation of r_s = .7286 between the two sets of results; the underlying sample size was not archived, so no significance level can be verified for this figure. Stating exactly what a result covers is what lets a buyer weigh it against their own case. That correlation is the only figure this comparison supports: agreement between one replication and one published study on one job-preference question. It is not a general accuracy claim, and it does not extend to other studies, domains, or experiment designs. ## What this comparison does not show Naming what the record does not contain is part of publishing the result straight. The archived record lists no attribute set and no full results table alongside the correlation figure, so this page does not reconstruct one. It does not claim a general accuracy rate for the underlying method, and it is not evidence of a healthcare-workforce product or a domain-specific service. It documents a single replication comparison, not a customer engagement. [figure: Four boundaries around r_s = .7286: one replication of one study on one question, no attribute set archived, no general accuracy rate, no healthcare-product evidence or guarantee for future studies.] ## Why does this comparison matter for a workforce study decision? A published, peer-reviewed discrete choice experiment gives a known answer to check a new method against before applying it to a real decision. A correlation this strong between an independent replication and the original published rankings is a reasonable basis for trusting the experiment design on a comparable job-preference question, not a guarantee that any new causal study will match every published result this closely. Teams evaluating a workforce, incentive-design, or job-preference question can review Subconscious's research and other case studies to see how the same methodology has been applied elsewhere, or book time to discuss whether a causal discrete choice experiment fits a specific workforce decision. --- # Test Sales Messaging Before the Enterprise Call, Not During It URL: https://subconscious.ai/blog/use-cases/test-sales-messaging-before-enterprise-call-not-during Published: 2026-08-08 ## Practice against a colleague proves nothing about a buyer A colleague roleplaying a buyer knows the product, understands the pitch, and makes the conversation easier. A manager grading a roleplay puts the rep in evaluation mode instead of learning mode. Neither behaves like the enterprise buyer who will sit across the table next quarter. The cost shows up later, not in the roleplay. A rep walks into a long, unrepeatable sales cycle having rehearsed against someone who never challenges them. If the deal stalls on an objection nobody prepared for, the team learns which response failed only after the pipeline value is gone. ## Which message actually changes the buyer's mind? Head of Sales Enablement and RevOps teams face a narrower question than "how do we practice more." It is: which specific messaging or objection response shifts a defined buyer's stated intent, before that response gets used on a deal that matters. Answering that requires comparing responses against each other under the same buyer definition, not judging one script in isolation and hoping it lands. ## Define the buyer and the objection set before testing anything A vague buyer definition produces a vague test. Before comparing messaging approaches, fix: - The role, seniority, and company profile of the buyer, for example a VP of Operations at a mid-market logistics company evaluating a tool their team already tried once. - The objection set specific to that deal stage: price, a prior failed implementation, competing priorities, or stakeholder buy-in. - The buying stage, since an early-discovery buyer and a late-stage buyer weigh the same objection differently. This is the same information a rep would want before a real call, but here it becomes the fixed condition for a controlled comparison instead of a mental checklist. ## Run the comparison as a causal test, not a guess With the buyer segment and objection set defined, a discrete-choice-style comparison puts two or more messaging or objection-handling approaches in front of that buyer definition and measures which one shifts stated preference. The question stays narrow (does leading with ROI beat leading with implementation risk for this buyer, on this objection), and the comparison isolates that one variable rather than judging a whole pitch at once. "Our findings reveal that proprietary LLMs can generate valid and behaviourally sound utility specifications, particularly when guided by structured prompts (CoT)." Sfeir, Nova, Hess, and van Cranenburgh, Journal of Choice Modelling (source) Research on using large language models for choice modeling finds that prompting strategy materially changes how closely a model's outputs track real preference structure, which is why this comparison needs a defined method, not an open-ended conversation with a simulated buyer (arXiv, 2507.21790). ## How do you move from a simulated comparison to a human check? Subconscious can test or validate studies with real human participants, so a team that wants a check beyond the simulated ranking does not have to redesign the study to get one. This step matters most on deals large enough that a wrong assumption is expensive; it is not required for every message comparison. ## Where this fits into deal preparation A team preparing for an enterprise call can use this ahead of a specific deal: - Define the buyer and the two or three objections most likely to surface. - Compare two or three response framings against that buyer definition. - Brief the rep on the response that performed better, and why, before the call. [figure: A five-step path: define the buyer and objection set, run a causal messaging experiment, compare objection responses, validate against a human baseline, then brief the sales team on the winning response and why.] ## What does this not replace? A messaging comparison does not rehearse tone, pacing, reading a room, or building rapport with a real person. It is only as good as the buyer definition and objection set built into it. Subconscious does not offer a turn-by-turn conversational rehearsal tool. The research methodology behind a messaging comparison is built for testing which action changes a defined buyer's decision. [figure: Two columns. Left: roleplay against a colleague, manager grading, no buyer definition. Right: a fixed buyer and objection set, two responses compared, outcome is which one shifted stated preference.] ## Before the next high-stakes call Reps still need to rehearse delivery. But the messaging and objection responses they rehearse can be chosen with evidence instead of instinct. Teams that want to see how a comparison like this is structured for their own buyer segment can review how Subconscious runs a study or look at comparable studies before setting up a session around an upcoming deal. --- # Is a Ranked Tool List Enough Evidence for a Launch Decision? URL: https://subconscious.ai/blog/use-cases/ranked-tool-list-enough-evidence-launch-decision Published: 2026-08-08 A product marketing or launch leader shortlisting AI audience-simulation tools ends up with the same artifact: a ranked list, each entry backed by a self-reported accuracy percentage. The question that matters is not which entry ranks first, but whether that list is sufficient evidence to commit launch budget, or whether the decision needs a controlled experiment first. It is not sufficient on its own. A self-reported accuracy percentage tells you how a vendor scored against its own benchmark, under conditions the vendor chose. It does not tell you how your specific launch alternatives would perform against your specific target population. Only the second question is the one a launch decision depends on. ## What does guessing wrong on a launch decision cost? Positioning, messaging, or budget gets committed on the strength of a vendor's accuracy claim or category ranking. The launch runs. Only after the spend and the go-to-market time are gone does the team learn whether the underlying read was correct. A ranking is a screening signal, not a decision input, and treating it as one moves the discovery of a bad call from before the spend to after it. ## What a shortlisting checklist actually verifies Before evaluating any audience-simulation platform, the criteria below separate a defensible pick from a plausible-sounding one: - Response fidelity. Whether outputs reflect how a defined segment actually reasons, or read as generic model output dressed in a persona label. - Segment comparison. Whether the tool can run the same question across multiple segments and show where they diverge. That divergence is usually where the insight lives. - Validation methodology. Whether the vendor publishes how it measures accuracy against real human responses, not just the resulting number. A platform that cannot describe its measurement method is asking you to trust a score it defines. - Time to first read. How long from setup to an actual answer, and whether that answer is directional or measured. - Compliance posture. Data residency and processor agreements, relevant for any team handling EU customer data. - Pricing transparency. Published tiers versus "contact us" indicates how a vendor treats disclosure, including on its accuracy claims. This category has marketed itself on turnaround: same-day or same-hour reads against multi-week traditional recruited-panel research, with a single self-reported accuracy score against a historical benchmark the vendor chose. Independent replication work complicates that pitch: a study comparing synthetic responses against academic survey benchmarks found agreement in some conditions and real gaps in others, depending on question type and population (Greenbook, "Testing Synthetic Data Against Academic Benchmarks: A Replication Study"). Treat any single blended accuracy number, from any vendor, as a claim to interrogate rather than a verified figure to shop by. ## Why doesn't a self-reported score answer the launch question? A ranked-list accuracy percentage is a single scored output, measured on the vendor's own historical benchmark. It answers "how well did this tool do on average, on the questions it chose to report." It does not answer "will alternative A or alternative B outperform for the audience I am about to launch to." The first two rows describe most of what a vendor-ranking exercise produces. The third is what a launch decision needs before budget moves. ## How does Subconscious test the launch decision instead? Where an open-ended simulation tool produces a single scored or ranked output, Subconscious runs a controlled discrete-choice experiment: it defines the actual launch alternatives, defines the population, and returns a measured causal effect with a confidence interval rather than a directional score. The output is not "this concept scored well." It is a measured answer to "which of these defined alternatives performs better for this defined population, and by how much." A number reported without its scope is marketing copy; naming the scope is what makes the number usable. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That figure describes the scale of the simulated population available for an experiment, not a recruitable panel of 800 million people willing to participate in a study. Recruited real-human validation is a separate, smaller step that uses actual participants. See the replication leaderboard for how these experiments are structured and the claims tested so far. ## Limitations Naming what a method does not cover is what lets a buyer check the claim before they rely on it. A controlled causal experiment answers the alternatives-and-population question. It does not replace direct customer discovery, observed in-market or beta behavior, or the judgment of the launch team making the final call. Publishing the extra step required at high stakes is what lets a buyer price the decision correctly. When a launch decision turns on it, Subconscious can move from a simulated experiment to real-human validation without changing the underlying causal question. That step matters when the stakes justify it; it is not required for every directional read, and running it does not turn the original causal-effect experiment into an observed usability session or a guarantee of market performance. ## Next step If a launch decision is riding on a ranked list of vendor accuracy claims, the smaller, cheaper move is to run the actual comparison first: define the alternatives, define the population, and measure the effect before the spend happens. See /how-we-work for the process, or /demo to walk through a specific launch decision. [figure: Self-reported vendor benchmark and directional simulated read feed a question mark over the launch decision; a controlled causal experiment with a measured effect and confidence interval feeds a confident decision.] --- # LLM-Simulated Panels: When a Persona-Conditioned Survey Result Is Enough to Act On URL: https://subconscious.ai/blog/methods-and-validation/llm-simulated-panels-when-persona-conditioned-survey-result-enough-act Published: 2026-08-08 A persona-conditioned survey result is enough to act on as a triage signal for general sentiment on common opinion questions among well-represented groups, before a causal experiment is needed to identify which specific change drives behavior. A method is only as trustworthy as the limits published alongside it. Conditioning a large language model on a demographic or psychographic backstory and asking survey questions produces an opinion distribution, not a causal answer. Argyle et al. (2023) showed that persona-conditioned sampling can track real survey distributions at the population level for common opinion and preference questions. That is a useful triage signal, not evidence about which specific change in a message, price, or product moves a decision, because nothing was varied and observed under controlled conditions. [figure: Decision path from a persona-conditioned opinion read, branching on general sentiment vs. which change drives behavior, routing the latter through a controlled experiment and human validation before shipping.] ## The question a buyer is actually asking A research, insights, or product leader facing a launch, price change, or repositioning decision needs one of two answers: what an audience thinks about an idea, or which version of that idea changes behavior. A persona-conditioned LLM read answers the first quickly; it cannot answer the second, because it was never designed to isolate one variable's effect on a choice. Naming a failure mode here is what lets a buyer check it before acting on a read. Treating a directionally-plausible read as proof that a specific message, price, or feature caused a shift is a mistake: it can look right in aggregate and still miss the lever that actually moved people, a gap that surfaces after launch, when the metric that was supposed to move doesn't. ## Where does the founding research draw its own line? A method's misses belong on the record next to its hits. [Bisbee et al. (2024) re-ran the same persona-conditioned sampling and found it overfits toward majority opinion, systematically under-representing minority subgroups, low-incidence intersections, and novel-category behavior. That narrows the method's claim rather than rejecting it: strong for reading common sentiment in well-represented groups, weak where a launch, pricing, or repositioning decision tends to be riskiest, in the segments most likely to drive real switching, churn, or backlash. ## What question does a controlled experiment answer? A controlled discrete-choice experiment presents a defined audience with systematically varied versions of an offer, price, or message and observes which version people choose. Because the variation is designed and randomized, the resulting difference in choice can be attributed to the variable that changed, with uncertainty reported over the simulated population the study design supports, not sampling error about the human population. ## How does Subconscious test the causal question? Subconscious runs randomized, controlled experiments against a simulation of a defined audience, establishing causal attribution within that simulation, then validates the same study design with real human participants to confirm the effect transports to human choice before a consequential decision, without re-deriving the study or changing the causal question being asked. See how that validation step works. Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people. That figure describes the pool a study can be run against, not a recruitable panel of 800 million respondents, and it matters only when audience scale, not method, is the constraint on a study; a larger pool narrows simulator variance, not bias against real human behavior. ## What still has to hold before you trust a result - A simulated experiment or a persona-conditioned opinion read estimates likely behavior; neither substitutes for observing what real people actually do. - Real-human validation confirms the same causal question. It does not turn a causal action test into a usability session, a clinical trial, or automatic proof of market performance. - No method here executes the resulting business action. A confirmed causal effect still has to be shipped, priced, or messaged by the team that owns the decision. - Stating a limit plainly is what turns a number into something a buyer can act on. The tail-failure evidence above applies regardless of which vendor or platform runs a persona-conditioned read, including the persona-conditioned choice tasks inside a simulated causal experiment: treat conclusions about novel categories, minority subgroups, and rapid attitude shifts as provisional. ## A sequence for the next decision - Name the specific action being decided: a message, a price point, a feature, a positioning claim. - Ask whether the open question is "what does this audience think" or "which version of this offer changes behavior." The first calls for a fast opinion read; the second calls for a causal experiment. - Run the causal experiment against a defined audience simulation, with uncertainty reported where the study design supports it. - Where the decision is consequential, validate the same study design with real human participants before committing budget, without changing the underlying causal question. - Treat the result as one input into the launch decision, not a substitute for observing what happens after the action ships, and see how a study gets scoped or browse how the method has been applied before starting one. --- # How to Test Messaging Before Launch URL: https://subconscious.ai/blog/implementation-and-procurement/how-test-messaging-before-launch Published: 2026-08-08 Test each message variant as a controlled comparison against a defined buyer audience before committing launch budget to one of them. The way to break an internal tie between competing headlines is evidence about which variant a target buyer responds to, not another round of opinions in a meeting. ## The decision this solves A launch team may have three message options and no evidence for choosing among them. Marketing prefers version A. Product prefers version B. A senior stakeholder wants version C. Without a comparison tied to buyer behavior, the loudest voice in the room wins by default. The cost of getting this wrong is not abstract. A launch window, ad budget, and sales enablement all get committed to a positioning that may not cause the intended response. Observed market evidence can arrive after the campaign has already consumed the budget or launch window. ## Why the traditional approach gets skipped Traditional message testing follows a known sequence: write several variants, build a survey instrument, recruit participants who match the target population, field the survey, and analyze the results. A historical planning example used 3 to 5 message variants, 200 to 500 recruited respondents, and 1 to 2 weeks of fieldwork before results were ready to analyze. Those ranges are not current Subconscious commitments or universal requirements for a valid study. Recruitment, instrument design, fieldwork, and analysis must fit the launch plan. When they do not, teams often substitute internal preference for buyer evidence. A controlled comparison can narrow the options before a consequential launch decision, while recruited research remains a separate validation step. ## Why use a causal comparison before spend? Subconscious tests message and go-to-market actions as causal experiments before capital commitment, estimating which message is more likely to cause better engagement or intent for each segment. This is a discrete-choice-style comparison of alternatives rather than open-ended opinion gathering. [figure: A five-step path from drafting three message variants through defining buyer segments, running the comparison, reading results by segment rather than average, and refining the winning variant before retest.] ### Step 1: Draft three candidate variants One variant gives you nothing to measure against. Five variants introduce enough additional variables to blur the comparison. Three can represent genuinely different approaches, such as benefit-focused, problem-focused, and proof-focused framing, rather than three versions of the same sentence. ### Step 2: Define the audience segments that matter As a planning example, define four buying contexts: a hands-on buyer evaluating day-to-day fit, a skeptical buyer weighing the message against competing priorities, a budget-constrained buyer, and a rigorous evaluator at a larger organization. Four is a workflow choice, not a product limit. Each definition should reflect the roles, constraints, and current alternatives that shape how the segment reads a message. ### Step 3: Run the comparison Expose each audience segment to each variant separately, so an earlier variant does not color the reaction to the next one. Ask direct questions for each pairing: What is the immediate reaction? What does the buyer think the product does? Does the message address a problem they have? What questions remain? How likely are they to take the next step on a defined scale? A 1 to 5 response scale is one planning example, not a prediction of market performance. ### How do you read the results by segment instead of just the average? Organize the results in a matrix of variant by segment rather than a single average score. A variant that wins overall but loses one high-value segment is not a clean winner. That is the finding worth acting on, not a footnote. ### Step 5: Refine and retest Take the strongest variant, address the objections that came up, and run it again against the same segments. Compare the revision with the prior winner rather than a fresh field of options. A planning example may budget 2 to 3 comparison cycles. That is a workflow choice, not a delivery commitment or proof that the result is ready to launch. ### Step 6: Move to real-human validation for the launch decision A comparison at this stage is directional. It narrows the field and surfaces objections before a team commits budget, but stated reactions to a message can diverge from what a buyer actually does when it counts, a pattern researchers call the say-do gap (Quirk's Marketing Research Review). That gap is why a launch decision with real consequences needs to close the loop with people. Subconscious can test or validate studies with real human participants. The team can move from the initial comparison to real-human validation without changing the underlying causal question: what causes the buyer to respond, not just which variant sounds better internally. ## Where else does this method apply beyond a headline? The same sequence works for email subject lines, ad copy hooks, product description clarity, pricing-page language, and the key slides in a sales deck. It also applies across markets: a message that works in one market may not resonate in another, so market-specific audience segments matter as much as the message itself when a launch spans regions. Combining approaches can answer both a directional and an observed-behavior question. As a planning example, use a controlled comparison to narrow 5 variants to 2, then run a quantitative test on the finalists with real traffic. The first step identifies the stronger causal hypothesis. The second tests behavior in the live channel. ## Where teams go wrong Comparing minor wording tweaks rather than genuinely different concepts. Swapping a verb is rarely worth a full comparison cycle. Testing two genuinely different value propositions is. Not defining "winning" before the test. Decide in advance whether the goal is comprehension, appeal, click-intent, or reducing objections. Each of those metrics can point to a different winning variant, so choosing the metric after seeing the results is not a real decision rule. Reading only the average. A variant that wins overall but alienates the highest-value segment is not a winner. Skipping the second round. A first comparison gives direction. A second comparison tests whether the response survives the revision. Launching on the first pass alone leaves objections unaddressed. ## Limitations A controlled comparison is directional evidence for narrowing options and surfacing objections. It is not a substitute for validation with real people before a launch decision that carries real budget or brand risk. Automated recommendation output, confidence intervals, and segment-level statistical validation are not standard claims here. Treat any specific number from a comparison as a result to investigate, not a final statistic. It should not be described as recruiting or fielding a study with real respondents; that is a separate validation step. ## Where to go next Start with how Subconscious tests decisions before launch, review the method behind the comparisons, or see case studies of decisions tested this way. To scope a message test for an upcoming launch, book time with the team. --- # Checking a simulated vaccine-allocation study against Duch et al. 2021 URL: https://subconscious.ai/blog/case-studies/checking-simulated-vaccine-allocation-study-against-duch-et-al-2021 Published: 2026-08-08 A consumer-insights team weighing a simulated discrete-choice result against a real human study needs a published study to check the simulation against. This entry compares a Subconscious-simulated discrete-choice run to a real, peer-reviewed study on COVID-19 vaccine allocation preferences. ## The published study behind the comparison Duch et al. ran a real conjoint experiment asking residents of 13 countries to rank vaccine-allocation priorities, meaning who should get a vaccine first and on what basis (Citizens from 13 countries share similar preferences for COVID-19 vaccine allocation priorities, PNAS). Subconscious ran a simulated version of the same discrete-choice question, restricted to U.S. residents, and compared the ordering of preferences against the published human result. ## What does the correlation say? The simulated and human orderings correlated at rs = .7996, p < .001. That is a Spearman rank correlation between how the simulation ranked allocation priorities and how the real study's U.S. respondents ranked them: a measure of whether the two agree on relative ordering, not a percentage-accuracy score. This single comparison is one entry in a public leaderboard, not a standalone accuracy claim. The leaderboard's point is to publish mid-range and weak results alongside strong ones, so a rs = .7996 entry sits next to others that score higher and lower. ## What does this comparison not establish? One correlation on one topic, vaccine allocation preference, does not establish accuracy for a different topic, a different country set, or a different decision. Do not round rs = .7996 up. This entry compares a simulation to a prior published study's results, not to a freshly recruited panel of respondents. When a decision genuinely turns on live respondent behavior rather than a published benchmark, Subconscious can test or validate studies with real human participants. ## Reading a leaderboard entry before trusting it for a decision Before treating any single comparison as sufficient grounds for a health-messaging, allocation, or policy-preference decision, check three things: the topic match between the comparison and the decision at hand, whether the correlation is a ranking measure or something else, and whether the source is one dated leaderboard entry or a claim of general accuracy. See the full leaderboard for how this entry sits alongside other published comparisons, and research for the underlying replication method. [figure: Comparison of the Duch et al. 2021 published human vaccine-allocation ranking and the Subconscious simulated ranking, connected by rs = .7996, p < .001, marked as one leaderboard entry.] A team deciding whether to trust a simulated result for its own vaccine-communication or allocation question should look for the closest topic match on the case studies page or the leaderboard itself, then confirm the comparison method matches the decision it's meant to inform. --- # Where a Causal Experiment Belongs When Anyone Can Ask AI for an Answer URL: https://subconscious.ai/blog/use-cases/where-causal-experiment-belongs-when-anyone-ask-ai-answer Published: 2026-08-08 A stakeholder can now open a chatbot, describe a pricing move or a new message, and get a confident-sounding answer in seconds. An insights leader's real decision isn't whether to allow that: it's which questions a fast AI answer can settle and which require a randomized experiment before anyone commits budget, roadmap capacity, or a public launch. ## Why a Confident Answer Isn't the Same as Evidence A number without its limits is marketing, so this gap gets named directly. A language model can generate a plausible reaction to a concept, a price, or a message. It cannot tell you whether that reaction predicts what a real market will choose once the alternatives are in front of people and something is at stake. A stakeholder who mistakes one for the other ships a decision that was never tested against an alternative. The misses belong on the record next to the hits, so this one gets stated plainly. The risk isn't that AI replaces research; it's narrower: a stakeholder bypasses the research function because a chatbot answers faster than the team can respond, and the team has no faster alternative that's still trustworthy. The Bureau of Labor Statistics projects 7 percent employment growth for market research analysts from 2024 to 2034, even as AI tools reshape daily workflow. The open question is where a team's evidence should sit on the ladder between "AI generated a plausible answer" and "we know what changes behavior." ## An Evidence Ladder, Not an AI Habit Most research questions don't need the same tier of proof. A useful way to sort them: Fast AI exploration is useful at the first tier, but it answers none of the questions it raises. The failure mode is treating a fluent exploratory answer as if it already occupies the second or third tier. ## Where Does a Randomized Experiment Belong? A randomized discrete-choice experiment belongs at the point where a stakeholder is about to act on a specific choice: this price against that one, this message against that one, this feature against the status quo. The question changes from "what do people think" to "which alternative moves the outcome, and by how much." Subconscious runs that experiment on a controlled synthetic population and returns a causal effect with a confidence interval for the specific action, not a single confident-sounding sentence about the general topic. That output is the middle tier: past exploration, short of a go/no-go decision that's expensive or public enough to warrant real-human confirmation. Current experimental methodology and how an engagement runs end to end show what that looks like in practice. When the decision is consequential enough, that same causal question can move to real-human validation without changing what's being asked. Subconscious can test or validate studies with real human participants. A team isn't forced to choose between speed and confirmation: it can get a causal read quickly, then confirm with real respondents before the decision becomes public or expensive to reverse. ## A Decision Rule for Insights Leaders The question to ask before any AI-generated answer reaches a stakeholder deck: does this claim change what the business does, and if it's wrong, what does that cost? If the answer changes nothing consequential, exploration is enough. If it changes a pricing, launch, or positioning decision, it needs a controlled comparison between the specific alternatives, not a general answer. If the decision is expensive enough to be hard to reverse, or public enough that a wrong call is visible, the causal result should be checked against real-human data before it ships. ## What Doesn't a Causal Experiment Replace? Naming a failure mode is what lets a buyer check it, so the boundaries are stated here. A causal experiment does not replace the exploratory or hypothesis-generation stage of research. Someone still has to decide what's worth testing, and no experiment picks its own question. It also doesn't replace human judgment on which decision is consequential enough to warrant real-human validation. Those calls stay with the insights leader, not with any tool downstream of them. ## Put the Ladder to Work Start with one live decision already on a stakeholder's desk. Write down the specific alternatives being weighed, not just the general topic. If a chatbot answer is already circulating about that decision, treat it as an exploratory hypothesis, not a settled result, and route the actual choice through a controlled comparison before anyone commits to it. A demo is the fastest way to see what that comparison looks like for a decision your team is facing this quarter. [figure: Four stages. Exploration proves nothing alone. Consequential claims need a causal experiment: an effect with a confidence interval on one alternative. Expensive or public decisions add real-human validation first.] --- # Testing In-App Upgrade Prompts Before They Reach a Live A/B Test URL: https://subconscious.ai/blog/use-cases/testing-app-upgrade-prompts-before-they-reach-live-b-test Published: 2026-08-08 A product or growth lead who owns in-app monetization copy has one real decision to make before a new upgrade-prompt variant ships: run it through a controlled pre-production experiment first, or send it straight into a live production A/B test. A live test exposes customers to every weak variant, with possible costs in churn, foregone revenue, and support demand. [figure: Four steps: draft upgrade-prompt variants, run a pre-production causal test returning effects and confidence intervals, narrow to strongest variants, send only those to a live A/B test measuring conversion and retention.] The useful answer is not to replace the production test. It is to decide which wording, price framing, and trigger conditions deserve production exposure. ## Decide what earns production exposure An upgrade prompt is part offer, part product interruption. The team is choosing what to say, when to say it, and which commercial action to ask a buyer to take. A weak choice also risks repeated dismissal of the prompt. Pricing experiments can affect customer trust, perceived fairness, and brand positioning, while subscription tests may also change retention and expansion behavior beyond the first conversion. Kameleoon's guide to pricing A/B tests describes those commercial and lifecycle effects. Pre-production and production experiments answer related but different questions: ## Turn the prompt into a causal question Start with the action the team wants to understand. For example: does naming the blocked feature increase upgrade intent relative to naming the paid tier? Does showing the price in the prompt change willingness to pay? Does the same offer produce a different response after a hard limit than after a successful task? Then hold the causal question steady while varying the parts of the prompt that the team can act on: - Wording. Compare the benefit, the blocked action, and the offer without changing the price at the same time. - Price framing. Compare how the commercial choice is presented while keeping the underlying offer fixed. - Timing. Test the trigger context as part of the treatment, not as background detail. - Buyer segment. Define who is making the decision. An individual buyer and a team administrator may respond differently to the same offer. - Decision output. Require confidence intervals and segment differences rather than a single ranked winner. Subconscious supports claims tests, pricing-sensitivity experiments, and monadic concept tests for this kind of comparison. The research approach is designed to estimate causal effects against a defined respondent population before a team commits a variant to a live test. ## Use planning examples without turning them into promises Inherited prompt exercises can still be useful when they are treated as planning inputs, not performance claims. A historical example used a twenty-word prompt to focus the review on one compact decision surface. Another planning exercise separated five buyer states and contrasted a one-month trial with a yearly commitment. Those quantities do not establish a benchmark, forecast conversion, or define the right offer for another product. Their value is practical: they force the team to specify the prompt, buyer, and commitment being tested. ## Keep the live test in the evidence plan A modeled willingness-to-pay signal is not observed purchase behavior, and this use case does not carry a named customer result or a specific accuracy figure. For a high-stakes pricing decision, carry the same causal question into recruited real-human validation before treating the pre-production result as final. The experiment process should define both the early screen and the production evidence needed to make the shipping decision. ## Test the draft already under review Bring the wording, price framing, trigger, and intended buyer for the next upgrade prompt. Book a walkthrough to map those choices into a controlled comparison and define what the later live test still needs to prove. --- # Persona-Based Research: When One Chatbot Answer Is Not Enough Evidence URL: https://subconscious.ai/blog/use-cases/persona-based-research-when-one-chatbot-answer-not-enough-evidence Published: 2026-08-08 A product or research leader has a decision to make: ship a feature, set a price, or launch a message. A persona-conditioned chatbot can help the team frame the buyer and rehearse possible reactions. Its answer should not decide the launch. A single conversation is a hypothesis about buyer behavior, while a controlled test of two or more alternatives is evidence about a defined choice. ## Use the persona to sharpen the question Persona-based research organizes what a team knows about a buyer group so that product and market questions become specific. The useful output is not a fictional biography. It is a sharper account of the buyer's goals, constraints, purchase context, and decision criteria. That account can help a team: - form an early product hypothesis; - draft messages for a defined buyer; - prepare questions for customer interviews; and - identify concrete alternatives that deserve a test. A number posted without its limits reads as marketing, so this one comes with its boundary attached. An inherited planning benchmark put conventional persona construction at weeks to months. Treat that figure only as historical context, not as a current Subconscious delivery claim or a market-wide norm. ## Do not mistake fluency for observed choice The limits of a method belong on the same page as its evidence, published where a buyer can check them. Published research reports that persona-conditioned language-model outputs can diverge from human survey and choice behavior through stated-preference distortion, reduced variation, and sensitivity to option order or labels (Distorted Perspectives of LLM-Simulated Preferences: Can AI Mislead Design?). Naming where a method breaks is what lets a buyer decide whether it fits their question. A separate reliability study found that agreement with human data varies by question type and population, so no single persona configuration should be assumed to reproduce human responses across decisions (Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents). This does not make the conversation worthless. The answer can expose assumptions and improve the alternatives. It does not measure how a target market chooses between those alternatives. ## Match the method to the commitment The required evidence should rise with the cost of being wrong. Early exploration can stay conversational. A decision that commits product work or launch budget needs a designed comparison. Subconscious fits the controlled-comparison side of this decision. For a study built around a concrete choice, it compares two or more alternatives in a controlled experiment on a simulated market and estimates which is more likely to change a defined outcome. Uncertainty is reported where the study design supports it. The scope of the study is also the boundary of the result. It does not automatically establish segments, rank every possible scenario, or prove market performance. When the cost of being wrong warrants another evidence layer, the same causal question can move from a simulated experiment to real-human testing without changing what the team is asking. ## A practical handoff Keep the persona conversation when the open question is: - Who is the buyer? - What constraints shape the purchase? - Which hypotheses or alternatives should we examine? Move to a controlled experiment when the open question is: - Which price point changes purchase choice? - Which message changes preference? - Which feature framing changes adoption intent? The experiment answers the buyer decision the conversation cannot. Review case studies for applied examples, or define a comparison for your decision in a research consultation. [figure: Two-column table: persona conversation vs controlled comparison, across framing the buyer, developing alternatives, a go/no-go call, and each method's boundary.] --- # Build vs. Buy: A Custom MCP Research Agent or a Causal Research Platform URL: https://subconscious.ai/blog/implementation-and-procurement/build-vs-buy-custom-mcp-research-agent-causal-research-platform Published: 2026-08-08 The deciding factor between wiring a custom MCP agent chain and using a causal research platform is whether the decision can tolerate an unvalidated verdict with no confidence measure or audit trail. A growth or marketing-ops team can wire a coding agent to a product-analytics tool, a synthetic-response tool, and a messaging tool, and have it post a weekly "ship or don't" verdict on a shipped feature. The harder question is not whether that pipeline can be built. It can. The question is what happens the first time the verdict is wrong and nobody in the channel can tell. ## What does tool-chaining through MCP actually do? The Model Context Protocol standardizes how an AI agent discovers and calls external tools, so one agent session can pull data from an analytics service, hand it to a second tool, and pass the result to a third, without custom glue code for each pair (Anthropic, "Introducing the Model Context Protocol"). An analytics vendor that ships an MCP server typically exposes its existing API surface as callable tools: querying events, cohorts, and dashboards through the same operations already available in its product (PostHog, "Model Context Protocol (MCP)"). Chaining several such servers behind one agent prompt is a real, working integration pattern. It is not an experiment design. ## Where the verdict breaks A common version of this pipeline: pull the cohort that used a feature in the last 30 days, describe that cohort to a synthetic-response tool, ask the resulting persona whether it would recommend the feature, and post the answer to a team channel. Each step works on its own. None of the steps checks the others. The persona's stated opinion is free-text output from a language model conditioned on a cohort description. It is not a controlled comparison between the feature and its absence, and nothing in the chain checks the stated opinion against how a real customer would actually behave. A team that treats that weekly post as a causal read on customer behavior can ship or kill a feature on an unvalidated verdict, with no held-out human check, no confidence measure, and no audit trail showing which step introduced the error. ## What does a structured causal question add? The fix is not more tools in the chain. It is defining, before any data is pulled, what would count as evidence and what would count as noise. Subconscious is the causal AI company. It runs randomized experiments on a simulation of a market and can validate the result against real human behavior, so the report answers which action drove the outcome rather than what a chat-style persona said it might do. A team can move from a simulated experiment to real-human validation without changing the underlying causal question, a step a hand-wired agent chain has no place for. Two related terms are easy to blur: "audience" and "panel." Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That audience graph is a pool the platform can draw a study against, not a recruited panel of 800 million people standing by to answer questions. ## Comparing the two approaches ## When the two are worth combining The build-your-own path is not wrong to attempt. Grounding a research question in real usage data is a legitimate pattern, and a team that already has product-analytics tooling in place can use it to decide which features are worth a proper test. Analytics data can narrow which cohort or feature deserves scrutiny; a structured, validated experiment should be what decides the action. ## What doesn't this analysis prove? This is a build-versus-buy framing, not a claim that a packaged, automated MCP workflow ships today. The comparison is about what each approach can and cannot validate, not turnaround time or per-call cost. ## Next step Before adding another tool to an agent chain, decide what would have to be true for the weekly verdict to be trustworthy: a defined causal question, a real-human check, and a confidence measure that travels with the result. Learn how Subconscious runs that kind of study, or see the research behind the method. [figure: Four-row comparison. MCP agent: question implicit, no validation, no confidence measure, only a tool-call log. Causal platform: question set upfront, human validation available, confidence reported, audit trail built in.] --- # An Automated MMM Says Shift Budget. Should You Act on It? URL: https://subconscious.ai/blog/methods-and-validation/automated-mmm-says-shift-budget-should-act Published: 2026-08-08 A marketing mix model (MMM) fits historical spend and outcome data to estimate each channel's contribution, then automates the data prep and Bayesian modeling choices behind that fit. Newer AI-driven MMM tools go further: upload data, get carryover and saturation assumptions selected automatically, and simulate "what-if" budget shifts in minutes instead of weeks. That speed changes the decision a CMO actually faces. The question is no longer "how long will this model take to build." It is: when the model says shift budget from channel A to channel B, which of the assumptions that would make that a causal claim actually hold? ## What the automation actually does An automated MMM agent typically handles three steps end to end: - Data prep. Reconciling platform-reported spend and conversions (TikTok vs. GA4, for example) into one modeling table. - Model fitting. Choosing Bayesian priors and functional forms for carryover and saturation effects, tailored to the uploaded data. - Scenario simulation. Running "what-if" budget reallocations against the fitted model and returning a projected ROI change. A number without its limits attached is marketing copy. Automating those three steps removes weeks of manual modeling labor. It does not change what the underlying model can prove. An MMM, automated or not, still estimates channel contribution from observational spend and outcome data. Faster fitting does not verify the assumptions, such as no unobserved confounding and correct adstock/saturation form, that would make those estimates causal. ## What is the gap between a faster model fit and a causal answer? Correlation vs. causation is the standard failure mode in marketing mix modeling: two channels that ran simultaneously, or a channel that rode a seasonal demand spike, can look like strong performers in a regression without actually driving the outcome (Recast, "How Do Causal Directed Acyclic Graphs (DAGs) Work in Marketing Mix Modeling?"). A model that reconciles TikTok and GA4 data automatically, then simulates a budget shift in seconds, is still answering "what pattern fit the historical data," not "what happens if I actually move the budget." Some MMM approaches address this directly by incorporating experimental data, geo-holdouts, or documented interventions into the model design (Lifesight, "Causal Marketing Mix Modeling (MMM): The Complete 2026 Guide"). The practical question for a marketing analytics lead is what to do with a specific reallocation decision, say moving spend from paid search to connected TV, when the MMM's recommendation for that exact move hasn't itself been tested. ## Where does a causal check fit before the budget moves? Subconscious runs causal, discrete-choice-style experiments that test how buyers respond to a specific offer or message directly, rather than inferring it from historical spend patterns. Applied to an MMM output, that means testing the buyer-facing consequence of the reallocation the model recommends, such as the resulting creative or offer change, before spend actually moves. The misses sit next to the hits in this description, on purpose. It does not test channel-level incrementality itself; a geo-holdout or lift test is the right instrument for that question. Naming what a tool doesn't do is what lets a buyer check the claim against the tool. This is a validation layer, not a replacement for the modeling step. Subconscious does not build or replace Bayesian time-series MMM software, does not do automated data wrangling across ad platforms, and does not generate model diagnostics or experiment-tracking artifacts. What a causal experiment adds is a second, independent read on the one decision that matters: does this specific reallocation actually move the outcome. A simulated experiment against a modeled population can test the reallocation decision quickly, with results scoped to that simulated population; when the decision carries enough budget risk to warrant it, Subconscious can test or validate studies with real human participants without changing the causal question. ## What this does not solve Stating the failure mode here is what lets a team check it before they rely on it. A causal check does not fix a badly specified MMM, and it will not diagnose why a model's priors were wrong. It answers one narrow question: does the specific action the model recommends hold up when tested directly. Teams still need the MMM to generate the reallocation hypothesis; the causal experiment is what a team runs before committing that budget. [figure: Two columns. Left: the MMM's job, reconciling spend/conversion data, fitting priors, simulating a hypothesis. Right: the causal check's job, testing that one move with buyers before spend changes.] ## Next step Before moving spend on an automated MMM's recommendation, isolate the specific reallocation decision and run it as a controlled test rather than accepting the model's fit as fact. See Subconscious's research for how these studies are designed, or talk to the team about testing a specific reallocation before it goes live. [figure: Five-step path: MMM fits spend data and recommends a reallocation; that move is tested directly with buyers; the result shows whether that specific move increased buyer choice; the team moves the budget or holds.] --- # When Is a Simulated Behavior Model Trustworthy Enough to Act On? URL: https://subconscious.ai/blog/methods-and-validation/when-simulated-behavior-model-trustworthy-enough-act Published: 2026-08-08 A causal effect estimated from a simulated buyer population is not automatically trustworthy. It becomes trustworthy after someone checks it against real behavior. That check, not the sophistication of the model, is what separates a decision-ready estimate from a plausible-looking artifact. ## The decision this answers A CMO or CSO looking at a launch, pricing, or messaging recommendation built on a simulated population has to decide whether to commit budget to it now or hold it for further validation. Committing before that check risks building a go-to-market decision on a model artifact instead of an actual buyer response. ## Why isn't simulation alone proof? Simulation-based models, whether they describe consumer choice or human cognition, work by approximating a process that is too complex to compute directly. The approximation can look convincing in isolation and still be wrong in ways that only show up when it is tested against data the model has not seen. Cognitive science research on likelihood approximation networks (LANs) illustrates the standard this way. LANs use a neural network to approximate the likelihood function of a cognitive model, making inference tractable for models that were previously impractical to fit. The published methodology includes parameter recovery: researchers simulate data from known parameter values, run the LAN-based inference procedure, and confirm it recovers those known values before applying the network to real behavioral data (Fengler et al., 2021, eLife). Recovery against known values shows the estimator is self-consistent under the model's own generative assumptions; it does not test correspondence to real behavior. ## The same logic applies to causal experiments on simulated markets Subconscious's causal action testing capability runs discrete-choice-style experiments on simulated populations to estimate which product, pricing, or messaging action is likely to move a target outcome. Structurally, this is a different validation problem than the LAN work: a LAN's approximation error can be checked against a known likelihood, while a simulated respondent population substitutes for the data-generating population itself, so the only check available is real human data. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result on direction of agreement, not a statement about effect magnitude, price elasticity, or interval calibration, and it is not a guarantee for a new market. It is published in the causal fidelity paper. The number matters less than the fact that it is measured and published at all. A team asking whether to trust a specific simulated estimate should ask the same question a cognitive modeler asks before trusting a LAN: has this class of model been checked against real outcomes, and does that check cover the situation at hand? ## What doesn't this validation prove? The LAN parameter-recovery standard comes from clinical cognitive-modeling research, not marketing or go-to-market decision-making, so its published accuracy figures do not transfer directly to a pricing or launch decision. It is evidence that simulation-based inference can be validated rigorously, not evidence about any specific business outcome. Uncertainty output is similarly bounded. Confidence intervals and posterior-style uncertainty estimates come out of any fitted discrete-choice model automatically; the caution is not whether an interval exists but whether it is well calibrated and whether the study design generated enough attribute variation to identify the parameter being estimated. An interval from a poorly identified or miscalibrated model is not more trustworthy for being a number. ## How do you move from simulation to a real-human check? The practical move is the same one cognitive modelers already make: run the simulation, then check it against real behavior before treating the result as settled. Subconscious can test or validate studies with real human participants, which lets a team move from a simulated estimate to a real-human check without changing the underlying causal question being asked. [figure: Branching path: is the model class validated? If no, hold. If yes, does that check cover this situation? If no, hold. If yes, was uncertainty estimated within the simulation or only assumed? If assumed, hold. If estimated, weigh that it excludes simulation-to-human transfer error before committing budget.] ## Where to start Review the replication methodology and current leaderboard before treating any simulated estimate as final, and see how the validation process works end to end before scoping a study that needs a real-human check built in. [figure: Five-step decision path: simulate a candidate behavior model, approximate the likelihood, recover known parameters, check against real behavior, then trust or re-validate the result.] --- # Will Your Claim Survive an Expert Audience? Test It Before They See It URL: https://subconscious.ai/blog/use-cases/will-claim-survive-expert-audience-test-before-they-see Published: 2026-08-08 A clinician, a security engineer, or a compliance officer reads a vendor claim differently than a consumer does. They know the failure modes, and what a number is hiding. If a launch claim cannot survive that kind of scrutiny, the moment to find out is before it reaches the real expert, not after. ## Why do expert audiences break the usual research playbook? Consumer research assumes an audience that reacts to messaging on instinct. A cardiologist evaluating a diagnostic claim, a network architect evaluating a security pitch, or a regulator evaluating a compliance framework brings domain knowledge that consumer research was never built to anticipate. Recruiting that audience at scale is also expensive and slow: specialists are hard to reach and reluctant to take vendor surveys. Physicians in particular remain openly skeptical of AI-driven claims even as adoption grows, which is the exact failure mode a technical or regulated launch has to plan for before it reaches them (Healthcare Dive, 2026). ## What decision does this affect? The question is not whether the product works. It is whether a specific claim, framed a specific way, holds up when a skeptical domain expert reads it. That decision sits with whoever owns the launch messaging for a technical or regulated product: marketing leadership, regulatory affairs, or both. Get the framing wrong and the cost is not a lukewarm response. It is a wasted advisory-board cycle, a delayed analyst briefing, or a messaging rewrite after the real regulator or reviewer has already formed an opinion. ## What causes a claim to fail with experts Claims fail with expert audiences for a narrow, diagnosable set of reasons: - Ambiguous precision. An accuracy claim that does not specify what it measures invites the obvious follow-up: accuracy of what, measured how, against what baseline? Experts ask that question immediately; consumers rarely do. - Wrong variable. A claim built around speed or convenience falls flat with an audience that cares about a different variable entirely, such as diagnostic accuracy or regulatory defensibility. - Generic framing. A claim written for a general buyer, not tuned to a named specialty, seniority, or regulatory context, reads as vendor noise rather than an argument built for that reader. ## How do you test a claim before an expert sees it? Subconscious runs a controlled, causal experiment comparing candidate claim framings against a segmented audience matching the real professional buyer type, such as clinicians in a given specialty, security engineers, or compliance officers. The output is a causal effect with confidence intervals showing which framing holds up and why, not a summary of impressions. This is a comparison of framings, run before the team commits budget or submits work for real expert review. Learn more about how Subconscious runs these experiments. ## Moving from a simulated comparison to real-expert validation A causal comparison of claim framings is not a substitute for the real expert's judgment. When the decision depends on it, a team can move from the simulated comparison to validation with real human participants without changing the underlying causal question. See how this fits into a broader research workflow. ## What this does not do - It does not grant regulatory clearance. A regulator's approval still requires the regulator. - It does not substitute for peer review. A methodology comparison is not a peer-reviewed result. - It does not produce a citable population statistic. The output is a causal comparison between claim framings, not proof that any expert body has endorsed the work. - It does not guarantee faster delivery or lower cost than any other approach; no such comparison has been run. ## Before the advisory board sees it The teams that get the most out of this kind of test are the ones who have already watched a technical or regulatory claim collapse on contact with a skeptical expert. Pick the specific claim that most needs pressure-testing, define the expert audience precisely by specialty, seniority, and region, and run the comparison before the advisory board, the analyst, or the regulator sees the work. For medtech, pharma, and other regulated launches, see how this applies to pharmaceutical decisions, or talk to the team about a specific claim. [figure: Five steps: pick the claim under pressure, define the expert segment, run a causal comparison of framings, see which survives and why, then validate with real experts.] --- # Cost-Plus or Value: How to Stress-Test a Price Before You Lock It In URL: https://subconscious.ai/blog/use-cases/cost-plus-value-how-stress-test-price-before-lock Published: 2026-08-08 Founders setting price for the first time usually reach for the same shortcut: add up costs, tack on a margin, call it a rate card. That answers what this costs to deliver, and skips what a customer will pay rather than walk away. Underpricing on a cost-plus basis gives away revenue on every contract signed at that number, and the loss compounds for as long as the contract runs. Overpricing without evidence risks losing winnable deals to a competitor the buyer never compares you against. Either mistake is expensive because price is hard to walk back once existing customers are anchored to it. ## Why does cost-plus hide the number that matters? Cost-plus pricing treats the customer as a passive recipient of a formula. It never asks what the customer is actually trading off: price against turnaround, price against a feature, price against the risk of switching providers. The standard method for recovering that trade-off is a discrete choice experiment (DCE): present respondents with choices that vary price alongside other attributes, and estimate willingness-to-pay from what they pick, rather than inferring it from cost math or anecdote, as documented in the peer-reviewed literature on discrete cost and random-coefficient willingness-to-pay estimation (Health Economics Review). ## What does a price stress-test look like in practice? A pricing decision becomes testable once it can be stated as a choice: which combination of price and feature set does the customer actually prefer, and by how much does demand shift as price moves. Subconscious runs this kind of test as a randomized experiment: respondents see controlled variations in price and other attributes that matter, and the resulting choices are the evidence, not a guess dressed up as a formula. ## What is this test, and what is it not? A discrete choice experiment requires a defined respondent sample and a designed choice task. It is a structured survey experiment, not a live back-and-forth with a simulated persona, and runs on a study cycle rather than producing an instant answer. Treat the output as evidence for the pricing decision in front of you, not a guarantee about how any one customer will behave. Subconscious can also validate the same study with real human participants once the simulated result narrows the decision, without changing the underlying causal question. That step matters when the pricing decision is large enough to warrant a second, independent confirmation before it goes into a contract. ## Before the rate card is final If the pricing decision has enough on the line, run the trade-off before the number goes into a contract. Review case studies on how this method has been applied, or book time to talk through the attributes worth testing for a given price point. For teams building the test internally, how Subconscious works covers the mechanics of setting one up. [figure: Two paths from a pricing decision. Cost-plus goes straight to a locked price with no demand signal. The choice-experiment path tests price against attributes first, producing a willingness-to-pay estimate.] --- # Can LLM-Generated Open-Ended Survey Responses Match Real Issue Distributions? URL: https://subconscious.ai/blog/methods-and-validation/llm-generated-open-ended-survey-responses-stand-real-verbatims Published: 2026-08-08 A consumer insights leader deciding whether to trust LLM-generated open-ended answers before fielding qualitative research needs one question answered: do those answers reflect the same aggregate distribution of issues real respondents raised, or only what sounds plausible? A published benchmark against a real survey gives a direct, if partial, answer. ## How did researchers test LLMs against a real government survey? The benchmark drew on ANES, the American National Election Studies, and specifically its 2020 Time Series Study: a survey run by the University of Michigan and Stanford University under National Science Foundation funding, which asked respondents, among other open-ended items, what they saw as the biggest problems facing the country, in their own words rather than from a fixed list. Researchers pulled a test set of 100 real respondents, took their demographic profiles (age, gender, race, education, income, occupation, religious affiliation, and self-placed political alignment), and prompted several large language models to generate an open-ended answer to that same question as if each model were that respondent, answering in 2020. Both the real and generated responses were then classified into a shared list of 18 recurring issues (healthcare, economy, immigration, and so on), and the resulting issue-frequency distributions were compared using three statistics: mean absolute error (MAE) between issue frequencies, Kendall's tau, and Spearman's rank correlation (the latter two measuring how well the relative ordering of issue importance matched, independent of exact frequency) (Scaling Open-Ended Survey Responses Using LLM-Paired Comparisons, Public Opinion Quarterly, academic.oup.com/poq/article/90/3/630/8551356). ## What held up, and what didn't The pattern is directional, though the paper reports no confidence intervals and the 100-respondent reference sample carries its own sampling error: no model exactly reproduced the real issue distribution, relative ranking survived better than issue share, and every tested model showed the same time-period drift. That drift is consistent with an LLM defaulting to its training-data-shaped view of an issue rather than the view a real respondent actually held, a pattern sometimes called temporal leakage; it is a distinct failure mode from demographic flattening, which compresses variance between personas rather than pulling answers toward more recent events. ## Why the ranking-versus-frequency gap matters for a research decision If a team is using LLM-generated text to explore likely themes before committing budget to fielded qualitative research, this benchmark suggests treating relative signal (which issues come up more than others) as more trustworthy than any specific rate or count the model reports. Frequency-level output looks precise but was the least reliable part of this result. That gap is why a single simulated pass isn't the last step for a consequential decision. LLM roleplay output isn't production-ready on its own; Subconscious runs the same behavioral question as a controlled study, then validates it with real participants, without changing the underlying causal question between the two steps (how we work). ## What are the limitations of this benchmark? This result comes from one demographic-conditioned prompting method, tested against one open-ended question, on one historical dataset. It speaks to issue-salience recall under a time-travel instruction, not to how LLM-generated text performs on other open-ended tasks, other demographic compositions, or unprompted (non-time-shifted) questions. The temporal-drift failure is worth planning around: any workflow that asks a model to represent a past or hypothetical state should expect the model to lean on more recent or more salient information unless that's specifically tested for and corrected. ## Where does this fit in a research workflow? Use LLM-generated open-ended text the way this study's authors frame it: a way to get an early read on likely themes, or to pre-test how a question might land with different groups, before spending on fielded qualitative work. Teams that want that baseline check built into the workflow can see how Subconscious structures controlled experiments and human validation on the research and leaderboard pages, or book time to discuss a specific question. [figure: Three-row list: issue ranking matched moderately to strongly; issue frequency matched weakly, with models naming more problems than real respondents; temporal accuracy failed, overestimating post-2020 issues.] --- # Persona Chat vs. Designed Panel Study: Which One Answers Your Next Decision URL: https://subconscious.ai/blog/comparisons/persona-chat-vs-designed-panel-study-which-one-answers-next-decision Published: 2026-08-08 A marketing or product research lead facing a go/no-go call has two very different tools available: a persona built from the analytics and CRM data a company already has, or a designed panel study that varies one thing at a time and measures the effect on stated behavior. Picking the wrong one doesn't just waste a research budget. It can put a pricing or positioning decision on an answer that was never built to carry that weight. [figure: Two columns: left, persona chat from analytics/CRM data, existing audience, no variation, no confidence interval. Right, panel study varying one thing, causal effect with confidence interval.] ## What an analytics-grounded persona can and can't tell you Several vendors build chat-style personas directly from a company's existing analytics and CRM data, letting a team query an audience segment in natural language (product overview). The persona reflects the audience a company has already reached: real segments, real behavior patterns. A number published without its limits is marketing. That grounding is also the boundary. It cannot tell a team how a market it hasn't reached yet, or a price, message, or feature it hasn't shipped, will move behavior, because there is no experimental variation behind the answer and no confidence interval attached to it. Ask the persona what a price increase does to renewal and it will produce a fluent, confident-sounding answer. Nothing in the underlying data tested that price. ## What does a designed panel study measure? A designed panel study starts from the opposite end. It holds everything constant except one variable under debate (price, message, feature, packaging) and measures how a defined population's choices shift. That is the same logic behind a randomized controlled experiment, applied to a market question instead of a lab question. Subconscious runs this kind of study as a controlled discrete choice experiment (McFadden DCE, Mixed Logit, ICLV) against a defined synthetic population, and reports the causal effect of the change along with a confidence interval. That is not a description of who is in an existing segment; it is an estimate of what a specific change does to behavior. ## Where each tool actually fits The table is a routing tool, not a scorecard. ## The failure mode this causes in practice Naming a failure mode here lets a buyer check for it before a decision ships on it. The costly mistake is not picking the wrong tool in the abstract. It is treating a persona chat's fluent answer as if it were a causal forecast. A team that asks a data-grounded persona how customers will react to a new price, and ships against that answer, has skipped the step where anyone varied the price and measured the response. The persona was designed to describe people the team already knows, not to test that change. The tell is in the question itself. "What does my audience look like" and "what does this specific change do to behavior" are different questions, and only one of them requires an experiment. ## Where does Subconscious fit, and where doesn't it? Subconscious is the right tool once the question is which specific change in price, message, or feature is likely to move behavior, not what an existing audience looks like. It isn't a substitute for a persona tool when the job is querying an audience already built from a company's own data. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. A team can also move a study from simulation to a recruited real-human validation on the same causal question, without redesigning the comparison. That is useful when the decision is large enough to warrant a second, independent check. ## Limitations The misses sit on the page next to the hits so a buyer can weigh both before trusting a result. A designed panel study still depends on someone specifying the right alternatives and the right population; a badly designed experiment produces a confident-looking effect estimate for the wrong question. A controlled study also only answers the specific comparison it was built for. It does not retroactively describe an existing customer base the way an analytics-grounded persona does. ## Next step An audience already reached calls for an analytics-connected persona tool. A specific price, message, or feature change calls for a designed experiment with a measured effect and a confidence interval. Review current replication results and how a study gets built, or talk to the team about the decision on the table. --- # 5 Alternatives to User Interviews When Recruiting Stalls URL: https://subconscious.ai/blog/comparisons/5-alternatives-user-interviews-when-recruiting-stalls Published: 2026-08-08 When you can't recruit enough of the right people for a full interview session, five other research methods still produce usable signal: support-ticket analysis, session recordings, contextual surveys, customer advisory boards, and simulated causal experiments. None of them replace a recruited interview panel on a high-stakes call: each trades some depth for speed, and the right choice depends on the question you're actually answering. ## The Bottleneck: Why B2B Interview Recruiting Fails Any product team running ongoing research eventually runs into one recurring obstacle: locating people who fit the profile and can actually make time to talk. - External recruitment is expensive at small scale. A study needing 12 participants still requires sourcing, screening, and scheduling each one individually. - Narrow segments are hard to fill. A target audience like "VPs of Engineering, B2B SaaS, 50-200 employee headcount" starts as a small global pool even before you factor in who's willing to make time. - Your most valuable customers have the least availability. Whoever you'd most want on the call tends to be the person least likely to have an open calendar slot for a research session. The method just has to match what recruiting can actually deliver in the time available. ## Five Ways to Get Signal Without a Recruited Panel [figure: Five research methods listed side by side as alternatives to a recruited interview panel, each with what it's best for and what it can't tell you.] ### 1. What Do Support Tickets and Feature Requests Reveal? Existing support conversations, ticket logs, and feature requests are unsolicited and contextual: customers describe problems in their own words, without a moderator in the room. It works well for surfacing pain points and ranking which bugs or feature requests matter most. It's weak on anything a customer hasn't already complained about, including new concepts or messages you haven't shipped yet. ### 2. What Are Session Recordings and Behavioral Analysis? Tools like Hotjar and PostHog capture how people actually move through a product, where they click, where they hesitate, where they leave. This is best for identifying UX issues, understanding navigation patterns, and confirming whether users find and use a specific feature. It shows what happened, not why, and it can only observe features that already exist. ### 3. What Are Contextual Surveys? Firing a brief survey at a precise moment in the product flow gauges satisfaction at key touchpoints and reveals why a feature does or doesn't get adopted. They're simple to trigger and easy to segment, but they depend on customers self-reporting in the moment, which limits how much reasoning a respondent will write down. ### 4. Customer Advisory Boards and Communities A standing group of customers who provide continuous feedback turns one recruitment effort into a consistently reachable panel. This is well suited to ongoing feedback loops, beta testing, and relationship-building that also generates research data over time. Naming the failure mode here is what lets you plan around it. The tradeoff is that an advisory board self-selects toward engaged customers, not your full target market. ### 5. Simulated Causal Experiments Rather than a single conversational persona, a simulated experiment compares defined actions, such as two messages, two concepts, or two prices, across a modeled population and reports which one moves the outcome and by how much. Subconscious runs this kind of controlled, causal comparison, and validates its simulated results against real human behavioral studies. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. A number without its limits is marketing. It is a validation result, not a guarantee for a new market. This method is best for concept validation, message testing, and comparing how different customer segments respond to the same stimulus, particularly for segmented B2B audiences that are hard to recruit at all. It is directional and hypothesis-generating, not a substitute for a recruited interview panel when a decision carries high financial or reputational risk. ## Which Method Answers Which Question ## What Simulated Experiments Can't Tell You Publishing the limit is what makes the method usable for a real decision. A simulated population is a modeled comparison group, not a recruitable interview panel: it cannot substitute for talking to your actual customers when the decision is expensive to get wrong. Treat it as a directional signal alongside support tickets, session recordings, contextual surveys, and advisory boards. When a decision genuinely needs recruited human interviews, such as entering a new market, a pricing change with contractual implications, or a claim that will appear in a regulated filing, recruit the interviews, even if it takes longer. ## Where to Go From Here Match the method to the question: use observation-based methods (tickets, recordings, surveys, advisory boards) to understand what's already happening, and use a simulated, causally-designed experiment to compare alternatives before you commit to a live study. See how the comparison holds up against published case studies, review how the causal experiments are built, or book time to scope a specific decision. --- # AI Research for Financial Services: Testing Client Decisions You Can't Survey URL: https://subconscious.ai/blog/industries/ai-research-financial-services-testing-client-decisions-cant-survey Published: 2026-08-08 Wealth management, insurance, and retail-banking teams have to decide which retention message, product proposition, price framing, or switching-moment intervention to fund, for clients who will not fill out a survey or join a focus group. Subconscious answers that decision by running a controlled experiment on a simulated version of the segment and measuring which alternative changes stated choice, with a causal effect and confidence interval attached. [figure: Three-item list: surveys reach hard-to-reach clients weakly; interviews need scheduled time; controlled experiments measure which alternative changes choice, reaching any segment without recruiting.] ## Why this decision matters Financial services firms hold more customer data than almost any other industry, and still struggle to explain customer behavior. Transaction data shows what happened. It does not show why. CRM data shows which products someone holds. It does not show whether they are satisfied, weighing a switch, or about to lapse. The cost of guessing wrong is concentrated in the accounts that matter most. A high-net-worth client considering a move rarely fills out a survey before leaving, a pattern documented in research on rich non-responders. A business owner juggling personal and business finances is not going to sit through a focus group. When a retention offer, a pricing change, or a new proposition is built on an untested narrative, the firm usually finds out it was wrong only after the client has already switched or the policy has already lapsed. ## What causes this outcome? Traditional research methods are constrained by the same three things in financial services: compliance review cycles, privacy rules on customer data, and the difficulty of getting time-poor or high-value people to participate. Those constraints do not remove the underlying behavioral question; they make it harder to answer with a survey, an interview, or a panel. A controlled experiment sidesteps the participation problem by comparing defined alternatives, not by asking a real HNW client or a real business owner to participate. The question shifts from "what do these clients say they want" to "which of these specific product, price, or message alternatives changes their choice, and by how much." ## Evidence Some client and product moments are hard to research for reasons beyond access. A firm cannot deliberately give a customer a bad claims experience to study how it affects loyalty, and it cannot manufacture a renewal-shock moment on demand to see who leaves. These are the moments where comparing controlled alternatives in a simulation, rather than waiting for a natural experiment to happen to a real customer, is the only practical way to get evidence before a decision ships. Examples of moments worth testing this way: - A renewal notice arrives roughly 15% higher than the prior term. Which retention message or discount structure holds the account, and which one accelerates the switch? - A claim takes three weeks and four phone calls to resolve. How much does that specific delay move stated loyalty compared with a faster, single-contact resolution? - A client has done all of their daily banking and investing through separate direct-to-consumer apps and has not been inside a branch in three years. What proposition, if any, gets them to consider a traditional provider? - A mass-affluent client is comparing a full-service advisory relationship against a lower-cost automated alternative priced at roughly half the fee. What specific value proposition justifies the premium, and for which segment does it fail to? Each of these is a comparison between defined alternatives, not an open-ended conversation. That distinction is what makes the result usable as decision evidence rather than as a plausible story. ## Options and trade-offs None of these methods replaces the others. A controlled experiment is the right tool once the firm has narrowed to a specific set of alternatives and needs to know which one is more likely to change behavior. ## Recommended decision process - Name the decision precisely: the segment, the specific alternatives under consideration, and the stated-choice outcome that counts as success (stated preference or a comparable stated-choice measure), and the real-world behavior, such as retention or uptake, it is intended to inform. - Define the population as narrowly as the real decision requires, such as HNW clients weighing a switch, next-generation wealth-transfer clients, or small-business owners evaluating banking relationships. - Run a controlled comparison of the alternatives against that population and measure the causal effect on stated choice, with uncertainty reported. - Route any claim, pricing, or compliance-sensitive language through the firm's compliance and legal review before it reaches a real client, regardless of how the alternative performed in the experiment. - Where the decision is consequential enough to warrant it, confirm the finding with real-human testing or validation before it ships, without changing the underlying causal question. ## Where does Subconscious fit? Subconscious runs randomized experiments on a simulation of the target segment, validated against real human behavior at 87% of the measured human ceiling (0.832 over 0.959; mean 0.73 across the 43 studies passing design filters; see the causal fidelity paper), to estimate which action is likely to cause a change in a defined outcome, for a defined population, with a confidence interval. For financial-services buyers, that means comparing named retention messages, product propositions, or price framings against a precisely defined segment, rather than running an open-ended simulated conversation. Subconscious can run controlled studies against a broad audience graph, which matters for financial-services segments that are narrow, high-value, and otherwise unreachable through recruitment. Where a firm needs to confirm a finding beyond the simulation, Subconscious can test or validate studies with real human participants. Learn more about how Subconscious structures and validates causal experiments. Firms evaluating this for a specific segment can read how the process works end to end. ## Limitations and failure conditions Publishing where the method stops is what lets a firm check a causal claim before it reaches a client. A controlled experiment does not replace compliance and legal review before a claim, price, or message reaches a real client. It does not replace the advisor or broker relationship, and it does not substitute for direct engagement with the firm's own customer data where that data is available and appropriate to use. Regulatory constraints specific to financial services still apply in full; a simulated result is evidence for a decision, not a compliance approval. Naming a failure mode here is what lets a buyer check it before treating simulated scale as validation. Running a study against a broad audience graph is not the same as recruiting real participants, and Subconscious keeps those two capabilities separate rather than presenting simulated scale as a substitute for recruited validation. A controlled experiment also will not tell a firm why a client is unhappy in their own words the way an interview can; it tells the firm which of the alternatives it already defined is more likely to change the outcome. ## Adjacent questions Can this replace a compliance-reviewed customer research program? No. It replaces the step where a firm would otherwise guess between alternatives, or wait for a real client interview slot that never gets filled. Does this require pulling individual customer records? No. The population is defined by segment characteristics relevant to the decision, not by processing an individual client's account data, which is part of why it is workable for hard-to-reach or sensitive segments. What is the fastest way to see if this fits a specific decision? Name one retention, pricing, or messaging decision for one defined segment and book a walkthrough to see the experiment design against that exact decision. --- # Lakmoos and Neuro-Symbolic Simulation: What a Regulated-Industry Buyer Should Actually Compare URL: https://subconscious.ai/blog/comparisons/lakmoos-neuro-symbolic-simulation-what-regulated-industry-buyer-should Published: 2026-08-08 An insights or pricing leader at an automotive, financial-services, or energy company evaluating a simulation vendor is usually comparing the wrong thing. The question isn't which tool produces answers that look most like a real customer's answers. It's whether the tool tells you which action moves the outcome you're deciding on, with enough certainty to commit budget or capital. Lakmoos is one of the more distinct vendors in this space, and deserves evaluation on its own terms, not as a stand-in for every simulation tool. ## What Does Lakmoos Actually Do? Lakmoos builds neuro-symbolic AI: neural networks paired with symbolic reasoning, aimed at modeling how people in specific regulated sectors think, feel, and decide (Science behind: Neuro-symbolic AI, Lakmoos AI). A symbolic reasoning layer can help a model apply an explicit decision rule, which matters where the actual choice a person makes is governed by one: a compliance threshold, a safety standard, a tariff structure. The company builds within three verticals: automotive, finance, and energy (About, Lakmoos AI). Inside those sectors it builds calibrated models meant to reflect industry-specific behavior patterns, regulatory context, and stakeholder dynamics, rather than one general-purpose simulation applied across any market. ## The gap a similarity score doesn't close The real category question, for any vendor whose central claim is how closely a simulated response matches a real one: does that similarity score tell you what to do? Matching how a population would likely answer is an input to a decision, not the same as identifying which action causes a shift in the outcome you care about, for which segment, with a quantified range of confidence. A pattern-matching tool can be well-calibrated to a sector's behavior and still leave open the question a pricing, launch, or positioning call turns on: if you do X instead of Y, does the target metric move, and by how much? The cost of skipping that step compounds in a regulated vertical. A pricing or claims decision that ships on a plausible-sounding simulated response, never checked against a controlled comparison or a real-human holdout, can trigger a compliance review after the fact, waste a launch cycle, or commit capital to the wrong segment before any real-market signal exists. ## Comparing what each approach is built to answer ## What proof should look like before capital moves If a vendor's central claim is a similarity or accuracy score, the follow-up question: reproduced against what, and how often? On this measure, Subconscious's best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters, drawn from a broader corpus of roughly 300 replicated human studies across 9 domains, the mean is 0.73 (the causal fidelity paper). That figure describes replication of study outcomes on a validation set, not a guarantee for a new market. The practical advantage: a team can start with a simulated experiment and, when the decision warrants it, move to a real-human validation study or holdout without re-deriving the causal question. ## Where Doesn't This Approach Help? Subconscious does not ship a packaged calibrated decision model for automotive, finance, or energy the way a vertical-focused vendor might. It is not a rule-based, neuro-symbolic reasoning architecture, nor a self-serve, instant-signup product tier. If the actual decision in front of you turns on a narrow, rule-governed regulatory logic question inside one of Lakmoos's three sectors, and the scope exists for a tailored vertical build, that architecture may be the more direct fit. Simulation, of either kind, has a hard boundary: a validated simulated study is not an observed usability session, a clinical trial, or automatic proof of market performance. Real-human validation extends the causal test; it doesn't turn it into a different kind of evidence. ## How Do You Make the Actual Decision? Before comparing vendors on similarity or accuracy claims, define the decision itself: name the specific action under consideration, the outcome it's supposed to move, and the segment it applies to. Then ask each vendor two questions. First, does the output estimate a causal effect of that action, or does it describe how closely a response matches a population pattern? Second, is there a path to check the result against real human behavior without restarting the analysis from scratch? How Subconscious structures that process, including the move from a simulated study to human validation, is covered in more detail in how Subconscious works, with methodology grounding in Subconscious research and applied examples in case studies. [figure: Two columns: Lakmoos answers how a respondent likely thinks or answers. Subconscious answers which action moves the outcome, for which segment, with quantified uncertainty. Both can extend to real-human validation.] --- # Should You Trust a Raw Tracking-Poll Average? A Hierarchical Model Answers That URL: https://subconscious.ai/blog/methods-and-validation/should-trust-raw-tracking-poll-average-hierarchical-model-answers Published: 2026-08-08 A raw tracking-poll average is not trustworthy on its own; a hierarchical model separates the real trend from pollster bias, method bias, and sampling noise, and only what remains after that separation deserves your trust. A tracking metric moved this month. Before a research or insights leader acts on that move, one question decides everything: is this a real change in the trend, or noise from which pollster ran the survey, which method they used, and how many people they sampled? Reading the raw average and reacting to it treats every source of noise as signal. [figure: Causal chain: raw poll scatter separated into pollster and method bias, feeding a random-walk latent state, corrected for overdispersion, pooled hierarchically, converging on a trusted signal.] ## The decision this affects Any team that watches a repeated measurement over time (approval, awareness, purchase intent, brand favorability) eventually has to decide whether a change is worth acting on. A raw average risks two mirror-image mistakes: treating noise as a real shift and moving a launch, price, or message in response to nothing, or dismissing a real shift as noise and missing the window to respond, because it never separates the latent trend from the biases and sampling error layered on top of it. ## Why Does the Raw Average Mislead? A public dataset of French presidential approval polls from 2002 to 2021, across ten pollsters and four survey methods (face-to-face, phone, internet, mixed phone-and-internet), shows monthly standard deviation that spikes well above what a single, stable trend would produce, particularly during one president's second term and the start of another's. Some of that variance is a real, if temporary, bump in approval after specific events. Some of it is not real at all: it is a property of which pollster asked and how. A pollster-by-method breakdown of the same data shows the pattern: face-to-face polls report systematically lower approval than average; phone polls report slightly higher. Individual pollsters carry their own persistent lean independent of method. None of this is political bias. It is statistical bias: a sampling method or a house's weighting choices nudging a number in a consistent direction relative to the average across pollsters and methods, not relative to some independently known true value. ## What Separates Signal From Noise? The fix is to build the reporting biases explicitly into the estimate instead of averaging around them. Political scientists use the same hierarchical, dynamic estimation approach to recover latent public opinion from noisy survey series (Caughey and Warshaw, MIT/Political Analysis) and to pool polls with contextual information for dynamic forecasts (Political Analysis, Cambridge University Press). - Treat the true trend as a hidden (latent) state. The model never observes the "real" approval level directly, only noisy polls that are a function of it. - Let that hidden state move as a random walk. Approval this month depends on approval last month plus some innovation, not on some independent draw each period. - Give every (pollster, method) pair its own bias term. House effects are estimated from the data rather than assumed away. - Model overdispersion explicitly. A random-walk-plus-bias model evaluated with a plain binomial likelihood still underestimates how much polls actually vary. A beta-binomial likelihood, adding one parameter to separate variance from the mean, closes much of that gap. - Partially pool across groups instead of fully pooling or fully separating them. Treating every president's trend as identical understates real differences between terms; treating each term as fully independent throws away shared information. A hierarchical structure, where each president's trend is drawn toward a common trend but free to deviate, does both jobs. - Constrain the bias terms. A free intercept alongside free house effects does not resolve the ambiguity between the trend's level and the average reporting bias; a sum-to-zero constraint, a reference pollster, or an informative prior on the house effects does. The published version of the model, correctly parameterized, tracks each president's approval trajectory through its natural cycles and recovers plausible pollster- and method-level bias estimates relative to the average across pollsters and methods: face-to-face confirmed low, phone confirmed slightly high, and individual pollster leans consistent with what a practitioner who collects these polls by hand already expected. ## How Does This Differ From a Controlled Experiment? The model above estimates a hidden state from repeated observational measurements. It never intervenes on anything. It answers "what is the true trend, net of measurement noise and reporting bias?" for an outcome that already happened. That is a different question from "which action caused the outcome to change?" A causal experiment requires a treatment, a comparison condition, and random assignment; a random-walk smoothing model requires none of those and cannot answer it no matter how well it fits. Subconscious sits on the causal side of that line: Subconscious runs randomized experiments on a simulation of your market, validated against real human behavior, to identify which action drives an outcome, reporting effect sizes with confidence intervals that quantify sampling variability within the simulation, not simulator-to-human bias, from repeated controlled measurement rather than a single point estimate. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. It is a validation result, not a guarantee for a new market. The shared discipline is not the method. It is the refusal to read a single number at face value: both approaches insist on quantifying uncertainty and separating a real effect from an artifact of how the data was collected. ## What this means for evaluating any tracked metric A tracked number that does not separate a real trend from house effects and sampling noise cannot answer whether it is safe to act on. ## Limitations A number without its limits is a marketing claim. Noise correction like this answers a narrower question. It tells you what the underlying trend probably is, given the measurements you already have; it says nothing about what would happen under a different action, price, or message, because it never varies the conditions the data was collected under. A team that needs to know whether a specific intervention moves an outcome needs a controlled experiment, not a smoother trend line. Even a well-specified hierarchical model inherits the limits of its inputs: retrodictive checks in the source model show it still underestimates sharp swings from real-world events. Naming where a method stops is what lets a buyer check it before they rely on it. Method boundaries also matter when validating against real people. Subconscious can test or validate studies with real human participants, letting a team move from a simulated causal experiment to real-human validation without changing the causal question. That step confirms an experiment's result against real behavior; it does not turn a random-walk trend estimate into a causal test, and does not substitute for one. ## Next step If the underlying question is which action actually changes the outcome, that calls for a controlled experiment, not a smoother read of an existing trend. See how a causal experiment differs from an observational read of the same market, or look at published replication results for a sense of how closely simulated experiments track real human behavior. --- # Single-Persona Chat or Multi-Segment Panel: Which Fits Your Research Question URL: https://subconscious.ai/blog/comparisons/single-persona-chat-multi-segment-panel-which-fits-research-question Published: 2026-08-08 A team comparing AI-persona research tools usually asks which tool is better. The question that actually decides the purchase is narrower: does it need one persona's read on a message, or does it need to see where several distinct segments or stakeholders diverge in reaction? Those are two different jobs, and the tool categories built for them look similar while answering different questions. [figure: Ranked list: single-persona chat shows one buyer's reaction; multi-persona panel shows group splits, uncontrolled; causal platform runs a controlled experiment with confidence intervals, then optional human validation.] ## Why this decision matters A single-persona conversational tool lets a team question one simulated buyer about messaging, objections, or positioning: describe a buyer, ask it questions, read its answers. That fits a marketing team stress-testing ICP language before a campaign goes out. A multi-persona panel platform is built around a different unit: a group of simulated participants reacting together, so a team can see where a customer segment, a set of stakeholders, or a panel of reviewers agrees and where it splits. That fits an agency prepping a pitch for a buying committee, or a product team checking whether a concept lands the same way across segments. Treating a single-persona chat output as a cross-segment read is the costly mistake: a conversation with one simulated buyer is one simulated viewpoint, and mistaking it for a representative signal across a market, or for a decision a buying committee will reach, produces a call built on one opinion with no way to see where segments disagree. The reverse mistake also happens: paying for a full multi-persona panel platform for a quick single-persona sanity check on one line of copy is over-scoped spend. ## What causes the difference in outcome? The difference is structural, not a matter of polish. A single-persona tool holds one viewpoint in the room, so there is nothing to compare it against. A multi-persona panel exposes several simulated viewpoints to the same message, concept, or pitch and captures their reactions side by side, which is what lets a team see agreement and disagreement across a segment or stakeholder group, no matter how good the one persona's answers are. Neither tool category is wrong for its job. A single-persona tool that only needs to stress-test one message against one described buyer solves that problem well; it does not need to become a panel to do it. ## Evidence Recent work on using large language models for choice modeling finds that prompting strategy and model choice materially change how closely an LLM-based persona approximates real decision behavior (Frank et al., arXiv 2025), a caution against treating one fluent persona conversation as a calibrated behavioral estimate. Separately, foundation models built to model human cognition broadly are a different object than a model built to estimate a specific behavioral treatment effect for a specific population (Nature, 2025); breadth of cognitive modeling is not the same claim as accuracy on one defined decision. Subconscious runs controlled discrete choice experiments on synthetic populations and reports causal effects with confidence intervals, built for the cross-segment, multi-stakeholder version of this question rather than single-persona dialogue. These figures are published in full, misses included, because the misses sit on the same leaderboard as the hits. Subconscious's best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper for the method. It is a validation result, not a guarantee for a new market. Where the decision depends on it, a team can move from that simulated experiment to real-human validation without changing the causal question. See the replication leaderboard for how the experiments and validation work. The line between simulation output and validated human data is published so a buyer can check which step a given claim rests on. A simulated panel result is simulation output; it becomes human-validated data only once the separate validation step runs. Audience reach, a simulated experiment, and recruited real-human validation are three distinct steps, and a comparison that blurs them into one claim is misleading. ## Options and comparison ## When should you use each tool? Choose a single-persona conversational tool when the job is scoped to one-on-one buyer dialogue before a campaign ships. The narrow focus is a feature there, not a limitation. Choose a multi-persona panel platform when a descriptive read of where a segment or stakeholder group agrees and diverges is enough for the decision at hand. Choose a causal decision platform when the decision has real cost if you get it wrong and needs an estimate of which action changes behavior, not just a description of reactions. That is the how we work fit: a controlled experiment against the specific action under consideration, with a result that reports its confidence. ## Recommended decision process Start with the question, not the tool. Write down whether it is "what does one buyer think about this" or "how do different segments or stakeholders diverge." A single-persona conversation answers the first; a panel read answers the second descriptively. Neither answers "which action will actually change behavior," because that requires a controlled comparison across actions, not just reactions to one input. If the decision is a launch, a price, or a positioning choice with real cost attached to being wrong, treat the descriptive read as a starting hypothesis and take the causal question to a platform built to test it. Book a walkthrough to see how a controlled experiment maps onto a specific pricing, messaging, or launch decision. --- # Predicting Swinging Strikes with Bayesian Additive Regression Trees URL: https://subconscious.ai/blog/methods-and-validation/predicting-swinging-strikes-bayesian-additive-regression-trees Published: 2026-08-08 Bayesian additive regression trees predict swinging strikes by summing many shallow trees fit through MCMC sampling over pitch-tracking features like velocity and spin rate, producing a whiff probability with a credible interval attached. A model that returns one number is asking for trust it hasn't earned. It earns that trust with an uncertainty band, checked against data the model never saw. A worked example from baseball pitch-tracking data shows what that check looks like and why a decision-maker should demand it before any go/no-go call. [figure: A four-step path from a raw point estimate through a calibration check and a holdout test on unseen data to a decision-ready estimate with a credible interval.] ## The decision this example stands in for Approving a launch, a price change, or a messaging shift on a model's output means trusting that output's error range, whether the model states one or not. A point estimate with no stated uncertainty hides risk: if the real result lands outside the range nobody wrote down, the team has no record of how much uncertainty it ignored to get there. The fix is a habit of checking calibration and holdout performance before treating any number as decision-ready. ## A worked example: predicting swinging strikes The pitch-quality problem in baseball illustrates the check cleanly because the raw data is public and dense. MLB's Statcast system, using Hawk-Eye cameras installed in all 30 ballparks, captures velocity, spin rate, movement, and release-point geometry for every pitch thrown. The pybaseball library turns that into a single function call returning roughly 700,000 pitch-level observations per season with over 90 tracked columns. The target: whiff rate, the share of swung-at pitches a batter misses, removes take decisions from the denominator, leaving a measure that still reflects which pitches batters chose to swing at. Four physical features, standardized to z-scores, form the baseline model: These relationships are non-monotonic: a 98 mph fastball with elite backspin plays differently than the same spin on a 91 mph pitch, and tree-based models capture that interaction where linear models can't. ## Why use a Bayesian additive regression tree instead of a gradient-boosted tree? Gradient-boosted trees such as XGBoost dominate pitch-quality modeling, but they build through sequential greedy optimization and return a single point prediction; getting an error bar out of them requires bootstrap or conformal add-ons. A Bayesian additive regression tree (BART), introduced by Chipman, George, and McCulloch in BART: Bayesian additive regression trees (The Annals of Applied Statistics, 2010), builds trees through MCMC sampling from a posterior instead, so every prediction ships with a full distribution and a built-in credible interval. BART's tree-structure prior favors shallow trees: the probability that a node at depth d splits follows With the library's default values (α = 0.95, β = 2), a depth-0 node splits with 95% probability, but a depth-2 node splits only about 10% of the time, an implicit brake against the overfitting that plagues greedy tree growth. This BART variant, implemented within the PyMC probabilistic-programming framework, combines a specialized particle Gibbs sampler for the tree component with standard gradient-based samplers for any linear pieces in the same model, such as venue-level random effects. ## The check that actually matters: calibration, then holdout Fitting the model is the easy half. The example runs two checks before trusting the output: Calibration. Pitches the model assigns a 10% whiff probability should whiff about 10% of the time in the data. A calibration report that only shows the hits is marketing copy. The baseline model calibrated reasonably well across most of its range, with visible deviation only at the extreme high end, where small sample sizes make any estimate noisy. Holdout generalization. Twenty pitchers were held out of training entirely and scored only afterward. The holdout result belongs on the record next to the training result, misses included. Calibration on that unseen set showed some deviation in small-sample bins and mild underprediction at low whiff rates, but held up well enough overall to indicate the model captured real pitch quality rather than memorizing quirks of the pitchers it trained on. Neither check is optional for a model whose output will drive a decision. A model that calibrates well on its own training data but has never been scored against holdout cases has only proven it can describe the past. ## Adding features changes the uncertainty, not just the number A second version of the model added release-point features (height, side, extension), an aerodynamic proxy called axis differential, a same-handed-matchup indicator, and venue-level random effects, growing from 100 to 150 trees. Comparing the two models by Widely Applicable Information Criterion, a Bayesian estimate of out-of-sample predictive accuracy that avoids running full cross-validation, showed the richer model with a better (lower) WAIC than the simpler one, though WAIC estimates leave-one-observation-out accuracy for the pitchers and venues already in the data, not generalization to new pitchers, and can be unreliable for hierarchical models with influential observations. The enhanced model's uncertainty bands were wider than the baseline's, on the same pitchers. That's not necessarily the model performing worse: added variance components, weaker per-parameter identification, and more trees can widen posterior intervals on their own, and the widening may also reflect genuine execution variability, such as a pitcher's release height or extension shifting from throw to throw. A model reporting a narrower interval than the data justifies is the riskier of the two, not the more admirable one. ## What this means for evaluating a causal-experimentation vendor The specific numbers in this example (baseline calibration quality, the WAIC improvement from adding features, any pitcher-level leaderboard) describe a public baseball dataset, not a Subconscious result, benchmark, or case study. What transfers is the standard: a claim that a predictive model is trustworthy should come with a calibration check and a holdout test against cases the model didn't see; a causal treatment effect requires separate identification evidence, since counterfactual outcomes are never observed to check against, not a single number asserted with confidence. Subconscious's stated approach to causal experimentation follows the same discipline: quantify uncertainty where the evidence supports it, and where relevant, validate a simulated result against real human participants without changing the underlying causal question being asked. See how the method works and current results on the leaderboard. ## Where does this method not apply? Naming a method's edge is what lets a buyer check it against their own case. Checking calibration and holdout performance validates that a predictive model's point-probability estimates are honest; it does not by itself establish that its credible intervals have correct coverage. It does not turn a simulated causal-effect estimate into an observed usability session, a clinical trial, or automatic proof that a decision will perform in the real market. Real-human validation, when used, confirms a causal question against real respondents; it's a separate step from the calibration discipline described here, not a replacement for it. ## The question to ask before trusting a number Before a launch, pricing, or messaging decision rests on a model's output, ask two things: has this been checked for calibration, and has it been tested on cases the model never saw. A vendor, or a method, that can't answer both is asking for trust on a point estimate alone. --- # 9 Steps to Turn a Marketing Goal Into a Causal Experiment URL: https://subconscious.ai/blog/implementation-and-procurement/9-steps-turn-marketing-goal-into-causal-experiment Published: 2026-08-08 Require a stated goal and a testable hypothesis before any team runs a comparison test. Skipping straight to a comparison produces a result nobody can interpret or act on, because there is no prior claim to confirm or reject against it. ## Why this decision matters A product, insights, or revenue-growth leader deciding whether to require this workflow is deciding who owns each stage of an experiment: what the team must define, and what a testing platform can carry out. Skipping the goal and hypothesis stage produces a pile of numbers with no stated cause attached, wasting the test cycle and the setup work behind it, whether that setup was a spreadsheet, a survey panel, or a causal-testing platform. Naming the goal and hypothesis first makes a later result interpretable, in line with the broader argument for structuring strategic decisions around an explicit theory before testing it, rather than mining data for patterns after the fact (Design- and Theory-Based Approaches to Strategic Decisions, Organization Science). [figure: A five-stage decision path showing a buyer moving from a stated goal, to a testable hypothesis, to a controlled comparison, to an estimated effect, to a decision.] ## The 9 steps ### 1. Get clear on the goal Start with the outcome that matters, not the tactic. Whether the question is which marketing message resonates with an audience, whether a new product feature moves the target behavior, or what shapes a segment's preferences, be specific. Example: a team launching a product wants to know which message (affordability, innovation, or sustainability) gets millennials interested. ### 2. Turn the goal into a hypothesis Convert the goal into a testable claim: what outcome is being measured, and what factor is being changed. For example, the claim might be that among millennials, leading with a sustainability message lifts product interest by 10%, naming both the outcome (interest) and the lever (message type), so the result can be judged as confirming, weakening, or reversing that claim. ### 3. Design a controlled comparison A hypothesis needs a comparison built to test it, not a single message shown to one group. A controlled comparison, where audience members are assigned to different message versions and their responses are compared, separates what a message caused from what would have happened anyway. Subconscious supports building this kind of comparison directly: a team enters its hypothesis and the variables it wants to test, such as different message types, without hand-building the test design from scratch. ### 4. Choose the audience and the scenarios Pick the audience the decision depends on, such as millennials, and set up the scenario variants the hypothesis calls for, such as a sustainability-led message against a price-led message. Subconscious runs these scenarios against a modeled audience built for causal experimentation, so the comparison reflects the segment the decision is about rather than a generic panel. ### 5. What happens when you run the comparison? Once the hypothesis, audience, and scenarios are set, the comparison runs and produces a result set the team can analyze. The value of this step is not speed; it is that the comparison was built to test the stated hypothesis, so the resulting numbers answer the question the team asked in step 2. ### 6. Read the results by mechanism and segment Look for which scenario moved the target outcome, and by how much, and whether different segments reacted differently. Subconscious's causal action testing and discrete-choice-style experiments estimate which action is more likely to move a stated outcome for the audience being tested, so a team can see whether a sustainability message worked because it aligned with a segment's values or because it stood out from what else was in market. Segment-level and uncertainty breakdowns beyond that estimate should be confirmed against current product documentation before a team relies on them, since coverage varies by experiment type. ### 7. Refine and re-test A single comparison rarely closes the loop. Use what the first test showed to sharpen the hypothesis, test a narrower set of variables, or run a follow-up comparison on a segment that responded differently. Multi-stage, adaptive test designs, where an early round narrows the field before a more focused follow-up round, make a business experimentation program more efficient over repeated rounds (A case for conducting business-to-business experiments with multi-arm multi-stage adaptive designs, PMC). "Our simulation results show that, compared to RCT, multi-arm multi-stage adaptive designs (MAMS) can reduce the sample size requirements and expected time to experiment completion whilst maintaining a similar level of statistical power." Legare, Yao, and Lo, Journal of Marketing Analytics (source) ### 8. Turn the estimate into a decision An estimate is only useful once it changes a decision: a marketing message, a product feature, or a pricing approach. Treat the causal estimate from step 6 as the basis for that choice, and be explicit about the confidence the estimate supports. Where a decision needs more certainty than a modeled comparison can offer, Subconscious can test or validate the same study with real human participants without changing the underlying causal question. ### 9. Why should the testing method stay privacy-conscious? A modeled audience lets a team explore a sensitive or early-stage question, such as a controversial message or an unlaunched product, without exposing real customers to the test or collecting their personal data in the process. That is a practical advantage, not a substitute for the human validation in step 8 when a decision needs it. ## What are the limitations of this method? A stated goal and hypothesis do not guarantee a well-specified experiment; a poorly chosen comparison can still produce an unusable result. Not every scenario comparison qualifies as a randomized controlled trial, and treating one as if it does overstates the certainty of the result. Subconscious does not optimize price or promotions automatically, does not output substitution or cannibalization matrices by default, and does not replace real-human validation when the decision in front of the team requires it. [figure: Two paths from one test result. Left: topline lift only, ending in a number with no cause attached. Right: broken down by mechanism and segment, ending in an explanation feeding a refine-or-re-test step.] ## Next step Read how the underlying method works in more depth on Research, see the workflow applied to a specific test on How We Work, or bring a live decision to a demo to see the comparison-design step firsthand. --- # The 48-Hour Agency Deck: Speed the Client Can't Poke a Hole In URL: https://subconscious.ai/blog/buyers/48-hour-agency-strategy-deck-simulated-buyer-experiments Published: 2026-08-08 A 48-hour deck survives client scrutiny when it keeps the same causal question fixed from the fast simulated read through a real-human check the agency names upfront. A client calls Wednesday afternoon. They need a competitive landscape read, an audience read, and a recommendation by Friday. The strategist's real decision is not whether to compress the timeline. It is whether to compress it in a way that survives the first hard question in the client meeting: "Is this real data?" ## The bind agencies are already in A traditional research cycle runs long because it stacks sequential stages: survey design and deployment (historically 1-2 weeks), fielding (2-4 weeks), analysis (1-2 weeks), and deck assembly (about 1 week). Add those up and a 5-9 week client timeline is common, a real disadvantage when a market question can't wait that long. The wrong fix is skipping the check entirely: running a fast simulated read, presenting it as validated market data, and shipping it. If the pattern doesn't hold against real buyers, the gap surfaces after the client has already acted on it, and the account absorbs the damage, not just the deck. ## Is "is this real data" a fair question to ask? Clients who ask whether a fast synthetic read is real are asking a sound methodological question, not a hostile one. Research bodies now treat it as a mandatory disclosure issue rather than a matter of agency judgment: an ESOMAR Congress paper on synthetic data in marketing studies documents that synthetic respondents can collapse variance and, in some conditions, flip the sign of key relationships relative to real respondents (ESOMAR, "Synthetic Data in Marketing Studies"). As of the 2025 ICC/ESOMAR Code, disclosure of synthetic-data use in a study is now mandatory, not optional (GMO Research, "ESOMAR 20 | GMO Research & AI"). That is the exact gap a hedge like "directional insight, not statistically representative data" gestures at without resolving. A disclaimer explains what the data isn't. It doesn't give the client a way to check it. ## Keep the causal question fixed from simulation to check Subconscious can test or validate studies with real human participants. The practical advantage is that a team can move from simulation to real-human validation without changing the causal question. A deck built this way carries a check the client can ask for, not a hedge that dares them not to ask. That distinction matters more than raw turnaround. A fast answer that can't be re-verified against real buyers is a bet the agency is making with the client's roadmap. A fast answer that names exactly how it would be re-verified is a claim the agency can stand behind. [figure: A five-step horizontal path: define the causal question, run a fast simulated read, decide if it needs a real-human check, re-run with real participants, write the recommendation with the check attached.] ## A workflow built around the check, not the deadline The compressed timeline agencies describe tends to follow the same shape, whatever the exact hour count: - Define the question before the method. Translate the client's business question into a specific, testable causal question: which action, tested against which alternative, for which buyer segment. Get this wrong and no amount of speed fixes it. - Run the fast read first. A simulated experiment can surface a directional pattern in hours instead of weeks. Treat the result as a hypothesis, not a finding. - Decide what needs a real-human check. Not every pattern needs one. A pattern that will drive a pricing decision, a message the client will run in market, or a recommendation with real budget behind it does. - Re-run the same question with real participants. Because the causal question didn't change between stages, the check confirms or corrects the same claim the deck is about to make, rather than answering a different one. - Write the recommendation with the check attached. State what was tested, on whom, and whether the simulated pattern held. That is what turns a directional read into something a client can act on. A turnaround number needs its source named alongside it to be checkable. Agencies that structure delivery this way have described roughly a two-day span for the full cycle, brief to deck, as a planning example from their own workflow, not a guaranteed turnaround. Treat any specific hour or day count from a vendor as internal to that vendor's process, not as a Subconscious commitment. ## Where does this approach fit, and where doesn't it? A savings number needs its limits published alongside it to count as evidence. Subconscious is not a packaged research-as-a-service product built for agency retainers, and there is no published figure for how many labor hours a causal-experiment workflow saves against a traditional cycle. There is also no case study yet of a full 48-hour agency engagement run this way. An agency evaluating this approach should treat any compressed-turnaround or cost-reduction number, from any vendor, as a claim to verify against its own pilot, not as a given. What is supported is narrower and more useful for the client conversation: a causal experiment, once designed, can run in simulation and then be checked against real people without redesigning the study. Read more about how that method works in research, and see the operating model behind it in how we work. [figure: Four boundaries: not an agency retainer product, no published labor-hours-saved figure, no case study yet of a full engagement, and any hour count is internal to one vendor's process.] ## Next step Before promising a client a compressed timeline, run one internal pilot: take a real, low-risk research question, run it as a simulated experiment, then check it against real participants before it ever reaches a deck. That pilot shows what the compression actually costs, in accuracy and in effort, before a paying client finds out first. Book a walkthrough to scope a pilot against a live client question. --- # A Latent Timing Segment Is a Hypothesis, Not a Reason to Move Budget URL: https://subconscious.ai/blog/methods-and-validation/latent-timing-segment-hypothesis-not-reason-move-budget Published: 2026-08-08 A model that groups customer activity by day and hour can hand you a clean story: "this segment is active Tuesday mornings." That pattern is already in the data. It does not tell you what happens if you change your timing, targeting, or messaging around it. ## What decision does this method force marketers to make? A CMO or head of marketing analytics who runs this kind of pattern-discovery model faces a narrow choice: treat the segment as an actionable driver and shift spend or scheduling toward it, or treat it as a hypothesis and test the action before committing budget. Get this wrong and a team shifts spend, send times, or messaging toward a segment whose weekly rhythm is correlational, sees no lift, and only then learns the pattern described what already happened, not what would change under a new action. ## How does the underlying model work? The technique is Latent Dirichlet Allocation (LDA), the same unsupervised topic model used in natural language processing, applied here to discretized calendar timestamps instead of words. The mapping: - Timestamps for an entity (a customer, a location, a driver) are discretized into calendar slots, such as day-of-week and hour-of-day. - Each entity's slot counts become a "document," the same role a bag of words plays in text topic modeling. - LDA learns a small number of latent time-of-week components, the role topics play in text: recurring patterns of activity concentrated in particular slots. - Each entity gets a mixture over those components, so its weekly rhythm is described as a blend rather than a single hard label. The result is a compact, interpretable summary of when different entities tend to be active, built entirely from timestamps already on hand. The foundational method is documented in Latent Dirichlet Allocation (Blei, Ng, and Jordan, Journal of Machine Learning Research, 2003). ## Where the method's own limits show up Two properties matter for how much weight a decision can put on the output: - A segment name carries little weight until its scope is stated next to it. It is descriptive, not causal. LDA groups co-occurring time-of-week behavior into latent components. It does not establish that intervening on one component (say, shifting an offer earlier in the week) changes an outcome for those entities. - Low-data segments are unstable. An entity with few observed events is assigned to a topic mixture that leans heavily on the model's prior rather than its own data, and that assignment can shift as more data arrives or prior settings change. Stating what the method proves and what it leaves untested is what makes the output usable. The output is a discovered pattern, not a tested mechanism. ## How do you turn the pattern into a tested action? Subconscious complements this descriptive, unsupervised pattern discovery with controlled experiments that test whether a specific action changes an outcome for a defined segment, with confidence language where the study design supports it. The practical sequence: - Use the latent-component output to name a candidate segment and a candidate action, such as messaging it at a different time or with different content. - Treat the pairing as a hypothesis, not a decision. - Run a controlled test of the action against that segment before shifting spend or scheduling to match it. When a decision depends on validating against real behavior, a team can move from a simulated experiment to a study with real human participants without changing the causal question being tested. [figure: Two-column diagram: entity with many events gets a stable assignment reflecting its own activity; entity with few events gets an unstable assignment leaning on the model's prior, which new data can flip.] ## A checklist before acting on a timing segment - Was the segment's assignment based on enough observed events, or is it leaning on the model's prior? - Does the plan treat the segment as a hypothesis, or has spend already moved on the strength of the pattern alone? - Has the specific action, not just the segment, been tested before it goes live? [figure: Path diagram: timestamps become latent time-of-week components, then a candidate segment and action framed as a hypothesis, then a controlled test, ending in a decision made with confidence where the test supports it.] Recurring weekly patterns are a starting point for a hypothesis, not a finished answer. See how Subconscious tests a specific action before it goes live. --- # Testing EV Buyer Segments, Feature Trade-offs, and Price Tiers Before Automotive Launch Spend URL: https://subconscious.ai/blog/industries/testing-ev-buyer-segments-feature-trade-offs-price-tiers-before-automotive Published: 2026-08-08 Automotive product, pricing, and marketing leaders can commit engineering, tooling, and campaign spend to an EV feature, price tier, or dealership message before knowing which buyer segment actually values it. A controlled experiment avoids this: compare the specific alternatives across precisely defined buyer segments and measure which one changes stated purchase choice, before the tooling order or campaign brief locks the choice in. [figure: Five rows, one per EV buyer segment, showing how one range or price figure lands differently: irrelevant, cost-compared, decisive, secondary, or ignored.] ## Why the decision matters Adding a feature such as a heads-up display or a larger touchscreen costs millions in engineering, tooling, and validation, and removing it after the decision is committed is close to impossible. A price tier or dealership sales script is cheaper to change, but a wrong call still shows up as dealership friction and wasted launch messaging that a team only discovers after the vehicle or campaign ships. The industry's model cycles used to make slow research acceptable. Annual brand tracking and multi-year product planning fit a world where a platform lasted roughly seven years between refreshes. That cadence is a poor match for a market where a competitor has changed pricing three times in a quarter and a new entrant can launch in an adjacent region with little warning. ## What causes the outcome EV buyers are not one audience. At minimum, teams typically distinguish five behaviorally distinct groups: an early adopter who bought an EV in 2019 and is shopping for a second one, a pragmatic switcher moving because the total cost of ownership now works in their favor, a reluctant switcher pushed by regulation or a company car policy, a luxury buyer who weighs badge and status ahead of powertrain, and a holdout who is not close to converting. Each group can respond to the same range figure, feature, and price differently, and a single combined research sample tends to average these responses together, hiding the split that actually matters for the decision. A controlled experiment is built to catch that failure mode. Instead of asking an open-ended question and summarizing what a mixed sample says, it holds the population's segment definitions fixed, changes one thing at a time, such as a HUD versus a larger touchscreen or a range framing versus a price framing, and estimates which segment's choice moves and by how much. ## Evidence A concrete range or price only means something in the context of who is being asked. A stated range of 500km reads as irrelevant to an early adopter who has already stopped thinking about range anxiety, decisive to a reluctant switcher who is still calculating commute margin, and secondary to a luxury buyer weighing the badge first. A price point near €80,000 clears a luxury buyer's threshold without hesitation while the same figure is compared, line by line, against a pragmatic switcher's current lease payment. These are illustrative examples, not a benchmark from any one study, but the same logic applies to any feature, message, or price point a team is testing. A result that only reports "63% preferred option A" without a segment breakdown throws away the information that actually drives the roadmap or pricing decision. ## Options and trade-offs None of these methods is obsolete: clinics and observed dealership behavior remain the only way to confirm what happens when a real buyer sits in the vehicle or walks the sales floor, and conjoint and MaxDiff studies remain a rigorous standard for a one-time, high-stakes feature trade-off. The gap they share is speed relative to a market where a competitor's pricing or positioning can shift inside a single quarter. ## Recommended decision process - Name the exact decision: which feature, price tier, range framing, or dealership script is being chosen between. - Define the buyer segments precisely enough that a result can be attributed to one segment and not another. - Run a controlled comparison of the alternatives against those segments and report the causal effect on stated choice, with uncertainty, rather than a single average preference score. - Treat the result as the first experimental pass, not the final word: confirm a consequential result with a vehicle clinic, an observed test-drive or dealership session, or an engineering and safety review before committing capital. - Repeat the comparison as the market shifts, since a pricing or competitive move can change which alternative wins. ## Where does Subconscious fit? Subconscious runs controlled discrete-choice experiments on a simulation of a defined market to estimate which action, such as a specific EV feature, price tier, or dealership message, is most likely to change stated purchase choice for a given buyer segment, with a causal effect and confidence interval. The output is a comparison between named alternatives, not a summary of plausible opinions from a generated persona. Subconscious can also test or validate a study with real human participants, which matters for a consequential decision like a feature investment: a team can move from the simulated comparison to a recruited human study without changing the underlying causal question. See current case evidence for how this has worked on pricing and go-to-market decisions in other categories, and book time to scope an automotive-specific comparison. ## What are the limitations and failure conditions? A controlled experiment on a simulated market still does not replace a physical vehicle clinic, an observed test-drive or in-dealership session, or an engineering and safety review. Nor is it a substitute for continuous, in-market brand tracking: a simulated experiment is not a panel of real people recruited on a recurring cadence, and a single comparison should not be read as an ongoing brand-monitoring product. The method works best when the decision is genuinely one alternative against another, with a defined buyer segment; it is a weaker fit for an exploratory, undefined question about how a market feels in general. ## Adjacent questions How many EV buyer segments should a team define before testing? Enough that a result can be attributed to a specific group's behavior rather than blended into a single average. Five is a common starting point (early adopter, pragmatic switcher, reluctant switcher, luxury buyer, holdout), but the right count depends on how differently those groups actually respond to the alternatives under test. Does this replace dealership training or sales scripts? No. A controlled comparison can show which script framing changes a stated decision for a given segment, but confirming it works on the sales floor still requires observing an actual dealership interaction. --- # Feature Prioritization Without Surveys: Choosing a Ranking Method URL: https://subconscious.ai/blog/methods-and-validation/feature-prioritization-without-surveys-choosing-ranking-method Published: 2026-08-08 Most roadmaps carry more candidate features than a team can ship in a single cycle: twenty on the list, capacity to build five, and a dozen stakeholders pulling in different directions on what should come first. The ranking exercise that decides which ones ship usually comes down to a survey, a stakeholder debate, or whoever argued loudest in the last planning review. What matters is not which tool gets used, but which ranking method tells you what will actually move an outcome like conversion or retention, versus what a group of respondents says it prefers. Getting this wrong is expensive in a specific way: a team burns a full development cycle building the feature that ranked highest on a flawed exercise, while the feature that would have moved the outcome sits unbuilt. ## Why stated-preference ranking is not enough A ranking method earns trust by naming where it breaks. A traditional prioritization survey asks users to rate or rank a list of candidate features. It has real, well-documented limitations: - Response bias. The people who fill out a survey are rarely a fair cross-section of the user base, skewing toward heavy users, frequent complainers, or those who simply have spare time to respond. - Question framing effects. How a feature is described in the survey changes how it gets rated. Small wording changes can flip the ranking. - No follow-up. A rating of "4 out of 5" for a feature carries no reason attached. There is no way to ask why. These are not reasons to discard stated preference. Surveys remain useful for discovery and for hearing customers describe problems in their own words. The limitation is specific: a stated ranking tells you what people say they'd choose, not what they would actually do if the feature shipped. That gap is documented in choice-modeling research: stated-preference methods are convenient to run but carry known response and framing bias, which is why researchers built revealed-preference methods to correct for it, and discrete-choice-experiment methods that force trade-offs to reduce it (Stated versus revealed preferences: An approach to reduce bias, Health Economics, Wiley). ## What is a synthetic ranking panel? One step up from a static survey is running the same ranking exercise against a simulated panel of user personas built from real segment data, then asking each persona to explain its top and bottom choices, producing reasoning alongside the ranking. If a persona ranks "bulk export" first and explains it saves 2 hours of manual work every Friday, that is a more useful signal than a bare rating. A common design for this kind of exercise: - Describe every candidate feature in a single sentence that states the benefit to the user, not an internal codename or spec. Keep the list to roughly 8 to 15 features; past that count, the ranking gets noisy. - Build separate panels for distinct segments, such as new users, long-tenured power users, and churned or at-risk users, since each group tends to value different things. - Present the full feature list to each panel and ask for a ranked order with reasoning for the top and bottom choices. - For a more rigorous version, run a max-diff exercise instead: present sets of four features at a time, ask which is most and least useful, and rotate the sets so every feature appears in multiple combinations. Running a series of these sets can produce a workable relative ranking for a list of roughly 12 to 15 features while avoiding the ordering effects that plague single-pass rankings. - Aggregate results across segments and look for universal priorities (features that rank high everywhere), segment-specific priorities (features one segment values and others don't), and surprises that warrant follow-up. - Cross-reference the resulting user-value ranking against revenue impact, retention impact, and strategic fit before locking a roadmap decision. That workflow fits a normal planning rhythm: a full ranking exercise quarterly, roughly 2 to 3 hours total, with a lighter 30-minute monthly check-in on the top few priorities. It is a real improvement over a single annual survey, but it is still a synthetic stated-preference exercise: personas are describing what they'd choose, not proving that shipping the feature changes behavior. ## Where a causal experiment changes the answer A synthetic ranking panel and a causal experiment answer different questions: Subconscious is built for the third row. Rather than asking a panel to rank features by preference, a controlled discrete-choice experiment on simulated segments tests specific feature alternatives against each other and estimates preference weights and choice probabilities that, through an explicit calibration model, indicate which one is more likely to move a defined outcome, with the uncertainty of that simulation-sampling estimate reported alongside the result, which does not capture how well the simulated population's preferences match real customers. That reframes the roadmap question from "which feature do people say they want" to "which feature, if shipped, is more likely to change what they do." Discrete-choice experiment designs have external-validity evidence behind them in adjacent fields: a systematic review of health-choice studies found that discrete-choice experiments systematically overpredicted real-world uptake, with the degree of overprediction varying by study design and context (How well do discrete choice experiments predict health choices?, The European Journal of Health Economics, Springer). That variability is why a roadmap decision built on a single ranking exercise, human or simulated, should be treated as one input rather than a verdict. [figure: Three-stage progression: survey and synthetic panel both measure stated preference from self-selected or simulated respondents; a controlled causal experiment adds randomized-design internal validity and feeds the roadmap decision.] Publishing what a method can't do is what lets a buyer check the claim against the result. A simulated ranking exercise is a reasonable way to triage a long list down to a handful of finalists quickly. It is not, by itself, proof that the winning feature will move the outcome. Subconscious can test or validate studies with real human participants, which lets a team move from a simulated experiment to real-human confirmation on the same causal question, without redesigning the test, before committing a development cycle to the result. ## What are the limitations of Subconscious? A number without its limits attached is marketing. Subconscious does not replace human research and does not output an automated roadmap recommendation; it estimates the causal effect of a specific, well-defined alternative on a specific outcome. Any claim about segment-level heterogeneity or willingness to pay for a given feature needs confirmation against the actual study design before it goes into a roadmap decision or gets repeated externally. ## A practical next step Pick the roadmap decision your team keeps debating without resolution: the one where a survey would take too long and a stakeholder argument won't settle it. Define the outcome that decision should move, such as activation, retention, or conversion, and run it as a controlled experiment on simulated segments before locking the next planning cycle. See how Subconscious approaches experimental design, review published methodology, or book a session to test one roadmap decision. --- # Concept Feedback vs. Causal Testing: Choosing How to Validate a Feature Before You Ship URL: https://subconscious.ai/blog/comparisons/concept-feedback-vs-causal-testing-choosing-how-validate-feature-before Published: 2026-08-08 A ship or no-ship decision has more than one kind of evidence available, each answering a different question. Single-user concept feedback tells you whether a simulated person reacts well to an idea. A multi-persona feedback panel tells you whether reactions differ across simulated segments. A controlled experiment tells you which version of a feature actually changes what customers choose. Picking the wrong one can send engineering time toward a feature that never moves a real decision, because the team validated a reaction instead of a cause. ## Two ways teams test a concept before real users see it Tools built for fast concept validation, such as Sanctum, a synthetic-user-testing product built for product teams (Y Combinator launch profile), present a feature or flow to a simulated user and collect structured feedback on it. That workflow sits between ideation and real user testing, answering one question well: does this concept read as confusing, appealing, or off, before a real person sees it. A related pattern extends that idea to multiple simulated personas, running the same concept past several segments in parallel to compare reactions. That catches a real problem single-user testing misses: a feature that reads well to one persona can confuse another, or land as a dealbreaker for a different segment. Both patterns tell you how a simulated audience reacts to something you show it, but neither isolates which specific change in the concept caused the reaction to shift, and neither tells you whether the reaction predicts an actual change in what customers do. ## What each method actually measures Reaction data and causal evidence are not interchangeable. A concept can score well in feedback and still fail to move an actual decision, because "people reacted positively" and "this specific change caused people to choose differently" are different claims. ## What is the cost of confusing reaction with causation? Positive concept feedback is not evidence that a feature will change what customers do. Shipping on the strength of reaction data, without testing whether the feature changes a real decision, risks the same outcome as shipping on gut feel: engineering effort spent on something that does not move the metric the business cares about. Concept feedback is not wrong so much as it answers a different question, and a ship decision needs the causal one. ## How does a controlled experiment answer the causal question? Subconscious is a causal behavioral platform built around a different question than concept-reaction testing: not "does a simulated user like this," but "which version of this feature causally changes what customers choose." It runs randomized, controlled experiments on a simulation of the market, estimating which alternative moves the decision-specific outcome. Where the decision depends on it, a team can move from a simulated experiment to a study with real human participants, without changing the underlying causal question being tested. That step matters when the cost of being wrong is high enough that a simulated result alone is not sufficient confidence for the ship decision. See how the platform works and case evidence. ## When is concept feedback still the right call? A controlled experiment is not a replacement for fast qualitative concept feedback. Early-stage exploratory work, surfacing confusion or gut reactions before a concept is fully formed, is exactly what single-user and panel-style feedback tools are built for. A randomized experiment is the wrong tool for that stage: it needs a defined set of alternatives to compare, and it does not observe someone using an interface the way a usability session does. Real-human validation does not turn a causal test into a usability observation, a clinical trial, or automatic proof of market performance. Treat concept feedback as the early filter and a controlled experiment as the test that precedes a ship decision with real stakes. Teams weighing this trade-off can review the research library or book a walkthrough. [figure: Path diagram: single-user feedback and multi-persona panel filter for confusion, both feed a controlled experiment that isolates which version causes a real choice change, feeding the ship decision.] ## Sources - Sanctum: Ship features to simulated users before real ones, Y Combinator - Sanctum product site --- # Pre-Testing Event and Conference Messaging Before the Booth Ships URL: https://subconscious.ai/blog/use-cases/pre-testing-event-conference-messaging-before-booth-ships Published: 2026-08-08 A trade show booth gets built, printed, and shipped weeks before a single attendee reads the tagline on it. By the time the team finds out whether the message made someone stop walking or blend into the noise, the budget is already spent. ## The decision a CMO has to make before the banner prints Trade show and conference budgets typically run five to six figures per event, committed to physical production (booth structure, signage, print) before any audience feedback exists (Vendelux, Event Marketing Statistics 2026). Exhibitors already treat this as a return-on-spend problem: industry analysis puts exhibitor ROI at roughly $20.98 returned per $1 spent on average, so the booth's messaging carries a large share of that return (TSNN, CEIR Report: How Do Exhibitors Spend Their Marketing Dollars?). A tagline that fails to register with someone glancing over mid-stride wastes that spend, however well the booth itself is built. The decision, then, isn't whether the booth looks good. It's whether to compare a small set of candidate taglines, pitches, and framings against the audience the event will actually draw, or finalize creative on internal consensus and find out at the show. ## Why don't standard pre-testing methods fit this channel? Surveys take weeks a print deadline doesn't have. A focus group typically costs thousands of dollars and still only returns a handful of opinions. Live A/B testing needs an audience that doesn't exist until the show floor opens. Event messaging also has to do something most other channels don't: stop someone mid-walk and earn attention in a few seconds, closer to billboard copy than an email subject line, with a live conversation attached once it works. ## Comparing messaging as a causal test, not a preference poll Subconscious can run a controlled comparison of a small set of messaging alternatives, such as tagline options, pitch framings, and opening hooks, against a defined target-buyer audience, testing which variant changes stated intent to engage rather than which one people say they like. This is a straightforward use of causal experiments: run the comparison, get a directional read on which framing moves intent, then finalize creative with that evidence instead of a debate inside the room. ## What does a messaging test catch that internal consensus won't? A messaging comparison surfaces reactions a team's internal debate tends to miss, such as a tagline that reads as interchangeable with every other vendor on the floor, or a framing that lands specifically because it names a problem the audience is already trying to solve. ## Where this extends beyond the booth tagline The same comparison logic applies to other event-adjacent decisions: a speaking-session title and abstract, an executive dinner invitation angle, a webinar registration headline, or a post-event follow-up sequence, each a small set of alternatives, a defined audience, and a decision that has to lock before a deadline. ## Where does this testing method run out? A pre-event message comparison can't fully reproduce the physical booth environment, the competitive noise of dozens of other vendors on the floor, or true walk-up attention span, so it estimates which message performs better in a controlled comparison, not the exact real-world lift in foot traffic or on-site engagement. Treat the result as directional, and pair it with observation at the event itself. Subconscious's real-human validation path lets a team move from that simulated comparison to a recruited-participant read on the same question without changing what's being tested; see how we work for the mechanics. ## Common ways this goes wrong - Testing copy without visual context. A booth is a visual environment; words alone tell half the story, so describe the full setup being tested, not just the tagline. - Asking about preference instead of action. "Which do you like" and "which would make you stop" produce different answers. Frame the comparison around the behavior that matters. - Ignoring the competitive floor. A booth doesn't stand alone at a conference, and a comparison that ignores the surrounding noise overstates how any one message will actually perform. ## A useful next step Before creative locks for the next event, run the candidate taglines or pitch framings through a controlled comparison against the event's expected attendee profile, and treat the result as one input alongside internal review, not a replacement for watching what happens on the floor. Recent case studies show how that comparison-to-validation path has played out on other campaign decisions, and a demo is the fastest way to see it run against a specific event audience. [figure: Two-column comparison. Left: "Which do you prefer?" yields stated aesthetic preference, the weaker signal. Right: "Which would make you stop and engage?" yields intent to act, closer to the real floor-traffic decision.] --- # Vendor Directory or Causal Experiment? Choose Based on the Decision URL: https://subconscious.ai/blog/comparisons/vendor-directory-causal-experiment-choose-based-decision Published: 2026-08-08 A research-ops or growth marketing lead should choose between vendor discovery and direct causal testing based on what remains unresolved. Use a directory or industry report when the team needs to understand the supplier landscape. Use a causal experiment when the team already has a product, pricing, or messaging action to test. A poor choice either adds procurement work that does not answer the decision or produces evidence that is too weak for the stakes. [figure: A branching path from "name the unresolved question." Unclear suppliers lead to vendor discovery, ending in a sourcing brief. A defined action and outcome lead to a causal experiment, ending in causal effect evidence.] ## Choose the path based on the unresolved question Vendor discovery answers, “Who can help us conduct this work?” Direct causal testing answers, “Which action changes the outcome we care about?” If the team cannot define the method, supplier category, or procurement criteria, sourcing comes first. If the action and outcome are already clear, the team can frame a controlled experiment without treating the question as a vendor search. ## When is the market landscape the missing evidence? Industry directories and trend reports help a buyer map research methods, supplier categories, and market practice before writing an RFP. An annual research-industry trends program is one public example of market-level evidence for that sourcing decision. That path fits an unfamiliar initiative, a formal supplier evaluation, or an internal case that depends on industry benchmarks. A sourcing brief that skips this line reads like proof it never claimed to be. Its output is procurement context. It does not itself establish whether a proposed price, message, product feature, or launch action will change buyer behavior. ## When is the action ready to test? Subconscious is a causal behavioral platform for controlled experiments on simulated populations. This path fits a team that can state the decision as a testable contrast, such as which price changes choice, which message changes consideration, or which offer changes preference. A simulated effect published without this line looks like a finished market result. Simulation is not automatic proof of market performance. Subconscious can test or validate studies with real human participants without changing the causal question. The team should set that evidence threshold before relying on the result for a high-stakes claim, price point, or launch decision. ## Compare what each route is designed to answer The routes serve different stages of the buying process. ## Set the evidence threshold before procurement Write down the decision, action, outcome, and consequence of a false conclusion. Then choose the path: - If the missing evidence concerns who can conduct the work, research the supplier market. - If the missing evidence concerns what buyers will choose under defined alternatives, frame a causal experiment. - If a simulated result would not meet the decision's proof standard, plan real-human validation on the same causal question. This sequence keeps the method attached to the decision. ## Turn the decision into a study See how a causal study moves from question to validation, or discuss the decision you need to test. --- # Pre-Testing the Renewal Deck Before the Client Reads It URL: https://subconscious.ai/blog/use-cases/pre-testing-renewal-deck-before-client-reads Published: 2026-08-08 An agency's quarterly account review is one meeting with no second chance. The room holds a day-to-day sponsor, a budget-holding executive who skims for the headline, a cost-focused finance reviewer, a skeptical outsider primed to push back, and sometimes a newcomer with no account history. One deck has to answer five questions in roughly forty minutes, and the standard prep, a partner read the Friday before, tests it against one reader, not five. The decision an account lead faces: run a segmented test of the deck's narrative and proof points against each stakeholder viewpoint before the meeting, or rely on an internal review that never simulates the room. Getting it wrong costs more than an awkward Q&A. A deck that reads as activity instead of proof to the person who signs the renewal, or that buries the objection the skeptical outsider was primed to raise, can stall the account with no chance to fix the story once the meeting starts. ## Why one review can't stand in for five readers Most client-facing decks have a single audience with a shared goal. A quarterly account review does not. The budget-holding executive walks in with three minutes of attention and no memory of the weekly status updates. The day-to-day sponsor, who lobbied internally to keep the relationship, reads every line for the wins they personally championed. The finance reviewer wants a ratio, not a narrative. The skeptical outsider has been hearing complaints from their own team and reads for the gap between the deck's story and that experience. The newcomer has no banked context and needs the deck to stand on its own. A single partner review can catch typos and tone. It cannot tell an account lead whether the executive's three-minute read lands on the same outcome as the sponsor's careful one. ## Five stakeholder viewpoints, one renewal decision Each viewpoint maps to a different stake in the renewal, not to an industry or function: Testing against all five before the meeting turns a single guess into five separate, checkable answers. ## How does Subconscious test this? Subconscious runs controlled causal discrete-choice experiments, including Mixed Logit models, that measure how clearly defined respondent segments respond to a message, framing choice, or set of proof points, reported as effect sizes with confidence intervals rather than one averaged read (ISPOR Conjoint Analysis Good Research Practices Task Force; Mixed logit). Applied to a renewal deck, that means testing the same slide, headline, or proof point against each of the five viewpoints as distinct segments, instead of asking one internal reviewer to imagine all five reactions at once. This is the same causal testing method Subconscious applies to other pre-launch message and framing decisions. ## What testing sequence should run before the meeting? One useful sequence works backward from the meeting date: roughly two weeks out, test whether the headline lands the same way for the sponsor and the executive; around ten days out, ask the finance viewpoint which number reads as proof and which raises a question; about one week out, ask the skeptical-outsider viewpoint whether the deck addresses the complaint their side has been raising; around four days out, ask the no-history viewpoint to read the deck cold and state what the engagement is for; and roughly two days out, ask every viewpoint what they think the meeting's real ask is. This sequence states where the schedule is fixed and where a team can adjust it, because a timeline presented as rigid when it isn't would mislead the reader. The spacing is a scheduling choice, not a fixed requirement. ## Patterns a segmented read catches that a single review misses A few problems repeat across account-review decks, and tend to surface only when the same page is read through more than one lens: - The wins slide is overcrowded. Packing twelve wins into one recap because none feels safe to cut buries the two or three that would land with the budget-holding executive, who remembers few and forgets many. - The data is framed for the wrong reader. The sponsor wants funnel metrics; the executive wants outcome-to-cost ratios; the finance reviewer wants cost per outcome. Decks tend to default to the metric the account team discusses weekly, rarely the one that convinces the person who signs the renewal. - The roadmap reads as a task list. Next quarter's plan described as activities lands differently than the same plan described as the outcomes those activities produce. - The renewal ask is either buried or too blunt. Leaving it unnamed leaves the executive unsure when that conversation happens; front-loading it turns the meeting into a pitch. A segmented read surfaces where the middle sits for a given account. - The skeptical outsider's concern has no slide. The team outside the day-to-day relationship almost never gets addressed directly, because the sponsor has no reason to have flagged the friction back to the account team. ## What does this testing not cover? Publishing where a test's reach ends is what lets an account lead check the claim before relying on it. A segmented panel test measures how a defined respondent segment, modeled on a stakeholder role rather than a specific named person, responds to a message and its framing. It is not a rehearsal of one client executive's individual politics or history, and it is not a recruited human focus group reading the deck live. It does not predict a specific renewal decision on its own and does not replace direct knowledge of the client relationship. It narrows where the deck's story is likely to lose a reader before the meeting. ## Where this fits beyond one account review The same five viewpoints, lightly adjusted, apply to any meeting where one deck faces a room of stakeholders who don't share a reading strategy: a renewal, a scope-expansion conversation, or a new-business pitch. An account lead can review how this fits into a broader testing workflow, or move directly to scoping a test against an upcoming renewal-critical review through Subconscious. [figure: Five-step timeline: sponsor/executive headline check, finance reviewer's number, skeptical outsider's objection, newcomer's cold read, then a check on the meeting's real ask.] --- # Persona Panels vs. AI-Led Interviews: What Evidence Supports a GTM Decision? URL: https://subconscious.ai/blog/comparisons/persona-panels-vs-ai-led-interviews-what-evidence-supports-gtm-decision Published: 2026-08-08 Persona panels and AI-led interviews can help a team form hypotheses. They do not, by themselves, establish whether a price, message, or launch action caused a behavioral outcome. When budget depends on the answer, require a controlled test of the action. Add real-human validation when the stakes justify it. [figure: Four-step path: transcript feeds a defined action and alternative, which feeds a causal test of a defined outcome, then an optional human-validation step before the budget decision.] ## What each research surface can tell you A simulated panel places several personas in one conversation. Its output is group commentary that a team can inspect, share, or use to generate questions. An AI-led interview focuses on a 1:1 conversation. The output is a transcript intended to surface stated motivations, language, and decision drivers. One current interview platform describes personas built from deep interviews and a conversational research workflow on its persona simulation page. Both surfaces can support exploration. Neither isolates the effect of changing one action while holding the rest of the decision constant. ## Why Is Fluent Output Mistaken for Proof? A detailed transcript can feel decisive because the reasoning is coherent. That fluency is not evidence that the modeled population will behave the same way in market. Before using simulated conversations to approve a pricing, positioning, or launch decision, check for three risks: - Variance collapse. Are responses converging so tightly that real disagreement may be missing? - Demographic flattening. Are group differences represented as labels rather than differences in behavior? - Over-rationality. Are respondents explaining choices more consistently than people tend to act? These checks define the limit of qualitative exploration, not its worth. A transcript cannot show that the proposed intervention will change behavior. ## The decision needs an intervention, an alternative, and an outcome Subconscious.ai is a causal behavioral platform. It runs controlled experiments on simulated populations to estimate which action moves which outcome. Its live method includes causal experimentation and discrete-choice-style modeling rather than free-form simulated roleplay. A useful study starts with a specific decision: - Intervention: the price, message, offer, or launch action under consideration. - Alternative: the action it must beat. - Population: the buyers whose behavior matters. - Outcome: the choice, adoption, trust, or preference the team needs to change. The research program is organized around that causal question. Subconscious can also test or validate a study with real human participants without changing the causal question. ## Choose the evidence that matches the decision ## When Is Open-Ended Conversation the Right Tool? Choose an interview-style workflow when the job is open-ended qualitative exploration of a simulated character. It can help a team collect motivations, vocabulary, objections, and hypotheses. Subconscious is not built for that job. It is also not a fully automated pricing optimizer, and it does not replace a defined experiment with an automatic recommendation. Human research remains necessary for discovery, emotional depth, regulated contexts, and real-world confirmation. ## Turn the GTM question into a test Use the transcript to decide what to test, not what to ship. Write the proposed action and its alternative. Name the buyer population and the outcome that would change the decision. Then review how a causal study is built or scope the decision before committing the launch or pricing budget. --- # Which Market Research Tasks to Automate, and Which Still Need Real Humans URL: https://subconscious.ai/blog/use-cases/which-market-research-tasks-automate-which-still-need-real-humans Published: 2026-08-08 Route fast, low-stakes exploratory questions to automated testing. Reserve real-human validation for decisions that are expensive to reverse or that the business will defend in public. That split is the decision a Head or VP of Market Research has to make once stakeholders expect AI to handle the first pass. ## Why the question is forcing a decision now Draft findings can land before a researcher finishes going through the data. A manager asks whether the team can just automate the first pass. None of that means research demand disappears; it means the mechanical parts of the job get faster, cheaper, and easier to access from outside the research function. The risk for a research team is narrower than "AI replaces researchers." It is spending most of the week defending manual first-draft work as craft, so stakeholders route around the function and act on unvalidated automated output. The opposite failure is just as costly: treating a fast, exploratory read as decision-ready evidence, then shipping a pricing, messaging, or launch decision on unvalidated signal and discovering the gap only after it's public or expensive to unwind. The U.S. Bureau of Labor Statistics' Occupational Outlook Handbook for market research analysts projects the occupation adding jobs steadily between 2024 and 2034. The job is not vanishing. The share of it spent on mechanical production is. ## How should each market research task be routed? A workable rule sorts tasks by what happens if the answer is wrong, not by how good the automated output looks. ## What should a research team build instead of running more tests? The strongest research teams in 2026 will be defined not by how many automated tools they use, but by a clear system that draws three lines: the boundaries of automated exploration, the findings a human has to check, and the claims that need validation with real people before the business relies on them. A workable version has four layers: - Exploration: generate hypotheses, objections, and alternative explanations quickly. - Directional testing: compare options fast, before committing time or budget to a full study. - Human review: check the audience definition, the framing of the questions, and whether the result answers the business question. - Validation: bring in real respondent data or fielded research when the decision is expensive or will be defended publicly. The value is not the exploratory output on its own. It is the disciplined path from a question to a decision the business can defend. ## How Subconscious supports that same progression This four-layer system maps onto how Subconscious works. Exploration and directional testing run as controlled, causal experiments: discrete-choice comparisons with quantified uncertainty, not a single fluent-sounding answer. Subconscious can run these controlled studies against a person-level audience graph covering 800 million real people for the exploratory stage. That audience graph is not a recruitable group of 800 million people, and it should never be described as one; it is the reach behind the exploratory stage. When a decision moves from directional to expensive or public, the same causal question can move to validation with real human participants, without redesigning the study or changing what is being tested. Keep the two claims distinct in any report: what the exploratory stage measured, drawn from the audience graph, and what real-human validation confirmed. Blending them into one number misrepresents both. ## The mistake that makes automated research dangerous That habit takes hold under pressure: speed is expected, the tool's answer sounds fluent, and a deck is due by end of day. A research function stays credible only if it can tell an output apart from actual evidence. Automated exploration can produce useful output. It cannot decide whether that output is valid for the decision in front of the business. The fix is to make limits part of the deliverable. State what the automated work was used for. State what it was not used for. State what still needs validation before the business acts on it. That reads as more disciplined, not less confident, because the limits are explicit instead of implied. ## What to do this week Start with one visible workflow, not a full rewrite of the research process. - Pick a real project with a live decision behind it. - Write the business decision in one sentence. - Define the audience and how expensive it would be to get the decision wrong. - Use automated exploratory testing only for the exploratory stage. - Go through the output by hand and flag each piece as useful, weak, or unsafe to act on. - Present the answer with a clear caveat and, when the decision warrants it, a recommended validation step with real participants. See how Subconscious runs that step. Run that cycle weekly across a month, and the team ends up with a working system that demonstrates speed, judgment, and quality control. [figure: A four-step path: exploration and directional testing run automated; human review checks framing and relevance; validation with real respondents gates decisions that are expensive or public.] ## Limitations This workflow does not remove the need for judgment about which questions deserve validation and which don't; that call still belongs to the researcher who understands the decision. Naming exactly what a figure covers is what lets a buyer check it against their own use case. Subconscious's audience-graph reach describes the population available for exploratory testing, not a guarantee about any single study's result, and it is not a substitute for real-human validation when a decision is expensive or public. Real-human validation confirms a causal question; it does not turn exploratory testing into a clinical trial or an automatic proof of market performance. Review Subconscious's current case studies and talk through your specific decision before committing a workflow to production. Book time here. --- # Checking a Synthetic Experiment Result Before You Act on It URL: https://subconscious.ai/blog/case-studies/checking-synthetic-experiment-result-before-act Published: 2026-08-08 ## The decision: trust the number, or check it first A synthetic discrete-choice experiment just told a research or insights leader which price, message, or feature wins. The next step commits budget: a launch, a pricing change, a positioning bet. Before that spend goes out, the question is whether this specific result is close enough to how real customers actually choose, or whether it needs an independent check first. Getting this wrong is expensive in a specific way: the decision looks well-supported at the time, and the gap only surfaces after the budget is committed and the launch has shipped. ## What does checking against published research look like? Subconscious designs synthetic experiments to be checked against independent, published choice research, not to stand on their own. A systematic review and meta-analysis of discrete choice experiments in health-related research reports agreement rates between stated-preference choices and real-world revealed-preference outcomes (PMC, National Library of Medicine), the same kind of prediction-accuracy comparison used here, not a bespoke internal metric invented for one vendor's dashboard. "Pooled sensitivity and specificity estimates were 89% (95% CI:77–95, I 2 = 97%) and 52% (95% CI:32–72, I 2 = 95%), respectively. The area under the SROC curve (AUC) was 0.81 (95% CI:0.77–0.84)." Zhang and colleagues, eClinicalMedicine (systematic review and meta-analysis of discrete choice experiments in health-related research) (source) Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. That figure is aggregate, not a per-run guarantee: it is measured across a body of replicated studies, not certified fresh for every new experiment before a buyer sees results. A single synthetic run can still diverge from real behavior even when the underlying method replicates well on average. ## When aggregate replication accuracy is not enough Aggregate accuracy answers "does this method generally match real choices." It does not answer "does this specific result, for this specific audience and decision, match real choices." For a high-stakes call, the second question sometimes needs its own check. Subconscious offers real-human validation as a separate service: a team can move a synthetic result into recruited real-human validation without changing the causal question being tested, so a high-stakes decision gets its own check instead of borrowing an aggregate number it was never measured against. Real-human validation is not a usability session, a clinical trial, or automatic proof that the finding will hold in market. It confirms the choice result against recruited participants answering the same causal question; it does not certify the launch, pricing, or messaging decision that follows. ## What to check before you commit budget - Does the result rest on a method with published, independent replication evidence, or only on internal claims. - Is the decision high-stakes enough that the aggregate replication number is not sufficient, and a dedicated real-human validation run is worth the added step. - Does the validation step test the same causal question as the original experiment, not a different one. [figure: Three stacked checklist items: published replication evidence exists, stakes justify a dedicated real-human validation run, and validation tests the same causal question as the original experiment.] ## Where does this fit before a launch decision? Run the original synthetic experiment, check the method's aggregate replication track record, and add real-human validation for decisions where being wrong is expensive. See the research methodology, the case studies of past runs, and the leaderboard of replication results across studies before deciding how much independent checking a given decision needs. [figure: A path from a synthetic result through a replication-accuracy check, a branch for high-stakes decisions, an optional real-human validation step, to the budget commitment.] --- # Social Listening for Market Research: Closing the Gap Between Signal and Decision URL: https://subconscious.ai/blog/implementation-and-procurement/social-listening-market-research-closing-gap-between-signal-decision Published: 2026-08-08 A social listening dashboard can tell a CMO that sentiment around a competitor's pricing move spiked. It cannot tell that CMO whether the counter-message their team drafted this morning would land with the audience driving that spike. That gap, between what a signal shows and what a decision requires, is where insights teams either move fast and guess, or move slow and miss the moment. ## Why isn't a sentiment spike an answer? A dashboard number means little without a plain account of what it cannot see. Social listening platforms are observational. They track volume, sentiment, share of voice, and emerging topics across public conversation. But the people generating those conversations never agreed to be surveyed, so a listening tool cannot put a new message, a pricing change, or a product concept in front of them and record a reaction. It can only report what already happened. Independent research on social media analytics in market research documents this same structural boundary: social data reflects who chooses to speak publicly, not a defined research sample. ESOMAR's guidance for buyers of social media research makes the same point from the buyer's side: Naming what a method was never built to measure is what lets a buyer check it before they rely on it. Before treating a listening output as a market-research answer, a buyer needs to know what population it represents and what it was never designed to measure. That leaves a real decision on the table. When a sentiment spike, a competitor misstep, or an emerging complaint surfaces, the team has to choose a response, a message, a feature framing, a pricing move, and commit budget to it. Three paths follow from there. ## The cost of each path Naming the cost of each path is what lets a team choose one instead of guessing. Acting on sentiment alone means shipping a response that was never tested against how the audience reacts to it. Commissioning a full human study for every candidate response is the opposite failure: rigorous, but slow enough that the market signal has often passed by the time results land. Waiting is its own decision, with a cost too. The middle path is a fast, structured test of the candidate response before committing budget, not a replacement for sentiment tracking or human research, but the step that sits between them. ## What does a decision-specific causal test add? Subconscious is a causal behavioral platform: it runs randomized experiments against a simulation of the market to estimate which candidate response moves a defined outcome for a defined segment. That means testing the counter-message, the feature framing, or the pricing move itself: a specific comparison tied to the decision at hand, not a general-purpose panel or an open-ended chat with a simulated persona. This is the step listening tools cannot provide: a controlled test of what happens if the team acts on the signal. When a decision depends on tighter validation than a simulated test can support, Subconscious can also test or validate studies with real human participants. A team can move from the simulated experiment to human validation without changing the causal question it is answering. That move matters when getting the call wrong would cost enough to warrant the extra time; it is not a step every response needs. ## Where the method stops A stopping point stated plainly is what keeps a fast method honest about what it does not cover. A causal experiment run against a simulated market is a fast first pass for narrowing options and iterating on messaging. It is not a substitute for recruited human participants on decisions that are regulated, high-stakes, or require exact measurement. Moving to real-human validation does not turn the test into a usability session or a clinical trial; it answers the same causal question with a different, more rigorous population. None of this replaces the listening layer. Detecting the signal still depends on the monitoring tools already in place. What changes is what happens the moment a signal demands a response. [figure: Five-step path: a signal is flagged, candidate responses are drafted, a causal test runs on a simulated market, high-stakes cases branch to human validation, then the path rejoins at budget committed.] ## Putting the sequence to use The practical sequence: let listening tools flag the signal, draft two or three candidate responses, test them against a defined segment before committing budget, and reserve recruited human validation for the decisions where getting the exact number right matters more than getting an answer quickly. Teams can see the underlying causal method, review how the testing process works, or look at applied examples before scoping a test against a live signal. --- # AI Buyer Journey Simulation: Awareness to Purchase URL: https://subconscious.ai/blog/use-cases/ai-buyer-journey-simulation-awareness-purchase Published: 2026-08-08 A buyer journey map is a bet before it is a document. Marketing and GTM leaders build one from a handful of retrospective interviews and whatever the sales team remembers about the last few deals, then commit content budget, ad spend, and a sales script to that pattern. If the trigger, the evaluation criteria, or the last objection in that map is wrong, every stage built on top of it misses the buyer. The map's biggest blind spot sits before any sales conversation starts. Gartner found that 67% of B2B buyers now prefer a rep-free buying experience, meaning most of a buyer's research happens away from any vendor, sales rep, or interview a team could later ask about (Gartner, 2026). [figure: Five stages of a B2B buyer journey shown in sequence, from trigger through discovery, evaluation, and decision objection, to post-purchase risk, each labeled as an assumption rather than a confirmed fact.] ## Five places the map can be wrong Each claim can fail independently, with a different cost. ### Trigger What event or problem sends this buyer looking for a solution? Get this wrong and pain-point content, search targeting, and top-of-funnel messaging aim at a problem the buyer doesn't have yet. ### Discovery Where does the buyer look first: a peer referral, LinkedIn, a search engine, an industry publication, a review site? This claim decides where awareness spend goes, and it is usually the least tested part of the map. ### Evaluation Which criteria carry the decision, what proof the buyer trusts, and what disqualifies a vendor outright. This shapes pricing pages, case-study selection, and how a sales deck orders its arguments. ### Decision objection Who joins the approval chain, what internal steps have to clear, and what the last objection is before signing. Get the objection wrong and a proposal answers a concern nobody in the room has. ### Post-purchase risk What "working" looks like across the early weeks after purchase, and which unmet expectation is most likely to cause early churn. This claim sets the onboarding checklist and the customer-success playbook. ## Turning each claim into a test A retrospective interview can describe what a handful of past buyers said they did. It cannot tell a team whether a new trigger message or a new objection response will move a defined segment. That requires a comparison, run before the content ships. Subconscious runs controlled discrete-choice experiments against a defined buyer segment (role, company size, industry, current solution, goals, and constraints held constant across the comparison) so any difference in stated intent or choice is attributable to the thing being tested, not the audience or the question (research). Each stage above becomes its own comparison: - At trigger, test two problem framings. - At discovery, test which channel cue shifts stated interest. - At evaluation, test competing proof structures or product concepts. - At decision, test which objection response holds up best. - At post-purchase, test which onboarding message reduces stated churn risk. A number without its setup is a marketing line. An example study design might run 3-5 buyer types, but that count describes one study's setup, not a fixed limit. The right number depends on how many segments the decision turns on. ## What does a completed test return? Run against a real decision, this kind of test produces a causal effect with a confidence interval for a specific comparison, not a stage-by-stage narrative of "what buyers are like." It tells a team which framing, criterion, or objection response moved the number, and by how much, for a named segment. A well-scoped test at one stage tends to surface the top 3 open questions the segment has there, alongside the effect itself. That combination is evidence a team can act on, rather than a story a team has to trust. [figure: Five arrows, one per journey stage, each pointing to its matching comparison test to run at that stage.] ## Where does the test stop? A causal effect only earns trust when its edges are published too. This method does not produce a free-form journey narrative, and it does not interview or roleplay a persona through open-ended conversation. It is not a replacement for qualitative discovery: win-loss interviews, sales-call review, and product analytics tell a team what already happened. What causal testing adds is a way to check a specific claim before betting a campaign or a sales process on it. Interviews with five customers who bought over a year ago and a structured comparison run today answer different questions. One tells a team what happened. The other tells a team what is likely to move a defined segment next. A team that wants both, the retrospective account and a tested answer to the specific claim carrying the most budget, can review case studies built the same way, or start with a demo scoped to the one claim worth checking first. --- # Before You Build the Survey: Testing What Actually Drives the Decision URL: https://subconscious.ai/blog/comparisons/before-build-survey-testing-what-actually-drives-decision Published: 2026-08-08 A controlled discrete-choice experiment, testing structured tradeoffs between attribute combinations, identifies which attributes actually move a decision before a survey measures them at scale. A research or insights lead who already runs closed-ended surveys faces a narrower question before the next one goes out: which attributes, price points, or messages are even worth asking about. A survey scores dimensions the team has already chosen. It cannot tell them whether they chose the right dimensions. [figure: A four-step horizontal path: a discrete-choice experiment feeds a causal ranking of attributes, then real-respondent validation, then a survey that asks only about the attributes proven to matter.] ## Where a closed-ended survey runs into its own design A survey question implicitly defines the space of possible answers before a single response comes in. Take a 1–5 rating on "How satisfied are you with our product?": the wording already treats satisfaction as the dimension that matters. If the real driver is "impressed by the capability but frustrated with the pricing model," a Likert scale cannot capture that split. SurveyMonkey's own product description is explicit about what the tool is for: distribution at scale, templated question types, and quantitative analysis across large samples: a 20-question survey sent to 5,000 people and segmented by demographic is exactly the job it's built to do well. Open-ended fields inside that same survey don't close the gap. Respondents typically write 5 to 15 words in a free-text box: "onboarding was confusing," "too expensive," which register as signals, not explanations. SurveyMonkey's own guidance on open-ended questions recommends them for added context, not as a substitute for structured follow-up: getting from "onboarding was confusing" to which part was confusing, what the respondent was trying to do, and what they did next requires a conversational follow-up a fixed-field survey form doesn't run. The result: teams that go straight to a survey often measure the wrong dimension with high statistical precision, then re-run research after launch to learn why a decision that looked solid on paper didn't hold up in the market. ## What does a controlled experiment add before the survey gets written? A discrete-choice experiment doesn't ask people to rate a dimension the researcher picked in advance. It presents respondents with structured tradeoffs, combinations of price, feature, and message attributes, and estimates which specific attributes and levels causally move the choice, each with a confidence interval on the effect size. That output is a ranked, quantified answer to "which of these things actually matters," not a satisfaction score on a dimension nobody has tested. Scope limits published alongside a method are what a buyer checks before committing budget. This is upstream of a tracking survey, not a replacement for it. A discrete-choice test on a synthetic population identifies which attributes are worth asking about at all; a SurveyMonkey-style survey then measures those specific attributes at scale, on a cadence, across a large sample. ## Where does the causal test stop? Naming what a method doesn't cover is what lets a buyer check the fit before they commit budget. A discrete-choice experiment is not an open-ended qualitative interview. It doesn't run sentiment analysis on free-text responses, and it isn't a survey-distribution or analytics platform: those stay squarely in SurveyMonkey's category. A causal test also doesn't answer questions where the researcher already knows which attribute matters and only needs to measure it at scale across a real population; that's a survey's job, not an experiment's. Publishing a result's validation status is what turns it from a claim into evidence a buyer can act on. The findings from a synthetic-population experiment are simulated-audience results, not observed market behavior, and they're only as trustworthy as their real-human validation. Audience reach and recruited real-human validation are two distinct claims: a synthetic experiment can run at a scale a recruited panel can't match, but confirming that its findings hold requires testing the same causal question against real respondents before a team commits budget to it. Where that validation step changes the answer, run it before scaling the survey; where it doesn't change the answer, the causal test result stands on its own. ## How do you sequence the two methods? A workable order looks like this: run a controlled experiment to find which attributes causally move the decision, validate the result against real respondents where the stakes justify it, then write the survey to track those specific attributes over time at scale. The research methodology explains how the discrete-choice design and confidence-interval reporting work; how a study moves from question to result covers the practical sequence a team runs through end to end. Starting with a causal test before the survey answers a narrower question than "what's the best research tool": which dimensions are worth quantifying at all, before the team spends survey budget on the wrong one. --- # Test Crisis Apologies and Press Releases Before They Go Public URL: https://subconscious.ai/blog/use-cases/test-crisis-apologies-press-releases-before-they-go-public Published: 2026-08-08 A crisis-communications lead has to choose which framing to publish first: an apology that leads with accountability, one that leads with the remediation plan, or one that leads with reassurance. Once the statement is public, the choice cannot be withdrawn or retested. The wrong framing can turn a manageable incident into a lasting loss of trust with the audience that matters most. ## Why do crisis statements usually ship untested? Most corporations have a crisis-communications plan, but the plan covers process, not content. It says who to call and when to issue a statement, not whether the statement itself will land the way the team intends. Testing a draft apology or press release means showing sensitive material to outside people before it is public, which creates confidentiality risk. It also runs into a timing problem: a live crisis compresses the review window, and there is rarely room to schedule a traditional study before a statement has to go out. The result is that most companies find out whether their crisis communications worked the same way they find out whether an ad worked: by watching the public reaction after the fact. ## What does a bad framing cost? The 2023 Bud Light boycott cost Anheuser-Busch InBev more than $1 billion in lost sales (CNN Business, 2024). United Airlines faced a comparable reckoning after a passenger was dragged off an overbooked flight in 2017, a response that became a defining case study in how a mishandled statement compounds an operational failure (Fortune, 2017). Neither company's underlying incident alone explains the damage. Both show how much a communications response can add to, or subtract from, the cost of a crisis. ## Framing this as a causal test, not a review The underlying decision is not "does this statement read well." It is "which framing changes how the affected audience responds." That is a causal question: hold the facts of the incident constant, vary the framing, and estimate which variant moves the audience's stated reaction most in the intended direction. Subconscious is built to test actions like this as a randomized experiment rather than a single reviewer's judgment call. A crisis-comms team can compare framings such as accountability-first, action-first, and reassurance-first against a defined audience segment and get a causal estimate of which framing performs better, with uncertainty attached to the result. ## Testing an apology: a worked example Consider a company that has to notify affected customers after a data breach and has drafted three candidate apologies: one that leads with accountability and a clear account of what happened, one built around the corrective steps the company is taking, and one that leads with reassurance about safeguards going forward. Running the three drafts as a controlled comparison against a defined audience segment answers a specific question: does this audience respond more strongly to accountability, to action, or to reassurance? The output is a causal comparison between the candidate framings, not a single reviewer's preference. ## Press releases face a multi-audience problem A crisis press release has to satisfy several audiences that do not want the same thing: the press wants news, investors want the financial picture, customers want reassurance, and regulators want compliance clarity. A release optimized for one audience can read poorly to another. The same experimental logic applies: define each audience segment explicitly and test whether a given framing moves that segment's stated response in the intended direction. A capability worth publishing includes the case where it does not apply. Segmenting by named professional role, such as journalist, investor, or regulator, is a study design choice a team should confirm is supported for its case, not a standing feature of every comparison. ## The confidentiality objection A common objection to pre-testing crisis material is that testing requires disclosing the incident before it is public. The answer is to test the anonymized framework rather than the specific disclosure: hold the company and incident details generic, and vary only the structural choice under test, such as whether an apology leads with action or with explanation, or whether a release states a specific remediation commitment. That structural question can be tested before any real crisis occurs, so a team enters an actual incident already knowing which framing pattern performs better for its audience. ## Moving from simulation to human confirmation Subconscious can run these controlled comparisons against a person-level audience graph covering 800 million real people. A number without its scope invites the wrong assumption. That audience graph is the population a study can define and query. It is not a recruitable panel of people signed up to participate. When a decision is consequential enough to warrant it, Subconscious can also test or validate a study with real human participants, moving from a simulated comparison to real-human confirmation without changing the underlying causal question being asked. ## When simulated testing is the wrong tool Naming where a method stops is what lets a team pick the right one. A causal comparison on a simulated audience is a first pass, not a replacement for every kind of crisis research. It fits best when a team needs to narrow several framing options quickly and the material is sensitive enough that showing it to outside recruits creates its own risk. It fits poorly when the stakes are existential enough that every element needs full validation, when a regulator or legal team requires a specific research method, or when the evidence needs to be defensible in later litigation. In those cases, traditional research with recruited human participants remains the right tool. ## Limitations A tool's limits belong on the page next to its claims. This is a framing-comparison tool, not a live, always-on crisis-monitoring workflow, and it does not guarantee a specific delivery time. It does not replace legal review, regulatory requirements, or human judgment about what a company can honestly commit to in a statement. Treat any output as one input to a decision that a communications and legal team still owns. ## Next step Teams that want to see how a causal comparison is structured for a specific audience and decision can review the research methodology or how Subconscious runs a study, or book a working session to scope a comparison against their own crisis-response playbook. [figure: Four steps: draft accountability-first, action-first, and reassurance-first variants; strip identifying details; run a randomized comparison; publish the framing that moved the response most.] --- # The Head of Research AI Adoption Checklist for 2026 URL: https://subconscious.ai/blog/implementation-and-procurement/head-research-ai-adoption-checklist-2026 Published: 2026-08-08 A head of research adopting AI tools in 2026 does not need to decide whether AI belongs in the workflow. It already is there. The decision that matters is where to draw the line between AI-assisted exploration and decision-grade evidence, and what governance sits around that line: which review gates apply, what disclosure language ships with an AI-assisted finding, and when a claim escalates to real-human validation. Get that line wrong in one direction and a stakeholder ships a launch, pricing, or positioning call on an unlabeled AI-generated read that was never checked. When it fails publicly, the research function's credibility, and the leader's job, absorb the blame. Get it wrong in the other direction and the team routes every fast question through a full fielded study, and the business starts working around research entirely. ## Why the Job Is Under Pressure, Specifically The threat to a head of research is not that AI eliminates the role. It is narrower: leaders are feeling pressure to bring AI in fast, before they have an operating model that would hold up to scrutiny. AI has moved from a novelty layer into daily workflows for analysis, reporting, data preparation, and self-service insight. Research demand has not disappeared alongside that shift. The U.S. Bureau of Labor Statistics still projects 7% employment growth for market research analysts from 2024 to 2034, faster than the average occupation (BLS, Market Research Analysts: Occupational Outlook Handbook). What has changed is the access advantage. Expertise used to live partly in knowing how to field a study, clean the responses, and package the finding. AI weakens that advantage: more people can now draft a survey, summarize a transcript, or generate a first-pass narrative. Once outputs are cheap for anyone to produce, the scarce skill becomes judging which one is trustworthy, and explaining why. ## Build Evidence Tiers Before You Adopt a Tool The leaders who hold their footing in 2026 will not be the ones using the most tools. They will be the ones with the clearest evidence system, one that spells out three things: what AI can do without oversight, where a human has to sign off, and which claims need real validation before they ever leave the building. A workable version has four tiers, moving from cheap and fast to slow and defensible: The value in this system is not the AI output. It is the disciplined, labeled path from a question to a decision the business can defend later. ## Where Does a Causal Experiment Fit in the Directional Tier? This is the tier where a platform like Subconscious sits, not as a replacement for the review and validation layers above it but as the mechanism inside it. Subconscious runs controlled, randomized experiments on a simulated audience so a team can test a pricing story, a message, or a feature framing before the slow or expensive part of research begins. Because the test is a controlled experiment rather than a single generated answer, it produces a causal effect with a confidence interval, not just a plausible-sounding read. See how the method works. Stating plainly what Subconscious does not decide is what lets a research leader check the claim before it reaches a live decision. That still leaves the human-review and validation tiers intact. Subconscious does not decide which claims are safe to leave in the exploration or directional tier versus which require a jump to real humans; that governance call stays with the research leader. And a causal experiment on a simulated audience is a distinct method from open-ended AI brainstorming with generated personas: it does not become a usability session, a clinical trial, or automatic proof of market performance just because the underlying question is causal. When a decision is expensive or public, the same causal question can move to real-human validation without being redesigned. ## What Adoption Mistake Actually Sinks Credibility? Teams get burned when they purchase tools first and only later figure out what success or risk should mean for their work. That error usually comes from pressure: the stakeholder wants an answer tomorrow, the tool produces a fluent one, and the deck needs a conclusion. What keeps research credible is a sharp line between something a tool spit out and something that counts as evidence. Where a tool's authority ends is a fact worth stating in the deliverable itself, so the reader can check it. A tool can produce useful output. It cannot decide on its own whether that output is valid for the decision sitting in front of the business. The fix is to make the limit part of the deliverable. Spell out where the AI-assisted work applies, where it stops applying, and what still needs validation before anyone treats it as fact. Leaders who do this consistently sound more credible, because they can explain exactly where their confidence has a boundary. ## Where Should You Start This Week? Do not rewrite the whole research function at once. Start with one visible workflow: - Pick a real project with a live, pending decision. - Write the business decision in one sentence. - Define the audience and the risk level attached to getting it wrong. - Use AI, or a controlled directional experiment, only for the exploratory and directional tiers. - Route the output through human review before it reaches a stakeholder. - Present the finding with an explicit caveat and a named next validation step. Repeat that loop on a second workflow the following week. Within a month, the team has something more durable than a list of approved AI tools: a working evidence system that a stakeholder can trust because its limits are stated out loud. Teams evaluating where a controlled, causal test fits inside that system can book time to walk through a specific workflow. ## The Bottom Line The pressure behind this question is rational. AI is changing what the fast, cheap part of research work looks like. Human judgment in research and strategy is still required; what shifts is the shape a defensible version of that role now takes. The safer position is closer to the decision, explicit about which tier produced each finding, and clear about what still needs a human, or a real respondent, before it becomes a claim. [figure: Four-step path: Exploration (AI-assisted, no gate), Directional testing (researcher reviews design and output), Human review (required before external framing), Validation (leader approves escalation to real data).] --- # PyMC-Marketing vs. Google Meridian: What to Check Before a Benchmark Moves Your Budget URL: https://subconscious.ai/blog/comparisons/pymc-marketing-vs-google-meridian-what-check-before-benchmark-moves-budget Published: 2026-08-08 A marketing analytics team choosing between PyMC-Marketing and Google's Meridian will find a published benchmark claiming one library is faster and more accurate than the other. Before that benchmark moves real ad budget, what matters is not which number is bigger, but who ran the test and whether anyone outside that team has reproduced it. ## The decision this comparison actually supports Reallocating spend across channels on the strength of a media mix model (MMM) is a real financial decision. Picking the wrong library on an unreplicated performance claim can mean trusting channel-contribution estimates that never held up outside the vendor's chosen test conditions. ## What are PyMC-Marketing and Meridian? Both PyMC-Marketing and Meridian are open-source Bayesian MMM libraries that handle multi-geo, hierarchical models. Meridian is Google's open-source MMM framework, which Google made generally available to advertisers. The architectural differences are documented: PyMC-Marketing supports multiple sampling backends (NumPyro, BlackJAX, Nutpie), while Meridian is built on a fixed TensorFlow Probability backend. The two libraries also differ in how they parametrize media response curves, model seasonality, and set default priors. ## What the published benchmark claims A benchmark authored by the team that builds PyMC-Marketing tested both libraries on synthetic datasets spanning four scales, from a single-market startup dataset (104 observations) to a global-enterprise dataset (10,400 observations), a 100x range in data complexity. The published results report large gaps in sampling speed, fit accuracy, and convergence, favoring the library built by the team that ran the test. The benchmark's author and the benchmark's favored subject are the same organization. That does not make the underlying numbers wrong. It does mean the numbers have not cleared the bar that would let a buyer treat them as settled fact. [figure: Five-row comparison of a self-reported vendor benchmark against independently replicated evidence, scoring who ran it, who set conditions, reproduction, correction mechanism, and code availability.] ## What to require before trusting the result ## Where this pattern shows up beyond MMM libraries The same structural problem, a vendor's own test deciding whether its own product looks good, applies to any tool evaluated on a benchmark its maker wrote. Subconscious does not build or benchmark media mix modeling software, and the parallel here is about evidentiary standard, not a competing product claim. Subconscious's best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). It is a validation result from a different domain, not a substitute for or a rebuttal of either MMM library's reported numbers. The relevant point of comparison is process: a claim backed by independent replication carries a different evidentiary weight than a claim backed only by its author. ## A practical checklist for the reallocation decision - Ask who ran the benchmark and whether they have a stake in the outcome. - Ask whether anyone outside that team has reproduced the result on different data. - Treat published code as an audit trail, not a substitute for replication. - If no independent replication exists yet, treat the claimed gap as a hypothesis, not a budget input. - Where the decision is large enough, run a smaller-scale internal test on your own data before committing spend to the recommendation. ## What are the limitations of this comparison? This comparison does not settle whether PyMC-Marketing or Meridian produces better channel-contribution estimates on any specific company's data. Both libraries have documented architectural differences worth evaluating in areas like sampler flexibility, memory footprint, and seasonality modeling. What is not yet established is the size of the performance gap claimed by the vendor's own benchmark, because that gap has not been independently reproduced. See Subconscious's approach to causal evidence and replication and its live results for how the same evidentiary question, does a claim replicate outside the team that made it, applies to causal experiment design more broadly. --- # Screen Landing Page Hero Copy Before You Spend Traffic on It URL: https://subconscious.ai/blog/use-cases/screen-landing-page-hero-copy-before-spend-traffic Published: 2026-08-08 A growth lead with six hero headline candidates and one landing page has a narrowing problem, not a testing problem. A live A/B test can compare two or three variants at a time. It cannot cheaply tell you which two deserve that traffic in the first place. For a B2B page that clears a few hundred sessions a week, a live test that starts with the wrong pair can run most of a quarter before it reaches significance, a quarter spent proving a headline was weak rather than shipping the one that wasn't. ## The candidate problem, not the test problem Hero copy carries outsized weight relative to how little of it there is: a headline, a subhead, a call to action. Small wording changes move signup and demo-request rates more than most landing page redesigns do, which is exactly why teams over-invest in debating a handful of options and under-invest in generating and screening more of them. The live test works fine once a team is down to two strong contenders. The expensive part is everything before that: picking which two out of six, eight, or twelve deserve real traffic. ## How do you narrow candidates before a live test? A structured pre-test workflow separates candidate generation from candidate validation: - Write out the candidate set. Vary one dimension at a time (outcome framing versus mechanism framing, a comparative anchor, urgency level, specificity) so the differences are legible rather than accidental. - Run a controlled causal experiment on simulated buyers matching the target ICP. Ask each simulated respondent to react to headline, subhead, and CTA as a real prospect would, and compare preference and stated intent across variants. - Take the top two into a live A/B test. The simulated round does the discovery work; the live test does the final validation, on a pair that has already cleared a bar. This is Subconscious's general fit for the decision: run the causal comparison on simulated buyers first, then move the same question to real-human validation without redesigning the experiment, so the variant reaching the live test has already been screened rather than guessed. ## What does a simulated round tell you that a live test can't? A live A/B test returns one number, conversion rate, and nothing about why the losing variant lost. A structured comparison of messaging variants can also surface qualitative reasons: whether a headline reads as generic, whether the value proposition lands as differentiated, or which specific objection the copy fails to answer. ## How much to trust the simulated ranking The limits below are published next to the workflow's claims for the same reason a leaderboard lists misses beside hits. A simulated preference ranking is a directional signal for narrowing candidates, not a statistically equivalent replacement for a live A/B test at scale. Methodology-level research on LLM-based synthetic respondents shows they can reproduce human survey response patterns at meaningful reliability (Maier et al., 2025), which supports using simulated preference this way. That is different from claiming a specific accuracy rate for hero-copy testing itself, a number Subconscious can't back for this workflow; the reliability finding describes the general mechanism, not a guarantee for any one comparison. "SSR achieves 90% of human test-retest reliability while maintaining realistic response distributions (KS similarity > 0.85)" Maier and colleagues, arXiv:2510.08338 (2025) (source) "SSR achieves 90% of human test-retest reliability while maintaining realistic response distributions (KS similarity > 0.85)" Maier and colleagues, arXiv:2510.08338 (2025) (source) ## Where a live test is still the right first move Three cases where screening with simulated buyers adds little: - The page already gets enough traffic to detect a large expected lift directly, so just run the live test. - The hero depends on a live product capability or integration a simulated respondent cannot evaluate without using the product itself. - The variant tests a pricing or social-proof claim the simulated respondent has no ground truth for. Outside those cases (headline framing, value-prop ordering, CTA wording, the sequencing of proof and benefit), screening before the live test is the cheaper path to a defensible shortlist. ## Where does the causal test fit into your stack? The simulated round is a step before an existing experimentation tool, not a replacement for it. Whatever platform runs the live A/B test, Optimizely, VWO, PostHog Experiments, or an internal flagging system, stays where it is. The screening step changes what goes into that funnel: a shortlist of two candidates that have already cleared a directional bar, instead of a set picked by internal debate. [figure: A four-step path showing a wide set of hero copy candidates narrowing through a simulated causal comparison on target-ICP buyers into a shortlist of two, which then goes into a live A/B test.] [figure: Two columns. Left, live A/B test: one output, conversion rate, no explanation. Right, simulated round: a ranking fed by three reasons: generic headline, undifferentiated value prop, unanswered objection.] ## Limitations and next step Naming this workflow's failure modes here is what lets a buyer check it before spending traffic on it. This workflow does not replace live A/B testing, does not carry a specific claimed accuracy percentage for hero-copy decisions, and is not a substitute for the time a proper causal experiment takes to design and run. It narrows the candidate set; the live test still decides the winner. For a team with more hero candidates than traffic to test them honestly, research covers the underlying causal methodology, and how we work walks through how a simulated comparison moves into real-human validation without changing the question being asked. Case studies cover applied examples across categories, and a demo is the fastest way to see the narrowing step run against a real candidate set. --- # 6 Alternatives to Focus Groups for a Concept, Message, or Pricing Decision URL: https://subconscious.ai/blog/comparisons/6-alternatives-focus-groups-concept-message-pricing-decision Published: 2026-08-08 Focus groups carry well-documented problems: groupthink, moderator bias, small samples, and long timelines. If the concept, message, or price you're testing can't wait weeks for a room of eight strangers to agree with each other, you have options. The right method depends on the decision in front of you: are you trying to understand a motivation, observe a behavior, or estimate which of several actions moves an outcome? Below are six methods, described honestly, including where each one runs out of depth. [figure: Five methods each paired with a question type: motivation, considered answers, in-context behavior, unnarratable workflows, and which action moves an outcome.] ### 1. Individual interviews One-on-one conversations with target customers, usually 30 to 60 minutes, following a semi-structured guide, giving each person an individual view instead of a group-influenced one. Interviews are strongest for exploring the "why" behind a decision: motivation, hesitation, and the mental model a buyer is using. A number without its limits is marketing. They're weakest on breadth: a typical study runs 8 to 20 interviews, and recruitment and analysis both take real calendar time. ### 2. Asynchronous qualitative platforms Participants answer questions, complete tasks, or record video responses on their own schedule, often over several days, instead of in a live session. Removing the group and the clock changes what people say: fewer social-pressure answers, more considered ones. This format fits concept testing, message testing, and lightweight longitudinal work, especially with a geographically spread audience. Publishing a method's limits is what lets a buyer check it against their own question. The trade-off is spontaneity: there's no moderator probing a half-answer in real time, so the quality of what you learn depends heavily on how well the questions were written going in. ### What are diary studies? Participants log their experiences, behaviors, or decisions over days or weeks, typically through a mobile app or written journal. Because behavior is captured in the moment rather than recalled afterward, diary studies avoid a lot of the recall bias that shows up in a single retrospective interview or survey. Diary studies are well suited to habits, routines, and customer-journey mapping. Nielsen Norman Group's overview of the method covers why in-context logging surfaces patterns a one-time conversation misses. The misses belong on the record next to the hits. The cost is participant burden: dropout rises with study length, and the data is unstructured and slow to analyze. "As a broad guideline, we recommend about $40 per hour of the total time commitment you expect for a study with a general sample of US-based participants." Nielsen Norman Group, Diary Studies: Understanding Long-Term User Behavior and Experiences (source) ### What is ethnographic observation? A researcher observes people in their natural environment instead of asking them to describe their own behavior, removing most self-report bias. It's the right call for understanding physical environments, workflows, or behaviors people can't easily narrate, common in health, retail, and industrial-design research. Naming where a method runs out is what makes the comparison usable. It's also the most expensive and least scalable option here, requires trained observers, and the findings take real interpretive work to become a product decision. ### 5. Simulated behavioral experiments Instead of asking people to describe a preference, a controlled experiment on a simulated population tests two or more alternatives (messages, prices, concepts) head to head and estimates which one is more likely to move a defined outcome. This is a causal-experiment method, not a moderated conversation: the output is a comparison of actions, not a transcript. Subconscious.ai runs this kind of experiment. Its best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). It is a validation result, not a guarantee for any single new question. When a decision is regulatory, high-stakes, or emotionally complex, the same causal question can be re-run with recruited human participants: the simulated result sets the direction to test, and the human study confirms it. It can't be treated as an interview with one named, recruitable person the way a qualitative panel can. ### What are micro-surveys? Very short surveys, one to five questions, deployed in-app, post-purchase, or as a website intercept. Unlike every method above, this one is quantitative: it captures a number in the moment instead of a story recalled later. Micro-surveys are the right tool for tracking satisfaction, monitoring NPS, or validating one specific hypothesis at scale. They give you "what," not "why": question design carries almost all the weight since there's no room to probe, and overuse trains customers to ignore the prompt. ## How the six compare These timelines are historical planning ranges, not current quotes for any specific method or vendor. ## Sequencing the decision Match the method to the question, not the other way around. A motivation question needs interviews or an async qualitative platform. A behavior-over-time question needs a diary study or observation. A "which of these options performs better" question is what a simulated experiment is built to answer quickly, before a slower or more expensive method gets scoped. Most research programs that get this right don't pick one method: they sequence them. Run a simulated experiment to narrow a wide field of concepts, messages, or prices down to the strongest candidates. Then take the finalists to individual interviews or a recruited study to confirm the result with real people before it ships. That sequence protects the research budget for the step that actually needs recruited humans, instead of spending it on options you were always going to cut. If the decision in front of you is which message, price, or concept to run with next, see how a causal experiment is set up or book time to walk through a specific decision. --- # When a Forecasted-Control MMM Can (and Can't) Answer a Budget Question URL: https://subconscious.ai/blog/methods-and-validation/when-forecasted-control-mmm-cant-answer-budget-question Published: 2026-08-08 A forecasted-control MMM can answer a budget question about known dynamics under expected future conditions, but it cannot tell a team whether an action its historical coefficients have never observed will work. A marketing analytics lead just allocated $30 million for next quarter's campaigns. The allocation plan depends on employment rates, temperature, and other external conditions the underlying Marketing Mix Model needs as inputs, for a quarter that hasn't happened yet. That's the scenario-planning paradox at the center of most Bayesian MMM work. The model is good at explaining what already happened: how much of past sales each channel is estimated to have caused, once seasonality and other controls are accounted for and given the model's identifying assumptions. It is not, by itself, able to supply the future values of those controls. ## Pairing a causal model with a forecasting model One way to close that gap is to let two different models do two different jobs. A Bayesian MMM (built in a framework like PyMC-Marketing) keeps the causal structure under its identifying assumptions, such as no unobserved confounding and media spend that isn't itself driven by expected demand: it encodes how advertising drives sales through mechanisms like adstock and saturation, and it returns a coefficient for each control variable. A separate time-series foundation model, for example Chronos-2, a transformer-based forecaster Amazon released for general-purpose time series prediction, projects the values of upcoming control variables like temperature or employment from historical patterns (Introducing Chronos-2: From univariate to universal forecasting). The pipeline runs in three steps: fit the MMM on historical media spend and controls to learn the causal coefficients, forecast the upcoming controls with the foundation model, then feed the planned media spend plus the forecasted controls back through the fitted MMM to get a sales prediction for the future quarter. The MMM's own causal structure, separating channel effects from control effects such as competitive activity, is itself an active area of methods work; a recent framework addresses channel-influence bias and cross-channel effects directly inside the MMM specification (A new framework for Marketing Mix Modeling). ## Why doesn't a forecasting error always wreck the prediction? What matters isn't whether Chronos-2 forecasts controls well, but how much a forecasting error in those controls moves the final sales prediction. Each MMM prediction is the sum of a media-channel contribution and a control-variable contribution, where the control contribution is coefficients multiplied by control values. Swap in a forecasted control instead of the true one, and the resulting error in the sales prediction scales with two things: how large the forecasting error is, and how much that control actually matters to the model. That gives a practical rule of thumb: multiply a control's forecast error rate by its share of the predicted sales level it contributes, and the result approximates the added error in the sales prediction. A control that swings the forecast but barely contributes to predicted sales doesn't matter, and neither does one that contributes to predicted sales but forecasts well. Only a control that is both influential and poorly forecast should worry a planner. The rule assumes the control variables behave in a relatively stable, well-patterned way. A structural break, such as a pandemic, a policy shift, or a new competitor entering the market, can throw off even a strong forecasting model, and a control with a clean seasonal pattern will forecast far better than one without. [figure: A decision path from a historical MMM's causal fit into forecasted control variables, cascading forecast error, and the point where it diverges toward an untested action needing a controlled experiment.] ## What can and can't this pipeline decide? The hybrid approach fits a specific set of planning problems: allocating a budget three to six months out under expected market conditions, and running scenario analysis across optimistic and pessimistic economic assumptions to see how conditions shift the forecast, though shifting the optimal media split further extrapolates the media saturation curves beyond their historical spend support. What it cannot do is tell a team whether an action the historical model has never observed will work. The MMM's coefficients are learned from what already happened. Forecasting the controls more accurately doesn't create a coefficient for entering a market the company has never entered, testing a price point it has never charged, or launching creative unlike anything already in the training data. [figure: A chain shows a control's forecast error rate multiplied by its share of explained sales, producing added error in the sales prediction, with a branch where a structural break invalidates the estimate.] ## Where the decision has to move from forecasting to testing That's the boundary that separates this pipeline from a controlled experiment. When the question is "how will this quarter's known dynamics play out given expected conditions," an MMM-plus-forecast pipeline that propagates control-forecast uncertainty through the coefficient posterior is the right tool, and a controlled lift-test experiment serves to calibrate the model's coefficients rather than duplicate what it already answers. When the question is "will this untested action work," there is no historical coefficient to extrapolate, and no amount of better control forecasting fixes that. A controlled causal experiment on a simulated population is built for that second case. Subconscious runs the tested action, such as a new market, a new price, or a new campaign, against a defined simulated population and reads out the effect within that population, rather than projecting a coefficient that doesn't exist yet. Subconscious can test or validate those studies with real human participants as well, moving from a simulated read to a human-validated one without changing the underlying causal question. Teams weighing which side of that boundary a given planning question falls on can see how a study gets designed and run. --- # Does a Simulated Yogurt-Choice Study Match Human Behavior? The Ares Replication URL: https://subconscious.ai/blog/case-studies/simulated-yogurt-choice-study-match-human-behavior-ares-replication Published: 2026-08-08 A simulated conjoint study on functional yogurt reproduced the attribute ordering found in a published human study, matching 87% of the measured human ceiling (0.832 of 0.959; see replication methodology). A replication score published without its limits reads as marketing. For a CPG or consumer-research leader deciding whether to trust a simulated study's ranking of choice-driving attributes, that result shows the ordering direction can replicate. It does not show individual-level prediction, calibration, or performance across subgroups. [figure: Two-column comparison. Left, "Confirmed": attribute ordering direction agrees. Right, "Not established": individual prediction, calibration, subgroup performance, cross-category generalization.] ## Why the decision matters A CPG team weighing a simulated conjoint study against a fielded human study needs to know whether the simulation recovers the same decision ordering, not just a plausible-sounding result. Treating a directional rank correlation as individual-level prediction, calibration, subgroup validity, or a guarantee across products risks a launch decision built on a simulated ranking that generalizes further than the replication supports. ## What did the replication compare? Ares et al. published a study examining how factors unrelated to taste or texture shaped whether shoppers picked functional yogurt over regular yogurt. A matched simulated study compared its attribute-ranking output against the same non-sensory factors. The replication fidelity between the two orderings is the outcome measured. ## Evidence The misses sit on the public leaderboard next to the hits. The replication figure of 87% (0.832 of 0.959) is a directional statement about whether the two orderings agree. It says nothing about how closely individual respondent choices matched, and it does not carry over to a different product category without its own replication. ## What are the options for validating a simulated ranking? Three ways exist to check whether a simulated ranking is trustworthy for a given category: trust the simulated study alone, run a fielded human study alone, or compare the two against a matched human study, as in this replication. Only the third produces a testable correlation instead of an assumption. ## Recommended decision process - Identify the published or fielded human study that covers the same product category and choice factors. - Run the matched simulated study against the same attributes and alternatives. - Compute the rank correlation between the two attribute orderings, and report the p-value alongside it. - Treat a significant, positive correlation as support for using the simulated ordering as a first-pass read, not as a substitute for category-specific validation. ## Where does Subconscious fit into this? The replication leaderboard shows how Subconscious tests studies against real human participants, letting a team move from a simulated experiment to real-human validation without changing the underlying causal question. Teams evaluating packaging, claims, or assortment decisions in CPG can use the same matched-replication approach before extending a simulated ranking to a new product line. The broader research method is documented on the research page. ## Limitations and failure conditions Naming the failure mode here gives a buyer something to check before relying on the result. One conjoint replication does not establish individual-level fidelity, calibration, subgroup performance, causal identification, or general validity across products. The rank correlation measures whether the ordering of factors agrees between the two studies; it does not measure preference magnitude, choice share, or any individual respondent's decision. Applying this result to a different product category, attribute set, or subgroup decision needs its own matched replication first. ## Sources - Ares, G. et al., study examining how factors outside sensory perception steered shoppers toward functional yogurt instead of regular yogurt, Food Quality and Preference --- # What Is AI-Driven Market Research? A Buyer's Definition URL: https://subconscious.ai/blog/use-cases/what-ai-driven-market-research-buyers-definition Published: 2026-08-08 AI-driven market research uses AI to generate responses to research stimuli, to analyze those responses, or both. It replaces parts of a workflow that traditionally needed real participants, manual analysis, and a long field calendar. The question a VP of Insights has to answer is not "what is this category" but "which of my research questions can move through it, and which need a human in the loop before I make a public claim." ## The decision this term is standing in for Every research team runs two kinds of question. Iteration questions ask which of several concepts, messages, or segments deserves more work. Commit questions ask what a team will put on packaging, in a regulatory filing, or in front of a board as a defensible population estimate. AI-driven methods are strong on the first kind and unproven on the second. Naming the category correctly routes a question to the right method before budget or credibility is on the line. ## What are the two capabilities inside AI-driven market research? The term covers two capabilities that vendors often bundle together: Generation. AI personas produce responses to a research stimulus, standing in for the recruitment-and-fielding stage of a traditional study. Analysis. Language-model tooling themes, summarizes, and compares responses, whether those responses came from AI personas or real participants, without the manual coding pass a human analyst would otherwise do by hand. A tool that only does analysis is AI-assisted. A tool that also generates responses is AI-driven in the fuller sense. ## Why did teams adopt AI-driven market research? Two forces made this practical rather than theoretical. Large language models became reliable enough that conditioned personas produce research-grade output instead of generic chat text. Peer-reviewed validation work gave the method academic footing: Argyle et al. (2023) showed that language-model-driven sampling could approximate real survey response distributions on directional questions (Political Analysis). That paper, not a vendor's self-reported accuracy figure, is the citable anchor for the "does this work at all" question. The third force is calendar pressure. Product and marketing cycles compressed, and a research process that runs on a multi-week fielding calendar cannot answer questions on a sprint cadence. ## Where does the boundary sit between simulation and validation? The slow iteration loop is the part these methods take over: narrowing a wide set of concepts to a few, running several message variants to find the strongest, and comparing segments to see where a positioning holds up. Work that used to need a dedicated field period can now happen inside a working session. The final validation step stays untouched for a decision that has to survive scrutiny after it ships: - Claims featured on packaging or carried in advertising copy - Submissions reviewed for regulatory or compliance purposes - Population estimates cited as a specific percentage of a named group, where the number has to be defensible - Sensory or emotional response to a physical product ## Where a causal platform changes the shape of this Most AI-driven research platforms stop at Layer 3: they simulate or predict what a described audience would say, then summarize the result. That answers "what would people say" but not "what would change their behavior if we altered one variable." Subconscious runs controlled, randomized experiments on simulated populations, so the output is a causal effect with a confidence interval rather than a generated opinion. When a decision is close enough to require it, the winning options can move to a real-human validation study without redesigning the study or changing the underlying causal question. That step is a check on the simulated answer, not a replacement for it, and it does not turn a causal experiment into a usability session or a clinical trial. ## What good practice looks like on a modern workflow The shape is consistent across platforms in this category, whether the output is directional or causal: Define the audience. Specific demographic and psychographic parameters produce a more useful answer than a broad description. Build the population. Enough respondents, stratified across the parameters that matter, to support the comparison being made. Design the instrument. A concept brief, message test, or comparison, structured the same way it would be fielded traditionally. Run the experiment. Submit the stimulus and the variations being tested; responses and comparative results come back together. Read the results, then decide what still needs a field study. Segment comparisons and directional reads answer most iteration questions on their own. A result the team plans to state publicly is the trigger for real-human validation, not an automatic next step for every study. ## Related terms - Silicon sampling, the academic name for the validation work behind AI-generated survey responses. - The say-do gap, the gap between what people report and what they actually choose, which any of these methods has to account for. - Randomized experiments on simulated populations, the mechanism a causal platform uses instead of a single generated opinion. [figure: A four-step horizontal path: Define the audience, Build the population, Design the instrument, Run the experiment, each step feeding into the next in a fixed order.] ## Where to go next To see how a causal platform frames a comparable study, read how Subconscious tests actions rather than describing sentiment. To decide whether a specific decision needs real-human validation before it ships, see how a study moves from simulation to validation. Teams evaluating recent studies can review case studies or book a walkthrough. [figure: A branching path starts at one question: will this be defended externally. Iteration questions route to AI simulation. Commit questions, like packaging claims, route to real-human validation.] --- # How to Tell If a Causal Vendor's Probability Model Is Auditable URL: https://subconscious.ai/blog/methods-and-validation/how-tell-if-causal-vendors-probability-model-auditable Published: 2026-08-08 A data science leader vetting a causal behavioral platform for a paid pilot needs one question answered before budget moves: can this vendor show the exact path from its modeling assumptions to a testable probability estimate, or does the output arrive as an unexplained number? A pilot spent on an opaque model cannot be defended internally when someone asks how the number was produced. ## Why isn't "AI insights" a good enough answer? Generic "AI insights" framing treats a probability estimate as a finished product, hiding the steps between the assumptions a model encodes and the number it reports. For a buyer, that gap is the actual risk: if a vendor cannot show how assumptions became a checkable estimate, there is no way to audit the estimate later, reproduce it, or defend it under internal scrutiny. The open-source Bayesian modeling ecosystem around PyMC makes the alternative concrete. PyMC models are built as a graph of random variables: a structured statement of assumptions with no probabilities attached yet. A separate conversion step turns that assumption graph into a joint log-probability graph, which is the object a model actually gets tested and sampled against. That conversion is what makes a Bayesian model auditable: every step from assumption to estimate exists as an inspectable graph, not a black box. ## From assumption graph to testable probability The pattern has three parts, regardless of which library implements it: - Define the random variable graph. Each unknown quantity in the model, such as a mean, a variance, or a probability of belonging to a segment, is declared as a random variable with a distribution and its relationships to other variables. - Convert the graph into log-probabilities. A dedicated conversion step walks the random variable graph and produces a joint log-probability graph: the mathematical object that says how likely any given set of parameter values is, given the model's assumptions and the observed data. - Differentiate and sample. Automatic differentiation turns the log-probability graph into gradients, which posterior sampling methods use to explore which parameter values are actually consistent with the data. PyMC's 4.0 release rebuilt this pipeline around a graph-based computational backend, replacing the discontinued Theano project with a new fork purpose-built for this workflow (PyMC 4.0 Release Announcement). That backend has since been renamed and continued as PyTensor, reflecting further consolidation of the graph-computation layer rather than a change in the underlying pattern (PyMC's backend fork to PyTensor). The library names in this ecosystem have changed twice since 2022; the architecture pattern has not: declare assumptions as a graph, convert that graph into a checkable probability object, then sample from it. ## What does this buy a buyer? The practical advantage is inspectability. A model built this way can be interrogated at each stage: what assumptions were encoded, how they were converted into a probability estimate, and whether that estimate can be reproduced from the same graph and the same data. A model that skips straight to a final number, with no equivalent intermediate object, cannot answer any of those questions. Where theta is the set of model parameters, y is the observed data, p(y | theta) is the likelihood implied by the random variable graph, and p(theta) is the prior. The conversion step is what turns the right-hand side from an abstract statement into a graph that can be evaluated, differentiated, and sampled. [figure: A left-to-right chain of five stages: a random variable graph converts into a log-probability graph, gets differentiated, sampled, and produces a final testable probability estimate.] ## How does Subconscious apply the same audit standard? Subconscious runs on causal experimentation and discrete-choice-style modeling, not generic synthetic-user roleplay, and validates its output against real human behavioral studies rather than treating a simulated result as self-evidently correct. The validation is concrete: our best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959; mean 0.73 across the 43 studies that pass design filters, drawn from a corpus of ~300 replicated human studies across 9 domains), detailed in the causal fidelity paper. The same audit instinct this article recommends for any vendor, asking how assumptions become an evaluable estimate, applies to Subconscious itself. A team can move from a simulated experiment to a real-human validation run without changing the underlying causal question, and compare results on the leaderboard that tracks replication performance across domains. ## Limitations Real-human validation confirms that a simulated study reproduces a prior human result; it does not turn a causal experiment into a usability study, a clinical trial, or an automatic guarantee of market performance. And the graph-based architecture described above is a property of the open-source Bayesian ecosystem this article uses to illustrate the pattern; it says nothing about which specific tools any given causal vendor, including Subconscious, runs internally. The useful question for a buyer is not which library a vendor uses, but whether the vendor can show, concretely, the steps between its assumptions and its estimate. [figure: A four-stage chain: a simulated experiment run leads to a check against the causal fidelity paper's studies, producing 87% of the measured human ceiling (mean 0.73 across 43 studies), then an optional real-human validation run.] ## What to ask before shortlisting a vendor Before committing pilot budget to a causal-inference or behavioral-simulation vendor, ask for the graph and the design, not just the number: what assumptions define the model, what conversion step turns those assumptions into a probability estimate, what randomized attribute design underwrites the causal identification, and what validation corpus checks that estimate against real outcomes. A vendor that can walk through that architecture in specific terms has given a buyer something to audit. A vendor that answers only with a final score has not. --- # The Consumer Analyst Skills That Matter in the AI Age URL: https://subconscious.ai/blog/buyers/consumer-analyst-skills-matter-ai-age Published: 2026-08-11 A CMO or VP of consumer research should judge analysts by how well they frame decisions, challenge evidence, and govern the handoff from exploration to action. The question is not whether a model can produce an answer, but what an analyst must review before that answer can influence pricing, positioning, or a launch. Without that gate, a plausible summary can become a business decision before anyone checks it against real behavior. The cost surfaces after the launch or spend is committed, when the analyst loses credibility with the stakeholders who acted on the finding. ## The value of the role is moving toward judgment The analyst role is not disappearing. The U.S. Bureau of Labor Statistics projects 7% employment growth for market research analysts from 2024 to 2034, with roughly 87,200 average annual openings (BLS Occupational Outlook Handbook). What is changing is where the role creates value. Models can draft questions, summarize transcripts, and produce an initial reading of a stimulus. The analyst is accountable for deciding whether the question matches the business choice, whether the evidence supports the conclusion, and whether the result is safe to act on. ## Five skills that protect the decision ### How do you frame the business choice? Start with the action under consideration, not the prompt: the price, message, feature, or launch decision. Then state what evidence would change that decision and how costly a false signal would be. ### 2. Separate exploration from validation A generated hypothesis or a simulated experiment can help a team explore, but neither is automatic proof of consumer behavior. Analysts must label the evidence stage clearly so stakeholders do not mistake a directional result for a validated one. ### How do you design a causal comparison? Descriptive reactions are not enough when the buyer needs to choose an action. A useful study varies the action under consideration, holds the rest of the setup steady, and asks what changed. This keeps the analysis tied to the decision instead of producing a collection of plausible observations. ### What should you review before a finding reaches a stakeholder? Before a finding reaches a stakeholder, review the audience definition, question wording, assumptions, source grounding, and alternative explanations. State what the work covered, what it did not cover, and what evidence is still needed. ### 5. Govern the handoff to action Define who can approve a directional finding, when real-human validation is required, and how caveats appear in the final recommendation. Expensive or public decisions deserve a stricter gate because the damage from an unchecked signal is harder to reverse. [figure: Five stages left to right: exploration, directional experiment, human review, real-human validation, action. A finding advances only after clearing its current stage's decision limit.] ## Make every stage carry a clear permission The distinction between stages is easier to govern when each one has an explicit purpose and decision limit. This is an evidence policy, not a ranking of tools. It gives the analyst a consistent way to explain why one finding can guide the next test while another is ready to inform action. ## Operationalize exploration and validation as separate stages Subconscious is one way to operationalize the separation between a directional causal experiment and real-human validation. A team can carry the same causal question from a simulated experiment into testing with real participants. The analyst still decides whether the setup fits the business question, whether the result is credible, and whether more evidence is needed before action (how the study process works). Causal experimentation does not eliminate the need for real-human validation on expensive or public decisions, and real-human testing does not turn a result into automatic proof of market performance. ## A planning exercise for the next decision Use the following as a one-week planning example, not as a delivery promise: - Pick a real project tied to a live decision. - Write the business decision in one sentence. - Define the audience and the cost of being wrong. - Keep generated hypotheses and a directional simulated study in the exploratory stage. - Review the output by hand and mark what is supported, uncertain, or unsafe to repeat externally. - Present the finding with a stated caveat and a named validation step. Repeat that once a week for a month. The output should be a working evidence policy: a clear rule for what can guide exploration, what requires analyst review, and what must be checked with real people before the business acts. --- # Build a Custom Persona Simulation or Buy a Causal Testing Platform? URL: https://subconscious.ai/blog/comparisons/build-custom-persona-simulation-buy-causal-testing-platform Published: 2026-08-11 Build a custom persona simulation when the team has engineering capacity for open-ended agent research, and buy a causal testing platform when the goal is a controlled test of which specific action moves a defined behavior. A research, data-science, or product-insights lead facing this choice has a Python library open in one tab and a budget approval form in the other. TinyTroupe is Microsoft Research's open-source, MIT-licensed library for LLM-powered multiagent persona simulation. It gives engineers full programmatic control over agents, environments, and interactions. The question worth asking before either path gets funded is not "code or no-code." It is: does the team need open-ended persona conversation, or does it need a controlled test of which action moves a specific behavior? ## What does TinyTroupe answer? TinyTroupe is infrastructure, not a finished study. A team defines agents, scripts their environment, and interprets whatever the simulation produces. That flexibility fits novel multi-agent dynamics research, population-level scenario modeling, and academic experimentation where the researcher is also the instrument designer. It does not ship a UI, a managed service, or a published validation benchmark, because that is not what a library is for. ## What does a managed causal platform answer instead? A managed platform trades that flexibility for a narrower question: which price, message, or feature change causes the outcome to move. Subconscious runs controlled, randomized experiments rather than open-ended agent conversation, so the output is a comparison a team can act on directly instead of a transcript that still needs interpretation. ## The real trade-off The comparison is not "product vs. library" in the abstract. TinyTroupe can be pointed at almost any simulation scenario an engineer can code; a causal platform is built to answer one narrower question well: does this change move this behavior, with a result the team can defend. ## Where this goes wrong Naming a failure mode in public is what lets a buyer check for it before it costs them. Two failure modes show up repeatedly. A team builds custom simulation infrastructure on TinyTroupe and, weeks later, still cannot say which action changed buyer behavior, because the library never asked that question. It answers "what happens in this scripted scenario," not "what causes this outcome." The opposite failure is buying a managed tool expecting a randomized experiment and getting directional persona chat instead, which reads like data but does not isolate cause. ## What a causal platform does not replace Publishing what a method cannot do is what lets a buyer check it before relying on it. A randomized causal experiment is not a substitute for engineering-controlled programmatic simulation research, novel multi-agent dynamics work, or academic experimentation. That is TinyTroupe's territory. It is also not a substitute for direct human usability research or engineering judgment about system design. Subconscious does not publish an accuracy benchmark comparable to any figure a competing persona-simulation vendor might quote, and none should be inferred here. ## Deciding before committing budget or engineering time If the team has engineering capacity and the open question is exploratory, "what could happen under these agent dynamics," TinyTroupe is the right tool, and it costs nothing but time. If the question is a specific buyer decision, which price, message, or feature to ship, and the team needs a result it can act on without first building and maintaining simulation infrastructure, that is what a causal experiment platform is for. Reviewing prior test results, reading more on how Subconscious works, or booking a walkthrough are reasonable next steps before either path gets funded. [figure: Two-column table contrasting a persona-simulation library and a managed causal testing platform across five rows: what each produces, who operates it, best fit, validation path, and cost driver.] --- # Where an Agent's Marketing Tool Chain Needs a Causal Check URL: https://subconscious.ai/blog/implementation-and-procurement/where-agents-marketing-tool-chain-needs-causal-check Published: 2026-08-11 Put one causal check in front of the step where an agent is allowed to spend money, change a price, or ship a message on its own. Every other tool in the chain can run on uncalibrated output, because a human still reviews the result before it reaches a customer. The step that removes that review needs the strongest evidence behind it. ## What changed when marketing tools became callable APIs The Model Context Protocol standardizes how an AI agent discovers and calls a third-party tool, so any compatible agent can reach a service's API without a custom integration (Anthropic, "Introducing the Model Context Protocol"). The specification defines how a server advertises its tools and how an agent negotiates a call against them (Model Context Protocol Specification). That single standard is why a marketing operations or RevOps team now faces a procurement question: which tools in the chain are safe to call and act on without a person in the loop, and which one has to stop and get checked first. ## Where is uncalibrated tool output a fine default? Most of a marketing agent's tool chain tolerates being wrong, because the output lands somewhere a person still reads before it changes anything customer-facing. The first five rows fail safely: a person catches the error before it reaches a customer. The last row does not, because nothing sits between the tool's output and the customer. ## The step that needs the check An agent wired to call a research tool as one more API in its loop can turn a wrong positioning read, a wrong price test, or a wrong channel call into a live action at machine speed, with no review step in between. Removing the human from that step raises the bar on what the agent is allowed to act on, because the tool feeding that decision is now the only check left. The evidence that belongs in front of a live action is a randomized comparison, not a single-pass tool response. A controlled experiment tests one change against a holdout, replicates the result, and reports a confidence interval instead of a point estimate. ## Where does Subconscious fit, and where doesn't it fit yet? Subconscious runs a randomized, replicated experiment on a simulated market and reports the result with a confidence interval, which is the check an agent's live-action step needs. Subconscious can also test or validate a study with real human participants, so a team can move from the simulated read to a real-human check on the same causal question without redefining what they're asking. Subconscious is not shipped today as a callable tool inside an agent's own loop. It is a decision-validation step that a person or a workflow invokes before the live action fires. A team should place it there: as the gate before the irreversible step, not as another unsupervised API call. ## What should you check before wiring the stack? Before connecting a research or analysis tool into a live-action chain, confirm three things: what the tool's output is based on (a single model pass, or a controlled comparison with a replication and a confidence interval), who or what acts on that output next, and whether that next action is reversible. If the action is a live price change, a live campaign change, or a live message to a customer, the tool feeding it needs the stronger evidence, and a human or a workflow gate belongs between the check and the action until that gate is proven safe to remove. [figure: Flow diagram: analytics, CRM, prospecting, and draft steps route to a human reviewer before reaching a customer. A live spend or price change step has no reviewer and routes through a causal check instead.] See how the underlying experiments are built and reviewed on the research page, and how a study moves from question to result on how we work. Ready to test one live-action step against this kind of check? Book a walkthrough. --- # Rehearsing a VC Pitch Isn't the Same as Proving the Numbers Behind It URL: https://subconscious.ai/blog/buyers/rehearsing-vc-pitch-isnt-same-proving-numbers-behind Published: 2026-08-11 The difference is evidence: rehearsal sharpens how a founder delivers a pitch, while proving the numbers behind it requires causal studies that validate the pricing, demand, or positioning claims the pitch relies on. A founder can rehearse every objection an investor might raise and still walk in with an unvalidated pricing, demand, or positioning claim under the pitch. Rehearsal tests whether the story holds together, not whether the market claims inside it are true. That gap is what a VC's pattern-matching finds. [figure: Four-stage timeline with two parallel tracks: rehearsal (build persona, run pitch, revise deck, final rehearsal) and evidence (flag unvalidated claims, start causal studies, review results, confirm evidence attached).] ## Why pattern-matching finds the gap Investor meetings are asymmetrical. The founder knows the company; the VC has sat through hundreds of similar pitches and evaluates each new one against that pattern history (Holloway, "Pattern Matching"). That pattern-matching is efficient for the investor and unforgiving for the founder: it surfaces the claims rehearsed least, such as why this price, why this segment, why now. Founders whose backgrounds or markets don't fit the pattern the VC has seen before face this asymmetry even harder: there's less precedent for the investor to match against (Forbes, "The Extra Hurdles Pattern-Breaking Founders Face With The VC Industry"). A rehearsed answer to a different question doesn't close that gap. ## What is objection rehearsal good for? Practicing against a simulated investor persona, built around a stage focus, sector thesis, portfolio context, and known concerns, surfaces objections a co-founder or advisor won't raise. A well-built persona pushes back the way a skeptical investor would: on market size ("your TAM includes segments you can't realistically reach"), competitive positioning (whether the roadmap survives an incumbent shipping it as a feature), unit economics (with a $500 CAC against a $1,200 ACV, what gets that ratio to 3x LTV/CAC), team gaps, and timing. A founder who runs the pitch, identifies the toughest objections, rewrites those sections, and runs it again tests whether the revised narrative holds up. The same rehearsal works for existing investors too, before a board meeting, a pivot, a down round, or a new fundraising round, to test how the current investor base is likely to react. ## What claims can't rehearsal validate? Rehearsal improves the story; it does not produce evidence for the numbers behind it. "That TAM assumes enterprise adoption rates that took a comparable company seven years; what evidence supports a different adoption curve for your category?" is a question a simulated persona can raise, but no amount of rehearsing an answer manufactures the underlying evidence. Walking into diligence with a well-rehearsed narrative and an unvalidated pricing or demand claim invites exactly that exposure, and the round stalls or re-prices down when it does. Subconscious runs randomized experiments on a simulation of the market, validated against real human behavior, to produce a causal answer to a pricing, demand, or positioning question, evidence a founder can point to when a VC challenges the claim, not the story around it. That is separate from rehearsing objections against a persona: not whether the pitch is well delivered, but whether the market claim inside it is true. Two case studies on Subconscious's research, Finta and New Age Floral, used causal pricing studies to settle a pricing decision with a quantified result before going to market. ## A two-track preparation timeline The two tracks run in parallel in the weeks before a raise: - Early. Build the investor persona from public research (blog posts, podcast appearances, portfolio, stated thesis). In parallel, identify which claims in the deck (pricing, demand size, positioning) lack evidence. - Mid-preparation. Run the full pitch through the persona. Log every question and objection as expected or new. Start a causal study on any claim that surfaced weak or unvalidated. - Before the final rehearsal. Revise the deck against the new objections and rerun the persona pitch with the updated version. Review study results as they come in. - Final pass. Final rehearsal focused on follow-up questions, not first-order ones. Confirm which claims now have evidence attached and which are still the founder's judgment call. ## Where the evidence ends Neither track proves the deal happens. Subconscious does not simulate a VC's judgment, portfolio thesis, or negotiation behavior, and it does not predict whether a given investor says yes. Persona rehearsal has a limit: it cannot replicate the body language, rapport, and chemistry of a real meeting, or access that Monday's partner-meeting conversation or a deal the investor watched fall through. Both are preparation tools, not crutches, and neither replaces being present, adaptable, and honest about what is and isn't proven when the meeting starts. [figure: List splitting what each track covers from what neither reaches: causal studies prove market claims; rehearsal sharpens delivery; investor judgment, negotiation, and rapport stay outside both.] ## Next step If a claim in the deck, a price point, a market size, a positioning bet, lacks evidence, that is the one to test first. See how the studies behind Finta and New Age Floral were run, or book time to scope a study against your own pitch deck. --- # How to Not Lose Your Market Research Job to AI in 2026 URL: https://subconscious.ai/blog/implementation-and-procurement/how-not-lose-market-research-job-ai-2026 Published: 2026-08-11 The job at risk in 2026 is not "market researcher." It is the researcher who only executes: drafting screeners, summarizing open ends, and assembling first-pass reports without deciding what evidence a stakeholder decision needs. The fix is not a new tool habit. It is a rule your team applies before any AI-assisted finding reaches a pricing, positioning, or launch decision: name which evidence tier produced the finding, and require the next tier up before anyone repeats it externally. ## Why the pressure is real but the framing is wrong AI has moved from a novelty layer into daily research work: drafting, summarizing, and assembling first-pass output at a pace no single person can match by hand. That produces a specific anxiety: a stakeholder wants an answer tomorrow, a draft report shows up before anyone has finished going through the data, a manager asks whether the team can skip the slow steps. The accurate risk is not that research demand disappears. The U.S. Bureau of Labor Statistics projects continued growth for market research analysts from 2024 to 2034, well above the average for all occupations. The risk is being treated as an expensive production layer once a machine can do the mechanical parts of the job. When drafting and summarizing get quick and low-cost, the person doing only that work has to move closer to the decision: better questions, better evidence choices, better caveats, better influence on what the business does next. ## What is a four-tier evidence gate? The researchers who hold their ground in 2026 can name, for any finding, which of four evidence tiers produced it, and require the next tier up before a claim leaves the building. The output of an early tier is not evidence by itself. It is a hypothesis that has passed one screen. Making that distinction explicit, in the deliverable itself, is what separates a defensible workflow from a fluent-sounding guess. ## Where does a modeled-audience read fit? A causal experiment run against a modeled population is useful for the directional tier: it produces a randomized, controlled comparison instead of a single fluent guess, before a team commits to the slow or expensive part of the research process (how the underlying causal method works). The workflow starts with the decision, not the tool. Write down what will change if the read points one way or another. Then define the audience precisely: who they are, the situation they're in, what they currently do, the alternatives they weigh, and the goal they're trying to reach. Run the comparison against a focused stimulus: a concept, message, pricing story, or feature idea. Ask for reactions, objections, and what would make the idea more credible, and compare across segments rather than stopping at the first answer. When the decision behind a directional read is expensive or will be stated publicly, the same causal question can move to a real-human validation study without being re-scoped: the question stays constant, only the evidence tier changes (how a study moves from a directional read to human validation). That escalation path, not the directional read alone, is what makes a four-tier gate defensible in front of a stakeholder who will ask "how do you know." ## What mistake breaks credibility? The mistake is presenting a directional, AI-assisted read as if it were a statistically fielded, validated study. It usually happens under pressure: the team wants an answer, the tool gives a fluent one, the deck needs a conclusion. A researcher's credibility rests on being able to tell an unqualified output apart from evidence a business can act on without caveats. The correction is to make the limit part of the deliverable, not an afterthought. State what the AI-assisted work was used for. State what it was not used for. State what still needs validation before anyone repeats the number outside the room. A researcher who does this does not sound less confident. They sound like someone whose confidence has boundaries, the more credible position with a stakeholder. ## A one-week audit to run now Do not start by rewriting the whole workflow. Start with one visible project: - Pick a real project with a live decision attached to it. - Put the business decision into a single sentence. - Name the audience and estimate what a wrong call would cost. - Restrict AI or a directional read to that early exploratory step. - Read through the output by hand and flag each piece as usable, shaky, or not safe to reuse. - Present the answer with an explicit caveat and a named next validation step. Do this weekly across a month. At the end, the payoff isn't a longer list of tools. It's a working evidence gate that demonstrates judgment, not just output. ## Limitations to carry into the rollout A limit stated in the open is what lets a stakeholder check the work instead of just trusting it. A four-tier gate does not remove the need for a validation step on claims that matter; it changes how a team moves from a directional read to a validated one without starting over with a different method. A directional read stays a hypothesis until it clears human review and, where the stakes require it, real-human confirmation. It does not substitute for a researcher's judgment about which questions matter or how to interpret a contradiction between sources. See how teams have staged this kind of escalation in case studies, or read more about the company behind this approach. [figure: Four steps: Exploration (internal ideation), Directional testing (early prioritization), Human review (low-stakes recommendations), Validation (expensive or public claims).] --- # Persona Simulation vs. Causal Action Testing: Choosing a Research Method for a Product Decision URL: https://subconscious.ai/blog/comparisons/persona-simulation-vs-causal-action-testing-choosing-research-method Published: 2026-08-11 A product or research leader comparing synthetic user simulation vendors is really deciding on a method, not a brand. Two claims recur across this category: a stated speed advantage measured in hours rather than months, and a stated accuracy or behavioral-match percentage. Neither claim tells a buyer whether the tool proves what changes behavior or only predicts what people might say. That distinction is the decision this article answers. ## What Do Product-Focused Persona Simulation Tools Do? One common vendor pattern, illustrated by Vectorial, builds synthetic user simulations aimed at product development: which features to build, how to prioritize a roadmap, and how to validate a concept before committing engineering time. A team presents options, the tool simulates user responses, and the output feeds prioritization. Vendors in this category typically describe their synthetic personas as trained on behavioral data and market a turnaround measured in hours rather than months, alongside a stated accuracy percentage. A vendor benchmark number needs its scope stated to mean anything. Those figures describe how closely a simulated response distribution tracked a prior human sample in the vendor's own test, not a guarantee about the specific product or pricing decision a buyer is about to make. ## What Question Do These Tools Not Answer? A persona simulation can report how a modeled user is likely to respond to a described feature. It does not, by itself, isolate which single change caused a shift in that response, versus everything else that varied in the setup. A model can track how people describe their preferences and still stay silent on what happens when one variable, and only one variable, changes. For a buyer choosing a research method ahead of a launch, pricing change, or message test, that gap has a real cost. Committing budget to a tool that predicts engagement, then discovering after launch that the modeled response did not hold under real market conditions, is a more expensive failure than a slower research cycle. ## Where a randomized experiment changes the answer Subconscious runs controlled discrete choice experiments, using McFadden discrete choice modeling, Mixed Logit, and ICLV, on a simulation of a target market. The method is built to isolate the causal effect of one action, such as a specific feature, price point, or message, rather than to produce a general-purpose persona a team can converse with. Subconscious can test or validate studies with real human participants, so a team that starts with a simulated experiment can move to real-human validation without changing the underlying causal question. A speed or cost claim only holds up once its limits are attached. This does not claim Subconscious is faster or cheaper than any specific competitor. It makes Subconscious the fit when the decision on the table needs a causal effect with a confidence interval behind one specific action, not a persona to interview or a stated behavioral-match percentage. ## Comparing the two approaches ## When Is Persona Simulation the Right First Step? Exploratory concept work and early roadmap ideation do not always require isolating a single causal variable. If a team needs a fast read on general reaction to a set of options before anything is locked in, a persona-based tool answering how an option would land can be a reasonable starting point. Causal action testing becomes the better fit once a specific decision, such as which price or which feature to ship, needs to be defended with evidence rather than a general impression. ## Limitations and failure conditions Naming where a method stops working is what lets a buyer match it to the right stage. Causal action testing is not a substitute for exploratory qualitative research or open-ended persona interviews, which serve an earlier stage of product thinking. It requires a study designed around one clear action to compare, not an open conversation about a product area. No comparison in this article ranks one commercial vendor as superior to another: the vendor capabilities described above come only from that vendor's own public product pages, and any accuracy or coverage figure a vendor publishes describes their own benchmark. ## Practical next step A team already leaning on persona simulation for early exploration does not need to abandon it. The decision point is the moment a specific action, a price, a feature, a message, needs to ship and the cost of being wrong is real. At that point, review Subconscious's published replication results and get a demo scoped to the exact action under consideration, rather than a general platform walkthrough. [figure: Two columns: persona simulation gives a simulated response and a vendor-reported accuracy percentage; Subconscious causal action testing gives a causal effect with a confidence interval, validated with real humans.] --- # Real-Human Panels vs. Controlled Synthetic Experiments: Choosing a Pre-Launch Validation Method URL: https://subconscious.ai/blog/comparisons/real-human-panels-vs-controlled-synthetic-experiments-choosing-pre-launch Published: 2026-08-11 Before a pricing, concept, or message decision ships, a research or product leader has to pick a validation method: recruit real humans, run a controlled experiment on a simulated population, or do both in sequence. The right choice depends on the decision's stakes and timeline, not on which method is newer. ## The cost of picking wrong Recruiting real humans for every iteration burns weeks and budget on decisions that do not need regulatory-grade evidence. Running simulation alone for a high-stakes, board-facing, or regulator-facing call risks shipping on evidence that was never checked against real human behavior. ## What is real-human research built for? Remesh is one example of this category: a recruited-human research platform, where a live group of real people is convened, and as they answer in their own words, AI clusters those answers into live themes and distills them into one representative voice, replacing manual synthesis a moderator used to do by hand (Remesh: Research Moderation & Data Collection). Recruited human research, whether a traditional facilitated focus group or a larger AI-moderated live discussion, produces qualitative texture and reactive discourse dynamics: people responding to each other in real time, revising positions, surfacing objections nobody anticipated. That dynamic is not something a controlled experiment is designed to reproduce. It is the right tool when a decision is regulator-facing, board-facing, or part of a longitudinal cohort study where the record needs real-respondent provenance. ## What is a controlled synthetic experiment built for? A controlled experiment run on a simulated population is a different instrument, not a faster version of the same one. Instead of asking a group of people what they think of an idea, it exposes a modeled population to controlled alternatives, such as a price, a claim, or a concept, and estimates which alternative moves the target behavior, for which segment, with uncertainty reported where supported. Subconscious runs this kind of study as a causal behavioral platform, answering which action changes an outcome rather than what a sample of respondents says it prefers. ## Comparing the two approaches ## When is real-human evidence the right call? Real-human research, live or traditional, is the right choice when the output will be cited to a regulator, a board, or the public and needs real-respondent provenance; when the study is part of longitudinal cohort tracking; and when the decision is a high-stakes brand or strategy pivot where the cost of being wrong justifies the cost and time of recruitment. ## Sequencing simulation and real-human validation Sophisticated research teams do not treat this as a one-time choice between methods. A common pattern: run a controlled experiment on a simulated population first, to screen concepts, prices, or messages and narrow a wide set of options down to the strongest candidates. Reserve real-human recruitment for the survivors, once a decision's stakes are big enough to earn that cost and speed tradeoff. This works because Subconscious can test or validate studies with real human participants on the same causal question a simulated experiment already ran, without redesigning the study. Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people, a modeling resource for defining and targeting a population, kept distinct from a recruitable panel of real respondents. ## Limits on both sides A controlled synthetic experiment is only as good as its causal design. Recruited-human panels carry the opposite limit: bounded by how many people can be recruited and scheduled, which makes them a costly way to test many candidate variants before a decision has been narrowed. ## Where to start For most pre-launch pricing, concept, and message decisions, running a controlled synthetic experiment first is the faster path to a defensible shortlist. Escalate to real-human research when the decision's stakes, audience, or regulatory context call for real-respondent provenance. See how Subconscious runs these studies, review case studies, or book time to scope a specific decision. [figure: Four-step path: a wide candidate set enters a synthetic experiment that screens it to a shortlist, which branches to real-human validation for regulator, board, or cohort decisions.] --- # Which Push Notification Copy to Ship, Before the Live Send URL: https://subconscious.ai/blog/use-cases/which-push-notification-copy-ship-before-live-send Published: 2026-08-11 A lifecycle or CRM marketing manager who owns push copy has one recurring decision: which variant goes to a defined cohort before the live send, and whether that cohort needs its own variant at all. Push is the highest-leverage channel most teams own and the one with the least copy review before it ships. A notification that reads as vague, pushy, or irrelevant does not just lose the tap. It costs the unsubscribe, and a user who turns off notifications is gone from that channel permanently. A wrong cohort-level call also burns send volume across the whole list on a variant that was never going to win. ## Why guessing or sequential testing is not enough Most teams resolve the copy decision one of three ways. The first two approaches only tell you what happened after you already sent something. A controlled experiment answers the question before the send: which copy change moves a defined cohort's stated intent to open or engage, with a confidence interval around that effect. [figure: Five steps: state the trigger and success condition, write distinct copy archetypes, build a matching cohort, measure open intent versus unsubscribe intent, then carry top variants to a live test.] ## What does a push notification test need to isolate? Push is short, fast, and emotional. A user sees it on a lockscreen, decides in under 2 seconds, and taps, swipes away, or turns off notifications entirely. That last option is the one most testing setups ignore. A copy test that only scores clarity, relevance, and curiosity will optimize for tap rate while missing the irritation signal that drives the unsubscribe. Subconscious treats irritation as a measured outcome alongside intent to open, because the two behaviors trade off against each other. ## The workflow - Define the send context. State the trigger event, the time of day the push fires, and what success looks like: opening the app, landing on a deep link, or finishing a flow already in progress. "User abandoned checkout six hours ago, evening send" is a different test than "user has not opened the app in two weeks, morning send, drive reactivation." - Draft a spread of variants, not near-duplicates. Mix archetypes: direct utility, curiosity gap, social proof, urgency, personalized address, a question, and plain status. Two near-identical variants only tell you which guess you prefer. - Define the cohort. Build the test audience around the real recipients of this send: active users, dormant users, a specific tenure band, or a job-title and team-size cut for B2B. The closer the test cohort matches the real send list, the more the result transfers. - Run the controlled experiment. Present each variant with its send context to the defined cohort and measure stated intent to tap, intent to ignore, and intent to turn off notifications from the app, with reasoning attached to each response. - Promote the top variants to a live confirmation test. Take the leading variants into your normal on-channel A/B test. As a planning example, teams commonly run this confirmation window over roughly 48 hours before committing to the full list. The controlled experiment tells you which variant should win; the live test still produces the open and tap-through number your forecast needs (Statsig). ## When does live testing still matter on its own? A pre-send experiment is not a substitute for live testing on high-stakes sends: the first push to a new user segment, a change in send volume, or an entirely new notification category the app has never sent before. For routine, established lifecycle sends, the experiment result is often enough to ship directly. ## Proof and limitations Subconscious runs push copy comparisons as controlled experiments: each variant is a treatment, the outcome is stated intent to open or engage against a defined cohort, and the result carries a confidence interval rather than a single point estimate. When a send is high-stakes or entering a new segment, the team can move from that simulated experiment to real-human validation without changing the underlying causal question: same treatment, same outcome, a real-participant sample instead of a simulated one. ## What should you do with the next send? Pick the next push queued to go out. Draft a spread of variants across the archetypes above, define the cohort that will actually receive it, and run the comparison before the send rather than after. See how the workflow runs end to end, review case studies from other teams testing before they ship, or book a walkthrough on your own send calendar. --- # When to Rerun a Causal Experiment Before You Act on It URL: https://subconscious.ai/blog/case-studies/when-rerun-causal-experiment-before-act Published: 2026-08-11 A research or insights lead who already ran a causal experiment eventually asks the same question: is the finding still good, or does the budget and strategy riding on it need a second look? A number without its limits is marketing, so this finding ships with the conditions that could break it. A single run is not proof of stability. Outside research on the psychology replication crisis found only a 36% replication rate for original findings, reason enough not to treat one experiment as a permanent answer (Nature, Communications Psychology). "Out of 100 independently performed replications, only 39% were subjectively labelled as successful replications, and on average, the effects were roughly half the original size." Open Science Collaboration replication study, cited in Korbmacher and colleagues, Communications Psychology (source) ## What does treating one experiment run as final cost you? A finding that was never checked for stability can quietly go stale while budget and strategy still ride on it. Market conditions shift, the audience changes, or the inputs that produced the original result stop holding. A rerun doesn't replace the first study; it checks whether the original answer still applies before more budget moves on it. ## Three triggers that justify a rerun Not every finding needs a repeat test. Three conditions do: ### Do results stay stable over time? Consumer preferences and market conditions move. Rerunning the same experiment design after time has passed checks whether the original result still holds. ### What happens when the target audience or inputs change? When the target demographic, product attributes, or pricing strategy shift from what the original study tested, a rerun with those adjusted variables shows how the outcome moves with them. ### Benchmark against a human baseline or new data Moving from a simulated experiment to real-human validation, without changing the causal question, is a direct way to check a finding. When new real-world data becomes available, rerunning the original experiment against it confirms whether the earlier causal insight still holds. ## What a rerun looks like in practice New Age Floral's engagement ran five iterative experiments across phases before arriving at its pricing recommendation, adjusting the question at each phase rather than accepting the first pass (case study). That pattern, rerun, adjust, check again, is what a replication check does for a causal action test: it holds the design constant while changing one variable, keeping the comparison clean. ## Limitations Naming where a rerun stops proving something is what lets a buyer check it against their own decision. A rerun checks a specific study's inputs and timeframe. It does not guarantee an outcome, and confidence-interval or uncertainty language only applies where the original study design supports it. Real-human validation confirms a simulated finding against real participants; it does not turn a causal action test into a usability session or a clinical trial, and it does not prove market performance. [figure: Five-step chain: original study design, isolate the single variable that changed, rerun with only that change, compare against the original result, clean read on what moved.] ## Next step If a finding is old, or the inputs behind it have changed, the fastest way to know whether it still holds is to rerun the study against the new condition. See how a causal experiment is structured, or book time to scope a rerun against your original design. [figure: A list diagram showing three triggers that justify rerunning a causal experiment: stability over time, changed inputs or audience, and benchmarking against a human baseline or new data.] --- # How to Use AI-Assisted Research Without Making Fake Strategy URL: https://subconscious.ai/blog/implementation-and-procurement/how-use-ai-assisted-research-without-making-fake-strategy Published: 2026-08-11 A research or brand strategy lead does not need to decide whether AI belongs in the process. It already is in the process. The decision that matters is narrower: which stage of the pipeline is an AI-generated read allowed to inform, and which stage requires a controlled experiment or real respondent data before it reaches a strategy deck. Get that boundary wrong and a team presents a fluent AI-generated narrative as if it were customer evidence. The recommendation is wrong, and the business commits budget on an unvalidated hypothesis. ## Why is the pressure showing up now? AI is no longer a side experiment; it sits inside the everyday research workflow, where teams lean on it to analyze, report, prep data, and pull self-service insight. That shift does not eliminate research demand: the U.S. Bureau of Labor Statistics market research analyst outlook forecasts headcount growth for market research analysts and marketing specialists over the 2024-2034 span. The real risk is narrower than "AI replaces researchers." It is that confident, fluent AI output gets treated as if it were evidence, and strategy gets built on top of it. As AI-assisted tools absorb more of the mechanical steps in a research job, whoever holds that job needs to sit nearer the actual call being made. That translates to sharper questions, more deliberate evidence choices, clearer caveats, and a more explicit call on what to trust. ## What changes for the person doing the work? Expertise used to live partly in access: knowing how to field a study, clean responses, and package a finding. AI-assisted tools weaken that access advantage. More people can now draft a survey, summarize a transcript, or generate an audience read on short notice. Expertise is not going away, but it is now checkable in a way it wasn't before. When anyone can produce a plausible-sounding answer, the person worth paying is the one who can judge which answer to trust, and who can spot a narrative that is generic, poorly grounded, or beside the point for the decision at hand. Concretely: frame the question before handing it to AI, and take responsibility for the caveat once AI has produced something. ## Build an evidence system, not an AI habit An evidence system spells out the scope AI-assisted work is permitted, flags what needs a human's eyes, and sets which claims need real validation before reaching a decision-maker. The AI-assisted output is not the value on its own. The value is a disciplined path from a question to a safer decision, with a defined point where the read either passes to a real test or gets set aside. ## Where a controlled experiment fits Once an AI-assisted exploratory pass has produced hypotheses or objections, those hypotheses still need to be checked against something more than a fluent model output. Subconscious tests the specific action a team is considering as a controlled experiment against a defined audience, using causal experimentation and discrete-choice-style testing to compare actions. That gives the directional read a causal comparison and a confidence interval before it informs a decision. Framed against the table above: an AI-assisted exploratory pass earns a place in "Exploration." A controlled experiment is what "Validation" looks like when the decision is expensive or the claim will be made publicly. Directional comparison and human review still sit between the two. ## What mistake makes this dangerous? Danger shows up when a team builds an audience read out of stereotypes or one fluent model output, then hands it over as if it were research. Deadline pressure is usually the trigger: the model returns something confident-sounding, a closing slide is still needed, and nobody stops to ask where the confidence came from. Credibility here rests on one distinction: can the team tell its own output apart from actual evidence? A model is capable of drafting something useful. Judging whether that draft holds up for the decision at hand is not something it can do. Limits belong inside the deliverable itself. Name what the AI-assisted work supported, name what it left unproven, and name the validation step that comes next. Spelling this out doesn't undercut confidence; it signals discipline, because the reader can see exactly where the confidence stops and starts. ## What to do this week - Pick a real project with a live decision attached to it. - Write the business decision in one sentence. - Define the audience and the risk level of getting it wrong. - Use an AI-assisted read only for the exploratory stage. - Go through the output by hand and flag what holds up, what's shaky, and what's unsafe to use. - Present the answer with an explicit caveat and a named next validation step. Repeat that weekly for a month. The payoff is not a longer tool list; it's a research system that visibly moves fast, exercises judgment, and holds a quality bar, with a clear record of which stage each finding passed through. ## Limitations This is a process discipline, not a shortcut. Subconscious does not replace the human review step; checking audience definition, prompt neutrality, and source grounding remains a person's job. Subconscious also does not generate the initial exploratory hypotheses or objections a team starts from; that work happens before a controlled experiment is worth running. A team still has to decide, stage by stage, what evidence a given claim needs. ## Next step Before a directional read reaches a strategy deck, name the stage it came from and the stage it still needs to pass through. Where that path leads to a controlled experiment, Subconscious's research approach explains how the causal comparison works, how we work covers the process end to end, and a live demo walks through a specific decision. Read more about Subconscious for the broader context. [figure: A four-step path from Exploration to Directional comparison to Human review to Validation, showing the evidence standard at each step, from none required to a controlled experiment or real respondent data required.] --- # Stress-Testing a First Monthly Investor Update Before It Ships URL: https://subconscious.ai/blog/use-cases/stress-testing-first-monthly-investor-update-before-ships Published: 2026-08-11 [figure: Four variables a founder can test in a monthly update against a modeled investor reader: which metric leads, tone, how the ask is framed, and whether risk is named or buried.] A monthly investor update shapes how investors remember a founder more than any pitch deck or board meeting. It is the document an investor skims on a Sunday night before thinking about the company on Monday. Over a year, the pattern of those updates decides whether a founder gets a warm intro at the next raise, a fast yes on a bridge, or a reply that never comes (Visible.vc, "Investor Updates"). Most founders write their first update without a single outside read. That is the gap a controlled comparison closes before the draft leaves the outbox. ## The decision the founder is actually making The question is not "did I write an update." It is which draft, tone, and metric emphasis should go out this month. Few things a founder writes carry as much weight per word as these five hundred, and most first-time founders write roughly five to ten before they find their voice. Investors rarely reply to the email itself, so the feedback loop is invisible. A founder who never gets outside signal drifts toward minimum-viable effort: a mediocre update is forgotten, but one that buries a problem instead of naming it costs trust that is slow and expensive to rebuild. ## What Does Choosing Poorly Cost? The cost does not show up in the moment. It shows up months later, when a lead investor who was never asked a clarifying question quietly loses conviction, or when a follow-on target reads several months of vague, inconsistent updates and takes a pass without saying why. ## Where Does a Controlled Comparison Fit? Subconscious can run a controlled comparison of two or more draft versions against a modeled investor audience on its person-level audience graph, and read which version changes the investor's stated reaction. That comparison can test: A useful comparison changes only the variable under test, the metric order, the tone, the ask, and holds everything else constant. That is what separates a causal read on the draft from a guess about which version feels better. ## What Is the Practical Advantage Over Writing Alone? A founder has too much context and emotional proximity to the numbers to read their own draft the way an outside investor will. A controlled comparison supplies that outside read without waiting months for investor feedback to accumulate. It surfaces the same failure pattern that recurs across first-time founder updates: a weak or missing ask, a risk section that stays buried, a metric that is flat or down and quietly drops from the draft, and a narrative with no arc, just a list of facts. ## Proof and where the method comes from The controlled comparison described here, testing one changed variable against a modeled audience and reading the effect, is the same method described on Research and How We Work. Naming exactly where proof stops is what lets a buyer check the claim. There is no case result or customer number specific to investor updates to cite; the description above applies the method to this use case, not a proven result for it. ## Limitations and what this does not replace A method that never states what it cannot do is just marketing copy. This tests message and tone reception only. It does not replace the founder's judgment, the board relationship, or an actual investor's real capital decision. A modeled investor's reaction to a draft is a read on how the writing lands, not a prediction of whether that investor will write a check. ## Next step Start with the update already sitting in a draft. Compare it against an alternate framing, a different metric order, a rewritten ask, or a named risk instead of an omitted one, and see which version a modeled investor reads as clearer before either goes out. See a demo or read more about Subconscious to understand how the comparison is built. --- # How to Evaluate Market-Simulation Evidence Before Committing Research Budget URL: https://subconscious.ai/blog/comparisons/how-evaluate-market-simulation-evidence-before-committing-research-budget Published: 2026-08-11 Evaluate a market-simulation vendor by the evidence behind the decision, not the largest accuracy figure in its pitch. The decisive questions are whether the study isolates an action, reports uncertainty, replicates, and can be tested with real people without changing the causal question. A single correlation snapshot cannot answer those questions. The cost of choosing poorly is not limited to the contract. A team can commit research budget, act on an unverified number, and discover after launch that the simulated result did not hold against customer behavior. [figure: Checklist of decisive questions before committing research budget: does the study isolate a defined action, report uncertainty, replicate, and can it be tested with real people.] ## The budget decision is about evidence Public case material shows that market simulation is being applied in wealth and asset management (EY). That is not, by itself, evidence that a result replicated or that a proposed action caused the measured outcome. The useful comparison is not a category label or sales model. It is the type of proof a vendor can put behind the decision. ## Compare the proof each method can provide A method's limits belong next to its claims, so a buyer can check them before committing budget. A correlation shows what moved together. It cannot tell a buyer whether changing a price, message, offer, or product feature caused the outcome. Random assignment and confidence intervals address that question; replication asks whether the result survives another test under a stated method. ## Questions that expose weak proof Ask for answers that can be reviewed before procurement: - What exact action and outcome did the study test? - Was assignment randomized, and what comparison condition was used? - How was uncertainty reported? - Was the result replicated, and under what population and market conditions? - Can the same causal question be tested with real people? - Which decisions fall outside the method's stated limits? If a deck repeats an inherited benchmark without the method behind it, label that number as a historical or planning example. Do not treat it as a current performance claim. ## How does Subconscious test the action? Subconscious research uses randomized experiment design and confidence intervals to estimate whether a specific action changes an outcome. When a decision warrants an additional check, a team can move from a simulated experiment to real-human validation without changing the causal question. It does not turn the study into an observed usability session, and it does not guarantee market performance. The Subconscious method is best suited to choices that can be expressed as a defined action, comparison, and measurable outcome. Subconscious can use a person-level audience graph representing 800 million people for targeting and simulation. A number without its limits functions as marketing, so the limit is stated plainly here. That reach is not a recruitable participant pool, and it must not be presented as a fielded sample size. Published benchmark results provide another artifact a buyer can inspect, but they do not remove the need to match the method to the decision. ## How do you set the procurement gate before reviewing proposals? Require a pre-specified causal question, the experiment design, confidence intervals, replication evidence, written failure conditions, and a real-human validation plan when the stakes justify one. Apply the same standard to every vendor. Naming what a study cannot establish is what lets a buyer check the claim against the evidence. This evidence cannot support a price or delivery-speed comparison. It also cannot convert an unpublished method or a promotional number into proof. If a supplier cannot provide the required artifacts, include the risk of an unverified result in the budget decision. Teams with a live market choice can discuss an appropriate study design. --- # Bayesian Media Mix Modeling for Marketing Optimization URL: https://subconscious.ai/blog/methods-and-validation/bayesian-media-mix-modeling-marketing-optimization Published: 2026-08-11 A marketing analytics team allocating budget across TV, paid social, and direct mail usually starts from last-touch attribution or a spend-to-revenue heuristic. Both approaches answer a correlational question: which channel's spend moved with the outcome. Neither answers the question that actually decides next quarter's budget: which channel's spend caused the outcome, and how much further can it scale before returns flatten. Bayesian Media Mix Modeling (MMM) is one answer to that causal question. It estimates how much each marketing channel contributes to customer acquisition or sales, using Bayesian inference to attach uncertainty to every parameter instead of returning a single point estimate. Google researchers introduced the Bayesian approach to MMM in 2017 (Jin et al., 2017). ## Why a linear regression undercounts channel effects A naive MMM treats acquisitions as a linear function of spend per channel. Real marketing data violates that assumption in two specific ways, and missing either one produces the wrong budget call. [figure: Left-to-right chain: channel spend, a reach/saturation filter, an adstock/carryover filter, then acquisition outcome. The two filters are the nonlinear steps a linear regression skips.] The reach function models saturation. The first dollars spent on a channel tend to produce the largest marginal gain in acquisitions. Additional spend on the same channel runs into diminishing returns as the audience becomes saturated with the message. A channel with room left on its reach curve and a channel near its ceiling can show similar week-over-week acquisition numbers, even though one has far more budget headroom than the other. The adstock function models carryover. A channel's effect on acquisition does not end the moment spend stops. Some channels decay within days; others carry an effect for weeks. A channel that looks weak in a short attribution window can still be earning acquisitions from spend several weeks earlier. The adstock function recovers that long-tail effect from the data. Cutting a channel that is actually mid-saturation, or one whose adstock decay hasn't finished, wastes the reallocated budget for a full cycle before the attribution error becomes visible in results. ## What does the Bayesian layer add over a standard MMM? A standard MMM fits reach and adstock parameters from historical data alone. The Bayesian version adds three properties that matter for a team making a budget decision: - Prior knowledge from experiments and expert judgment can enter the model formally. A marketing manager's accumulated intuition about a channel, or the result of a completed lift or incrementality test, becomes a prior distribution rather than a fact the model has to rediscover from scratch in the current data. - Uncertainty propagates coherently through the nonlinear saturation and adstock transforms into ROI and optimal-allocation estimates, and priors act as regularization under the severe collinearity typical of channel spend data. A channel estimated with wide uncertainty is a signal to run a targeted lift test before committing more budget to it, not a number to treat as settled. - The model updates as new spend and outcome data arrive, sharpening the saturation and carryover estimates for a channel within the period the model was fit on. Letting those estimates drift as market conditions change requires an explicit time-varying-coefficient or state-space specification, not just posterior updating. ## Where fitting a Bayesian MMM gets hard in practice Naming a failure mode before a team hits it is what makes the method usable. Building a Bayesian MMM is conceptually simple and operationally demanding. The practical stumbling points are consistent across implementations: - Likelihood choice. Customer-count data can be modeled with a Poisson, negative binomial, normal, or longer-tailed distribution such as Student's t, and the choice changes how the model treats outliers and overdispersion. - Prior specification. State-of-the-art models can carry hundreds of parameters across regression coefficients and reach/adstock shape parameters. Weak or poorly chosen priors are a common source of convergence failure. - Reach and adstock parameterization. The functional form chosen for each affects how interpretable the resulting curve is and how directly it can inform a spend decision. - Model comparison. Dozens of candidate MMM variants may need evaluation during development. - MCMC convergence. Correlated chains, sampling divergences, and over-specified models can all produce a model that looks fit but has not actually converged. That failure mode is invisible without checking chain diagnostics directly. None of these are unique to marketing. They are the standard costs of fitting a hierarchical Bayesian model with a large parameter count, and a team evaluating whether to build or buy MMM capability should budget for them. ## Where pre-spend causal testing fits before an MMM does MMM is retrospective: it explains what already happened to a channel's spend and outcomes. It cannot tell a team what would happen with an untested message or creative, or with a budget split outside the spend range it was fit on. Subconscious's causal experimentation runs a randomized, discrete-choice-style test of a specific channel, message, or budget-allocation action before a team commits spend to it, producing a directional causal comparison with uncertainty language where the study design supports it. That is a complementary step upstream of MMM, not a substitute for it: Subconscious does not build or replace a Bayesian MMM, does not fit reach or adstock curves, and does not run MCMC inference. Where the decision depends on more than a directional read, a team can move from a simulated test to real-human validation of the same action without changing the causal question being asked. [figure: A branching path: a channel-effect estimate leads to a check of its uncertainty width, splitting into narrow (commit budget) and wide (run a lift test first).] ## What limitations should you keep in view? Publishing where a model breaks is what lets a buyer check its number against reality. An MMM, Bayesian or otherwise, is an observational regression: its causal interpretation depends on no unobserved confounding and exogenous spend, assumptions routinely violated when budgets are set in anticipation of demand (seasonality, budget-follows-sales, competitive response). It only measures what happened in the spend and outcome data it was fit on, and its uncertainty estimates are only as trustworthy as its priors and convergence diagnostics. Pre-spend causal testing narrows a different kind of uncertainty: which of several untried actions is worth taking. For a related discussion of causal versus predictive modeling, see how Subconscious approaches causal action testing and published case studies. --- # Funnel-Aware MMM: A Bayesian Architecture for Full-Funnel Marketing Optimization URL: https://subconscious.ai/blog/methods-and-validation/funnel-aware-mmm-bayesian-architecture-full-funnel-marketing-optimization Published: 2026-08-11 A standard marketing mix model (MMM) treats every channel as independent: spend goes in, conversions come out, and each channel gets its own response curve. That assumption breaks when upper-funnel activity, such as demand generation or video, creates awareness that shows up weeks later as branded and non-branded search. If a model cannot see that path, it can recommend cutting the upper-funnel budget that was quietly driving the lower-funnel result. ## The path a funnel-blind model can't see Two dynamics make a standard MMM undercount upper-funnel value. First, upper-funnel spend has both a direct effect on leads and an indirect effect that flows through lower-funnel spend it helped create. Second, lower-funnel channels like paid search often carry daily budget caps: when demand exceeds the cap, the surplus clicks and leads are never observed, so the data understates true demand. A funnel-aware Bayesian MMM architecture encodes both problems in its structure, though the direct-plus-indirect decomposition it estimates is identified only if there's no unmeasured confounding of the lower-funnel-spend-to-leads relationship, an assumption the model specifies rather than proves. It models lower-funnel spend as an outcome of upper-funnel activity and as an input to lead generation at the same time, and it uses a censored likelihood so capped days are treated as "demand was at least this high," not "demand was exactly this much." This mirrors the mediation structure documented in pymc-marketing.io's guide to measuring upper-funnel impact, which walks through the same direct-plus-indirect attribution logic on the open-source PyMC-Marketing library. ## Inside the funnel-aware architecture The core extension is a two-stage causal structure with four components: - Media transformations on both stages. Adstock (decay) and saturation (diminishing returns) are estimated separately for the upper-to-lower relationship and the lower-to-lead relationship, since a channel can decay slowly toward awareness but saturate quickly toward search volume. - Time-varying baselines. A Gaussian-process-based baseline lets lower-funnel spend drift with seasonality and organic growth that has nothing to do with upper-funnel spend, though when a campaign recurs alongside a seasonal peak the two are collinear and the split between them depends on GP kernel and prior choices rather than being fully determined by the data. - Channel-specific controls. External factors that move one lower-funnel channel, such as a promotion or a policy change, are modeled separately from global controls so a one-off spike doesn't contaminate the upper-to-lower effect estimate. - Censored likelihood. Days where a budget cap binds are treated as censored observations rather than exact counts, which recovers a larger upper-funnel effect than a model that reads capped spend at face value. [figure: Diagram: upper-funnel spend splits into a direct arrow to leads and an arrow to lower-funnel demand, which hits a budget cap. A censored-likelihood step treats capped demand as a floor before it reaches leads.] ## Where a funnel-aware model changes the optimization question A standard optimizer answers "spend more here, less there" across channels it treats as independent. A funnel-aware optimizer treats lower-funnel budget caps as a decision variable alongside channel spend, because raising upper-funnel investment only pays off if the lower funnel has room to capture the demand it creates. ## How does this connect to Subconscious's approach? Subconscious does not build marketing mix models and does not run, replicate, or validate this architecture. The two approaches do share one discipline: model the causal path an action takes before trusting the number the model reports. Subconscious applies that same discipline by testing a budget, channel, or messaging decision as an explicit causal comparison on a simulated market and, where it changes the answer, checking that comparison against real-human validation. Read more about that testing approach at /how-we-work or the underlying method at /research. ## What are this method's boundaries? This is a time-series Bayesian MMM fit on historical spend and lead data with a censored likelihood, a different method from a decision-specific causal experiment run on simulated or recruited buyers. It estimates attribution and budget response from observed spend history; it does not test a new message, price, or concept that has never been run before. Extending this kind of architecture to production, such as handling campaigns that change strategy mid-series or channels that enter partway through the data, is engineering work layered on top of the core model and is out of scope here. ## What should you check before you cut upper-funnel spend? Before trusting a recommendation to cut demand-generation or video spend on weak direct ROI, confirm the model that produced it accounts for two things: whether upper-funnel spend has an indirect path through lower-funnel channels, and whether any of those lower-funnel channels were capped. If the model can't answer both, the "weak" channel may simply be one whose effect was absorbed and hidden downstream. --- # Pre-Testing a Conference Keynote Thesis Before It Reaches the Stage URL: https://subconscious.ai/blog/use-cases/pre-testing-conference-keynote-thesis-before-reaches-stage Published: 2026-08-11 [figure: A left-to-right sequence of four steps: write the thesis as one sentence, compare thesis variants, compare the outline's supporting claims, then compare opening lines.] Choose the thesis before building the slides. Compare candidate claims against the audience segments the event actually draws, then commit stage time to the version more likely to change belief or intent. ## The decision: which claim goes on stage A flagship keynote is one of the highest-leverage public moments an enterprise brand or communications leader gets in a year. One historical planning example uses these quantities: forty minutes on stage, six months of inbound attention, a dozen inbound partnership conversations, and forty hours of prep. A number without its limits is marketing. Those figures are a planning example, not a current Subconscious benchmark, customer result, or promised outcome. The question the executive's team has to answer before a flagship talk is narrow: which single claim should the executive defend in front of this specific room, and will it land with the analysts, enterprise buyers, press, and partners in the seats, fall flat, or backfire. A stage moment cannot be retested once the executive has said the sentence out loud. ## Why is the cost of guessing asymmetric? The room contains more analysts, prospects, and press in one sitting than most other calendar events combine. A thesis that misfires there does more than waste the slot. It can damage the executive's standing with the people in that room, and misalign the market narrative for a launch or campaign timed to the appearance. ## Four ways a thesis fails before anyone reads a slide The failure modes are predictable enough to name in advance and test before prep time goes into the talk. All four are failures of misjudging how a specific audience will receive a specific claim, the kind of question a controlled comparison can answer before the room does. ## How does causal message testing fit before the thesis is locked? Subconscious helps teams test product, pricing, messaging, and go-to-market actions before committing capital, time, or reputation to them. Applied to a keynote thesis, that means comparing candidate thesis statements against an audience segment built from the analysts, enterprise buyers, press, and partners the specific event actually draws, to estimate which framing is more likely to change belief or intent before the executive commits stage time to it. This sits inside message and positioning testing: comparing claims against a causal response, not collecting generic opinion. Pre-testing a message against its intended audience before broad release is established practice outside AI-assisted testing too; public-health communicators routinely pretest message drafts with representative audience segments before a campaign goes live, since a flawed message is cheaper to catch in draft than after it reaches the room (The Compass for Social and Behavior Change: How to Conduct a Pretest). The keynote case applies that same logic to a single high-stakes claim rather than a campaign. A workable sequence for a flagship talk: - Write the thesis as a single sentence. It cannot be a topic or a title; it has to be one declarative claim the executive is ready to stand behind. Test that sentence against the audience segment before a single slide gets built. - Compare thesis variants. Write a conservative version, a provocative version, and a more contrarian version of the same claim, and compare how the audience segment ranks them. The ranking regularly surprises the executive: a provocative claim can lose to a conservative one when the audience reads the provocation as unearned, and a contrarian claim can win when the executive was underselling their own point of view. - Compare the outline's claims. Once the thesis is fixed, break the talk into five to seven supporting claims and check which the audience already believes, which are new, and which read as filler. This is where redundant setup gets cut and the genuinely new claims earn more time. - Compare opening lines. The first ninety seconds decide whether the room leans in. Write three opening lines and compare them against the audience segment before rehearsal, turning the choice into a comparison instead of a guess. - Surface the strongest objection. Ask which claim in the talk the audience segment would challenge first, and pre-empt that objection inside the talk itself rather than discovering it live during Q&A. ## What this does not cover The misses go on the public page next to the hits. Causal message testing answers which claim is more likely to change belief or intent for a defined audience segment. It does not evaluate delivery, stage presence, vocal performance, live Q&A handling, or room chemistry on the day, all of which require human rehearsal and real-room feedback, not a claim comparison. A thesis that tests well can still be delivered badly. Naming this failure mode is what lets a buyer check the method before booking the talk. The audience segment used to compare claims is not the same as recruiting real people to sit through a mock talk. Subconscious can test or validate studies with real human participants, and a team that wants to move from a simulated comparison to a live read-through with real people can do so without changing the underlying question being asked, but that is a separate step, not something the claim comparison itself performs. ## How do you build a library of tested claims over time? An executive who runs this comparison ahead of every flagship appearance builds, over a year of regular speaking, a catalog of thesis framings that have already held up against a given audience. That catalog compounds: the claims that survived testing and then survived the room become the basis for the next talk, the next byline, and the next appearance, instead of starting from a blank page each time. The keynote stops being a one-off bet and becomes the most visible entry in a longer sequence of tested claims. If a flagship talk is on the calendar this quarter, start by writing the thesis as one sentence and comparing it against the audience segments the event actually draws, before a slide gets built. Learn more about how Subconscious runs a causal experiment, see current research, or start a comparison before the next stage commitment. --- # AI Research for Pharma: Pressure-Test Positioning Before Launch URL: https://subconscious.ai/blog/industries/ai-research-pharma-pressure-test-positioning-before-launch Published: 2026-08-11 [figure: Four-item list: positioning, KOL preparation, patient journey mapping, and formulary strategy, each pressure-tested with a causal experiment before real-people research.] A pharma commercial or brand launch lead has one shot at first-impression positioning with prescribers, and the people whose reactions matter most, KOLs, formulary committees, target patients, are the hardest population to reach on a launch timeline. A full KOL advisory board takes months to convene, and physician time for pharma advisory work is compensated at negotiated hourly rates that scale with specialty and seniority, which is part of why teams ration how many real conversations they can afford before locking a strategy. Payer market research is a mature category, mock P&T panels, payer advisory boards, and recruitment of sitting and former P&T members, but the real committee's group deliberation and vote can't be observed, and sample for a specific plan's sitting members is thin. The result is a structural gap. Positioning, KOL prep, patient-journey assumptions, and access strategy all get locked well before a team can test them against the population that will actually react to them. ## Which reaction is worth testing before you commit The decision this fits is narrow: which prescriber, KOL, patient, or formulary-committee reaction should a team pressure-test with a controlled experiment before locking launch positioning, an advisory-board narrative, or an access strategy. A wrong positioning bet burns the launch window, and a wrong read on committee objections burns scarce KOL goodwill and advisory-board budget on the wrong questions. Subconscious runs controlled experiments on defined prescriber, patient, or payer populations to compare positioning, message, or access alternatives before a team commits budget. Point of proof: the causal experiment method and replication record, not case claims from any single prior vendor. A simulated reaction from this kind of experiment is a hypothesis to test, not evidence. It requires validation with real human research before it informs a regulatory or promotional claim, and it does not replace real KOL relationships or clinical evidence. ## How does drug launch positioning research work? Traditional launch research runs sequentially, HCP qualitative, message testing, conjoint analysis, sometimes an ATU study, and produces results in batches on a fixed timeline. That sequence still matters. What it doesn't offer is a way to iterate on positioning language between formal rounds. A causal experiment lets a team compare defined positioning alternatives against a defined prescriber population before locking the detail aid: lead with efficacy versus safety versus convenience, and see which framing produces a measurably stronger directional response, and where that response differs across prescriber segments. The output is a comparison between tested alternatives for a population specified in advance, not a prediction of what any individual prescriber will do. ## Why is real KOL time so scarce and valuable? Real KOL time is scarce and relationship-sensitive. Every conversation used to test an argument is goodwill not spent on substantive advisory work. Independent analysis of KOL engagement describes the recurring tension between a KOL's role as an independent expert and their financial relationship with the sponsoring company, which is part of why real advisory conversations are worth protecting for issues that need a real expert's judgment, not for rehearsing how an argument lands. "Two recent business intelligence reports on how drug companies identify, recruit, train, and pay their opinion leaders state that influential doctors can earn up to $400 an hour." Ray Moynihan, BMJ (source) A controlled experiment against a defined set of prescriber or KOL profiles, grounded in publication record, therapeutic-area focus, or stated position where that information is available, can surface which parts of a discussion guide are likely to generate real debate before the actual advisory board meets. Naming this boundary lets a team judge what the exercise can and can't tell them. This is preparation, not a substitute for the advisory relationship: what a team learns is where an argument is weak, not what a specific named KOL will say. ## Patient journey mapping Traditional patient-journey research recruits real patients, conducts depth interviews, and synthesizes findings, and it stays necessary, especially for rare diseases where patients are hard to recruit, and chronic conditions where the journey spans years longer than a single research cycle can observe. A causal experiment can compare hypotheses about where patients experience friction at each journey stage, pre-diagnosis, diagnosis, first treatment, switching, long-term management, against a defined population, before the team commits its real-research budget to the stages that turn out not to matter. The result narrows where depth interviews should focus. It doesn't replace them. ## Why is formulary committee research so hard? Formulary committees are structurally hard to research directly, which is exactly why access strategy often gets built on assumption rather than evidence. A controlled experiment against a defined set of formulary-relevant profiles, a cost-focused reviewer, a clinically oriented physician member, an outcomes-oriented medical director, can test how a cost-effectiveness argument, a competitive-positioning claim, or a specific prior-authorization framing lands, before the real committee sees it. The limit sits next to the comparison in the table above so a buyer can weigh both at once. This does not predict what any real committee will decide. It surfaces which objections and questions a team should be ready to answer. ## Where this stops being a hypothesis A method proves itself by publishing what it can't do alongside what it can. A simulated prescriber, KOL, patient, or committee reaction answers one question: is this argument, positioning, or framing strong enough to be worth testing further. It is a hypothesis-generation and comparison tool, not a source of clinical or regulatory evidence, and a result from it should never be attributed to a real named individual. Before any insight from this kind of experiment informs a regulatory claim, a promotional claim, or a final launch decision, it needs validation with real human participants, real prescribers, real patients, or real committee-adjacent reviewers, depending on the question. The causal question being tested stays the same across that step; what changes is the population answering it. Start with one launch or access decision your team needs to make, name the population and the alternatives you're weighing, and bring it to a Subconscious working session. --- # Bayesian Marketing Measurement When Individual Tracking Weakens URL: https://subconscious.ai/blog/methods-and-validation/bayesian-marketing-measurement-when-individual-tracking-weakens Published: 2026-08-11 When individual-level attribution loses coverage, marketing teams should not treat the remaining tracked journeys as the whole market. Channel-aggregate models can estimate contribution without reconstructing individual paths, while customer-value models can keep acquisition tied to longer-run value. Bayesian methods make the uncertainty and assumptions in both decisions explicit. They do not rescue poor data or turn estimates into causal proof. For a marketing analytics leader allocating budget, or a data science lead building measurement in-house, the decision is whether to move budget allocation and customer-value targeting onto aggregate, uncertainty-aware methods. ## Why did individual-level attribution stop being reliable? Regulatory constraints (GDPR, CCPA) and platform-level changes (Apple's iOS 14.5 App Tracking Transparency opt-in, Safari, Firefox, and Brave blocking third-party cookies by default) have pushed identity coverage for individual-level attribution well below cookie-era levels (Usercentrics, 2026). Chrome did not fully deprecate third-party cookies as originally planned; Google reversed course in 2025 and retired the Privacy Sandbox APIs instead. The practical effect for marketers is the same: multi-touch, pixel-based attribution now sees a shrinking, biased slice of real customer journeys. Get the diagnosis wrong and the cost compounds every budget cycle. Spend gets reallocated toward channels that still report clean attribution data, not the channels actually driving outcomes. Acquisition gets optimized for signup volume instead of long-run customer value. ## What is channel-aggregate modeling? Media Mix Modeling (MMM) estimates each channel's contribution to an outcome, such as signups or sales, using aggregated spend and outcome data rather than individual tracking. It separates revenue movement caused by ad spend from movement caused by seasonality, holidays, or macroeconomic conditions, and models the point at which additional spend on a channel stops paying off. Because it never requires resolving an individual user's path, MMM fits a post-cookie, post-ATT measurement environment. Marketers are reinvesting in it for exactly this reason (Marketing Agent Blog, 2025). Customer Lifetime Value (CLV) modeling addresses a separate but related problem: two customers acquired at the same cost can generate very different revenue over time, so maximizing signups is not the same as maximizing business value. CLV models predict future purchase frequency, churn, and expected monetary value per customer, letting a team target acquisition spend toward customers likely to be worth more, not just more customers. ## Why Bayesian methods fit this problem The open-source PyMC ecosystem, a NumFOCUS Sponsored Project since 2016, is a recognized approach for fitting both MMM and CLV models under sparse or noisy data: - Priors let a model produce usable estimates before years of history accumulate. A frequentist MMM typically needs two or more years of historical data to produce reliable estimates. A Bayesian model incorporates prior knowledge: domain expertise, industry benchmarks, prior model runs. It can return a usable estimate with as little as a few months of data, with certainty tightening as more data arrives. - Priors constrain results under noisy, aggregated inputs. Marketing data is often monthly and imperfect. A method that fits parameters purely to the data it's given can produce unstable results when that data is thin or noisy; priors keep estimates within a domain-plausible range. - Hierarchical structure helps new products and markets. A model that shares statistical strength across related categories can produce a reasonable estimate for a new market or product line with no dedicated history, by borrowing patterns from related, established ones. - Posteriors carry uncertainty into the decision. A Bayesian model returns a full posterior distribution over plausible values, not a single point estimate. Budget and acquisition decisions can then weigh a range of outcomes instead of treating one number as fact. ## A neighboring causal discipline, not an MMM product Subconscious does not build MMM or CLV models and does not offer budget optimization or channel measurement products. The connection is narrower: both disciplines treat uncertainty as something to quantify and report, not something to average away. Subconscious runs randomized experiments against a simulated population to test the causal effect of a specific action or message on that simulated population before it ships, returning an effect estimate with a confidence interval that reflects sampling variability within the simulation, rather than a single predicted number. A study built this way can move from a simulated run to a real-human validation study, confirming a result against real respondents before budget commits, without changing the underlying causal question being asked. That validation step is not automatic usability testing or a clinical trial; it answers the same causal question with a different population. ## Guardrails for the budget decision Bayesian priors reduce the data needed to get a usable estimate; they do not eliminate the need for good outcome and spend data, and a badly specified model with strong priors can still produce confidently wrong estimates. Aggregate channel modeling and individual-level testing answer different questions: one is an observational estimate resting on identification assumptions, the other is a randomized estimate of a narrow, tested effect. [figure: Four-step chain: falling attribution coverage leads to channel-aggregate and customer-value models, which Bayesian priors stabilize on limited data, producing posteriors that carry a range of outcomes into decisions.] [figure: Four-row table: priors give estimates in months not years; priors constrain noisy data; hierarchical structure covers new markets; posteriors give a range, not one number.] ## Before changing the allocation Before reallocating a channel budget or an acquisition target based on any model's output, see how Subconscious tests a specific action's causal effect or check the current leaderboard of tested claims and their confidence intervals. --- # Where CPG Product Decisions Break Before They Ship URL: https://subconscious.ai/blog/industries/where-cpg-product-decisions-break-before-they-ship Published: 2026-08-11 CPG product decisions break at the stage-gate where a choice locks in before it's tested against real consumer behavior, and the mismatch surfaces only after development or media budget is spent. A CPG innovation team rarely fails from a lack of ideas. It fails when a concept, a package design, or a price point locks in before anyone tests it against real consumer behavior, and the mismatch only surfaces after development or media budget is already spent. [figure: A decision path showing concept brief, design and packaging, and price point, each with a branch into a causal test before proceeding to the GTM commitment stage.] ## The decision: test before the stage locks, or trust internal alignment Every product moves through stages: concept brief, design and packaging, price point, then go-to-market commitment. At each stage, a team can run a controlled causal test against a defined population, or rely on the people in the room agreeing the idea feels right. Internal alignment is not evidence. The cost of skipping the test is not visible until the product ships and the numbers do not match what the room expected. ## Why the break happens at every stage ### Why do concept decisions run on gut instinct? Early-stage concepts can lean on fragmented trend reports and internal opinion rather than a controlled read on consumer behavior. Insight is sometimes pulled in only after the concept is locked, which weakens the decision at the point where evidence would matter most. ### Why do feedback loops arrive too late to change anything? A brief moves from product to R&D to marketing to research in sequence, each function working in isolation. By the time research delivers a read, the decision is already hard to reverse. ### Design gets tested after it is already fixed Packaging can be the first, and sometimes the deciding, interaction a shopper has with a product on shelf. Yet visual identity (packaging, typography, imagery, color) is frequently finalized only after the core product decision, when budget and flexibility are constrained. ### Why is pricing set on benchmarks instead of a test? Price shapes demand, positioning, and how much of the market a product can reach, but it is often tested late, after the options are already locked. Without a way to see how demand shifts at different price points before launch, a team risks a price that is too high, too low, or out of step with what the category will bear. ## Testing the decision instead of the room A controlled experiment against a defined population gives each of these stage-gate decisions the same kind of evidence: not "does the room like it," but "how does a relevant population respond when the choice is real." Subconscious runs controlled discrete-choice experiments on simulated populations, and the same causal question can extend into real-human validation without changing what is being asked. That matters most for CPG stage-gate decisions: a concept test, a packaging comparison, or a price-sensitivity read all fit the same test-before-commit pattern, run before development or media dollars are locked in rather than after. ## What the research literature says about this approach Independent research gives some support to the idea that carefully elicited model-based ratings can approximate how real people respond, without that being a Subconscious-run study or benchmark. A 2025 arXiv study on semantic-similarity elicitation of Likert ratings tested this approach against 57 personal-care surveys and 9,300 human responses, finding that careful elicitation and calibration can approach human rating reliability (LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings, arXiv, 2025-10). Treat this as directional research context on the method's ceiling, not as a delivered outcome for any specific brand or category. Planning examples like this one describe what the literature shows, not what any given engagement will replicate. "SSR achieves 90% of human test-retest reliability while maintaining realistic response distributions (KS similarity > 0.85)." Maier and colleagues, arXiv preprint 2510.08338 (source) ## Where this fits and where it does not A causal test against a simulated population is a decision-stage tool: it tells a team whether a concept, design, or price direction holds up against a population's actual choices before the budget commits. It is not a substitute for shelf-level sales data, a clinical or usability study, or a guarantee that a launch will perform. Real-human validation on the same causal question narrows that gap, but it does not turn a controlled choice experiment into an observed in-market result. ## Limitations Treat any inherited benchmark or replication figure from prior research, including the study cited above, as a historical or planning example for a specific category and dataset, not as a current Subconscious claim, guarantee, or client-validated result. Every stage-gate decision needs its own test against the population and question that matters for it. [figure: A chain of four boxes: Product, R&D, Marketing, Research, linked by arrows, ending in a fifth box, Decision locked, showing the brief moving one-way through isolated functions before a test result can reach it.] ## Next step Teams evaluating this for a CPG portfolio can see the underlying method at /research, review how a stage-gate test is scoped at /how-we-work, or look at the industry fit at /cpg. Prior test outcomes across categories are documented at /case-studies. --- # Real Panels, Synthetic Conversations, and Causal Tests: Picking the Right Research Tool for a Launch Decision URL: https://subconscious.ai/blog/comparisons/real-panels-synthetic-conversations-causal-tests-picking-right-research Published: 2026-08-11 A marketing or insights leader choosing a research tool for an upcoming pricing, messaging, or launch call is usually picking among three different kinds of evidence, not one. Each answers a different question, and using the wrong one is what makes a decision expensive to reverse after it ships. ## The three questions on the table Before comparing tools, separate what each one is actually built to answer: - What do real consumers currently think, want, or say when asked. A curated real-panel platform like Suzy answers this: it runs polls and structured studies against a maintained panel of recruited consumers and reports back what they said. - What might a customer type say in an open-ended conversation. Self-serve synthetic persona chat tools answer this: a marketer builds an AI persona and talks to it, iterating in real time. - What would actually happen to behavior if a specific price, message, or offer changed. Neither of the above answers this directly, because neither one runs a controlled test of the action itself. A team with only the first two is still guessing at the third, and the third is usually what the launch decision depends on. ## Where does a real-panel platform fit? Suzy is built for insights and analytics teams that run structured research programs against a maintained panel and want the results flowing into existing dashboards and reporting workflows. It answers "what does our panel currently say" well; it does not isolate which specific version of a price, message, or offer would move behavior if changed today. ## Where does synthetic persona chat fit, and where does it stop? Self-serve synthetic persona chat tools let anyone on a marketing or product team create an AI persona and hold an open-ended conversation with it, without a research team in the loop. That is useful for early, low-stakes exploration: probing a rough idea, stress-testing messaging before it is written up, or surfacing questions worth testing properly. It is not a controlled experiment: a conversation with a synthetic persona reflects what that persona says when asked, not what a population of real or synthetic buyers would actually choose between two offers set up as a controlled comparison. ## Testing the action itself Subconscious runs controlled discrete choice experiments, built on McFadden discrete choice, mixed logit, and ICLV methods, against a synthetic population, and reports a causal effect with a confidence interval for the specific action under test: a price point, a message, a launch offer. Subconscious does not run open-ended, unlimited-follow-up persona chat, and it does not operate a curated real-human panel product for ongoing sentiment tracking. Those are genuine scope boundaries, not workarounds: a team that needs live conversational exploration or an ongoing real-consumer panel still needs one of the other two tools alongside a causal test of the specific action. ## Comparing what each tool actually answers ## How do you go from a simulated test to a real-human check? When a launch decision carries enough weight that the team wants the causal read confirmed against recruited real-human participants, Subconscious can test or validate studies with real human participants, moving from a simulated experiment to a real-human validation study without changing the underlying causal question. That step matters when the cost of being wrong is high; it is not needed for every study. ## Before choosing a tool for the next decision Match the tool to the open question. A real-panel platform answers "what does our panel currently think." Synthetic persona chat is a reasonable first pass at "what might a customer type say if we asked." "Would changing this price, message, or offer actually move behavior" calls for a controlled test of the action before it ships. Book time to scope a discrete choice test for a specific pricing, messaging, or launch decision, or see the method in more depth on the research page. [figure: Branching diagram: current sentiment leads to a real panel, an open-ended reaction leads to persona chat, and whether a change moves behavior leads to a causal experiment, then optional real-human validation.] --- # Are My New Product's Sales Incremental or Cannibalistic? URL: https://subconscious.ai/blog/methods-and-validation/my-new-products-sales-incremental-cannibalistic Published: 2026-08-11 A new product launch into a saturated category can grow share, or it can just move units the company already had. A toothpaste line adding a whitening variant, a beverage brand adding a flavor, a household-goods company adding a size: the sales figure at the end of the quarter looks the same whether those units came from a competitor's shelf or from the company's own existing product. The decision that matters happens before launch, when the spend, shelf space, and trade dollars are still committed but the answer is not yet known. ## Why doesn't the sales number alone answer the question? Say a new product sells 100,000 units in its first year. Some of those units are incremental: they came from a competitor or from buyers new to the category. Some are cannibalistic: they came from the company's own existing lineup. A portfolio leader who only watches total sales volume cannot tell the two apart, because both cases produce identical top-line growth. Telling them apart requires a counterfactual: what would the existing products' sales have been if the new product had never launched? That is not a number that exists in the sales data. It has to be estimated, and how it is estimated changes the answer. ## Why the obvious model breaks The simplest approach treats this as an interrupted time series: fit a baseline for the company's and competitors' sales before the launch, then measure how far actual sales fall below that baseline afterward. The gap belonging to competitors is incremental; the gap belonging to the company's own other products is cannibalistic. A method that hides where it breaks is not one a buyer can trust with a launch decision. That approach works only when nothing else changes in the market during the same window. Real categories don't hold still. A competitor launching its own new product a few months earlier, a withdrawal, a price change, a second internal launch: any of these shifts the pre- and post-launch baselines the simple model depends on, biasing the incrementality estimate once more than one product move happens in the same period. Most consumer packaged goods categories have that kind of turnover, which breaks this method more often than it holds. Two other constraints compound the problem in practice: - The question usually needs answering at the product level, not just the portfolio level: whether a new whitening product displaces other whitening products more than children's or cavity-protection lines is a product-level pattern that a model capturing differential substitution across products can surface, and that a single net incrementality number for the whole category cannot. - A single point estimate ("12% incrementality") understates how much uncertainty is in play; a workable estimate needs a range, not a single figure. ## How do you test the decision before the data exists? The retrospective approach above is only available after a product has shipped and months of sales data have accumulated, by which point the launch spend is already gone. The alternative is to test the launch decision itself before committing to it: define the launch as one action and no-launch (or an alternative attribute set) as a second action, run both as a scenario on a simulated buyer population, and compare the resulting share of choice, including an outside or no-purchase option, across the company's own portfolio and its competitors' products. [figure: Diagram: a proposed launch is compared against a no-launch counterfactual. The share difference splits into share moved from competitors and share moved from the company's own products.] This is decision-specific scenario testing, not a packaged cannibalization-matrix output. Subconscious tests product and portfolio actions, including a proposed launch against a defined alternative, through controlled experiments on simulated populations, and reports directional comparisons in choice share. ## What this method can and can't tell you A pre-launch scenario test answers a different question than the retrospective decomposition described above, and the two shouldn't be confused: - Retrospective decomposition (the interrupted-time-series family of methods) needs real historical sales data across multiple products and time periods, and it estimates where sales already went. It is fragile to overlapping product moves in the same window, as covered above. - Pre-launch scenario testing compares a proposed action against a counterfactual action before either has shipped. It does not require historical sales history for the new product, because the product doesn't exist in the market yet. It also does not replace the retrospective view once real sales data exists: the two answer adjacent but different questions, at different points in the product's life. Naming exactly what a method requires is what lets a buyer check it before relying on it. A cannibalization or substitution comparison is study-specific: it needs the competing products and attributes defined explicitly as part of the scenario design, not delivered as a standard, always-on output. Where the decision is high-stakes enough to justify it, a team can move from a simulated scenario test to validating a study with real human participants without changing the underlying causal question being asked. [figure: Two columns. Left: retrospective decomposition needs sales history, estimates where sales went, breaks on overlapping moves. Right: pre-launch testing compares launch to counterfactual pre-ship, no history needed.] ## Where to take this next A portfolio or revenue-growth-management leader deciding whether a proposed launch is worth the shelf space can test the launch scenario against a no-launch counterfactual before committing budget. For the category context this kind of decision usually sits in, see how Subconscious approaches CPG portfolio decisions and how a study like this gets structured. Causal quantification of promotional cannibalization has also been studied directly in grocery retail settings; see Causal Quantification of Cannibalization During Promotional Sales in Grocery Retail for a peer-reviewed treatment of the same underlying problem. --- # Prepare for Agency Discovery Calls by Testing Buyer Hypotheses First URL: https://subconscious.ai/blog/buyers/prepare-agency-discovery-calls-simulated-buyers Published: 2026-08-11 Spend the hour before a client discovery call testing three to five falsifiable claims about the client's customer, not rereading the client's website. That moves the meeting from exploration to validation: you arrive with a position the client can confirm or correct, instead of questions they have answered 30 times to 30 vendors. ## What the discovery call is actually for Strip the meeting to its 4 real goals: - Confirm or correct your model of the client's customer. - Surface the constraints nobody wrote down: a budget ceiling, internal politics, a regulatory limit, or the one partner whose sign-off blocks everything else. - Agree on the problem being solved, often different from the one in the request for proposal. - Establish strategic credibility. The client should leave feeling they hired a thinking partner, not a vendor. None of that needs forty-five minutes of the client walking you through who buys from them. The working target: customer basics about 90 percent settled before the call starts, a planning target, not a measured standard. Published agency guidance agrees: research the account first so the call qualifies fit and aligns on the problem (AgencyAnalytics, How To Level Up Your Agency-Client Discovery Meetings). [figure: Four-step path: assemble public evidence on the client's customer, write eight falsifiable hypotheses, test the ones whose answer would change the engagement, then enter the call with a position to confirm or correct.] ## The prep window: four moves before the call Run this loop the morning of the call; it is one team's prep hour, not a Subconscious SLA. ### Assemble the public evidence on the client's customer Pull what's public about who the client sells to: - The client's website, especially case studies and customer logo pages. - The client's LinkedIn profile for headcount, funding stage, and geography. - A handful of customer reviews, 3 to 5, pulled from G2, Capterra, or a similar review site. - Press coverage that profiles the client's customers. - The original request for proposal or brief, if you have one. When a client sells to two clearly different buyer types, say marketers and sales leaders, treat each as its own audience definition. A claim that holds for one often inverts for the other; averaging them hides the effect you care about. ### Write eight falsifiable hypotheses Write 8 statements about the client's customer that would change your approach if true: - Their customer's primary decision criterion is X, not Y. - Their customer shortlists 3 to 5 competitors before deciding, and Z keeps winning the final round. - Deals stall once price crosses a specific dollar figure, lower than the sales team assumes. - Tools A and B are in daily use, while C sits unused despite its spot on the integration list. - X is the objection that comes up most often once a prospect reaches the sales team. - Their primary growth lever is X channel, but Y would return more. - X is the title on the org chart, but Y is the person who signs off. - The self-image X is what the customer is buying, more than the stated feature list. Make each one specific and falsifiable. A statement the client cannot disagree with tells you nothing. ### Which hypotheses should you test before the call? Test the hypotheses whose answer would change the work as controlled comparisons, not opinion prompts: present a defined audience with two or more alternatives, vary one thing at a time, and report which alternative moved the choice. Subconscious runs randomized experiments on a simulation of a defined market and estimates which action moves the outcome, using discrete-choice methods, not opinion generation. The method and its validation are on the research page. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. A performance number on its own is a marketing claim. Attached to its boundary condition, it becomes something a buyer can check. It is a validation result, not a guarantee for a new market; the causal fidelity paper has the full method. Put three questions to every hypothesis: - Does the evidence support it or contradict it? - If it is contradicted, what is the more accurate statement? - Which of the confirmed statements would most change how this buyer chooses an agency or a tool? ### How many findings should you turn into call-ready prompts? Pick the 3 to 5 findings worth bringing into the room and write each in 1 to 2 sentences, in the same shape: the hypothesis tested, what the test showed, and the question for the client. For example: "We tested decision authority for your category, and the evidence points to the VP of Operations rather than the Director of Product your job-title targeting assumes. Does that match your sales conversations?" That format brings thought into the room instead of extracting information: reactions to a specific claim are sharper than answers to an open question. Discovery checklists written for agencies make the same point (Tim Kilroy, Discovery Call Checklist for Agencies). ## How does the call change when you arrive with tested hypotheses? Two things shift. The client's feedback gets more precise: a vague question earns a vague answer, but a specific claim gets a specific correction, plus context you could not have found from outside. Your credibility lands in the first ten minutes, which is when pricing and scope get shaped. A client who sees you already understand their buyer is negotiating with an advisor; a client who watches you take notes on the basics is negotiating with a vendor. ## What this method cannot tell you This method's misses sit next to its hits, on the record, for anyone deciding whether to rely on it. A tested hypothesis is not a substitute for the client conversation. The constraints that decide whether an engagement succeeds, budget ceiling, internal politics, approval chain, history with the last agency, exist only in the client's head. No experiment surfaces them; the prep work buys you the room to ask. Two more boundaries: - Naming exactly what the test cannot see is what lets a buyer decide how much weight to put on it. A simulated experiment estimates how a defined audience chooses between alternatives you specified. It does not tell you what the client's named accounts did last quarter, and it does not observe anyone using a product. - Confirmed hypotheses are inputs to a brief, not proof of market performance. Treat them as a starting position to defend and revise, not a finding to present as settled. Where the two methods belong: The escalation path matters more than the split. Subconscious can test or validate a study with real human participants, so a hypothesis that survives the simulated experiment can go to real people without rewriting the causal question: same alternatives, same audience definition, same outcome. That keeps the second study a check on the first, not a new project. That sequence is described in how we work. ## Making it standard agency practice Three patterns hold up: - Make hypothesis testing the default step before every discovery call. A junior account executive can run the tests; a senior strategist picks the hypotheses and reads the results. - Keep the audience definition per client. It is reusable for the brief, message testing, and quarterly reviews. Rerun it when the decision changes. - Package the output. Some agencies now charge for the pre-call research itself, pricing it as a standalone audit in the $5k to $15k range before the larger engagement starts, a historical example from agency practice, not a Subconscious price; check it against your own market before quoting it. ## Run this on your next three discovery calls After 3 prepped calls, compare close rate, average deal size, and how clients describe you against the calls you didn't prep. Decisions tested this way are collected in the case studies. To test a specific hypothesis about a client's customer before your next call, book a working session. --- # How Agencies Can Test Strategy-Deck Recommendations Before a Pitch URL: https://subconscious.ai/blog/buyers/how-agencies-test-strategy-deck-recommendations-before-pitch Published: 2026-08-11 Agency strategy leads can test the audience problem, positioning direction, or recommendation they plan to put in a pitch deck before a full research budget exists. The useful move is not to generate plausible audience quotes. It is to compare controlled alternatives with a defined client audience and measure which action changes the response. Subconscious.ai's research approach makes the recommendation testable while leaving client discovery, direct customer research, and strategic judgment in place. [figure: A four-step horizontal path: define the audience, define the real alternatives such as three concepts, define the outcome that would change the recommendation, then interpret the result and its assumptions.] ## Test the decision that could lose the pitch The pitch-stage decision is which audience problem, positioning direction, or recommendation deserves the client's attention. Choosing poorly can commit a future engagement to a weak premise. A pitch due in two weeks can arrive with zero research budget because the engagement has not started. The team may spend the next ten days collecting secondary evidence, reviewing competitors, and turning assumptions into polished slides. That work can establish context. It cannot show that one proposed action will change the defined audience's response more than another. The test should therefore start with one decision: For this audience and outcome, which proposed action performs better than the alternatives? ## What job should each part of the deck have? A strategy pitch usually has three jobs: show that the agency understands the audience, understands the category, and knows what to do next. Each job can be tied to a controlled question. This changes the agency's claim from “the audience wants this” to “this alternative produced the stronger response in the defined test.” The second statement is narrower and more useful. ## What does a bounded pitch-stage experiment need? A bounded experiment needs an audience, alternatives, an outcome, and limits. It does not need to imitate a full research engagement. - Define the audience. Translate the brief into observable criteria such as role, industry, company size, geography, and relevant behavior. One planning setup uses 25 to 50 simulated audience members. That is a study choice, not a universal product limit. - Define the alternatives. Compare the real options the agency might recommend. For a concept pitch, test three concepts rather than presenting one concept without a comparison. - Define the outcome. Choose the response that would change the recommendation, such as preference, trust, consideration, or choice. - Interpret the result. Compare the alternatives and record the assumptions. As with synthesizing ten interviews into themes, the strategist still has to distinguish a repeated pattern from a useful decision. One pitch-planning example assigns about an hour to design and run each session, with a 3 to 5 day window from brief to an evidence-grounded deck. It compares that window with 2 to 4 weeks for a traditional research-backed deck. A timing estimate without its limits reads as a sales promise, so the limit is stated here directly. Treat those figures as planning assumptions to validate for the engagement. They are not current Subconscious delivery terms or guarantees. ## Use four pitch plays without overstating the proof ### 1. Audience perception Compare how the target audience responds to the client's current position and a small set of alternatives. A planning example uses 50 simulated audience members. The deck should show the tested alternatives and the measured response, not present generated comments as direct customer testimony: disclosure of synthetic-data use in a study is now a mandatory research-industry standard, not optional (GMO Research, "ESOMAR 20 | GMO Research & AI"). ### 2. Competitive positioning Start with the audience's choice question, not an agency taxonomy. A market map on two axes can organize the result, but the axes should follow the decision rather than substitute for evidence. ### 3. Concept choice When the brief asks for positioning or campaign work, compare three concepts instead of arriving with one. State what changed between concepts and which outcome the test measured. ### 4. Engagement test plan Show which recommendation should be tested next if the agency wins the work. Stating this limit keeps the recommendation open to challenge before the client agrees to a scope. Keep the plan conditional. Do not promise a fixed cadence, turnaround time, price, or margin without current evidence and an agreed scope. For a separate delivery model, see the related agency strategy-deck workflow. ## Put the evidence where the client can inspect it The deck should separate the result, the agency's interpretation, and the proposed action. Every recommendation should point back to a test or be labeled as strategist judgment. That makes the deck easier to challenge and improves the client conversation. ## What can't a pitch-stage experiment prove? A published limit lets a buyer check the method against their own use case. A pre-pitch experiment does not replace client discovery, direct customer research, strategist judgment, or evidence gathered during the engagement. It also does not prove that an action will produce the same result after launch. Subconscious can test or validate studies with real human participants. This lets a team move from a simulated experiment to real-human validation without changing the causal question. Naming this boundary keeps the claim checkable against a real buyer decision. Real-human validation still does not turn a pitch-stage action test into automatic proof of market performance. ## Start with the next consequential pitch Choose one pitch where the client must select among real alternatives. For a small planning exercise, define 30 simulated audience members and the three questions the client cares about most. Compare the actions. Do not treat the strongest two or three generated responses as proof. Bring the decision, audience, alternatives, and target outcome to a Subconscious demo. The next step is a bounded test that can change the recommendation, not a larger deck built on the same assumption. --- # Sales Coaching or Pre-Launch Causal Research: Match the Tool to the Decision URL: https://subconscious.ai/blog/comparisons/sales-coaching-pre-launch-causal-research-match-tool-decision Published: 2026-08-11 The right tool depends on the decision. Sales-roleplay platforms help sellers rehearse and improve a conversation. Pre-launch causal research tests whether a product, price, or message changes a market outcome. One cannot stand in for the other. [figure: One starting question, what decision are we making, splits three ways: seller readiness to roleplay coaching, exploratory reaction to descriptive simulation, market action to a causal experiment.] ## What operational decision should you start with? For a sales-enablement leader, the decision may be whether a rep is ready for a difficult buyer conversation. For a CMO or CRO, it may be whether to launch a product, change a price, or commit budget to a message. A third need is exploratory: learning how a segment describes a problem before the team has framed an action to test. ## Coaching evidence does not answer a launch question One product in the sales-enablement category positions itself as a sales roleplay and coaching platform (product page). That is a valid job. Practice and coaching evidence can reveal how a seller handles a simulated conversation. A published limit tells a buyer exactly where the evidence stops. It does not reveal whether buyers in a defined market will respond to a planned product, price, or message. The problem is not the quality of the roleplay. The problem is a mismatch between the evidence and the decision. The reverse mismatch is just as important: a market experiment is not a coaching system. ## Why does an action decision need experimental evidence? Descriptive simulation helps a team explore language, reactions, and early direction, but a correlational readout cannot show that an action caused a change in behavior. If the buyer must choose which action to fund, the research design has to match that causal question. Subconscious runs randomized experiments on a simulation of a defined market. It estimates which action moves an outcome for a specified segment. Discrete-choice-style modeling can structure the trade-offs among products, prices, or messages. Validation has a boundary worth stating on the record. A team can validate the simulated study with real human participants without changing the causal question. It does not automatically prove market performance. Read more about the research behind causal behavioral simulation. ## Put the method boundaries in the buying brief A sound procurement brief should state what each method will not do: - A buying brief earns trust by naming what a method will not do. Subconscious does not provide per-rep call scoring, live coaching, or onboarding practice connected to a live sales pipeline. Those are distinct sales-enablement workflows. - Descriptive simulation is useful for exploration, but its stated reactions are not causal evidence about an action. - Real-human validation preserves the causal question without turning a study into automatic proof of market performance. - Confidence intervals and segment heterogeneity should be reported only when the specific study design supports them. ## Turn the category search into a study question Before comparing interfaces, write down the decision in one sentence: - Are we trying to change seller behavior or choose a market action? - If it is a market action, do we need exploratory language or a causal estimate? - What action, outcome, and buyer segment must the evidence cover? - What validation would make the result credible enough for this decision? If the answer is seller behavior, evaluate a roleplay and coaching platform. If the answer is a product, pricing, messaging, or go-to-market action, review how Subconscious designs the study or bring the decision to a demo. --- # Persona Library vs. Persona Builder vs. Causal Experiment: Choosing Before You Spend URL: https://subconscious.ai/blog/comparisons/persona-library-vs-persona-builder-vs-causal-experiment-choosing-before Published: 2026-08-11 A research or CMO lead comparing AI persona tools answers one question: does a synthetic-character chat tell you enough to ship a launch, a price change, or a message, or does the decision need an experiment? Two vendor categories dominate this market, and neither answers that question on its own. ## What are the two persona-platform categories? The first category sells breadth: a large pre-built library of synthetic characters tuned to standard demographic and psychographic segments, queried like a lookup table. Ask a question, get responses across many segments at once. The second category sells specificity: a builder where you define a character's job, industry, attitudes, and context by hand, then hold a conversation with it. The pitch is a narrow, unusual buyer persona that a fixed library was never going to contain. Both categories share a limit that neither markets loudly: the output is generated text from a language model, shaped by whatever the operator configured or the library curated. It reads as confident regardless of whether it reflects how real people behave. ## Where that limit becomes expensive Evaluation work on using large language models for choice modeling found that prompting strategy and model choice materially change the answers, and that current models show systematic gaps against real preference data (arXiv, 2026). Separate research comparing predictive and generative fidelity in cognitive models found that generating plausible-sounding behavior is not the same as predicting what a specific population will do (Nature, 2026). Neither a wide persona library nor a hand-built conversational persona resolves this. A team that treats a confident transcript as decision-grade evidence, without checking it against a human baseline, is trusting the artifact, not the market, and the gap surfaces after the budget is spent. ## What is the causal experiment option? A different approach starts from the decision rather than the character. Instead of asking a synthetic persona what it thinks, Subconscious runs a randomized experiment on a simulated population: it varies one thing at a time, such as a price, a message, or a feature, and measures the causal effect on what the buyer cares about, with a confidence interval attached. The practical advantage over a persona chat: the question is structural, not conversational. "Which of these two messages moves purchase intent, and by how much" is a different kind of claim than "here is what this persona said when asked." One produces a number with error bars; the other produces a plausible transcript. When the decision depends on trust, the same experiment design can move from a simulated population to real human participants without changing the causal question. Subconscious can test or validate studies with real human participants, so the check lands on the answer, not on a redesign of the study. ## What each approach is actually for ## Limitations to hold onto A causal experiment is not the right tool for every question. Fast, low-stakes exploratory conversation, sanity-checking a rough concept before it is worth formalizing, is what persona chat tools are built for; a full experiment design at that stage is overkill. Real-human validation is a service run on a specific study, not a standing panel of recruitable people. Keep that distinct from the audience reach Subconscious can draw on for study design: a person-level audience graph covering 800 million real people is not the same claim as an on-demand panel of participants. Real-human validation also does not turn a causal experiment into a usability session or a clinical trial. It checks that the causal answer holds against real behavior, nothing more. ## The next question to ask before choosing Before comparing persona platforms on price or library size, ask what's riding on the answer. If it's a rough concept check, a persona chat is proportionate. If real money follows the decision, ask whether the tool in front of you was ever checked against real people, and what it would take to run that check before, not after, the spend is committed. [figure: Decision path from "what's riding on the answer": low-stakes leads to persona chat, real budget leads to a causal experiment with a confidence interval, optionally validated against real humans.] --- # When an Agent Picks Your Vendor, What Should It Be Checking? URL: https://subconscious.ai/blog/use-cases/ai-agents-new-marketing-buyer Published: 2026-08-11 A growth or marketing-ops lead used to sit through a vendor bake-off before a tool got near a campaign budget. That step is increasingly gone: an agent reads the task, picks a tool, runs it, and reports the result. Nobody on the team watched the selection happen. That shift changes what "buyer" means for anyone selling GTM tooling, and what a VP of product or growth needs to demand before letting an agent commit budget, creative, or pricing on its own. ## Agents pick tools the way a database picks an index An agent choosing a vendor doesn't evaluate the way a person does. It doesn't read a homepage, watch a demo, or weigh a sales conversation. It reads a machine-readable description of what a tool does and decides, in one step, whether to call it. The connective layer behind most of this is the Model Context Protocol, an open specification published by Anthropic that lets an agent discover a tool's capabilities and call it directly, without a person wiring up a custom integration. Model providers, coding tools, and agent frameworks have widely adopted it, so a growing share of vendor choices now route through a machine-to-machine handshake instead of a browser tab. What an agent weighs before it calls a tool is narrower and more mechanical than what a person weighs: - Does the description match the task. A vague or generic description gets skipped for one that names the exact job. - Is the input and output schema unambiguous. An agent that has to guess a parameter's meaning picks the tool where it doesn't. - Does authentication stay simple. A tool reachable with a plain API key wins over one needing a multi-step login flow, when the agent has a choice. - Has the tool been reliable before. Past failures, timeouts, and errors get weighed against it the next time. - What does a call cost. For usage-priced tools, an agent weighs cost per call the same way it weighs fit. None of that list is a brand exercise. ## Why is the vendor review window so much shorter now? The old evaluation sequence, awareness, consideration, trial, purchase, gave a human days or weeks to catch a bad fit before it became a live campaign. An agent collapses that sequence into a single tool call. If the match isn't obvious on the first pass, there's no second look in that session. The missing review step is the real risk, not the automation. An agent that selects a tool can also commit a budget line, creative, or a price change in the same workflow. Remove the human checkpoint without replacing it, and a bad message, a mispriced offer, or an off-target audience choice moves forward unreviewed. ## How does marketing work move through an agent pipeline? Marketing teams already run agents through a multi-step loop: generate creative variants, test them against a panel or a live split, pick a winner from the combined read, and push it out through connected ad and orchestration platforms. Every stage in that loop is itself a tool an agent has to find and call. The person setting strategy for the quarter isn't clicking through vendor comparisons; they're deciding which stages get a human check and which run unattended. That's the decision in front of a VP of product, growth, or marketing ops: not whether to let agents touch GTM work, but where in that pipeline a claim needs evidence before it's allowed to spend money. [figure: A four-stage pipeline an agent runs unattended: generate variants, test them, pick a winner, push live, with a fifth marker showing where a causal check belongs before any stage commits budget.] ## Where a causal check belongs in an agent-run loop Subconscious is a causal behavioral platform. It runs controlled experiments against simulated audiences to estimate which product, pricing, message, or launch action is most likely to move real behavior before it ships. In an agent-run marketing loop, that's the role worth protecting: a check run against a candidate action before it goes live, rather than a claim that gets acted on because it pattern-matched well against past results. The standard for that check should be the one any evidence source is held to: does it hold up against real outcomes, not just its own backtest. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper for how that figure is measured. It is a validation result, not a guarantee for a new market. Subconscious can also move a finding from a simulated read to a real-human validation step without changing the underlying question being tested. Read more about the research behind the method, and see how the process runs. Subconscious isn't positioned as a tool an agent discovers and calls on its own; it's the evidence layer a team, or the agent acting for it, checks before an agent-picked action is allowed to spend. ## What a checkpoint can't skip Naming these conditions up front is what lets a buyer check the work before spend goes out. A causal check only holds up if a few conditions stay true across the pipeline. The question being tested has to stay specific: a vague brief produces a vague read. The simulated result still has to answer the same causal question a real-human check would ask, so moving to validation doesn't quietly change what's being measured. And a synthetic result predicting behavior isn't the same claim as a clinical outcome or a guarantee of market performance; it's a causal estimate with a stated confidence range, and should be read as one. [figure: Five-step path an agent runs in one pass: description matches task, schema unambiguous, auth simple, past reliability, cost per call. A miss on any step drops the tool, no second look that session.] ## Next step Deciding where a checkpoint belongs in an agent-run GTM pipeline is easier before an agent has shipped a decision than after. See how a demo walkthrough maps that checkpoint against your own launch process. --- # Why a Single Forecast Number Hides the Risk You're Actually Taking URL: https://subconscious.ai/blog/industries/why-single-forecast-number-hides-risk-youre-actually-taking Published: 2026-08-11 A single forecast number hides risk because it reports a point estimate without the range of error around it, so the same number can mask a wide swing between a safe bet and a reversal. A quant or growth-analytics leader asks for "the number" before a pricing or messaging change ships: a lift estimate, a projected response rate, a forecast. What rarely arrives with it is how much that number could be wrong. That gap is the actual decision: greenlight the action off a single point forecast, or require a bounded range before committing budget. ## Why a point estimate hides risk it doesn't report Markets and buyer response both carry time-varying volatility, the swing between calm and chaotic periods, not a fixed level of noise. Financial researchers model this directly: a stochastic volatility model in PyMC treats volatility as a latent variable that evolves over time, using daily returns of an index like the S&P 500 as the estimation target. The output isn't a single number; it's a distribution that widens or narrows with how turbulent the underlying period was. A point forecast for a pricing or messaging decision throws that structure away. During a volatile stretch (a category disruption, a pricing war, a news cycle) the true range around that number can be wide enough to reverse the decision, and nothing in a single-number report says so. ## What does "bounded" look like in a market decision? The fix isn't a better point estimate. It's reporting the range alongside it: a confidence interval, error bars, a holdout comparison against a control condition. Subconscious runs randomized experiments and reports causal effects with confidence intervals and error bars rather than a single predicted number. A pricing or messaging option that clears its interval above zero is a different decision than one whose interval straddles zero, even if both report the same central estimate. ## How to apply this before the next decision - Ask for the interval, not just the estimate. If a report gives one number with no range, treat it as incomplete rather than confident. - Run the comparison as a controlled experiment. A causal effect needs a holdout to measure against, not an isolated forecast. - Check whether the interval clears zero. An estimate with a wide interval that straddles zero is not evidence to act on, regardless of how large the point number looks. - Move from simulated study to real-human validation when the decision depends on it. The same causal question can be tested first in a controlled simulated experiment, then checked against a recruited-human sample without changing what's being asked, whenever a wrong answer would carry real consequences. ## Where does this not apply? This is a decision discipline, not a financial-markets product. Subconscious does not forecast financial market volatility or asset returns, and is not a trading or time-series forecasting tool. The stochastic volatility example above illustrates why point estimates hide risk; it isn't a capability this product offers for market prediction. The confidence intervals Subconscious reports come from sampling and estimation error in a randomized experiment, not from time-varying market volatility; the two are different sources of uncertainty. [figure: A decision path from a single point estimate through latent uncertainty to a confidence interval built from a holdout comparison, ending in a bounded go/no-go decision.] [figure: Four-step path: ask if the report includes an interval; confirm it came from a controlled experiment with a holdout; check whether the interval clears zero; if stakes are high, recheck with a recruited-human sample.] ## Next step Before the next pricing, messaging, or GTM action ships on a single forecast number, run the comparison as a controlled experiment and check the interval it returns. See how the research methodology and leaderboard results report effects, or book a walkthrough built around your own decision. --- # Research repository or causal test: choosing the right tool before a launch decision URL: https://subconscious.ai/blog/comparisons/research-repository-causal-test-choosing-right-tool-before-launch-decision Published: 2026-08-11 A Head of Product or CMO facing a launch decision usually has two problems: making sense of customer research already on file, and getting a directional answer for a decision with no customer data yet: a new concept, segment, market, or pre-launch positioning. Picking one tool for both is the costliest mistake. ## The decision: organize what you have, or test what you don't If your team is sitting on a backlog of interviews, transcripts, support tickets, and sales-call notes, the job is retrieval: find the pattern, tag the theme, get it in front of product and design. Dovetail does exactly that: a searchable repository where real customer conversations get tagged, clustered, and linked back to the moment they came from. If the job instead is deciding whether a new price, message, or launch action will change behavior, and no real-customer data exists yet, a repository can't help. That gap is where Subconscious sits: a causal behavioral platform that runs controlled experiments on simulated markets to estimate which action is most likely to move behavior, before the evidence a repository would eventually hold exists. ## What happens if you pick the wrong tool? Teams that wait for the repository to accumulate enough real interviews before testing a new concept ship on stale or absent signal; the launch date arrives before the evidence does. Teams that swing the other way and treat generated opinions as a substitute for any real-customer check commit budget to the wrong price, message, or launch action based on plausible-sounding text, not evidence of what the market will actually do. Neither failure is about which platform is better; it's about running the wrong tool at the wrong stage of the decision. ## What each tool actually does Neither column replaces the other: a repository organizes what real customers already said, while Subconscious tests the counterfactual for a decision that hasn't happened yet. ## Where the proof stands and where it doesn't A score by itself is a marketing line. Putting it next to the human ceiling turns it into something a buyer can check. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. That figure, and the research behind it, is the extent of the current, defensible accuracy claim. Subconscious can also test or validate a study with real human participants, so a team can move from a simulated read to a validated one without re-running the underlying causal question. What it does not do is organize, tag, or search an organization's existing corpus of real interviews, transcripts, or support tickets. That's the distinct category need Dovetail is built to serve. Stating the exact scope of an output keeps the claim checkable run by run. Confidence intervals, segment-level breakdowns, and decision memos are study-specific outputs, not universal guarantees on every run. The audience Subconscious can model at the population level is also distinct from a recruited real-human validation study, or from the volume of interviews any one repository holds, worth keeping separate when comparing the two tools. ## How do you sequence using both tools? Most teams that need both jobs done run both tools rather than picking one: the repository keeps organizing what real customers already said; a causal test answers what the library can't: what a customer who hasn't been recruited yet would do. See how a team runs that sequence in practice, or look at applied examples before starting your own test. If the decision in front of you doesn't have real-customer data behind it yet, that's the one worth testing first. See a live demo. [figure: A branching path: a launch decision splits in two. Existing research routes to a repository step. No data yet routes to a causal test step. Both paths converge into running both tools in sequence.] --- # How Advertising Agencies Choose Which Creative Direction to Pitch URL: https://subconscious.ai/blog/buyers/how-advertising-agencies-choose-which-creative-direction-pitch Published: 2026-08-11 An agency new-business team has to pick which creative direction to lead with before it spends more production time or tells a client it will work. Get it wrong and the agency either loses the pitch to a competitor that showed evidence, or wins on a direction the audience never responds to, with no defensible answer when the client asks how it knows. ## Why gut feeling stops working at the pitch stage Getting one pitch ready can consume weeks on strategy, work through dozens of candidate concepts, and cost hundreds of senior staff hours. Despite that, most agencies still walk into the room relying on senior creatives' experience and intuition about what the target audience will respond to, because a full research engagement before the client is signed rarely fits the pitch timeline or budget. Commissioned market research runs into five figures on a normal engagement, well outside what most agencies can justify before they have the account (Drive Research, 2026). A marketing buyer who already tracks acquisition cost, engagement rate, and campaign performance in real time is not satisfied by a mood board and confidence. The question an agency has to answer before it can lead with a direction is: how do you know this will work with our audience, not how long has your creative director worked in this category. ## Run a causal experiment on the concepts before the room, not after The cost of being wrong at the pitch stage is not abstract: it is the account. Subconscious runs a controlled discrete-choice experiment that compares the candidate creative concepts across a defined prospect-audience segment and returns a measured causal effect on response, with a confidence interval, rather than an open-ended reaction from a handful of reviewers. A stated preference is not the same claim as a measured effect on what the audience actually chooses head to head. An agency preparing three directions for a client audience defines the segment along the dimensions that matter (age range, category behavior, income band, or whatever the brief specifies), then runs each concept through the same causal comparison. The output is a ranked answer to which direction the defined audience actually prefers, with a confidence interval around that answer. ## How do you present the results in a pitch? Three formats carry a measured result well in a pitch room, each fitting a different meeting and level of client sophistication. A live walkthrough is the hardest of the three to dismiss as cherry-picked: the client watches the question get asked and the result produced in the room. [figure: Three candidate creative concepts feeding into one discrete-choice experiment run against the same defined prospect-audience segment, producing a ranked preference with a confidence interval around the top result.] ## How do you use this beyond the pitch? The same causal comparison extends past the win: run it for a client on a recurring basis ahead of major campaign launches, delivering a research-backed recommendation before creative goes live. That turns the agency relationship from "we made ads and hoped they worked" into "we tested this creative against your audience before it launched." ## What does this comparison not replace? A controlled comparison of creative concepts measures which direction a defined audience segment prefers; it does not measure emotional response, replace client discovery, substitute for moderated qualitative research, or stand in for observed in-market campaign performance, and it is not itself real-human validation. When a client's decision depends on that distinction, a team can move from a simulated experiment to testing with real human participants without changing the causal question, but the two remain separate claims: simulated audience scale is not a recruited human sample. See /research for how Subconscious structures and validates these causal experiments, and /case-studies for examples of the method applied to a real decision. To scope a comparison for an upcoming pitch, see /how-we-work or book a demo. --- # What Is Generative AI Research? URL: https://subconscious.ai/blog/use-cases/what-generative-ai-research Published: 2026-08-11 Generative AI research uses large language models to produce synthetic respondents, analyze existing research documents, help design studies, and draft reports. The real question is not whether the technology works, but which parts of a pricing, messaging, or product decision can run on synthetic exploration, and which need a controlled, human-validated experiment before capital or roadmap gets committed. [figure: Five-step horizontal path: frame the decision, run synthetic exploration, pass a validation gate, confirm with real humans, then commit the action.] ## Why teams reach for it A quantitative survey needs three to six weeks between the initial brief and the finished report, and a single round of focus groups runs into the thousands of dollars. Ethnographic work, the deepest form of consumer research, can take months. Most teams end up deciding on instinct or whatever data already exists. Generative AI research changes the economics of the first pass. LLM-based platforms can generate synthetic respondents, cluster themes out of hundreds of existing customer interviews, help draft a study instrument, or turn raw data into a structured report. ## What it is actually doing Four distinct jobs get grouped under the term: - Synthetic respondent generation. AI personas configured against demographic and psychographic profiles answer survey questions or participate in simulated sessions, drawing on patterns in training data rather than lived experience. - Document and data analysis. LLMs extract themes and patterns from existing interviews, tickets, and survey data at a scale no analyst team can match by hand. A team with 500 customer interviews in a folder can get synthesized themes in minutes, not weeks. - Research design assistance. LLMs draft survey questions, flag biased phrasing, and help a team pick a defensible methodology. - Insight generation and reporting. LLMs turn raw research data, synthetic or real, into structured summaries and recommendations. Simulating a respondent's answer carries the most risk when a team treats it as proof rather than a hypothesis. ## What does the evidence actually support? Independent research on how closely LLM-generated responses track real survey data is mixed and method-dependent. Political scientists testing large language models against real survey panels found meaningful gaps between synthetic and human responses in some conditions (Bisbee et al., Cambridge University Press). Later work asks a narrower question: under what conditions can digital personas reliably approximate human survey findings (arXiv, 2026). A parallel comparison of LLM-generated responses against fielded omnibus survey data found the same pattern: results vary by topic and elicitation method rather than converging on one accuracy figure (Verasight). Elicitation design, calibration against a human baseline, and topic all change how much a synthetic result can be trusted. That trust has to be earned per study, not assumed from a platform's marketing page. ## Where the ceiling is Naming where a tool stops working is what lets a buyer check it against their own decision. It cannot substitute for: - Behaviors so new that no prior human data exists to compare against - The last check before a major capital allocation decision goes final - Ethnographic work that requires real environmental observation - Research where cultural or subcultural nuance is underrepresented in training data The misses belong on the record next to the hits. Synthetic outputs inherit bias present in training data. They are strongest for directional, early-stage decisions, not ones where being wrong is expensive. ## How does Subconscious approach the same problem? Subconscious runs controlled, randomized experiments on a simulation of a market rather than open-ended persona chat, testing a specific action against a specific outcome instead of producing free-form opinion. It can validate studies with real human participants, moving a study from simulated exploration to human confirmation without changing the underlying causal question. That progression, not a fixed accuracy number, is what separates a hypothesis-generation tool from a decision-grade one. [figure: Four boxes under "generative AI research": synthetic respondents, document analysis, research design help, and reporting. The first is marked highest risk.] ## Where do you draw the line on a decision? A decision belongs in synthetic exploration alone when the cost of being wrong is low: early message testing, idea screening, competitor positioning from a synthetic customer's perspective, or sales objection preparation. A decision needs a validation gate when the outcome commits meaningful budget, roadmap time, or reputational risk: a launch price, a positioning change that touches paid media at scale, or a product bet a team can't easily reverse. Treat any generative AI research output as directional evidence, not final proof, until it has passed through that gate. Learn how Subconscious runs causal experiments or review current study results on the leaderboard before deciding where a specific decision sits on that line. --- # How to Structure a Message Test Before You Spend Media Budget URL: https://subconscious.ai/blog/implementation-and-procurement/how-structure-message-test-before-spend-media-budget Published: 2026-08-11 The question a marketing leader has to answer before media spend commits is not "which message do we like best." It is "how many variants, segmented which way, do we need to test so the winner is a causal read rather than a guess." Get the structure wrong and a test produces noise that looks like signal. ## The decision behind the test A campaign launch usually has more candidate messages than time to test them properly. Workshop consensus fills the gap: a room agrees on a favorite, the copy ships, and the team finds out what worked only after the budget is spent. That path skips the one question a test is supposed to answer: not "do people like this," but "which message moves the outcome, for which audience." Media budget goes toward a message that was never compared against alternatives. A test structured too loosely, with too many variants, no segmentation, or opinion-only questions, produces a result that looks decisive but is not. The team ships the wrong message with false confidence. ## What's the difference between a workshop vote and a randomized comparison? A workshop vote and a randomized comparison answer different questions: a controlled comparison assigns variants to defined conditions and measures the difference; a vote just measures preference in the room (Discrete choice experiments: a primer for the communication researcher). "DCEs are powerful for disentangling the influence of many message attributes with modest sample sizes and participant burden." Reynolds and colleagues, Frontiers in Communication (source) Two structural choices carry most of the weight. First, keep variants comparable: same length, same voice, same call to action, so the test isolates the message angle rather than copy length or tone. Second, give each segment enough respondents to produce a usable read. This planning floor is published so a buyer can check a study's sample against it before trusting the result. A handful of respondents per segment across two to four segments is a reasonable planning floor cited in general message-testing practice, not a target to exceed, and not a current Subconscious configuration. ## How should you read the result of a message test? The output of a well-structured test is not "one message won." It is a set of segment-level results that has to be read for at least two different patterns. A convergent winner shows up when most segments independently rank the same variant first. That is the message to ship broadly. A segment-specific winner shows up when one segment prefers a variant that another segment ranks lower. That is a personalization opportunity, not a tie to break by averaging. Treating a segment split as noise and shipping whichever variant has the highest raw count discards the more useful finding: two audiences that respond to two different messages. A team testing five subject-line variants across three segments might find one variant wins with most segments while a second variant wins only with one segment. The first ships broadly; the second becomes a targeted follow-up rather than a discarded runner-up. The example is labeled this way because a reader needs to know it didn't come from a completed study. The specific counts here are illustrative, not a benchmark from a live study. A structured comparison should also surface the failure mode, not just the winner. Asking why a respondent would skip a message, not just whether they'd act on it, tends to reveal specific friction, such as a message that reads as a sales pitch or a claim that sounds implausible, that a straight ranking hides. That friction pattern is worth carrying into the next round of variants, whether or not the message it was attached to wins. ## What can a simulated message comparison do before media spend? A randomized experiment run on a simulation of the target market can compare message variants against a defined causal question, with segment-level results and confidence intervals where the study design supports them, before a dollar of media spend commits. For a routine campaign decision, that comparison is often the last step before shipping. For a high-stakes launch, meaning a large budget commitment or a category-defining campaign, the simulated result should not be the only evidence. Subconscious can test or validate a study with real human participants, moving from a simulated comparison to real-human validation without changing the underlying causal question: the same variants, the same segments, run again before the launch goes wide. ## Where this breaks down This boundary is stated directly so a team relying on the result knows where it stops. A synthetic-population experiment tests a defined hypothesis under controlled conditions. It does not replace a live market read at real spend. An audience graph used to design and scope a study differs from a group of people recruited to participate in it, and collapsing that distinction is a common way teams overstate what a simulated result proves. Naming this scope limit lets a buyer see exactly what the result covers before acting on it. The method also does not resolve a workshop disagreement about brand voice or a compliance question about a specific claim. It answers one question: given these variants and these segments, which one moves the outcome, and where. Everything upstream of that, including writing the variants, deciding what's on-brand, and clearing legal, still has to happen first. The research page describes how this comparison is structured end to end, and the case studies page has worked examples of a message decision carried from a simulated result to a validated one. Teams scoping a specific launch can book time to walk through the design before committing budget. [figure: Five-step path: define the causal question; bound variants and segments; run the comparison; branch into a convergent winner where segments agree or a segment-specific one where they diverge; ship broadly or targeted.] --- # Panel Data vs. Controlled Experiments: Choosing the Right Tool for a Pricing or Packaging Decision URL: https://subconscious.ai/blog/comparisons/panel-data-vs-controlled-experiments-choosing-right-tool-pricing-packaging Published: 2026-08-11 A brand manager deciding whether to change a price, a package, or a claim usually starts with the same question: what do we already know? The answer sits in three different places, and each one answers a different part of the decision. ## What syndicated panel data tells you Platforms like Numerator aggregate purchase and shopping behavior across a consumer panel and deliver it through dashboards built for research and analytics teams; Numerator positions itself as a consumer and market intelligence platform (Numerator). That kind of data answers a retrospective question well: what have customers actually bought, and how has that shifted over time. A comparison earns trust when the miss sits next to the hit for panel data too. It does not answer a prospective one: if a price or a pack changes, will behavior move, and by how much. Structured research programs on this model typically run days to weeks from question to answer. That cost is reasonable when the question is descriptive and the data needs to cover the full market. It is the wrong tool when the open question is which specific version of a change will move behavior before money is committed to it. ## What's the gap between what happened and what would happen? Descriptive purchase history and syndicated panel data can show that share dropped after a competitor cut price. Neither one isolates which lever (price, pack size, claim wording) would move the outcome if pulled today. Answering that requires holding everything else constant and varying only the one thing under consideration: a controlled experiment. This is the sequencing most teams get wrong: they either ship a launch straight off descriptive data, or commission a new correlational survey that still can't isolate cause from coincidence. The useful order is descriptive data first, then a controlled test of the specific action under consideration. ## Three ways to answer a pricing or packaging question The first two rows describe or predict. The third isolates cause. A brand team that only has the first two is still guessing at the third question. ## Testing the action, not reconstructing the market Subconscious runs controlled discrete-choice experiments against a synthetic population to test the effect of a specific action, a price point, a pack change, a claim, on choice, returning a causal estimate with a confidence interval. There is no published case study for this exact comparison; the argument rests on the research method itself: a controlled experiment isolates the variable under test in a way observational and panel data cannot. For a CPG brand deciding between two package designs or three price points, the panel answers "what's the current trend," and the controlled experiment answers "which of these specific options would move the number." ## What does a controlled experiment not replace? Naming what a tool does not do is what lets a buyer check the fit before they commit budget. Subconscious does not aggregate historical purchase transactions or syndicated retail panel data. Teams that need that baseline still need a panel data provider; a controlled experiment tests the next decision. ## Before committing the budget Ask which of the three questions above is still open. If it's "what has been happening," panel data answers it. If it's "which specific action will change what happens next," that calls for a controlled test of the actions under consideration before they ship. Book time to scope a test for a specific pricing, packaging, or claims decision. [figure: A decision path from the open question about a pricing or packaging change, branching to panel data, survey research, or a controlled experiment, converging on testing the action before it ships.] --- # How to write a causal question before you scope a behavioral experiment URL: https://subconscious.ai/blog/implementation-and-procurement/how-write-causal-question-before-scope-behavioral-experiment Published: 2026-08-11 ## Start with the decision, not the topic Before an insights, product, or growth team scopes a first behavioral experiment, or evaluates a platform to run one, the question that anchors the study has to name a cause and an effect. A topic is not a question. "Car buying" is a topic. "How does fuel efficiency impact car buying?" is a question, because it names a candidate cause (fuel efficiency) and an outcome (the buying decision). This distinction is the difference between a study that can support a decision and one that cannot. A team that commissions research around a topic instead of a causal question usually gets description back: rates, preferences, sentiment. None of that tells them what to change. ## Why is "what causes X" a better question than "tell me about X"? A causal question requires two things a purely descriptive question doesn't: a candidate cause that could plausibly come before the outcome, and an outcome that can move in response to it. Research design guidance on causal study design describes this as establishing temporal precedence and covariation between the proposed cause and effect before a study is built around it (Research Connections, Causal Study Design). Test a candidate question against two checks: - Does it name a cause and an outcome? "What causes car buying?" is broader than "How does fuel efficiency impact car buying?" Both can work as a starting point, but the broader version will need to be narrowed again once the study design starts, because "car buying" bundles together dozens of separate influences. - Is the cause something a study can actually vary or compare? A factor a team can present in different versions to different audience segments (price, framing, a feature claim, a policy change) is testable. A vague attitude or trend is not, until it's translated into something concrete. ## Narrowing a broad topic into a testable question Most teams start broader than they need to. Work through the topic in order: - Name the behavior of interest. Not "the market" or "our customers," but the specific action: buying, switching, renewing, recommending. - List the factors that plausibly influence that behavior. Price, a feature, a message, a competitor's move, a policy change. - Pick one factor and state it as a cause. "Does [factor] change [behavior]?" - Check the question against the study you'd actually need to run. If answering it requires comparing at least two versions of something in front of a defined audience, it's ready to move to audience and design decisions. A causal action test on Subconscious is scoped the same way: a decision prompt, a target audience, and the specific actions or concepts being compared. Getting the causal question right at this stage is what makes that scoping possible. ## What doesn't this step decide yet? This step is scoped on purpose, and naming that scope is what lets a buyer see where its authority ends. Framing the causal question doesn't determine who the audience is, how the comparison will be built, or how results get interpreted once the study runs. Those are separate decisions, made after the question is set. It doesn't guarantee the resulting study will be well-powered or well-designed; a clear causal question is necessary but not sufficient for that. And it makes no claim about the timeline, price, or delivery terms for scoping a study. [figure: A decision path narrowing a broad topic step by step: to a specific behavior, then a candidate cause, then a stated causal question, then a check for whether it implies a comparison a study could run.] [figure: Five stages left to right: causal question set, audience decided, comparison built, study run, results interpreted. First stage highlighted as this article's scope; the rest marked as later, separate decisions.] ## Where to go from here Once the causal question names a behavior and a candidate cause, the next decisions are who the audience is and what versions of the cause to compare. Research covers how Subconscious structures causal action tests once that question is set, and How we work walks through the process from question to result. Teams ready to scope a specific study can book time to talk through it. --- # Agency vs. in-house marketing measurement: where causal action testing fits URL: https://subconscious.ai/blog/methods-and-validation/agency-vs-house-marketing-measurement-where-causal-action-testing-fits Published: 2026-08-11 A marketing analytics or data-science leader deciding how to measure effectiveness usually frames the choice as agency versus in-house. That framing hides a second, more consequential decision: whether the team can test its next pricing, message, or channel action before committing budget, not just explain last quarter's spend. ## The build-versus-buy question most teams ask An option's upside is marketing until its cost sits next to it in the open. In-house marketing gives a team control over data and domain knowledge tailored to the brand. It also means hiring and retaining specialized talent, and maintaining a measurement practice as methods evolve. Agency partnerships offer diverse expertise, established tooling, and an outside perspective, with less transparency into the underlying model. Open-source, Bayesian tools have narrowed that tradeoff for the modeling layer itself. PyMC-Marketing is an open-source Python package for marketing mix modeling (MMM) and customer lifetime value (CLV) built on Bayesian methods, which gives a team transparency into assumptions and uncertainty that a black-box agency model does not expose (PyMC-Marketing documentation). Choosing to build on tools like this is a real option for a team with the statistical and engineering capacity to run it. ## What does that decision actually buy you? Marketing-mix modeling, whether run by an agency or built in-house on an open-source stack, fits response curves to media and pricing history that support both retrospective channel-contribution estimates and forward-looking budget-allocation scenarios across known channels. A model's fit to historical spend is only useful once its boundary is stated plainly. It does not answer a different question a team faces every week: whether a specific new message or channel choice, one with no spend history yet, will move the outcome before the team commits capital to it. Committing engineering headcount and a multi-quarter build to a measurement stack, or a multi-year agency retainer, while still deciding the next pricing or message change on guesswork, is the cost of treating build-versus-buy and pre-commitment testing as one decision instead of two a team can resolve separately. ## Where the two approaches sit relative to each other Subconscious.ai is a causal behavioral platform built for the second row: it helps teams test product, pricing, messaging, and go-to-market actions before committing capital. Stating what a tool does not do is what lets a buyer check the fit before they commit budget to it. That is not a replacement for marketing-mix modeling software. Subconscious does not attribute historical media spend, and it does not provide agency-style campaign execution or management. ## A concrete way to think about the split A team weighing agency versus in-house measurement can keep both tracks: - Use marketing-mix modeling, agency-run or in-house on a package like PyMC-Marketing, to understand how existing channels have performed and to allocate an established budget. - Use a controlled action test against a defined audience to decide whether the next price point, message, or channel is worth funding at all, before that spend becomes part of next quarter's contribution estimate. Treating measurement and testing as competing purchases, rather than complementary steps, is what leads teams to over-invest in retrospective tooling while still guessing on the decision in front of them. ## Where does this fit for a team already building the case? A team that has already decided to build or buy marketing-mix measurement can still test the specific action under consideration before it becomes part of that measurement history, independent of whether the modeling stack is finished. Reviewing recent case studies or the underlying research is a reasonable next step, and a short demo shows what a single action test looks like in practice. [figure: A path splits into agency MMM and in-house MMM, both explaining past spend. A causal-action-test branch sits before budget commitment, testing the next price, message, or channel choice first.] --- # Prove Research Impact When AI Makes Everyone Faster URL: https://subconscious.ai/blog/implementation-and-procurement/prove-research-impact-when-ai-makes-everyone-faster Published: 2026-08-11 When AI-assisted tools let anyone produce a research draft in minutes, the question a research leader has to answer is not "how do we go faster." It is "where does a finding stop being a draft and start being safe to act on," answered before a fast, plausible output reaches a business decision. [figure: Branching path: an AI-assisted read leads to a decision point asking if the finding carries real cost of being wrong. Low-stakes goes straight to action. High-stakes routes through a causal test first.] ## The risk AI exposes first Research teams now reach for AI daily. It drafts surveys, summarizes transcripts, and produces a first-pass read on a concept or message in minutes. That speed is not the problem. The problem is what happens when speed gets measured instead of decision quality: a team ships a conclusion built on a fast, unvalidated read because the deck needed one, not because the underlying claim was tested. That risk is narrower than "AI replaces research." A research function does not disappear because output gets cheaper to produce: the U.S. Bureau of Labor Statistics projects continued growth for market research analysts and marketing specialists through 2034 (BLS Occupational Outlook Handbook). What does disappear is the safety margin between a fluent answer and a proven one, unless a team draws a deliberate line between the two. ## What two jobs get compressed into one deliverable? An AI-assisted read and a validated finding answer different questions. Treating them as interchangeable is the mistake that makes fast research dangerous. One is built for speed and breadth: it surfaces hypotheses, objections, and directional reactions before a team commits budget to a slower method. The other is built for weight: what a team should be willing to defend in front of a stakeholder who will act on it. Collapsing these into one undifferentiated "the AI said" output is what turns a productivity gain into a credibility risk. ## Where does the validation gate belong? The decision a research or research-operations leader has to make before this reaches procurement is not whether to adopt AI-assisted tools. It is whether to build or buy a defined validation step that sits between AI-assisted exploration and any decision with real cost of being wrong. Skipping that gate has two failure modes, and both are expensive. A team ships a decision on an unvalidated AI-assisted read that turns out to be wrong. Or a team overcorrects and validates everything uniformly, spending budget testing low-stakes calls that never needed it. Neither failure is solved by using AI less or more. Both are solved by defining, in advance, which findings need a test and what that test has to prove. A controlled causal experiment is that test. The mechanism, described at /research, isolates one action and measures its effect with stated uncertainty, which is what turns "this seemed to resonate" into a claim a team can stand behind. ## What the validation step is not A controlled causal experiment does not automate the exploratory stage itself. It does not generate the hypotheses, draft the stimulus, or decide which findings are safe to act on and which are not; that judgment stays with the research team. It is not a dashboard for tracking how many projects went through a validation step, and it does not report on research-team productivity. It supplies one specific thing: a test of a specific action, run as a randomized intervention, with a result a team can cite. That boundary matters for procurement: expect it to gate findings before they reach a consequential decision, not to replace the exploratory workflow. ## How do you move from a simulated test to real-human validation? For decisions where a simulated test is not enough on its own, the same causal question can move to real-human validation without being redesigned: Subconscious can test or validate studies with real human participants, raising the evidentiary weight without rebuilding the test. That matters most for decisions above a stated risk threshold. Method boundaries hold in both directions. Moving to real-human validation does not turn a causal action test into a usability session, a clinical trial, or automatic proof of market performance. It answers the same question the simulated test asked, with a stronger evidentiary basis behind it. ## Setting up the gate A team does not need to rebuild its whole workflow to start. The smallest useful version is to take one live project, write the decision it is meant to inform in one sentence, and decide in advance what risk level makes that decision worth a controlled test. Everything below that threshold can stay in AI-assisted exploration. Everything above it goes through validation before anyone acts on it. /how-we-work walks through how that test gets designed and run, and /case-studies shows the kind of decision this threshold is built for. A team ready to define its own threshold can start with /demo. The point is not to slow research down. It is to make sure that when research moves faster, the decisions built on it are still ones the team can defend. --- # Alternatives to Recruited-Participant Panels for Qualitative Research URL: https://subconscious.ai/blog/comparisons/alternatives-recruited-participant-panels-qualitative-research Published: 2026-08-11 Screener-based interview panels solve a real problem: finding qualified people for qualitative research. A team that needs to interview B2B buyers, niche professionals, or a specific demographic segment can use a recruitment platform to connect with pre-screened participants who match the criteria. The trade-off is time and cost, which raises a question worth asking before every study: does this research question require a freshly recruited human panel, or can the team get a useful answer without one? Screening, scheduling, and no-shows are consistently reported as the slowest part of qualitative recruitment, adding real calendar time before a single interview happens (Norstat, "Participant Recruitment in Qualitative Research"). [figure: A path diagram: a research question feeds a simulated experiment, whose findings split into confirmed answers and high-stakes ones, with only the high-stakes findings routed to a recruited-participant study.] ## How do recruitment and simulation approaches compare? Buyers evaluating qualitative research options are usually choosing among structurally different approaches, not competing vendors of the same one. Subconscious sits in the last row. Subconscious can also test or validate the same study with real human participants without changing the underlying causal question: a team is not choosing between simulation and human validation as separate platforms, they are choosing when in the process to bring each one in. ## When simulation is the right first step Not every research question needs real participants recruited on day one: - Exploration and hypothesis generation. When a team is still figuring out which questions to ask, a simulated experiment can explore the space before anyone books a screener call. - Cross-segment comparisons. Comparing how five different buyer segments respond to the same question can run at once against a simulated population; a recruitment panel requires sourcing five separate groups in sequence. - Speed-sensitive decisions. A product or marketing team may need a directional read in days rather than the weeks a screener-based panel typically takes to fill. - Budget-constrained teams. Early-stage teams without a standing research budget still need a first read on customer behavior before deciding whether a recruited-participant study is worth funding. ## How do you sequence simulation and recruitment together? Research teams that use recruited-participant panels have historically sequenced two steps: simulate first to sharpen the question, then recruit real people to validate what emerged. A team might run an early round of simulated interviews to identify which two or three findings are ambiguous or high-stakes enough to warrant a recruited-participant study, reserving screener time and budget for the claims that need confirmation with real people, not the whole research question. That sequencing matters most when the decision is high-stakes: a launch, a pricing change, or a claim made publicly. Simulated experiments estimate causal effects on a modeled population; they are not a substitute for a recruited-participant study when only a specific named human population's response is decision-relevant, such as a regulatory, legal, or safety-critical claim about real customers. Recruitment methodology research shows that even well-designed studies can struggle to reach and retain some populations (BMC Medical Research Methodology, "Widening participation: recruitment methods in mental health randomised controlled trials"). For teams weighing whether a research question needs a recruited panel today, Subconscious can run the simulated first pass and carry the same causal question into a validation study when the decision warrants it. Book time to scope a study. --- # Validate a Business Idea Before You Build It URL: https://subconscious.ai/blog/use-cases/validate-business-idea-before-build Published: 2026-08-11 [figure: A branching path showing an idea moving through four checks, market reality, buyer pain, business model, and risk profile, each scored green, yellow, or red, converging on a proceed, pivot, or kill decision.] You have a business idea, but you don't know whether its buyer problem is real enough to support a business. Before committing to a build, hiring, or fundraising, turn the idea into four testable questions: market reality, buyer pain, business-model viability, and failure conditions. ## The decision, and what it costs to get wrong The decision in front of a founder before any interviews or spend happen is not "is this a good idea." It's whether to proceed, pivot, or kill a specific idea before committing real time, hiring, or a fundraising cycle to it. CB Insights ties roughly four in ten startup shutdowns to no market need or poor product-market fit, the exact failure mode a founder is trying to catch before it costs a runway. (CB Insights: The Top 20 Reasons Startups Fail) ## Four checks before you commit A useful triage sequence runs the idea through four separate pressure points. ### Check 1: Market reality Ask what the market actually looks like: how big it is, who else is already in it and what they do well or poorly, what's driving demand, and whether the space is growing, flat, or shrinking. A market that reads as tiny, saturated, or declining is information worth having before anything else gets built. ### Check 2: Buyer perspective Ask the people who would actually buy the product, not a general audience, what their biggest unmet pain is in this area, how they cope with it today, whether they'd pay for a product that did what you're proposing, and what would make them switch from their current option. ### Check 3: Business-model pressure test Present the rough model (product, target market, proposed pricing) and ask whether the unit economics work, whether customers can be acquired profitably, and where the biggest risk in the model sits. ### Check 4: Critical challenge round Ask for every reason the idea could fail, what's being missed, and what would have to be true for it to work. This is the same set of questions investors, competitors, and the market will eventually ask. ## Score the result Two or more reds points toward a pivot. All greens is a reason to move toward real customer validation. A mix means addressing the yellow and red areas before going further. ## What doesn't this triage answer? This sequence is a first pass, not a full diagnosis. It's no substitute for actual conversations with prospective buyers, a prototype built and put in front of users, or a live market experiment. What it does is tell a founder whether those next steps are justified. It also doesn't resolve the one question that decides whether the idea is worth building: does the specific value proposition or price point actually move a defined buyer group's stated preference. That question needs a fixed choice set and a named audience, not an open-ended "is this a good idea" conversation. ## How do you test the claim that actually decides it? Where an unstructured panel discussion gives a subjective read on an idea, Subconscious runs a controlled discrete choice experiment against a defined target-buyer population, so the value proposition or price point at the center of the idea gets tested for whether it actually drives stated preference, with a causal effect and a confidence interval attached (methodology). Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It requires a defined decision and audience segment, such as one value proposition tested against one named buyer group, rather than an open-ended discussion with no fixed choice. Subconscious can test or validate a study with real human participants, which lets a founder move from a simulated read of the value proposition to a real-human check on the same question without changing what's being tested. ## How do you move from a scored idea to a tested one? A green-heavy score from the four checks is a reason to commit to real customer validation, not a substitute for it. Reviewing the method behind these tests or seeing how a study is set up is a reasonable next step once the idea has cleared triage. A demo walks through setting up that test against your own value proposition and audience, and past studies show what the results look like. --- # How to Choose Among 10 AI Ad Creative Testing Approaches in 2026 URL: https://subconscious.ai/blog/use-cases/how-choose-ai-ad-creative-testing-tools-2026 Published: 2026-08-11 Choosing among the ten AI ad creative testing approaches means matching the decision a team needs to make, whether high-volume screening, observed reaction, network-level read, or causal comparison, to the category built for that job. A performance marketing lead facing ten finished creative variants has one real decision: which two or three earn a media budget. Sending all ten straight to paid channels burns spend on the losers a pre-test would have caught. The category built around that problem has grown large enough that buyers now compare methods, not just vendors, per the IAB's State of Data 2026 report. ## What a pre-flight test should cover A creative test earns its keep when it ties a stimulus to a decision measure, not a preference score. Screen these before launch: - The hook: whether the opening seconds of a video or the first line of copy earns attention - Headline and call-to-action wording - Image or thumbnail response - Full video pacing and narrative arc - Landing-page reaction after the click - Whether the read changes across audience segments Preference alone answers "which do people like." A decision-linked test answers "which action moves behavior for this segment," the question that should gate spend. ## Ten approaches, sorted by the job they do Vendors in this category specialize; none covers every job below. The roster shifts often, so evaluate by category first and confirm current scope directly with any vendor under consideration. ### What is a decision-specific causal experiment? Some platforms run controlled comparisons between named creative actions, such as one hook against another or one price frame against another, and report an effect with a confidence interval rather than a single score. Fits a team that already knows the decision it needs to make and wants to know which alternative caused the outcome change for a defined segment. ### 2. Broad population simulation Other tools model a wide synthetic crowd to gauge how a campaign might land across a general consumer or media audience. Suits early-stage directional reads before a team narrows to specific creative alternatives. ### 3. Behavioral spread modeling A third category focuses on how a reaction propagates through a network rather than an individual response in isolation. Matters when a campaign depends on sharing or word-of-mouth effects, not just first-exposure reaction. ### 4. Lightweight real-human panels Some services route a stimulus to paid human respondents for first-click, preference, or short-exposure tests. Work well as a cross-check after a synthetic pass has narrowed the field, trading speed for an observed human reaction. ### 5. B2B decision-maker audiences A narrower set of tools recruits or models specific professional roles, such as finance, IT, or procurement buyers, for creative aimed at business audiences rather than consumers. ### What is qualitative UX-adjacent testing? Another category runs open-ended simulated interviews against product pages, onboarding flows, or in-product creative, useful when a team wants reasoning behind a reaction rather than a single score. ### 7. Lower-cost simulated focus groups Some entrants offer a lower-cost, lighter-weight substitute for a traditional focus group, aimed at teams that need a directional read without a full research budget. Confirm current plan pricing before procurement, since it moves often in this category. ### 8. Observed on-page behavior Heatmap and session-recording tools do not run a pre-launch test at all. They record what real visitors did on a live landing page, closing the loop after a creative has already shipped rather than screening it beforehand. ### 9. Regulated-industry workflows A smaller set of vendors build around the audit trail that claims-heavy industries need, such as finance, insurance, healthcare, or automotive, where every tested claim has to be traceable for compliance review. ### 10. Product-launch response The last category is purpose-built for gauging reaction to a new product announcement or launch creative rather than ongoing ad rotation, closer to a concept test than a media pre-flight. ## One published operating sequence One publicly described workflow sequences these categories rather than picking one: produce a wide set of variants, run a synthetic pass to cut the field to a handful of finalists, spot-check the strongest two or three with real respondents, then commit paid spend only to the winner of that narrowed set. Post-launch performance is compared against the pre-launch read to calibrate the next round. Treat this as one documented sequence, not a benchmark any specific vendor guarantees. [figure: Four starting questions, each branching to one approach: high volume to a scoring tool, observed reaction to real-human panels, network read to population simulation, one action's effect to a causal experiment.] ## How do you match testing evidence to the decision? Pick a scoring tool when hundreds of assets need routing through triage before spend commits. Pick real-human testing when an observed reaction is a requirement, not a nice-to-have. Pick a population simulator when a high-budget campaign needs a network-level read before commitment. Pick a causal experiment when the question is which specific product, price, message, or GTM action changes an outcome for a defined segment, and the team needs to name its assumptions and uncertainty rather than trust a single score. Subconscious runs this kind of study as a controlled comparison among creative alternatives, and the same causal question can move from a simulated read to real-human validation without a redesign when warranted before spend commits. No category above removes the need for live evidence after launch. The strongest setup treats pre-flight testing as a filter and post-launch behavior as the check on whether the filter worked. [figure: Five-step path: wide variant set, synthetic pass cuts to finalists, real respondents spot-check the top two or three, spend commits to the winner, post-launch results compare back to calibrate the next round.] --- # Customer Panel Software: Sequence the Method to the Decision URL: https://subconscious.ai/blog/comparisons/customer-panel-software-sequence-method-decision Published: 2026-08-11 A consumer insights, product, or marketing leader without a dedicated research team faces one recurring decision before a concept, price, or message ships: run a controlled simulated experiment first, or go straight to a real-respondent panel study. The wrong call in either direction is expensive: shipping a launch decision off an unvalidated synthetic read that doesn't hold up with real customers is one failure mode; burning weeks and an enterprise research budget on a real-respondent panel to answer a question a validated simulated experiment could have settled first is the other. ## What are the three shapes of customer panel research? Panel research for consumer, product, and marketing questions comes in three structural shapes, independent of any single vendor: - Real-respondent panels. A provider recruits actual people from an established panel and fields a survey, interview, or structured study against them. This produces real-respondent provenance and stated-preference data, typically over a period of days to weeks, usually under an enterprise contract. - AI synthetic panels. A platform builds AI personas from public data and psychological models and aggregates their simulated answers, usually within a single working session. - Hybrid platforms. Real respondents are recruited through panel partners while AI tooling supports study design, moderation, or analysis of what those respondents say. Independent research on whether large language models can stand in for human choice behavior finds that model-based predictions can track some patterns in stated preference but should not be treated as an automatic substitute for measured human choice without validation (arXiv, 2026). That finding is why the sequencing question below matters more than the category label on the tool. ## Compare by evidence, not by interface [figure: A four-step path: assess the cost of being wrong, run a controlled simulated experiment to explore quickly, confirm the result with a real-respondent panel, then ship the decision.] Two structural tradeoffs sit underneath this table. Synthetic methods win on speed and iteration count: a team can run a new version of a study as soon as the last one raises a new question. Real-respondent methods win on stakeholder perception and regulatory standing, because the answer came from an accountable, recruitable person rather than a model. Most teams that use both are not choosing one over the other; they are sequencing exploration first and confirmation second. ## Where a controlled simulated experiment fits Subconscious runs controlled experiments on a simulated population and reports validation results from the causal fidelity paper: its best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against the published human result, versus 0.959 between two independent samples of real humans). Across all 43 studies that pass design filters, drawn from roughly 300 published human studies across 9 domains, the mean is 0.73. That is the evidence a team can act on before committing budget to a real-respondent panel: does the simulated read hold up against a known outcome, not whether the tool feels convincing. Subconscious can also test or validate the same study with real human participants without changing the underlying causal question. That sequencing, not a claim that either method alone is sufficient, is the practical advantage over choosing a single panel type up front. ## What does the simulated pass not cover? A controlled experiment on a simulated population is not a self-serve price table. The person-level audience graph behind the simulation covers 800 million real people, but that graph is a modeling input, not a pool of individually recruitable respondents who can be scheduled for a survey. Where a decision needs a real, individually recruited human to answer directly, a real-respondent panel or interview is the right instrument, whether that happens before or after a simulated pass. Real-human validation also has its own boundary. It confirms or corrects a simulated read; it does not turn a causal test of one action into a usability session, a clinical trial, or automatic proof of how something will perform after launch. ## Four questions that decide the sequence - What is the cost of being wrong? A reversible, low-stakes decision, such as an early concept screen or a message iteration, can usually run and ship on a simulated read. A launch-scale, hard-to-reverse decision should route through real-human confirmation before spend commits. - Does the decision depend on a number, or a direction? Directional reads travel well through simulation. A specific willingness-to-pay figure or conversion estimate needs the real-human pass for the number itself. - Who owns the research function? A self-serve simulated experiment fits a team without a dedicated research operator. A real-respondent panel program generally needs someone accountable for recruiting, fielding, and interpreting it. - How many iterations does the question need? A question that will be re-asked as the concept changes benefits from starting in simulation, where a new version can run again without re-recruiting a panel. For most teams, the workable pattern: use a controlled simulated experiment to explore and narrow the field of options, then confirm the option that is actually going to ship with real human participants before the decision that carries the real cost. See how the study process works end to end, review research methodology and validation results, or book time to walk through where a specific decision sits on this sequence. --- # Persona Documents vs. a Randomized Concept Test: Which One Answers Your Question URL: https://subconscious.ai/blog/comparisons/persona-documents-vs-randomized-concept-test-which-one-answers-question Published: 2026-08-11 A persona document tells a marketing or insights team who the audience is. It does not tell the team what happens when message A runs against message B, or which price a buyer actually picks over another. Those are two different jobs, and choosing the wrong tool for the job is how a campaign ships on an assumption that was never tested against an alternative. ## Two starting points for understanding an audience Research tools that build interview-derived persona documents start from a corpus of past interviews or transcripts. They organize what a team already learned into a structured profile: a name, a set of quoted motivations, a summary of pain points. One tool built around this workflow describes itself as turning existing customer-interview datasets into shareable persona profiles. Tools built for randomized concept testing start from the decision, not the archive. A team defines two or more concrete options, a message, a price, a feature set, a positioning line, and runs a controlled comparison against a simulation of the market to see which option performs better and why. The persona document answers "who is this person." The controlled test answers "which version wins." ## What can't a persona document tell you? A persona document is a description, not an experiment. It can summarize what interviewees said they wanted last quarter. It cannot show a team what a specific new headline does against a specific alternative headline, because no comparison ran. A team that treats a persona's stated preferences as a stand-in for that missing comparison is making a causal claim, "this message will work because this persona would like it," without ever having tested causality. ## Comparing the two approaches ## How does Subconscious run a controlled comparison against a simulated market? Subconscious runs randomized experiments that compare concrete options, messages, prices, features, against a simulation of the market and returns which option wins and why, with confidence intervals attached to the result. It is closer to "if we ran this as an A/B test, which version would come out ahead, and how confident should we be." Teams evaluating how the underlying method works can see how a randomized comparison is built before any spend goes into a campaign. Subconscious can also test or validate studies with real human participants. Publishing the point where simulation hands off to real people is what a buyer checks before betting budget on the result. A team can start with a simulated comparison, then confirm the winning result against a recruited human sample, without changing the underlying causal question, when the decision is consequential enough to warrant that extra step. ## When is an interview-derived persona document still the right call? Choose a persona document when the job really is systematizing a corpus of interviews a team already has, organizing scattered qualitative notes into something shareable across the organization. Stating where a method reaches its edge is what lets a buyer size the decision correctly. A controlled comparison is not a substitute for having talked to customers in the first place. ## Making the choice If the open question is "who are we building for," a persona document built from real interviews answers it directly. If the open question is "which message, price, or concept should we ship," that requires a comparison with an actual winner, not a description of a type of person. Teams that need to see how this fits into a broader research and testing process can review the workflow, or look at documented outcomes from past comparisons before deciding which approach fits the decision in front of them. A team ready to run a controlled comparison against its own concepts can start with a working session. [figure: Two-column diagram: left, interview data producing a persona document. Right, options run through a randomized concept test against a simulated market, producing a causal winner, with optional human validation.] --- # Choosing AI Feature Disclosure Wording Before It Ships URL: https://subconscious.ai/blog/use-cases/choosing-ai-feature-disclosure-wording-before-ships Published: 2026-08-11 A product or trust-and-safety leader signing off on a new AI feature must choose the exact sentence beside the activation control: which verb voice describes the data use, whether a retention window is stated, and how visible the opt-out is. That choice should be tested before the feature flag flips. Once the sentence ships to the full user base, reversing a poor choice is expensive. ## The decision comes down to a few specific wording choices The same underlying data-use fact can be written several ways, and each version lands differently with a reader deciding whether to trust the feature. Publishing what each review method covers, and what it leaves untouched, is what lets a buyer check the fit before relying on it. Legal review can determine whether each statement is complete and true. It does not by itself estimate how a reader will interpret each version or how that interpretation will affect opt-in behavior. Those are behavioral questions. ## Getting the wording wrong is costly in both directions The Federal Trade Commission says required disclosures must be clear and conspicuous, taking into account their placement, proximity, prominence, and whether their language is understandable to the intended audience (FTC, How to Make Effective Disclosures in Digital Advertising). Federal agencies have also stated that existing consumer-protection law applies to automated systems and their marketing (FTC AI Policy Statement). Wording that under-discloses invites regulatory attention and, separately, drives complaint volume and feature abandonment once users feel misled after the fact. Wording that over-discloses, by hedging or stacking qualifiers out of caution, can suppress adoption of a feature people would otherwise use. Both failures are hard to walk back once the copy is live in front of every user, which is why the specific wording deserves a causal answer before launch rather than a legal read followed by silence until support tickets arrive. ## What does disclosure copy do? Most product copy does one job: it explains a feature. Disclosure copy does two jobs in the same sentence: it informs the reader about a data practice, and it asks the reader to decide whether to keep using the feature next to it. Research on how users perceive and respond to manipulative interface language in AI products shows why technically truthful wording can still feel evasive or coercive to the person encountering it (The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models). ## How do you test wording before it ships? Subconscious can run a controlled experiment comparing the exact disclosure-copy variants a team is deciding between, such as the verb-voice options above, against a simulated population, and measure the behavioral effect on an outcome like opt-in rate. The causal question, which wording moves the outcome and by how much, gets set up before the copy goes live. This is a message-comparison study: it estimates which specific alternative performs better and reports the uncertainty around that estimate. That comparison runs against a person-level audience graph covering 800 million real people. A reach number needs its limits stated next to it, or it reads as marketing rather than a method description. This is a statement about experimental reach, not a claim that all 800 million form a recruitable pool for interviews (research). When a decision is high-stakes enough to warrant it, the same causal question can move from the simulated comparison to a study with real human participants without changing what is being measured, so the team is not re-deriving the question from scratch at the higher-stakes stage (how we work). ## What does this kind of test not replace? Naming exactly what a test does not cover is what lets a buyer check for the gap before relying on the result. A behavioral comparison of wording variants measures how a specific population responds to specific sentences. It does not substitute for legal or compliance review of procedural completeness: whether a stated retention window matches the real practice, whether every third-party processor is correctly named, or whether the disclosure meets requirements in every jurisdiction where the feature ships. Legal sign-off is still required regardless of which wording performs best in the comparison. The two reviews answer different questions, and skipping either one leaves a gap the other was never designed to close. [figure: Five steps left to right: verb voice choice, whether a retention window is stated, opt-out visibility, a causal test comparing variants against a simulated population, then shipping the wording that wins.] ## Where to start Most product teams have at least one AI feature moving toward a launch decision, and it ships with disclosure copy that has to make a specific wording choice. Reviewing case studies of how a causal comparison is set up, or booking a demo to walk through a specific disclosure decision, is a reasonable next step before that copy goes into the screen. --- # What to Use After HubSpot's Make My Persona URL: https://subconscious.ai/blog/comparisons/what-use-after-hubspots-make-my-persona Published: 2026-08-11 A marketing or product team that has already run HubSpot's Make My Persona template has a document, not a decision. HubSpot's template asks demographic, goal, and objection questions and produces a one-page profile at no cost. The real question for a team past that step is what the document is for next: better sourcing, a way to be questioned, or a test against a specific action the team is about to take. ## What the free template gets right, and where it stops HubSpot's template is a useful starting artifact for a first buyer profile. What it does not do is let a team check the profile against current customer records, question it, or find out how a messaging, pricing, or positioning choice lands with the buyer it describes. It is a static document that cannot be updated or verified without redoing the exercise by hand. ## Three ways teams extend it Teams outgrowing a static profile move one of three ways: - Data-grounded profile generators. These rebuild the profile from CRM records, website analytics, or social signals, producing a document plus a behavior-data dashboard. The output is a document; it is better sourced. - Conversational profile tools. These let a team pose questions to a modeled version of the profile and read back a plausible answer. They are fast to set up and useful for early-stage exploration, but a plausible answer is not a measured one, and most of these tools do not publish how the model's answers compare with what real customers say. - Causal experiment platforms. These start from an existing profile or segment and run a randomized comparison of specific actions (a headline, a price point, a positioning statement) on a modeled population built from it, returning which action moved the outcome. Subconscious is in this third group. A team choosing between them should start from the decision in front of them, not from which output looks most sophisticated. ## Where does a causal test pick up after a profile is built? A profile, however it was built, describes a buyer; it does not tell a team whether swapping one headline for another, or one price point for another, changes what that buyer does. That is what Subconscious tests for: once a team has a profile or segment defined, Subconscious runs a controlled experiment on a modeled population built from that segment to compare specific actions and estimate which one moves the outcome, with quantified uncertainty where supported. Subconscious does not generate a persona document, and it does not offer a conversational interface for talking to a modeled buyer. It answers a narrower, later-stage question: which action wins, not what the buyer looks like on paper. ## How do you move from a profile to a tested decision? A causal question sometimes needs to move from a modeled population to real customers before it ships. Subconscious can test or validate a study with real human participants, carrying a question from a controlled experiment into human confirmation without redefining what is being measured. It follows the controlled comparison rather than replacing it, once the modeled result is worth confirming. ## Choosing between the three The decision comes down to what the team is stuck on: - If the team has never built a buyer profile before, the free HubSpot template is the right first step; building a heavier tool on top of no profile is premature. - If the profile exists but keeps going stale because it is disconnected from actual customer records, a data-grounded generator closes that gap. - If the team wants a fast, exploratory read on how a modeled buyer might react to an idea, a conversational profile tool is the cheapest way to get one; its answer is a hypothesis, not a measurement. - If the team already has a profile and is choosing between two or more specific actions, a headline, a price, a positioning line, and needs to know which one changes behavior, that is a causal test, not a profile exercise. A profile that has never been checked against how buyers decide is a hypothesis about the market. A causal experiment turns that hypothesis into an estimate a team can act on before it commits budget to the wrong version. [figure: Branching diagram from a persona document into four routes by blocker: no profile, stale profile, fast exploratory read, choosing between specific actions.] See worked examples, or book time to walk through a specific decision. --- # A Staged Evidence Model for High-Stakes Research URL: https://subconscious.ai/blog/implementation-and-procurement/staged-evidence-model-high-stakes-research Published: 2026-08-11 High-stakes research needs separate gates for exploration, causal comparison, human review, and real-human validation. A number without its limits is marketing, so the limit gets stated up front. Exploratory output can narrow what deserves testing. It should not become the basis for an executive, pricing, launch, or public claim until the evidence matches the cost of being wrong. ## What is the operating-model decision? A Head of Research or VP of Consumer and Market Research has to decide whether those gates are formal policy or an informal judgment made under pressure. A plausible but untested recommendation reaches an executive or goes public, then fails. The research function loses the credibility it needs to shape future decisions. The U.S. Bureau of Labor Statistics projects employment growth for market research analysts from 2024 to 2034 (Occupational Outlook Handbook). The durable value in that role is judgment about questions, evidence, and claims. A staged operating model protects that judgment. ## Four layers, four different jobs Each layer answers a different question. [figure: Four gates in sequence: exploration narrows hypotheses, causal comparison shortlists actions, human review checks framing, real-human validation confirms the effect before a public claim.] The handoff between layers needs an owner and an acceptance rule. Without both, exploratory material can become decision evidence simply because it is already in the deck. ## Keep the causal question intact Subconscious is a causal behavioral platform for testing product, pricing, messaging, and go-to-market actions before a team commits capital. Its method uses causal experimentation and discrete-choice-style comparison rather than open-ended roleplay. That method fits the directional comparison layer. A team defines the action, audience, alternatives, and outcome. Subconscious compares the actions under a controlled study design. When the claim justifies another gate, the team can move from a simulated study to real-human validation. The practical advantage is continuity: the team preserves the intervention, comparison, population, and outcome, so the evidence stays tied to the original decision through validation. ## Set the validation trigger before the study Before work begins, write down four questions: - Which business decision will this evidence support, and what is the cost of choosing poorly? - Which audience, action, alternative, and outcome define the causal question? - Which result may remain exploratory, and which result requires human review? - What evidence must exist before the claim reaches an executive or goes public? The fourth answer is the validation trigger. Set it before anyone sees a preferred result, so the standard cannot shift once an appealing answer appears. ## Why should boundaries be part of procurement? Subconscious is one stage in this evidence system. A published boundary lets a buyer check the method before relying on it. It is not an automated decision engine, a packaged catalog or pricing optimizer, or a substitute for human review. It does not automatically produce confidence intervals, decision memos, or recommendations for every study. A researcher still has to define the audience, check neutral framing, examine source grounding, and decide whether the stakes require real-human confirmation. Real-human validation also has a clear boundary. Naming that boundary is what lets a buyer check the claim against the method. It does not turn a causal action test into an observed usability session, a clinical trial, or automatic proof of market performance. These limits should appear in vendor evaluation and study acceptance criteria. A buyer should be able to identify what the method compares, who reviews the design, what evidence advances the claim, and which decision remains with the accountable human. ## Run a one-month operating exercise Treat this as a 2026 planning example, not a delivery commitment: - Pick one live decision. - Write the decision and cost of choosing poorly in one sentence. - Define the audience, alternatives, outcome, and risk level. - Use causal comparison only for the directional layer. - Review the design and result against the four validation questions. - Present the result with its current evidence label and next gate. Apply the sequence once a week for a month. At each review, record the current layer, the evidence available, the accountable reviewer, and the condition for advancing. At the end of the month, inspect every handoff. The useful result is not a larger volume of output. It is a repeatable boundary between a hypothesis, a directional comparison, and evidence ready for a high-stakes decision. ## Put one consequential claim through the gates Bring one product, pricing, messaging, or launch decision to a working session. Define the action, audience, alternative, outcome, and validation trigger. The goal is a defensible path from a question to a decision, with human accountability at every gate. --- # Customer Chatbots, AI Audience Interviews, and Causal Experiments Compared URL: https://subconscious.ai/blog/comparisons/customer-chatbots-ai-audience-interviews-causal-experiments-compared Published: 2026-08-11 A customer-facing chatbot, a self-serve AI audience-interview tool, and a causal behavioral platform get compared as if they compete for the same budget line. They don't. Each answers a different question, for a different counterparty, at a different point in the decision. The buyer's job is to name which question is actually stuck, not to pick the tool with the bigger category name. [figure: A branching path from "what job is stuck" splitting into three: a chatbot for live support, a self-serve AI interview tool for cheap exploration, or Subconscious running a causal experiment before launch.] ## What a customer-facing chatbot does A chatbot persona builder serves live conversations with your own customers. You configure a brand voice and a knowledge base from product documentation and FAQs, then a deployed bot handles support and sales conversations on a website or messaging channel, in the pattern of a platform such as Makebot. The buyer here already has an operational workflow, usually a support queue, and needs a tool that plugs into it well. The chatbot's value is deflection rate, resolution time, and customer satisfaction on tickets that already exist. Publishing this limit is what lets a buyer check the fit before committing budget. It does not tell a team what to build, price, or say before launch. ## What does a self-serve AI audience-interview tool do? A self-serve AI audience-interview tool lets a team define a target customer profile, brief a structured conversation in plain English, and get directional answers back from a calibrated model in minutes. The counterparty is internal: your team is talking to the tool, not the customer. This is fast, cheap to iterate, and useful for early-stage hypothesis generation. It is not the same as a randomized experiment, and it is not a substitute for talking to real customers when a decision is expensive to get wrong. ## What does Subconscious do instead? Subconscious runs randomized experiments on a simulation of the market to estimate which action, price, message, or launch decision drives a target outcome, before a team commits budget or build time. Subconscious can test or validate studies with real human participants, so a team can move from a simulated experiment to real-human validation without changing the causal question being asked. ## The decision that actually separates these tools Is the job to deflect or answer a live customer conversation, or to decide what to build, price, or say before spending the budget to find out? Buying a support-facing chatbot when the real need is validating a decision before launch leaves the decision unvalidated. Buying a pre-launch research tool when the real need is ticket deflection leaves the support queue backed up. ## When is a chatbot the right call? A customer-facing chatbot fits when the team has a high-volume support queue, a stable FAQ, clear deflection ROI, and someone with the standing to set how an AI surface talks to customers on the brand's behalf. ## When a causal experiment is the right call A causal experiment fits when the cost of guessing wrong on a product, pricing, messaging, or launch decision is high enough that a team needs an estimate of which action drives the outcome, not just a directional read. Run the experiment, then extend the same causal question to real-human validation when the decision warrants it. ## Where a directional AI interview tool still helps Naming where a tool stops is what keeps the recommendation checkable. A self-serve AI audience-interview tool remains useful upstream of both of the above: for generating hypotheses and rough concepts cheaply before deciding which question deserves a full causal test or real-respondent fieldwork. ## The bottom line Match the tool to the job, not the category label. A support queue needs a chatbot. Early-stage hypothesis generation can use a directional interview tool. A decision that is expensive to get wrong needs a causal experiment, with a clear path to real-human validation when the stakes justify it. See how Subconscious runs that experiment, or talk to the team about a specific decision. Learn more about Subconscious. --- # PyMC-Marketing vs. Meridian: What a Baseline-Modeling Benchmark Shows About MMM Attribution URL: https://subconscious.ai/blog/comparisons/pymc-marketing-vs-meridian-what-baseline-modeling-benchmark-shows-about-mmm Published: 2026-08-11 A marketing mix model can hit a strong R² and still get channel attribution wrong. Those are two different questions: does the model predict total sales well, and does it correctly separate how much of that outcome came from each media channel versus the underlying trend. A team that checks only the first question can reallocate budget away from a channel that actually worked, because the model absorbed that channel's true effect into a flexible baseline term instead of assigning it to media. That gap is the subject of a third-party benchmark comparing two open-source MMM libraries, PyMC-Marketing and Google's Meridian, on synthetic data with known, true channel contributions. Because those contributions are known, the benchmark can score each library on whether it recovered the right answer, not just whether it fit the observed sales curve. ## The two questions a fit metric can't answer Goodness-of-fit metrics like R² and MAPE measure how closely predicted sales track observed sales, not whether that variation was assigned to the right source. A flexible baseline term can track short-term sales fluctuations closely, improving fit while quietly absorbing variation a media channel actually caused. Contribution-recovery metrics (bias, standardized root-mean-squared error (SRMSE), and CRPS on the recovered channel effects) measure the second question directly, because the benchmark's synthetic data has a known ground truth for each channel's contribution. ## What did the benchmark measure? The benchmark evaluated PyMC-Marketing and Meridian across three small, growing, and medium-sized synthetic datasets on three dimensions: sampling convergence, in-sample predictive fit, and contribution recovery. Meridian's newest release automates spline-knot selection for its time-varying baseline; this round tested that change against the same datasets and generating process as the library's earlier round. The two libraries model trend and seasonality differently. PyMC-Marketing uses explicit Fourier terms for seasonality with an optional Gaussian process for longer trend. Meridian models both trend and seasonality with a single spline baseline, now sized automatically by the new knot-selection routine. The table reports the benchmark's own published figures as historical results from that specific run of PyMC-Marketing v0.17.0 against Meridian v1.2.1. They describe one synthetic-data study, not a live measurement of either library's current performance. In this benchmark's own account, Meridian's automated knot selection improved in-sample R² and lowered MAPE across every dataset size after the update. In the same runs, Meridian showed higher bias, higher SRMSE, and lower Durbin-Watson statistics than PyMC-Marketing on contribution recovery, and the benchmark reports greater leakage of media signal into the baseline term as knot flexibility increased. [figure: Two-column comparison. Meridian: better fit, higher R2, lower MAPE, paired with worse attribution, higher bias and SRMSE. PyMC-Marketing: worse fit paired with better attribution.] ## Why can a more flexible baseline look better and mislead at once? A lower Durbin-Watson statistic means residuals are more autocorrelated: the baseline tracks short-term noise instead of modeling seasonality as an explicit structure, which can inflate in-sample fit while obscuring how much of a lift came from media versus the baseline. Out-of-sample performance was not part of this benchmark round. ## What is the saturation problem that no library escapes? The benchmark identifies a separate, structural limit that applies to both libraries: channels that saturate quickly are hard to recover accurately. Contribution recovery depends on spend varying within a channel's non-saturated response range; past that point, extra spend barely moves the outcome, so spend and sales stop moving together, and the model loses its ability to tell that channel's true effect apart from the baseline, even when the data still has plenty of spend fluctuation. A more flexible baseline makes this worse, because it can more easily absorb the variation that should have been assigned to the saturated channel. ## Checking your own model before moving budget This benchmark's warning applies past these two libraries: fit metrics and attribution metrics answer different questions, and passing one says nothing about the other. Before reallocating spend on a model's contribution numbers, the model's recovered contributions need their own check, separate from R² or MAPE. Subconscious approaches this by running controlled experiments on simulated buyers, and Subconscious can test or validate a study's findings with real human participants, moving from simulation to real-human validation without changing the causal question. See how these experiments are structured on how Subconscious works and how results compare against real outcomes on the replication leaderboard. ## Limitations of this benchmark This is one third-party synthetic-data study of two specific open-source library versions, not a Subconscious study, a real-company dataset, or a ranking of MMM vendors in general. No pricing, speed, or accuracy claim about either library should be read as current: neither library was verified, run, or endorsed by Subconscious in this benchmark, and both continue to change with new releases. The figures above describe the versions tested at the time of that study. ## Next step If your team is deciding whether to trust an MMM's channel attribution before shifting budget, see what Subconscious tests differently and where a controlled experiment can check a contribution number a fit metric alone cannot confirm. --- # How to Pre-Test a Retainer Renewal Deck Before the Client Meeting URL: https://subconscious.ai/blog/use-cases/how-pre-test-retainer-renewal-deck-before-client-meeting Published: 2026-08-11 A retainer renewal deck should be tested as a decision instrument, not reviewed only as a presentation. The account team needs to decide whether to keep pricing flat, raise it, or restructure scope, and which sequence of past results, future value, and pricing evidence will cause the room to support that choice. Getting that decision wrong can produce a flat renewal, a scope cut, or a quiet move to RFP. A historical planning example may put a single renewal at six to seven figures of annual revenue, depending on retainer size. The relationship can matter for years: Agiled's 2026 compilation reports an average client lifespan of 56 months for agency retainers, compared with 24 months for project-based relationships (Agiled, Client Retention Statistics for Agencies). [figure: Five-step path: name the commercial action, create controlled deck variants, define audience by decision-maker role, choose the behavioral outcome, then randomize exposure and compare results.] ## What must a renewal deck defend and earn? A renewal deck explains what the retainer delivered during the past twelve months, then makes the case for what the next twelve months should become. As a planning example, imagine a draft that gives 70 percent of its slides to past results and 30 percent to the future. The client receives a detailed recap but little reason to expand the relationship. Reversing the ratio without enough evidence creates the opposite problem: ambition without a defensible record. Pricing sits between those two jobs. A flat price can suggest that the current scope is the ceiling. A higher price can look like more money for the same work. A restructured scope can create confusion unless the deck explains what changes and why. The experiment should test the pricing frame together with the narrative that supports it. ## Five decision-makers read five different decks The same slide can create a different response depending on who reads it. A useful experiment represents the distinct roles that influence the renewal, including people who may review the deck after the meeting. This defines five buyer roles, the choices they influence, and the evidence each needs, not five invented biographies. ## Turn deck review into a randomized experiment A useful pre-test changes one decision variable at a time. The team can then estimate which intervention changes support for renewal rather than collecting reactions to an entire deck. - Name the commercial action. Choose among keeping pricing flat, raising it, or restructuring scope. Do not test an undefined goal such as "make the deck stronger." - Create controlled variants. Compare past-results-first with future-value-first sequencing, different pricing frames, or different placements of ROI evidence. Hold the account facts, proposed work, and visual quality constant. - Define the audience by role. Include the five decision-makers above in proportions that reflect the actual buying group. Keep known facts separate from assumptions about people who have not spoken directly. - Choose the behavioral outcome. Measure which variant changes support for renewal, willingness to defend the spend, or preference among scope options. Aesthetic preference is not the commercial outcome. - Randomize exposure. Show each participant or simulated buyer one controlled variant, then compare outcomes across conditions. The research approach is designed to identify which action changes the decision and where uncertainty remains. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. A number without its limits is marketing. That figure describes audience reach, not a recruitable group of 800 million participants. The leading experiment can then be validated with real human participants without changing the causal question. ## The five reads that expose a weak renewal argument The experiment should preserve the practical questions that an experienced account team would ask in a live review. Past-results read. Show the results section on its own. Ask which contribution matters most and whether the agency's attribution is credible. The day-to-day contact may remember the work, the sponsor the metric, and finance the cost. Narrative read. Show slides one through the end. Ask what changed during the year and what that change makes possible next. If three out of five decision-maker roles infer different stories, the deck is still a collection of slides rather than a coherent renewal case. Future-value read. Isolate the next-year proposal. Ask what new buyer or business outcome it is designed to cause. A list of three or four new capabilities is not a growth case unless each one connects to an outcome the client already values. Price-fairness read. Place the pricing slide beside past results and future scope. Ask whether the price follows from the value and work proposed. Two or three engagement options can make the choice explicit, but they should represent real scope differences rather than cosmetic packaging. Side-conversation read. Ask what each decision-maker would tell a colleague after the meeting. The deck needs to give internal advocates language they can defend on their own. ## Plan enough room to act on the result The value of the test depends on whether the team can change the deck after seeing the result. One inherited planning example reserves half a day for review three days before the meeting. That is a workflow illustration, not a Subconscious delivery commitment. The purpose is to leave space to revise the narrative, change an option, or remove an over-claimed result. A night-before review can identify the same weakness without leaving room for a considered response. The revision should follow the evidence. If a future-first sequence increases support while pricing remains fixed, change the sequence. If support falls when the deck adds capability language, remove or reconnect that language to the client's stated outcome. If uncertainty remains high across all conditions, do not force a winner. Return to the account facts or ask the actual client. ## What can and cannot the experiment decide? A published limit lets a buyer check the tool before they rely on it. A causal test can compare defined renewal-deck interventions. It cannot guarantee that the client renews, predict the exact behavior of a named executive, or settle contract terms on its own. It does not replace direct conversation with the actual decision-makers, the account team's knowledge of the relationship, or legal and financial review. The result is one input into the renewal decision. It is strongest when the alternatives are concrete, the audience definition reflects the buying group, and the measured outcome matches the commercial action. Naming the failure mode is what lets a buyer catch it before it does damage. It is weak when the team uses it to confirm a preferred story, treats generated language as evidence, or hides material account facts from the design. ## Bring one live renewal decision Start with the next consequential renewal on the calendar. Bring the current deck, the pricing or scope choice, the roles that influence the decision, and the outcome that would change the account team's plan. Book a working session to scope the variants, experiment, and human-validation boundary before the client meeting. --- # Interrogating a Simulated Persona vs. Testing a Causal Action URL: https://subconscious.ai/blog/use-cases/interrogating-simulated-persona-vs-testing-causal-action Published: 2026-08-11 Interrogating a simulated persona means asking one simulated character its opinion, while testing a causal action means running a controlled experiment that compares defined alternatives across a defined population and measures which one changes behavior. Sales enablement, account management, and product teams reach for an interrogatable AI persona: a simulated stakeholder you can question before a meeting, or a simulated customer type you can run a mock advisory session with. That conversation feels informative, but it is one simulated character answering whatever it is asked; it is not a controlled test of what changes a real buyer's behavior. The decision to make before relying on that conversation: is the answer evidence for a specific commercial choice, such as a pitch angle, an ICP definition, or a segment's likely objection, or does that choice need a controlled experiment that compares defined alternatives across a defined population? ## What a persona conversation gives you, and what it does not A simulated persona is built to hold a consistent point of view across a chat: it can be asked what would make it buy faster, what a competitor does better, or how it would react to a pitch. The conversation is fluent and can surface a plausible-sounding objection. What it does not give you is a comparison. Nothing in an open-ended conversation defines two or more alternatives and measures which one changes the simulated party's stated position more; the output is shaped by whatever was asked, not by a controlled design. Marketing-research writing on this category makes the same point: these tools extend the A/B test, not replace it, because a single simulated conversation has no built-in comparison (GreenBook, on why most executives can't define this category, 2026). [figure: Two columns. Left: persona interrogation, one character answers a question, no alternatives or population. Right: controlled experiment, defined alternatives tested across a population, yielding a measured effect.] ## Why does the difference matter for the decision? Treating a single simulated conversation as decision-grade evidence carries a specific cost: a fluent objection or preference with no defined alternative and no defined population behind it can send a team into a high-stakes meeting on an assumption, or lock in an ICP definition, that was never tested. The gap surfaces only after the deal or launch fails. ## How does Subconscious approach the same buyer questions? Subconscious runs a controlled discrete-choice experiment: it compares defined message, offer, or ICP-criteria alternatives across a precisely defined population and returns a measured causal effect with a confidence interval. That reframes the use cases sales and account teams reach for a persona chat to solve: - Sales-meeting prep. Instead of asking a simulated stakeholder what it thinks of a pitch, define the competing pitch angles and test which one moves the buyer segment's stated preference. - ICP definition. Instead of treating a persona's self-description as the ICP, define the candidate criteria as alternatives and test which set of criteria predicts the behavior that matters. - Customer advisory input. Instead of a mock advisory session with simulated customer types, define the roadmap alternatives and test which one the defined population responds to. The Subconscious research program explains how these experiments are structured and validated, and the how-we-work page walks through the method. ## What does this not replace? A controlled causal experiment does not replace live discovery calls, account relationships, sales judgment built from real conversations, or role-play training with real people. When a decision warrants it, a team can move the same causal question from a simulated experiment to real-human validation without changing what is being tested. ## Keep the terms distinct Audience reach and participant recruitment are two different things. A structured, population-scale simulation is not the same as a recruited panel of real respondents, and neither one is the same as an open-ended chat with a single simulated persona. ## Next step If the team can name the alternatives, the population, and the behavior that matters, bring that decision to a Subconscious working session. If the answer is still exploratory, the case studies page shows how other teams moved from an open question to a testable one. --- # Bayesian Modeling for Modern Marketing: Solving Real-World Attribution & CLV Challenges URL: https://subconscious.ai/blog/methods-and-validation/bayesian-modeling-modern-marketing-solving-real-world-attribution-clv Published: 2026-08-11 Cookie deprecation and shrinking first-party tracking have made channel attribution harder to trust. A marketing analytics lead choosing between a linear-regression attribution model and an uncertainty-aware Bayesian one is really choosing between a single confident number and a range that says how confident that number actually is, conditional on the model; neither a coefficient nor a posterior resolves the confounding introduced when spend tracks expected demand or when tracking data itself is missing. ## Why does the point estimate break first? A linear regression fit to customer acquisition cost (CAC) or channel effect returns one coefficient per channel. That coefficient looks precise, but it hides how much of it is signal and how much is noise from a shrinking, cookie-restricted data set. A budget reallocation built on that single number can move spend into a channel whose true effect range includes zero; a regression's standard errors and confidence interval can show that, but interpreting what that range means, and propagating it through the nonlinear adstock and saturation transforms typical of a media response curve, is where the linear approach runs out of road. Black-box machine learning models make the same mistake in a different shape: they can fit the historical pattern well and still give no account of which inputs the prediction actually depends on, or how sure the model is. ## What does a Bayesian model add? A Bayesian approach to media mix modeling starts from a stated prior, expert knowledge about plausible ranges for a channel's effect, then updates that prior against observed data, and returns a posterior distribution instead of a point estimate. Google's applied research on media mix modeling with carryover and shape effects documents this approach for advertising response curves that persist and decay over time (Bayesian Methods for Media Mix Modeling with Carryover and Shape Effects, Google Research). A parallel treatment applies the same hierarchical structure to marketing attribution specifically, modeling channel similarity and time-varying effects rather than a single fixed coefficient per channel (Bayesian Modeling of Marketing Attribution, arXiv). The practical difference shows up in three places: ## Where the shared thesis lives, and where it stops The underlying argument, report which action moves which outcome with quantified uncertainty rather than a single confident estimate, is the same argument behind Subconscious's approach to testing marketing and product decisions: controlled experiments on simulated markets with causal-effect estimation, reporting uncertainty where the study design supports it. A scope stated plainly is what lets a buyer check whether a tool fits their problem before they adopt it. Subconscious does not build media mix models, does not estimate carryover or adstock effects, and does not forecast customer lifetime value. That distinct econometric workflow means a team evaluating a Bayesian MMM specifically should read this as a framing explainer, not a comparison of interchangeable tools. What does transfer is the discipline of testing a decision, such as a price change, a message, or a positioning claim, as a controlled experiment and reporting the confidence interval around its effect, a coverage guarantee that rests on the randomization in the experimental design, distinct from the credible interval a well-specified Bayesian attribution model reports around a channel's contribution, which rests on the model and its priors. Teams that want to see how that discipline applies to a specific action can review the research behind the approach or see it applied on the leaderboard. ## Limitations The misses belong next to the hits, so a reader can see exactly what this piece does not claim. This explainer is about method fit, not tool selection. It does not recommend a specific Bayesian modeling library, replace an attribution or CLV vendor evaluation, or claim that causal experiments on simulated markets substitute for time-series marketing mix modeling. Teams with an active attribution or CLV modeling need should evaluate that decision on its own econometric merits; the how we work page describes the separate class of decision Subconscious tests. [figure: Four-step chain: a stated prior belief about a channel's effect, updated by a Bayesian model against observed data, producing a posterior range that shows whether the effect could be zero, feeding the budget decision.] --- # PyMC in the Browser: Why Deployment Architecture Isn't a Causal Validity Signal URL: https://subconscious.ai/blog/implementation-and-procurement/pymc-browser-why-deployment-architecture-isnt-causal-validity-signal Published: 2026-08-11 A vendor demo opens a browser tab and runs a Bayesian model with no server and no install. The demo is real. A modeling stack like PyMC can now run entirely client-side. The question a technical evaluator scoping a causal-experimentation or Bayesian-modeling purchase should ask is different: does removing the server change whether the experiment underneath produces a valid causal estimate? It does not. ## What changed to make browser-native modeling possible? Running a full Python data-science stack in a browser tab, rather than a stripped-down subset, depends on a Python distribution compiled to WebAssembly and run inside the page (Pyodide). That runtime lets a modeling library like PyMC substitute a pure-Python execution mode for its usual C or JAX compilation path, at a cost in raw sampling speed. It does not change what the model is doing: the same priors, likelihood, and sampler run, just on the visitor's own machine instead of a shared server. ## The three-step pattern behind the demo The deployment mechanics are simple, worth naming because that simplicity is what makes them easy to over-credit: - Declare packages. A page-level configuration block lists the libraries to install into the browser session, the modeling library, its tensor backend, and a plotting library, pulled the same way as from the Python Package Index. - Embed the model. Python code defining priors, a likelihood, and a sampling call sits directly in the page markup. A beta-binomial coin-flip model, an uninformative Beta(1,1) prior on the probability of heads, a Binomial likelihood connecting n observed flips (for example, n=10) to k observed heads (for example, k=5) through that probability, and No-U-Turn sampling to draw from the posterior, is a common first example. The same structure applies to conversion-rate and other binary-outcome estimation problems. - Open the page. No server deployment step exists. The runtime installs packages on load and the model is ready to sample. Three steps, no infrastructure ticket. That is a real reduction in setup friction for a demo, a workshop, or a teaching notebook. It says nothing about whether the beta-binomial model, or whatever model replaces it in a real evaluation, was the right experimental design for the decision being made. ## Where is this pattern genuinely useful? Browser-native execution has concrete uses beyond the demo: interactive notebooks that run without a local Python install, adaptive assessments that select the next question from a live posterior instead of maintaining two parallel implementations (one for batch fitting, one for real-time serving), and self-service webapps where a hierarchical model is fit against data a visitor uploads directly, without backend infrastructure. Each is an infrastructure win: fewer moving parts, faster iteration. None of them is a substitute for asking what the model is estimating and how that estimate was validated. A workshop notebook that lowers the bar to trying Bayesian modeling is not the same claim as a procurement-ready causal estimate. ## The evaluation checklist this replaces When a deployment-architecture claim shows up in a vendor evaluation, three questions separate it from an evaluation of the underlying method: - What is the model estimating, and is that a causal effect or a correlational fit? - How was the experimental design validated, independent of where it runs? - Can the same causal question move from a simulated setting to real-human testing without changing what's being measured? Subconscious is built around that third distinction. /research and /how-we-work document how that experimental design and validation process work, separate from any question of client-side versus server-side execution. [figure: Path from a browser demo through three questions, what's estimated, how design was validated, real-human testing, to a decision. Deployment simplicity sits aside, not feeding it.] ## Limitations A scope claim only counts if it says what it excludes. Subconscious does not ship a client-side or browser-embedded modeling runtime, and this is not a comparison of Subconscious's architecture to any browser-based Python runtime. No accuracy, speed, or cost claim about either approach should be read into this article. [figure: Three browser-native use cases, notebooks, adaptive assessments, self-service webapps, all pointing to "infrastructure win," with a separate branch showing none reaches a validity claim.] ## What to check before you sign off If a deployment-architecture claim is part of what's being evaluated, ask the vendor to show the experimental design and validation evidence on its own terms, separate from the demo. If that evidence exists only as "it runs in the browser," the procurement decision is being made on infrastructure, not on causal validity. --- # Simulated vs Recruited Research: Choose the Evidence Your Decision Needs URL: https://subconscious.ai/blog/comparisons/simulated-vs-recruited-research-choose-evidence-decision-needs Published: 2026-08-11 Choose the method by the consequence of being wrong. Use a simulated experiment to compare actions and narrow a directional question. Use real-human research when a consequential decision needs human validation, documented sampling, or observed behavior. When the stakes justify both, carry the same causal question from simulation into a human study. [figure: Branching path: a directional question routes to simulation; a consequential claim routes to a recruited study; a decision that is both routes through simulation, recruitment, then a benchmark check.] ## Match the method to the claim The useful distinction is the claim each method can support, not simulated versus real. Publishing where a method stops working lets a buyer check it before betting a launch on it. A recruited survey still records stated answers. It becomes behavioral evidence only when the design observes a click, purchase, abandonment, or another real action. Stated preferences can diverge from revealed choices, so a simulated answer should not be presented as observed behavior. The study compares stated and revealed preferences directly. ## When should you use a simulated experiment? A simulated experiment compares controlled alternatives before committing capital: concept rounds, message variants, audience exploration, competitive framing, pre-research scoping. The practical advantage is discrimination. A randomized experiment holds the decision context steady, varies an action, and estimates which alternative changes the outcome. That beats asking a modeled buyer for an unstructured opinion. Hard-to-reach buyer roles can still be represented in an early experiment, but representation is not recruitment: a modeled senior buyer helps form and test a hypothesis, not prove one participated. ## Does recruitment guarantee clean data? A method's failure modes belong published beside its results. Real participants matter, but the recruitment label does not guarantee clean evidence. Nonprobability surveys face documented risks from fraudulent respondents, bots, inattentive answers, and professional survey-taking. NORC's literature review summarizes these quality threats and the controls researchers use to detect them. Judge a recruited study by its sample, screening, quality controls, instrument, and connection to the decision. Use it when: - A single launch, pricing, or positioning call is expensive to reverse. - The market context is novel or has changed enough that prior patterns may not hold. - The result will become a public statistic and needs documented human sampling. - The study can connect stated reasons to analytics, sales data, or an observed action. A recruited study is not ground truth when its sample or instrument is weak. ## Carry one causal question through both stages The strongest sequence does not run two unrelated studies. A simulated experiment sharpens the alternatives, outcome, and target population, then hands that same causal question to a human validation study. Subconscious supports controlled causal experiments on a simulation of the market and can validate them with real human participants. The question carries over unchanged, so a decision-maker can compare the simulated result with a human baseline instead of reconciling different instruments. The replication record provides a defined benchmark for that comparison. A score is evidence only when its limits are published beside it. The misses sit on the leaderboard next to the hits. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market, and it does not remove the need for decision-specific validation. Read the causal fidelity paper and its methodology. Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people. Naming what a figure does not cover is what lets a buyer check the claim before relying on it. That figure describes population definition and audience reach, not a recruitable pool of study participants. ## What should you do before choosing a research method? Before choosing a method, write down the action, the outcome, and the evidence required to release capital. - For a reversible directional choice, run the controlled simulated comparison and state its boundary. - For a hard-to-reverse decision, require real-human validation of the same causal question. - For a behavioral claim, observe behavior rather than treating stated preference as a proxy. - For a public number, require documented sampling and methods. - For either path, define failure conditions before seeing the result. The leaderboard is useful when replication evidence and failure modes matter to the procurement decision. If the decision is already defined, bring the action, alternatives, and required evidence to a working session. --- # Building an AI Governance Layer for Market Research URL: https://subconscious.ai/blog/company-and-trust/building-ai-governance-layer-market-research Published: 2026-08-11 [figure: A four-stage path showing a finding moving from AI-assisted exploration through directional testing to human review, then escalating to a controlled experiment before it reaches a decision.] A research leader who lets AI-assisted output move straight into a stakeholder deck is making a governance decision, whether or not anyone wrote it down. The real question a governance layer has to answer is narrower than "can we use AI here": it is whether a given finding is strong enough evidence to act on, or whether it has to be escalated to a controlled experiment first. Publishing where a directional read stops keeps a number from turning into marketing. A directional AI read gets treated as validated evidence, a decision gets made on it, and the gap surfaces later, after budget or launch spend is already committed. The cost lands on the research function's credibility with the stakeholders who acted on the finding, not on the AI tool that produced it. ## Why this needs a rule, not a habit AI has moved from a novelty layer into daily research work: drafting analysis, formatting reports, preparing data, producing first-pass summaries. That does not mean research demand disappears; it means the mechanical parts of the job get faster and cheaper, which pushes the person doing the job closer to the decision. The exposure is that a research function which only produces output, without a rule for what that output is allowed to support, cannot defend a call after the fact. A workable governance layer needs four ordinary pieces, staged by how much the decision costs to get wrong. That is the same staged-review principle the NIST AI Risk Management Framework's Generative AI Profile sets out for AI-assisted work generally: "The NIST AI Risk Management Framework (AI RMF) is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems." NIST, AI Risk Management Framework (source) The value is not the AI output by itself. It is the traceable path from a question, through AI-assisted exploration, to the review and validation step that earned the right to inform a decision. ## Where a controlled experiment fits in that path Naming what a directional session cannot do is what lets a buyer check the claim against the record. An open-ended AI or synthetic-panel session is useful for the exploration and directional-testing layers: it surfaces hypotheses and rules out weak ones quickly, before anyone commits to fielding a study. It does not produce a defined comparison across defined alternatives, so it cannot answer the escalation-layer question on its own. That is the tier a controlled discrete-choice experiment sits in. Subconscious runs a controlled experiment comparing defined alternatives across a defined population and returns causal effects with confidence intervals. That gives a governance framework a defined comparison with quantified uncertainty, sitting between an open-ended AI read and a fully fielded human study, produced ahead of the slower, costlier fielding stage. Details on how those experiments are structured and validated are on how we work. A controlled experiment is one input to a governance framework, not the framework itself. It does not write disclosure text, approve which use cases are permitted, or supply an escalation workflow. Keep the scale of a simulated experiment distinct from a recruited human sample: a defined population is not the same thing as a set of participants recruited for a study. ## When to add real-human validation When the decision is expensive or public enough that a causal estimate alone is not enough evidence, the same defined comparison can move to real-human validation without changing the underlying causal question. That step matters most when a governance policy specifically calls for it, not as a default addition to every study. The misses sit right next to the hits in this record, and this is one of them. Real-human validation does not turn a causal experiment into an observed usability session, a clinical trial, or automatic proof of market performance. It answers the same question the simulated experiment asked, with a recruited sample, so a research function can show a stakeholder how the finding held up outside the simulation. ## Writing the rule down The mistake that makes this dangerous is writing governance after the first incident, once a fluent-but-unvalidated answer has already shaped a decision. The fix is not a policy document nobody reads. It is a workflow default: state what is being decided in a single sentence, name the audience and how much risk is attached, restrict AI or a directional panel to the exploratory stage only, have a person check the output, and label the evidence stage before anyone presents it externally. That labeling step is not optional. A finding presented as "directional" carries a different weight than one backed by a controlled experiment with confidence intervals, and a stakeholder who cannot tell which one they are looking at cannot make an informed call on it. ## Where this fits, and where it does not A governance layer built this way still leaves room for human judgment in research. What it changes is what a defensible research function looks like: closer to the decision, faster to generate a defined comparison, and explicit about which findings still need validation before anyone repeats them outside the room. Case studies show how that escalation path has worked for specific pricing, messaging, and positioning decisions, and a demo walks through how a specific decision would move through the layers above. This limit is stated here so a buyer can check the claim before acting on it. The limitation carries through every layer: a causal effect from a controlled experiment answers the question it was designed to test, for the population it was run against. It does not certify that a decision will succeed in market, and a governance framework that treats it as final proof rather than the strongest available evidence has the same defensibility problem it started with. --- # Validate a Product Idea Before You Commission Formal Research URL: https://subconscious.ai/blog/implementation-and-procurement/validate-product-idea-before-commission-formal-research Published: 2026-08-12 An innovation or insights lead facing a vague product idea has two bad options: commission an expensive formal study around an idea that isn't sharpened yet, or skip evidence and let a stakeholder's gut call the decision. The better path is a middle tier: directional, AI-assisted exploration that clarifies the idea first, then real, controlled validation once the decision is expensive or public. [figure: Four steps left to right: Exploration, Directional testing, Human review, Validation. A line between Human review and Validation marks where directional output becomes evidence.] ## Why the pressure to skip evidence is rising AI has moved from a novelty layer into daily research work. Demand for evidence has not disappeared: the U.S. Bureau of Labor Statistics puts market research analyst and marketing specialist roles on a growth track for the 2024-to-2034 decade (BLS, Occupational Outlook Handbook). The real risk is narrower: a team spends real money on a formal study while the underlying idea is still fuzzy or poorly framed. Automating the mechanical parts of research does not shrink the cost of a bad question. That cost arrives later, once the budget is gone. ## Should you pick a better AI tool or build an evidence system? The fix is not choosing the right AI tool. It is setting the rules up front: the scope of what AI-assisted output may do on its own, where a person must sign off, and the claims that need real validation before they touch a launch, pricing, or positioning decision. A workable system has four layers: - Exploration. Generate hypotheses, objections, and alternative framings for a loose idea. - Directional testing. Compare options quickly against a synthetic audience to see which framing holds up before spending on a fielded study. - Human review. Check the audience definition, prompt neutrality, source grounding, and business context before trusting any output. - Validation. Move to real respondent data, behavioral data, or expert review. The output of the first two layers is not evidence. It is a faster route to a sharper question: better segments, clearer use cases, and named objections a formal study can build around. ## Where should the line be drawn between AI output and real validation? The danger is treating directional, AI-assisted output as a substitute for real usage or demand testing. A fluent answer from a synthetic audience is not proof real buyers will act the same. Credible research separates output from evidence, then labels each: "directional read," "hypothesis for validation," "requires real-human testing before external claim." A controlled causal experiment fits here. Subconscious runs the same causal question, what action actually changes a buyer's choice, as a directional test first, then can test or validate studies with real human participants. Real-human validation does not turn a causal action test into an observed usability session, a clinical trial, or an automatic proof of market performance. It answers the same causal question with a different evidence source. ## What are the limits of directional AI testing? A directional or causal read does not eliminate the need for a human to define the audience and judge whether the output is credible. An audience graph built for controlled experiments is not a recruitable panel for open-ended qualitative work: it answers a specific causal question, not every research question a team might have. [figure: One causal question splits into two branches: synthetic audience gives a directional read, real participants give validation, not a usability test, trial, or proof of market performance.] ## What to do with the next idea on the list Do not rebuild the research process. Pick one real project with a live decision. State the business decision in a single sentence, pin down the audience and the risk at stake, and keep directional AI-assisted exploration to the exploratory stage. Review the output, mark what's useful or unsafe, and name the validation step that must happen before treating the answer as proof. When the cost of being wrong is high, a launch, a pricing change, a public claim, that validation step means running the causal question again with real participants before it ships. See how that step works in Subconscious's research process, or talk through a specific decision. --- # How to Evaluate a Synthetic Respondent Platform Before You Trust Its Output URL: https://subconscious.ai/blog/comparisons/how-evaluate-synthetic-respondent-platform-before-trust-its-output Published: 2026-08-12 Before signing with any synthetic-respondent or AI-research vendor, require one thing: proof that its simulated studies reproduce real human study outcomes, not just a claim that they do. A platform whose output looks plausible but has never been checked against real human behavior is a bad foundation for a pricing, messaging, or launch decision. ## What "synthetic respondent" covers The category spans three shapes of product. Conversational panels build AI personas and let a team question them directly. Behavioral simulations run larger population-level models and report distributions rather than individual answers. Survey-shaped tools route the same synthetic personas through a structured questionnaire so the output slots into existing survey-reporting workflows. The shape a vendor sells determines what kind of evidence you can reasonably ask it to produce: a conversational panel should be able to show you a transcript; a behavioral simulation should be able to show you the comparison it ran. ## The evaluation checklist before you sign Every vendor pitch collapses into a small set of testable questions: use them before scale, not after a launch depends on the output. ## Is correlation the same as causal evidence? An accuracy percentage against a historical panel tells you how often a vendor's synthetic answers matched what people already said. It does not tell you which of two actions, such as two prices or two messages, would cause a better outcome going forward. A number without its limits is just marketing. Independent researchers who have reviewed published experiments with synthetic-user studies have found the results inconsistent across studies and question types (A Review of Experiments with Synthetic Users, MeasuringU), which is one reason a single accuracy number should not be the deciding factor. Replication against an academic benchmark, checked study by study, is a sturdier bar than a marketing-page percentage (Testing Synthetic Data Against Academic Benchmarks: A Replication Study, Greenbook). Subconscious runs the checklist above against its own product. Its randomized experiments compare specific actions rather than reporting a single accuracy score, and every published result on its leaderboard is a study, not a headline percentage. Subconscious can also test or validate studies with real human participants, so a team can move from a simulated experiment to real-human testing without changing the causal question it started with. Publishing what a method is not is what lets a buyer check it against the claim. Its simulated audience is a person-level graph covering 800 million real people, which is a modeling scale, not a recruitable panel, and the two should never be presented as interchangeable. ## Where does simulation alone stop being enough? Simulated exploration fits early-stage work well: concept testing, message testing, and segmentation, where a same-day answer matters more than recruiting real respondents for every iteration. The failure modes belong on the record next to the successes. It fits poorly, on its own, with regulated decisions, deeply sensory product tests, or any situation where a stakeholder will demand real human signal before approving the outcome. For those cases, pair the simulated study with a smaller real-human validation step rather than treating the simulation as the final answer. ## Put the checklist to work Run the five questions above against any vendor before signing a contract. To see how Subconscious's own controlled experiments and validation evidence hold up against it, start with how the platform works or book a walkthrough. [figure: A five-item checklist, each item a criterion paired with the question to ask a vendor and why it matters: validation method, comparison type, audience basis, path to real-human validation, and disclosed limitations.] --- # AI Research vs Real Users: A PM Decision Framework URL: https://subconscious.ai/blog/comparisons/ai-research-vs-real-users-pm-decision-framework Published: 2026-08-12 A product manager chooses a research method for a specific decision: a feature bet, a price change, a message, or a positioning call. The method should match the evidence that decision requires. Use a simulated behavioral experiment when the question is about causal choices across defined options and the result will be treated as directional evidence. Use real participants when the decision depends on actual product use, spending, lived context, or proof that a person understood something. Sequence both when a simulated finding will drive a consequential action and needs a study-specific check against human responses. Choosing poorly creates two risks: shipping on a simulated signal that was never checked against real behavior, or committing to real-user fieldwork for a reversible, low-stakes call that did not require it. This timeframe is stated as scheduling context so a team can check the assumption before committing to a date. Two to four weeks is a historical planning assumption some teams have used for recruiting and scheduling, not a current service commitment. ## The core tradeoff A controlled behavioral experiment on a simulated population suits breadth: testing many concepts, framings, or messages before committing budget to one. A simulated respondent does not carry a real budget, a real procurement process, or years of lived context behind a workflow. Real-user research, including interviews, usability sessions, and surveys, is grounded in responses and behavior nobody had to model. It can capture physical interaction with a product and the weight of a real financial or emotional decision. ## When simulated testing is the right call Early-stage concept exploration. A team narrowing five feature ideas to two can use a controlled behavioral experiment to compare all five under one defined causal question. The result can reduce the option set, but it does not prove how people will use the final feature. Reversible decisions with bounded consequences. A feature flag, message, or concept suits simulated testing when the team can reverse the action and will treat the result as directional, not conclusive. Sprint-speed calls. A prioritization decision due before a fixed deadline, with no time to recruit and schedule participants, gets a directional read from a simulated session it would not otherwise have in time. Copy and messaging testing. Which value proposition changes preference, or which feature name is clearer, can be framed as explicit choices for a simulated population. The output is comparative evidence, not proof of market performance. Pre-validation before a human study. A simulated test can identify which concepts deserve a real-human check. The human study still carries the burden of validating the selected causal question. Competitive positioning. Modeling how buyers choose between two product descriptions across several framings can reveal which causal contrasts deserve further validation. ## When you need real users High-stakes pricing decisions. Willingness to pay depends on a real budget and a real pain of paying. A simulated population can model price sensitivity directionally, but a revenue-affecting price call needs a check against people who are actually spending money, and a defensible quantitative number needs a properly sized sample, not a handful of respondents (NN/g: Quantitative User Research: Study Guide). Usability testing with complex interactions. Observing someone click through a multi-step workflow and hit real edge cases requires a real person in front of a real prototype. Simulated testing can evaluate a described flow, not the physical and cognitive experience of using the software. The reverse holds for open-ended usability problems: a small qualitative sample, as few as five participants, can already surface most of a workflow's usability issues, even though that same small sample would not support a quantitative claim (NN/g: Why 5 Participants Are Okay in a Qualitative Study, but Not in a Quantitative One). Emotional and behavioral nuance. Whether someone trusts a feature with sensitive data, or how they feel about a change to a workflow they have used for years, involves personal context a model approximates rather than replicates. Regulatory or compliance validation. Proof that users understood a consent flow or disclosure requires documented human evidence. Simulated research does not satisfy that bar. Checking a simulated finding before it drives a decision. Periodically running the same question through both methods and comparing results is how a team learns which questions its simulated signal answers reliably. ## Can you sequence both without redesigning the study? Subconscious can run a controlled behavioral experiment on a simulated population, then check the same causal question with real human participants without redesigning the study. The human step is a study-specific replication check, not a general endorsement of every simulated result. A funnel that uses both stages this way: run a simulated experiment across 10 concepts to narrow to 3, then run 5 real-user interviews on those 3 to narrow to 1, then commit a full usability study to the winner. Each stage filters, so recruited-participant time goes only to what already cleared simulated screening. The public replication metric behind that validation step comes from the causal fidelity paper: our best configuration reaches 87% of the measured human ceiling on one study, 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters, drawn from roughly 300 published human studies across 9 domains, the mean is 0.73. This number is published next to the studies where the match came up short, so a buyer can check the claim before acting on it. That figure describes the "does simulated signal match a real-human study" step specifically. It is a validation result, not a guarantee for a new market, and not a claim about every possible research question. ## Four questions before choosing a method - How reversible is this decision? Easily reversible, such as copy or a feature flag, favors simulated testing. Hard to reverse, such as pricing, core architecture, or brand positioning, needs a real-user check. - What does the deadline allow without lowering the evidence standard? Time available before deciding is a planning constraint, not evidence that one method is valid. If the decision requires observed human behavior and that evidence is unavailable, narrow, postpone, or mark the decision unresolved. - Does the decision touch money or emotion? If people are paying for something, or the change reaches a workflow they rely on personally, lean toward real users. - Are you exploring or confirming? Exploring a wide option set favors simulated testing. Confirming a final call before it ships favors real users. [figure: A decision path: name the decision, then branch to a simulated experiment for reversible directional calls, real-user research for real spend or lived context, or a check against both for consequential findings.] ## What can't the validation step settle? Real-human validation, run this way, is a study-specific replication check on one causal question. It is not open-ended usability testing of a live prototype, and it does not walk a person through a multi-step UI. A simulated population built from broad audience data is a targeting and simulation input, not a recruitable panel for interviews. These outputs are described precisely here so a buyer knows exactly what to reconfirm before acting on them. Confidence intervals, segment breakdowns, and decision-memo outputs are specific to each study and need confirming per engagement, not treated as a standing guarantee. ## Make confidence question-specific One historical planning example is to compare simulated findings with existing human research for two to three sprints, recording where the methods agree and diverge. The origin of this cadence is stated directly here so a buyer can weigh it against current practice. That cadence is an inherited planning benchmark, not a standing delivery commitment. Trust in the simulated signal should be earned for the specific question, not assumed globally. ## Put the evidence burden on the decision Do not use simulated testing merely because a deadline is close. Use it when the causal question, action, and evidence standard fit the method. When actual spending, physical interaction, compliance, or lived emotional context determines the answer, collect human evidence or treat the decision as unresolved. See how one causal question can move from simulation to real-human validation, or bring a live decision to a working session to identify which evidence it requires. --- # Grounded AI Conversation Tools vs. Causal Experiments: Which One Answers Your Launch Question URL: https://subconscious.ai/blog/use-cases/grounded-ai-conversation-tools-vs-causal-experiments-which-one-answers Published: 2026-08-12 A grounded conversation tool is useful for exploring what a type of buyer might say. It is not evidence for what that buyer will do. Those are different questions, and mixing them up is how a directionally clean-looking test ships a launch decision the market doesn't confirm. [figure: Two-column table: conversation tool gives one transcript, checked against prior knowledge; causal experiment gives a measured comparison, checked against a human baseline.] ## What does a grounded conversation tool actually do? This category of tool builds a structured model of a customer type: role, context, history, values, decision patterns, communication style. It then grounds that model in real data such as interview transcripts, domain knowledge, or product-usage patterns. A consistency layer keeps the model's answers stable across different questions and phrasings, which is what separates it from typing "act like a procurement manager" into a general-purpose chat model. The output comes through a conversation interface: one exchange, a multi-model session comparing several customer types at once, or a structured interview script. A synthesis layer then compares the responses and pulls out themes. That stack answers qualitative questions well: what objections would this buyer raise, what language lands, what's confusing about a concept. It is not built to answer a quantitative one, such as what share of buyers would pay a given price, because a single grounded model produces one consistent point of view, not a distribution of real buyer behavior. ## Where does the accuracy break down? Grounding quality drives the ceiling: a model calibrated on real customer interviews produces better signal than one built from assumptions, and the discipline is younger than traditional survey methods, so calibration practice varies by vendor. Independent research on this exact failure mode found that pushing a language model to sustain many distinct conversational identities compresses their diversity over long sessions, with output drifting toward a smaller set of default responses even when the prompts describe different people (Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models, arXiv). Separate work on aligning generated profiles to real population distributions treats that alignment as an open research problem, not a solved one (Population-Aligned Persona Generation for LLM-based Social Simulation, arXiv). Neither result makes the category useless. Both mean a conversation transcript from a grounded model is a hypothesis about a market's reaction, not a measurement of it. "Counter-intuitively, the models achieving the highest per-persona fidelity consistently produce the most stereotyped populations" Xiao and colleagues, "The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models," arXiv (source) ## The comparison that matters for a launch decision ## What does a causal test add once exploration is done? Subconscious runs controlled experiments on simulated populations to estimate which action moves a specific behavioral outcome, with uncertainty reported where the study design supports it. Those studies run against a person-level audience graph covering 800 million real people, an audience definition for the experiment rather than a chat window to interview one buyer type at a time. Subconscious can also validate a study with real human participants when the decision calls for it, without changing the underlying causal question. See how the experiments run. ## What this doesn't cover yet Subconscious doesn't package a general-purpose multi-model conversation interface for qualitative exploration as a standalone product. A fast round of objection-drafting or message brainstorming across several buyer types in a chat window calls for a different tool at a different research stage. Discrete-choice-style experiment types, uncertainty ranges, and decision write-ups are specific to how a given study gets configured; they describe what a particular test can produce, not a standard feature of every engagement. ## Deciding which stage you're actually in If the open question is "what would this buyer say," a grounded conversation tool is the right instrument, and a research team will still want to read the transcript before designing a study. If the open question is "which version of this launch, price, or message actually changes behavior," that's a measurement problem, and it calls for a controlled test with a way to check the result, against a human baseline or against real participants. See how a study moves from design to a validated result, or bring the decision to a working session. --- # Empty States and Error Copy: Test Before You Ship, or Trust the Design Review? URL: https://subconscious.ai/blog/use-cases/empty-states-error-copy-test-before-ship-trust-design-review Published: 2026-08-12 Open a product in a fresh account and count the empty states and error messages a new user hits before finishing one task. That copy usually shipped untested. For a Head of Product weighing how much rigor first-session copy deserves, the real question is narrower: does this text change what a new or existing user does next, or does it just look fine in a design review? ## Why untested copy is a real cost, not a style problem Empty states are often the first meaningful text a new user reads after signup, whether onto a bare dashboard, an empty inbox, or an empty project list. Error messages arrive when trust is most fragile, when something has broken and the user does not know whose fault it is or what happens next. Nielsen Norman Group's usability research finds that unclear error messages increase task abandonment and erode trust in the product and the brand, and that effective error copy needs plain language, a specific description of the problem, and a constructive next step rather than an apology (Nielsen Norman Group, "Error-Message Guidelines"). A related NN/g breakdown of the "help users recognize, diagnose, and recover from errors" heuristic makes the same point about diagnosis and recovery, not just wording (Nielsen Norman Group, "Usability Heuristic 9"). "Usability heuristic #9 emphasizes the importance of good error-message design: "Help Users Recognize, Diagnose, and Recover from Errors."" Nielsen Norman Group, "Error-Message Guidelines" (source) Neither source is a Subconscious result; no Subconscious-run experiment, customer result, or benchmark exists yet for empty-state or error-copy testing specifically. That gap matters: this is a smaller, more tactical call than the launch, pricing, or positioning decisions causal experiments typically inform. The case for testing rests on the cost of guessing wrong at a first-session moment, not a proven lift number. ## What does a controlled comparison actually isolate? Design review answers "does this look right to us." A controlled comparison answers a narrower question: does this variant change what a defined audience segment says it would do next, compared with the version currently shipping. Testing copy against a defined segment is message and copy testing, not a broad usability study, clinical evaluation, or proof of market performance. Kept narrow, a comparison isolates three things: - Comprehension. Shown the empty state a new user lands on after signup, does the segment correctly describe what the product does and what to do next? - Motivation. What in the copy would make someone act, and what would make them close the tab instead? - Variant preference. When shown two or three candidate versions side by side, which one the segment says changes their stated next action, not just which one reads better. ## Errors are not one message: they are three different situations Most products use near-identical language across error types that call for different tone and next steps. A useful comparison separates the situations before testing the copy: Treating all three the same, using the generic "something went wrong, please try again," is the pattern Nielsen Norman Group's guidance argues against: it is neither specific about the problem nor constructive about the next step (NN/g, "Error-Message Guidelines"). ## How do you go from a simulated comparison to a real answer? A copy comparison against a defined segment narrows candidates fast, before anything ships. When the decision warrants it, Subconscious can also validate the same comparison with real human participants, moving from a simulated read to a real-human one without changing the underlying causal question. Use that step when the empty state or error sits on a path with real revenue or retention consequence; for most microcopy decisions, the simulated comparison alone is enough. ## Where this fits, and where it does not This stays a tactical, first-session decision. Treat any win-rate or before/after conversion figure attached to a specific copy variant as unverified unless it comes from a real test against a real segment; general claims like "one phrasing beats another" are not evidence for your audience. Run the comparison against your own defined segment rather than borrow someone else's result. Teams that test their worst empty state or error message first, the one everyone already flags internally, get a concrete answer: keep the current copy, or ship the variant that changed what the segment said it would do. See how this fits into a broader research workflow, or how Subconscious works before scoping a first test. [figure: Path: "Copy ready to ship" to "Design review," then "Controlled comparison" (comprehension, motivation, variant). Splits at "High-stakes path?": most ship there; high-stakes copy adds "Real-human validation."] Ready to test the copy your team already argues about internally? Book a walkthrough or read how other teams structured a first comparison in case studies. --- # Should Your Agency Add a Research-Validated Retainer Tier? URL: https://subconscious.ai/blog/buyers/add-decision-research-agency-retainer Published: 2026-08-12 Clients on a retainer expect proof the work will land before the media budget goes out, not performance metrics after it does. Impressions, clicks, and conversion rates are trailing indicators. An agency that wants to charge for confidence, not just output, needs a way to test creative before it ships and defend that test when a client asks what it actually proved. [figure: Standard Retainer branches into two paid-up tiers: Research-Validated Retainer, backed by Pre-Campaign Validation, and Full-Service Research Retainer, backed by ongoing Quarterly Strategic Research.] ## Why the standard retainer pitch stalls Most agencies have tried the same fix: run a survey or focus group ahead of each major campaign to validate the creative direction. It reads well in a proposal and breaks down for three recurring reasons. Cost. A panel provider's price for even a quick 300-respondent quantitative read swings between roughly $5,000 and $15,000, and that's before layering on a proper qualitative round (Drive Research's 2026 market research cost guide). Multiply that by four campaigns a year and the research line item can exceed what the client pays in total monthly retainer fees. Speed. Traditional research runs four to eight weeks from brief to results. An agency often has two weeks to produce creative for a launch, with no room to commission a study and get results before the work goes live. Frequency. A focus-group finding from three months ago describes a market that has since moved. Consumer sentiment, competitive dynamics, and cultural context shift continuously, so a quarterly snapshot often answers a question that has already changed by the time the campaign runs. ## The real decision: how much rigor sits behind the extra fee Agencies that solve the cost and speed problem still face a second decision: what kind of evidence backs the "validated" label. Two different products hide under that word. One version runs a synthetic-audience read and reports what an AI-modeled persona said it liked. That's directional: useful for catching an obviously wrong creative direction early, weak as a defensible claim once a client asks why the campaign underperformed. The other version runs a controlled discrete choice experiment: it holds a creative concept against defined alternatives, measures which one changes stated buyer behavior, and reports a causal effect with a confidence interval rather than a preference score. Subconscious runs studies this way, and a team can move the same causal question from a simulated run to a real-human validation step without re-scoping it. That distinction matters commercially: an agency charging a premium for "validated" creative is exposed the moment a client asks what the validation demonstrated. A directional opinion and a causal estimate are not interchangeable answers to that question. ## What the extra tier is worth charging for The retainer-tier idea is increasingly common: a 2026 review of agency retainer models found AI-assisted research moving from a pilot add-on toward a standard planning line item (AI Smart Ventures on the 2026 agency strategy retainer). The structure agencies have converged on has three tiers: - Standard Retainer: creative production and campaign management, no attached research claim. - Research-Validated Retainer: the above, plus pre-campaign validation and a recurring audience-intelligence report. - Full-Service Research Retainer: ongoing panel or audience management, competitive intelligence, concept testing, and quarterly strategic research. From one agency's public pricing, a client paying $10,000 per month for a standard retainer was quoted $2,000 to $4,000 more per month for the research-validated tier. Treat that spread as a starting anchor to test with your own clients: it reflects one vendor's retainer economics, not a benchmark Subconscious sets or a Subconscious price. ## Positioning it as risk reduction, not research Clients rarely pay more because "we added research." They pay more because the research answers a question they already worry about: will this work before we spend the media budget on it. Framing that tests well: - "Before your media budget goes out the door, we can already tell you whether this creative changes behavior." This reframes the fee as insurance against wasted spend, not a research line item. - "We catch the creative problem before your customers do." No client wants to learn a campaign underperformed three weeks after launch; a pre-campaign test catches it while it's still cheap to fix. Keep the claim behind that framing honest. If the test behind the tier is a directional synthetic read, say so and price it as directional. If it's a controlled experiment with a causal estimate, that's the claim that survives a client asking "prove it." See how a comparable test reads in a published case study. ## How to add the tier without overselling it - Pick two or three clients on the roster where pre-campaign validation would matter most. Longer sales cycles, higher-stakes launches, or a recent history of creative disagreement are the strongest candidates. - Scope one testable causal question per client, not a general "audience read." Define the two or three creative alternatives worth comparing before building anything. - Run a pilot on the next live campaign concept and show the client the difference between a validated decision and an unvalidated one, with the actual result, not a hypothetical. - Price the tier to the evidence, not the label. A directional synthetic read and a controlled experiment with a confidence interval are not the same product and shouldn't carry the same premium. - Build the report into the recurring client meeting, so the research becomes an expected part of the retainer rather than a one-off deliverable. [figure: Two columns. Left: synthetic-audience read gives a preference score, no confidence interval. Right: discrete choice experiment measures which creative changes behavior, gives a causal effect with a confidence interval.] ## Where this breaks down A claim without its edges is marketing. This tier has edges. A research-validated tier is not a substitute for a fast, embedded creative-feedback loop built for weekly retainer cadences, and nothing here should be read as a same-day or minutes-level delivery promise. It also isn't proof of campaign performance: a causal estimate from a controlled experiment shows which concept is more likely to change behavior in the test population, not what happens once the campaign is live, subject to media placement, creative execution, and market conditions the study didn't model. Real-human validation, when it's used, confirms the same causal question against real participants. It doesn't upgrade a concept test into a guarantee. Agencies that sell the tier as certainty instead of evidence recreate the exact credibility problem the tier was supposed to fix. To see how the underlying method works before pricing a retainer amendment around it, book a walkthrough. --- # AI Campaign Effectiveness Research: Test the Strategy First URL: https://subconscious.ai/blog/use-cases/ai-campaign-effectiveness-research-test-strategy-first Published: 2026-08-12 A brand and marketing leader who commissions a six-figure campaign usually finds out whether the message worked only after the media is bought, the creative is running, and a brand-lift study has had four to eight weeks to close. By then the budget is spent, and the next planning cycle is already underway. The decision that mattered, whether this message, this creative direction, and this channel plan would move the target audience, was never tested before it became irreversible. ## Why does the standard measurement cycle arrive too late? A number without its limits is marketing copy, so this cycle's limit gets named plainly. A brand-lift study is built to confirm impact after a campaign runs, not to compare alternatives before one is chosen (Happydemics). Marketing-mix models need several quarters of spend history before they produce anything a planner can act on. Post-campaign surveys carry recall bias and self-report distortion on top of the wait. None of these tools were built to answer a question before the money commits. The misses sit on the public record next to the hits, and cost is one of them. Cost compounds the timing problem. A rigorous lift study is expensive enough that most campaign budgets cannot fund one for every message, creative route, or channel plan under consideration (InfluenceFlow). Teams fall back on click-through and conversion metrics, which describe direct response, not the perception shift a brand campaign is built to create. Naming a failure mode is what lets a buyer check it, and this one gets named directly. There is a third failure that gets less attention: scope. Creative gets tested on its own. Copy gets tested on its own. Media plans get scored on reach and frequency. Almost nobody tests whether the creative, the message, and the channel work as one system, or whether a claim that reads as credible on a podcast still reads as credible on Instagram. ## Why is a campaign one behavioral system, not three separate tests? Testing a finished ad in isolation answers a narrower question than testing the strategy behind it. Campaign effectiveness depends on how the audience definition, the strategic concept, the creative route, the channel context, and the exposure sequence interact, not on how each element scores by itself. That means comparing alternatives before anything is finished: the raw concept ("we are telling this audience X, through these channels") against a competing concept; a creative direction built on humor against one built on customer proof; a message placed on LinkedIn against the same message placed on a podcast; an awareness-then-consideration sequence against a version that skips straight to retargeting. Each comparison isolates one variable so the result stays interpretable. ## Running the comparison as a controlled experiment Subconscious runs this as a causal behavioral experiment rather than a single reaction check. The workflow starts with defining the target audience and the decision at stake, then compares campaign alternatives, message against message, creative route against creative route, channel plan against channel plan, against that audience under controlled conditions. The output is a causal effect and, where the study design supports it, a confidence interval for each alternative, not a single plausible-sounding answer. A score published without its ceiling is marketing, so the ceiling gets published too. That method is validated against real outcomes, not just internally consistent. The underlying causal methodology reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. And because the comparison runs against a person-level audience graph covering 800 million real people, kept separate from any recruited panel, a marketing team can define a specific audience segment rather than settle for a generic one. [figure: A five-step decision path moving testing before the media spend: define the audience and decision, test message alternatives, test channel fit, lock the strategy, then confirm impact after launch.] ## What does this replace, and what does it not replace? This process replaces guessing at message-channel fit before a campaign locks. It does not replace a post-campaign brand-lift study with real respondents. Subconscious can move from a simulated experiment to a real-human validation study without changing the underlying causal question. Used this way, the earlier test and the later study answer the same question at two points in time: does this message, in this creative form, through this channel, change how the target audience thinks about the brand. A campaign that clears the earlier test still needs the later confirmation. A campaign that fails the earlier test never has to reach the expensive one. [figure: Two columns: simulated test before launch vs real-human study after launch, both asking whether the message changes audience perception. A failed left result skips the right column; a passed one still requires it.] ## Where this changes a marketing leader's decision The clearest use is a brand campaign built to shift perception rather than drive an immediate click, since perception is the hardest outcome to measure after the fact and the costliest to get wrong before it. Product launches carry the same weight for a different reason: the first impression a launch campaign makes is difficult to undo, so testing the launch strategy, not only the launch ad, lowers the risk of a rollout that misses. Repositioning work benefits because the real question, whether a new message actually shifts an established perception or just bounces off it, can only be answered by testing message and audience together. And a performance campaign that has plateaued despite creative iteration often has a strategic problem, not a tactical one, which single-ad testing will not surface. A marketing leader carrying a campaign decision this quarter can see how a study like this gets scoped before committing the media plan, rather than after. --- # Test Investor Cold Email Copy Before You Burn a One-Shot List URL: https://subconscious.ai/blog/buyers/test-investor-cold-email-copy-before-burn-one-shot-list Published: 2026-08-12 A pre-seed or Series A founder writing cold investor outreach has one real constraint: most named investors on a target list read a first email once. A weak opening line or an unclear framing burns that contact. The founder has spent a scarce shot on a bad draft while the fundraising calendar keeps moving and the reply rate stays low. [figure: Five steps left to right: draft distinct email framings, run a causal comparison against a defined investor segment, identify the stronger framing, validate with real readers, send the winning version to the list.] ## Why the copy decides more than the channel Founders often blame the channel: cold email is dead, only warm intros work. That diagnosis fits founders who already have a network dense enough to route around cold outreach. For everyone else, the channel is not optional, so the real lever is the copy itself: the subject line, the first sentence, and whether the ask feels proportionate to a stranger's inbox. Reply rates on cold outreach vary widely by list quality, targeting, and how personalized the message is. Third-party benchmarking shows the same spread across cold email generally, not just investor outreach (Instantly, Cold Email Benchmark Report 2026). That spread is the argument for testing copy before sending: on a list a founder cannot easily re-run, closing even part of the gap between a generic email and a tuned one changes how many first meetings a raise produces. ## What does an investor decide in the first few seconds? An investor's inbox decision is fast and mechanical: glance at the subject line, skim the first sentence, then reply, archive, or "come back to this," which usually means archive. That sequence rewards a small set of things a founder can test directly before the list ever sees the email: - Whether the subject line earns an open - Whether the first sentence earns a second sentence - Whether the reader understands what the company does and why it matters to them specifically - Whether there is a reason to reply now instead of later - What reads as generic, pattern-matched, or low-effort That last point is the one founders are worst at judging on their own, because they wrote the email and cannot read it as a stranger who has seen thousands of similar pitches. ## How do you test which email framing works? The useful reframe is to treat "which version of this email gets more replies" as a testable causal question about a defined audience segment, not a matter of instinct or a single friendly opinion. Subconscious.ai runs controlled discrete-choice experiments on synthetic populations and returns causal effects with confidence intervals for comparisons like this, for example comparing two or three framings of the same email against a segment built to resemble the investor archetype a founder is actually emailing. That is a message-testing application of the same causal-experiment method Subconscious uses for pricing, positioning, and creative testing elsewhere, not a dedicated investor-outreach product. Read more about how Subconscious structures and validates a comparison like this. The misses get published next to the hits so a buyer can check the method. Two or three friendly investor contacts giving a quick read is not a substitute for this. It is one useful, informal data point, but it is not a controlled comparison, and it will not tell a founder which specific line is costing them replies across the segment they actually plan to email. ## What this does not tell a founder A number without its limits is marketing. Testing copy answers a narrower question than founders often expect. It does not predict: - Whether one specific named partner replies on a given day, since that depends on the state of their inbox that morning - Whether a category is fundable right now, since that is a market-timing question, not a messaging question - Whether the underlying traction is investable, since a copy test evaluates the email, not the business behind it Naming the failure mode here lets a founder verify the result before trusting it. For those questions, a founder still needs direct investor conversations and market signal. Copy testing is for the part of the outreach a founder actually controls: what the email says and how it is structured. Published research on the underlying causal-testing method covers how these comparisons hold up outside message testing as well. ## Can a simulated comparison be validated with real readers? When a founder wants more confidence than a simulated comparison provides, the same causal question, which framing gets a stronger response, can move to real-human testing without changing what is being measured. Subconscious can test or validate studies with real human participants, letting a team confirm a simulated result with recruited readers before committing a full list to one version of the email. [figure: Three questions a copy test cannot answer, set against the one it does answer: which framing reads better to the reader.] ## What to do before the next batch goes out - Pull the current investor cold-email template and identify two or three genuinely different framings, not several minor rewrites of the same structure. - Define the investor archetype being emailed: stage, check size, category focus, and fund profile. - Run a controlled comparison of the framings against a segment built to resemble that archetype. - Ship the framing with the strongest predicted response, and track the real reply rate against the previous batch to see whether the result holds. Founders who want to scope a specific outreach test before committing a list to it can book time to talk it through. Background on the company is at About. --- # AI Personas vs. Buyer Personas: When to Use Each URL: https://subconscious.ai/blog/comparisons/ai-personas-vs-buyer-personas-when-use-each Published: 2026-08-12 Buyer personas and AI personas both try to answer who your customer is, but neither one tells you whether a specific message, price, or feature will actually work with that customer. That gap matters most right before a launch: a marketing or insights leader has a persona deck everyone already aligns around, and must decide whether to ship the next campaign on that deck's assumptions or test it against real audience response first. [figure: A persona deck splits into a buyer persona that aligns the team and an AI persona that stress-tests one character. Both converge on a gap closed only by a controlled experiment yielding a causal result.] ## What are buyer personas? Buyer personas sketch an ideal customer as a semi-fictional character, built from market research and existing customer records. The sketch typically notes demographics, behavior, motivations, goals, and friction points, bundled into a name and short backstory. Buyer personas are designed for alignment. They hand a team a common vocabulary for talking about customers, so a label like "Sarah, the Enterprise Procurement Manager" stands in for a whole set of assumptions the group can point to instead of re-explaining them each time. A well-built persona sharpens marketing copy, gives sales useful context on the buyer, and helps product teams prioritize which features matter most. A persona deck without its limits is just marketing copy. The limit: a buyer persona is static. It gets written once and sits in a deck. It cannot answer a new question or surface a reaction nobody anticipated, so a launch built entirely on it can fail in a way the team had no early warning for, after the budget is already spent. ## What are AI personas? A conversational AI persona is a model built to behave like one type of person, not describe them: define a role, context, history, beliefs, and behavioral patterns, optionally grounded in real customer data such as interviews, CRM notes, or support tickets, and the result is a character a team can question, challenge with scenarios, or pitch to. An AI persona of this kind is built for interrogation, not alignment. It is useful when a team has a specific question and wants a quick, exploratory read from a simulated character. Naming where a tool stops is what lets a buyer check it before betting budget on it. A simulated conversation with one character, however well grounded, is still a role-play, not a controlled experiment with confidence intervals around the result. ## The key differences ## When to use buyer personas - Bringing new hires up to speed on who the customer is - Getting marketing, sales, and product to agree on the target customer - Creating shared vocabulary across a large team - Producing something shareable and presentable to stakeholders ## When to use a conversational AI persona - Quickly stress-testing an idea against one simulated character before it's ready for formal testing - Preparing talking points for a sales meeting or investor conversation - Drafting hypotheses about how a customer type might react, to be tested later ## Can either persona type settle the decision? Neither tool measures how a defined audience actually responds to a specific choice, and neither replaces a controlled experiment when real budget is on the line. A persona deck can't be interrogated for something it never anticipated, and a single simulated conversation can't produce a causal, confidence-interval-bound answer about how a broader audience would respond. Subconscious.ai runs controlled discrete-choice experiments against a defined audience instead of describing or role-playing one. Where both personas stop at a plausible description, this approach tests the actual decision, message, price point, or feature choice against measured behavioral response and returns causal effects with confidence intervals (see the causal fidelity paper: 87% of the measured human ceiling on one study, mean 0.73 across the 43 studies passing design filters). Some of that testing gets validated further with recruited real-human panels, so a team can move from a large-scale simulated read to confirmation with real respondents without changing the underlying question being asked. See how this fits into a research workflow. ## Limitations The misses belong on the page next to the hits. This is a decision-testing tool, not a persona-chat product, and it is not a substitute for the alignment work a buyer persona does inside a team. Treat audience-reach scale and recruited real-human validation as distinct claims: one describes the scale of the simulated experiment, the other a separate, smaller confirmation step with actual people. ## Next step If the team already has a persona deck and is deciding whether to ship on it, the fastest way to reduce that risk is to run the specific message, price, or feature choice as a controlled experiment before committing budget. See recent results on the case studies page, or book time to scope a test against your own audience. --- # How Partial Pooling Supports Decisions from Sparse Survey Data URL: https://subconscious.ai/blog/methods-and-validation/how-partial-pooling-supports-decisions-sparse-survey-data Published: 2026-08-12 A sparse survey can support a segment decision when the model shares information across related groups and the result carries its uncertainty. It cannot support the decision when a thin cell is presented as a precise standalone fact. ## A decision case built on thin cells A public-opinion team needs an estimate for a demographic or geographic segment. The overall survey is useful, but the segment has only a few observations. A direct cross-tab would let those observations dominate the answer. Collecting more data could help, but the team first needs to know whether the existing sample contains enough structure to support a model-based estimate. That leaves two costly errors: - Treat noise in a thin cell as a stable segment preference, then commit messaging or resources to it. - Discard useful evidence and commission more sample before testing whether a hierarchical model can support the decision. ## How does partial pooling change the evidence? Multilevel regression and post-stratification (MRP) has two parts. First, a multilevel model estimates responses across demographic and geographic groups, allowing related cells to inform one another through partial pooling. Second, post-stratification weights those cell estimates to the composition of the target population (Using Multilevel Regression and Poststratification to Estimate Dynamic Public Opinion). A data-rich cell is influenced more by its own observations. A thin cell is influenced more by the shared pattern in the hierarchy. The model does not erase the sparse segment. It makes the amount of direct and borrowed evidence explicit. [figure: Left-to-right chain: thin cells feed a multilevel model sharing information across groups, reweighted to the population, producing an estimate with an uncertainty range that branches to action or more testing.] The method answers a different question from a direct cross-tab or a causal experiment: ## What can the estimate authorize? The estimate should change the decision only to the extent that its uncertainty permits. A concentrated range may support a bounded action. A wide range may support a provisional test, more data collection, or no action. Reporting only a point estimate hides that distinction. Structured priors can help when an MRP model contains many interactions and sparse cells, but they do not remove the need to test whether the model is well specified (Improving multilevel regression and poststratification with structured priors). A number without its limits is marketing. Three limits remain: - The misses belong on the record next to the hits. The hierarchy can be wrong. Partial pooling helps only when the grouped segments share meaningful structure. - Population weights are not a cure for missing variables. Post-stratification adjusts for included population dimensions. It cannot repair selection bias tied to factors the model does not represent. - An estimate is not an intervention effect. MRP can estimate segment opinion or preference. It does not establish which message, product, or policy action caused a behavioral change. ## The validation gate before action For a product, pricing, or messaging choice, the next question is causal: which action changes the outcome for the segment? Subconscious uses controlled experiments on simulated populations to compare actions, with uncertainty reported where the study design supports it. The MRP estimate can help define the segment and prior evidence. It does not replace the experiment. When the cost of a mistaken segment decision warrants another gate, Subconscious can test or validate studies with real human participants without changing the causal question. Human validation checks the decision against real behavior. It does not turn a model-based estimate into automatic proof of market performance. Before acting, require a direct answer to four questions: What information was borrowed across groups? Which population counts shaped the estimate? How wide is the uncertainty? What evidence would cause the team to change its decision? --- # Directional Read or Decision-Grade Evidence? A Buyer's Test Before You Spend on Synthetic Research URL: https://subconscious.ai/blog/use-cases/directional-read-decision-grade-evidence-buyers-test-before-spend-synthetic Published: 2026-08-12 If a synthetic panel gives you a plausible-sounding read on a campaign, a price, or a message, the question that matters before you commit budget is not which platform produced it. It is whether that read is a directional guess or a causal effect you can defend. Commit spend on the guess and the failure shows up later, as a mispriced offer or a message that tested well but flopped with real buyers. ## Why does the synthetic research category exist? Traditional human research is slow and expensive in exactly the places a marketing or insights leader needs to move fast. A focus group typically runs 3 to 4 weeks from brief to readout (historical planning example). Recruiting a hard-to-reach segment, such as senior buyers or regulated professionals, can run into the thousands of dollars per study (historical planning example). A team limited to a handful of human sessions gets 5 to 10 questions answered; a faster method can ask hundreds in an afternoon (historical planning example). That speed and reach gap is real. It is why synthetic panels became a normal part of the research stack. Speed does not make a read decision-grade. It makes it fast to obtain and easy to over-trust. ## The two things a "read" can mean Before shortlisting any research method, separate two outputs that look similar on a slide: - A directional signal. A model's best guess at how a segment would respond, reported as a single score or a plausible-sounding summary. Useful for early exploration. Not evidence a specific action caused a specific outcome. - A causal effect estimate. A quantified answer to "if we change this input, how much does the outcome move, and how confident are we," reported with a confidence interval, not a point score. A buyer who cannot tell which one is on their desk cannot tell how much budget risk it is safe to attach to it. ## What is a synthetic read weak at? A number published without its limits functions as marketing. External validity research on discrete choice and vignette methods is the relevant evidence here, not vendor marketing. Controlled comparisons between stated-preference experiments and real-world behavior find that these methods track actual choices reasonably well on routine, low-stakes decisions, but the fit degrades on unfamiliar or high-stakes choices (Hainmueller et al., PNAS). A systematic review of discrete choice experiments against real health-related choices reports similar external-validity gaps depending on the choice context (Springer, The European Journal of Health Economics). The pattern holds for synthetic panels built on top of these methods: directional accuracy is highest on routine, low-precedent-free questions, and weakest on emotionally charged behavior, novel markets, and decisions driven by social contagion, where there is no calibration data for the model to lean on. That is not a reason to avoid synthetic methods. It is a reason not to treat their output as a finished answer on anything where being wrong is expensive. ## Four questions to ask before you trust the number - Is this a score or an effect? A single plausible number is a signal. An effect with a confidence interval is evidence. Ask which one you are looking at. - What decision is riding on it? Early concept exploration and message iteration can run on directional signal alone. A pricing change, a launch decision, or a claim going into a regulated channel should not. - Can the method be checked against real people without re-asking the question? A study designed to run as a synthetic experiment and be checked against recruited human respondents, without changing what is being tested, is a materially different guarantee than a synthetic-only read with no path to verification. - Does the source publish where the method fails, not just where it works? A published failure mode (emotionally charged behavior, novel markets, network effects) is a sign the benchmark is honest. A claim with no stated limitation usually has one it isn't disclosing. ## Where does a causal experiment change the answer? Subconscious runs controlled discrete choice experiments, including McFadden discrete choice, mixed logit, and integrated choice and latent variable models, and returns a causal effect estimate with a confidence interval rather than a single predicted score. That answers question 1 directly. Subconscious can also test or validate studies with real human participants, which answers question 3: a team can move from a simulated experiment to a real-human check without changing the causal question being asked. Recruiting real respondents for that check is distinct from the scale of any audience model behind the simulation; keep those two claims separate when you evaluate a research method's pitch. This matters most exactly where the PNAS and European Journal of Health Economics findings above say synthetic-only methods are weakest: unfamiliar, high-stakes, or emotionally loaded decisions. A confidence interval tells you how much to trust a single run. A human check tells you whether the causal question still holds outside the simulation. ## Where this does not change the answer Not every decision needs this level of rigor. If the question is whether a headline direction feels closer to what your last three campaigns tested, a directional synthetic read answers it fine, and a full causal experiment with human validation is overkill. Reserve the four-question test above for decisions where a wrong answer costs more than the research would have. See how the underlying method works in more detail in our research documentation and how a study moves from design to a validated result in how a Subconscious study runs. ## What this does not settle - Naming where risk remains lets a buyer check the claim before spending. A confidence interval does not eliminate risk on novel markets with no calibration precedent; it quantifies the uncertainty that remains. - Real-human validation checks the causal question against recruited respondents. It does not turn a discrete choice experiment into a clinical trial, a usability study, or a guarantee of market performance. - No accuracy figure, from any source, is a substitute for reading its stated failure modes. Treat a benchmark that hides its weak cases as an incomplete benchmark. ## Run the test on your next decision Before committing budget on a synthetic read, apply the four-question test above to the last panel result you were handed. If it fails on question 1 or 3, that is the signal to ask for a causal effect estimate with a confidence interval, and a human check, before the spend goes out the door. See recent case evidence for what that verification path looks like end to end, or book a walkthrough of a controlled experiment on your own decision. [figure: Two columns. Directional Signal: single score, no interval, no verification, for early exploration. Decision-Grade Evidence: effect with confidence interval, checkable against real respondents, for pricing and launch.] --- # How to Roll Out Causal Experimentation Past a Single Pilot URL: https://subconscious.ai/blog/implementation-and-procurement/how-roll-out-causal-experimentation-past-single-pilot Published: 2026-08-12 An insights or research-ops lead who has run one causal experiment on Subconscious faces a harder question: how far to commit before the method has proven itself. Buy full integration too early and you spend procurement cycles on a workflow that has not yet earned trust. Treat the pilot as a one-off study and the answer never becomes a repeatable input to launch, pricing, or messaging decisions. The way out is a staged rollout rather than a single decision. Each stage answers a specific question before the next stage asks for more budget or more workflow change. ## Stage 1: How do you prove the causal question on a general population? Start with one controlled discrete-choice experiment against a general population. The goal is narrow: confirm the causal question is well formed and the experiment produces a clear answer about which action drives the outcome, not to validate every downstream workflow at once. This mirrors standard practice in discrete-choice-experiment methodology, where pilot testing precedes any scaling of sample size or design complexity (Improving Methods for Discrete Choice Experiments to Measure Patient Preferences, NCBI Bookshelf). A small pilot surfaces design problems before they get expensive to fix. "Small problems in a pilot study may not have drastic effects on the main DCE results, but certain DCE designs (eg, those with correlated attributes) may require special care, such as added sensitivity analyses." Alan R. Ellis, Improving Methods for Discrete Choice Experiments to Measure Patient Preferences, NCBI Bookshelf (PCORI) (source) ## Stage 2: How do you test against your own customer data? Once the general-population experiment holds up, the second stage repeats the same causal question against the organization's own customer data. This is where the staged approach earns its name: you are not committing to a new platform, you are asking whether the same causal pattern holds for the customers you actually have. Pilot-to-scale sequencing like this is well documented in the discrete-choice-experiment literature, including guidance on how pilot results should inform subsequent survey design decisions rather than being treated as final (Improvements to Survey Design from Pilot Testing a Discrete-Choice Experiment, The Patient, Patient-Centered Outcomes Research, Springer Nature). Subconscious can also validate a study with real human participants at this stage, moving a team from a simulated experiment to real-human validation without changing the underlying causal question. ## Stage 3: When should you decide on a recurring cadence? Only after the causal pattern holds across a general population and a team's own customer data should the team decide whether the question is worth asking on a recurring cadence, rather than as a one-time study. This decision belongs to the team, not the vendor: a pattern that holds once is evidence, but a pattern worth tracking quarterly or before every major launch is a different commitment. ## Stage 4: Embed into existing workflows, deliberately Embedding an experimentation cadence into an existing research or product workflow is a real commitment, and should follow the three stages above rather than precede them. A team that embeds first and validates later risks building process around an untested causal claim. ## What this staged approach is not None of the four stages above should be read as promises about automation. Deciding to run experiments on a recurring cadence does not mean CRM data flows into the platform automatically, and it does not mean the platform runs continuously in the background generating findings without a team defining the next causal question. [figure: Two branches: committing to deep integration too early wastes procurement cycles and stalls buy-in; treating the pilot as one-off leaves a real answer unused.] ## Choosing the stage you are actually ready for The cost of getting this sequencing wrong runs in both directions: committing to deep workflow integration before a single experiment has proven the causal question wastes procurement cycles, and treating the pilot as a one-off leaves a real answer sitting unused instead of becoming a repeatable input to decisions. [figure: A four-step path: a general-population pilot experiment, testing the same question against the org's own customer data, a team decision on recurring cadence, and embedding that cadence into existing workflows.] Map your rollout to the stage you are actually ready for, not the one furthest along. Start with research to see how a single experiment is structured, or book a session to scope which stage fits your team's current data and workflow. --- # A Quick Persona Draft vs a Tested Decision: Choosing the Right Tool URL: https://subconscious.ai/blog/comparisons/quick-persona-draft-vs-tested-decision-choosing-right-tool Published: 2026-08-12 A persona sketch from an in-browser writing assistant is fast, but speed is not evidence. Before greenlighting a positioning, messaging, or pricing change, the real question is whether that sketch is grounded in anything tested, or whether it just reflects whatever pattern the underlying model defaults to. ## What the two tools are actually built for Voilà is an in-browser AI writing assistant that surfaces inside email, docs, social, and CRM tools to draft messaging, summarize text, and sketch quick persona profiles on the fly. It is a general productivity assistant, not a research or experiment platform. Subconscious is built for a different job: testing a specific product, pricing, or messaging action against a defined buyer population before a team commits budget to it, using a randomized experiment run on a simulation of that market and checked against real human behavior. Neither tool is a worse version of the other. They answer different questions. ## Why is one generated persona sketch not decision-grade? A single persona produced by a general-purpose assistant reflects one pass through a language model, with no comparison group and no check against how real people actually respond. Published 2026 reliability research on persona-conditioned language models documents demographic flattening and reduced response variance when models are asked to stand in for a population: the generated respondents cluster around a narrower, more "average" answer than the people they are meant to represent (Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents, ACM Web Conference 2026 Companion Proceedings). A companion study on when digital personas can reliably approximate human survey findings reaches a similar caution about treating one generated profile as a stand-in for tested demand (When Can Digital Personas Reliably Approximate Human Survey Findings?, arXiv). That is the gap that matters for a positioning or pricing call. A persona sketch can help a team draft copy faster. It cannot tell them whether the message actually moves the buyer, because there is no test built into producing it. ## How does a causal test close that gap? Subconscious runs a randomized experiment against the actual action under consideration, such as a specific price point, message, or launch decision, across a defined buyer population, rather than generating one profile and treating it as representative. The output is a causal read on which action moves the outcome, with the comparison built in from the start instead of added afterward. A simulated result stated without its validation step is a marketing number. When the decision calls for it, a team can move from that simulated experiment to a real-human validation check without changing the question being tested. Whether that extra step is worth taking depends on the stakes: a wrong call has to be expensive enough to earn it. For a routine positioning draft, the simulated read is usually the point where the team already has enough to decide. ## Where each tool fits ## What Subconscious does not do here Naming what a tool does not do is what lets a buyer check the fit before they commit budget. Subconscious is not a browser extension or an in-workflow writing copilot. A causal test still needs a defined decision, a defined audience, and a comparison set before it can run; it does not replace real-human validation when a decision calls for it, and it does not produce a persona sketch as a standalone deliverable the way an in-browser assistant does. Teams that want both, a fast draft for internal alignment and a tested read before spending against it, typically use each tool for what it is built for. [figure: Two-column comparison: a persona sketch, one ungrounded generation with no comparison group, against a Subconscious causal test, a randomized experiment against a defined buyer population before budget is spent.] ## Which tool should you reach for? Reach for an in-browser assistant like Voilà when the job is a fast first draft, a messaging idea, or a persona sketch to align a team internally, and nobody is about to spend budget on it yet. Reach for a tested decision when the next step is shipping a positioning change, a price, or a launch message, and being wrong is expensive enough that a draft is not sufficient evidence. Recent example studies show what that test looks like in practice; a walkthrough of a specific decision is the fastest way to see whether it fits the one in front of you. --- # Toluna vs a Causal Experiment Layer: Panel Research or Test-First? URL: https://subconscious.ai/blog/comparisons/toluna-vs-causal-experiment-layer-panel-research-test-first Published: 2026-08-12 An insights lead who already runs studies through a recruited panel vendor faces a recurring choice: commit the next product, pricing, or messaging question to a full panel cycle, or test the candidate actions first and reserve the panel for confirmation. Getting this wrong is expensive either way. A panel study built around the wrong question burns weeks and recruitment budget, and a decision shipped on stated-preference survey answers alone can miss the gap between what people say and what they actually do. ## What a recruited panel delivers Toluna operates as a consumer-panel and research platform, connecting brands to a recruited base of real respondents for structured studies, tracking, and reporting (Toluna). That model fits discovery work, ongoing brand and category tracking, and studies where a client needs confirmation from an identifiable human population, not just a directional read. Toluna also offers a self-serve research product built for teams that want to launch and manage studies against its panel directly rather than working through a managed-service engagement (Toluna Start). Either route still depends on recruiting, fielding, and waiting for real respondents to answer: a cycle with real value, but still a cycle. ## The gap a panel cycle alone doesn't close A recruited panel is strong at telling a team what respondents say they think or prefer. It is not built to isolate which specific action (a price point, a message, a feature) causally moves a target behavior before that action goes into the field. Running every candidate variant through a full panel study to find the ones worth fielding is slow and expensive, and stated preference alone does not reliably predict behavior. ## Where does a causal testing layer fit? Subconscious is a causal behavioral platform: it runs controlled experiments on simulated populations to estimate which action moves a target behavior, before a team commits to the full research cycle. It is not a panel-recruitment replacement: recruited human panels remain valuable for discovery, tracking, and confirmation. Existing survey or panel data can be used as an input, extending the workflow into experimental decision testing. Where scale matters, the same experiments can run against a person-level audience graph covering 800 million real people: a modeled comparison population, kept distinct from a panel a vendor recruits and manages directly. When the decision depends on it, a team can move from a simulated experiment to a study validated with real human participants without changing the underlying causal question. That step confirms a result before it drives a launch decision. ## Comparing the two motions ## What does a causal experiment layer not replace? A causal experiment layer is not a substitute for recruiting real respondents when the decision requires it. Regulatory research, brand tracking, and studies where a client needs a confirmed human data point still belong with a recruited panel. And moving a study from simulation to real-human validation does not turn it into an observed usability session, a clinical trial, or an automatic guarantee of market performance; it changes who answers the same causal question, not what kind of evidence the question needs. ## Sequencing the two together A workable order for the buyer weighing this decision: define the candidate actions, run them as a controlled experiment to identify which ones are worth fielding, then send only the narrowed set to a recruited panel for tracking or confirmation. That sequence spends panel budget on questions already worth asking, rather than on the full space of things that might have worked. Book time to walk through a specific decision before committing the next study to a full panel cycle. [figure: Four steps left to right: define candidate actions, run a controlled experiment to see which action moves the target behavior, narrow to worthwhile actions, send that set to a panel to confirm.] --- # What a Weak Replication Correlation Actually Tells a Product Team URL: https://subconscious.ai/blog/case-studies/what-weak-replication-correlation-actually-tells-product-team Published: 2026-08-12 A weak replication correlation, like the .5213 result below significance found here, tells a product team the simulation partially tracked human preference and points to attributes worth testing further before a launch decision. ## The decision this replication is meant to inform A CPG or food-brand product leader is weighing which attributes to test before a launch or reformulation: organic certification, growing region, production method, price. The practical question is whether to run a causal simulation first or go straight to a fielded human study. Committing budget to a fielded study, or to a product change, on the strength of an untested simulation method is the expensive mistake this check is meant to prevent. ## The published study used as the check Skreli et al.'s 2017 study ran a discrete choice experiment on Albanian consumers, testing preference and willingness to pay for tomatoes across four dimensions: whether the tomato was organically grown, its growing region, whether it was hothouse grown, and its price (Spanish Journal of Agricultural Research). Subconscious ran the same attribute set through a simulated discrete choice experiment and compared the results. ## What does the correlation actually say? The reported agreement between the simulated result and the published human study was rs = .5213, p = .1008, a moderate correlation that did not clear the conventional p < .05 threshold. That is neither a strong match nor a failure to replicate: on this attribute set, the simulation tracked some of the same preference structure as the human study, without a demonstrated significant relationship. ## Why is reporting a weak result useful? A vendor that only shows its best replications is not showing a product team enough to make a real decision. Publishing a case where the correlation is moderate and non-significant is the evidence that matters: it shows where a simulation-first approach tracks published human preference data and where it does not, on a specific and checkable attribute set, rather than asserting a general accuracy claim and asking a buyer to trust it. ## What does this result not establish? This single replication does not establish general accuracy for the method, or show that it reliably reproduces human choice on other attribute sets, categories, or markets. A result like this one is useful for narrowing which attributes and levels are worth testing next. It is not a substitute for a fielded study when the decision is high-stakes, such as committing to a reformulation or a launch price. ## What to do with a result like this Use a simulation pass to decide which attributes are worth testing at all, not as the final word on which one wins. When the decision is expensive to get wrong, pair the simulation with a fielded human study on the same attribute set before locking the choice. Review the research methodology behind these comparisons, browse other case studies to see where correlations are strong and where they are not, or talk to the team about designing a study that pairs a simulation pass with real-human validation. [figure: Comparison of a simulated discrete choice experiment against a published Albanian consumer study across organic, growing region, hothouse, and price. Overall agreement is rs=.52, p=.10, below conventional significance.] --- # Survey-Replication Tools vs. Persona Platforms vs. Causal Experiments URL: https://subconscious.ai/blog/comparisons/survey-replication-tools-vs-persona-platforms-vs-causal-experiments Published: 2026-08-12 A consumer insights, product, or pricing leader evaluating synthetic-research tools usually asks the wrong question: which tool is fastest or cheapest. The question that matters is narrower: does this decision need to know what people say they would do, or which action changes what they do? Those are different research problems, and most tools on the market solve only the first one. ## Three categories, three different questions Synthetic-research platforms split into three methods that answer three different questions. Confusing them is the expensive mistake: shipping a decision based on a stated answer when it required evidence about what changes behavior under a controlled alternative. The first two methods are both forms of stated preference: they tell you what a simulated respondent claims about an idea. Neither, by construction, tells you which of two actions moves adoption, conversion, churn, trust, or preference. That gap is why a research question that looks survey-shaped or persona-shaped can still produce the wrong answer for a decision about cause and effect. [figure: Branching diagram: survey-replication and persona-conversation tools both branch to "stated preference." Causal action platforms branch to "causal effect with uncertainty."] ## When is a survey-replication tool the right choice? Survey-replication tools fit teams that already run a formal market-research function and want to keep an existing survey or structured-interview methodology, running it faster. If your research question maps cleanly onto a questionnaire item, and stakeholders expect traditional survey or focus-group deliverables, it's a reasonable fit. The tradeoff is the same one traditional surveys carry: a fixed set of questions can't chase down an unexpected response, dig into why a respondent feels that way, or pivot as the conversation unfolds, and it stays scoped to the research team, not product, marketing, or sales. ## When a persona-conversation tool is the right choice Persona-conversation platforms trade statistical breadth for open-ended depth. Talking to a single synthetic persona, or a small panel of personas, is useful for early-stage ideation: pressure-testing a positioning angle, surfacing objections before a launch, or generating hypotheses worth testing with real customers later. That step is useful, and Subconscious doesn't replace it. But a persona conversation produces a plausible opinion, not evidence that a specific action changed behavior, and shouldn't be treated as a substitute for validation. ## Where do both methods stop? Neither tool is designed to answer the question that drives most product, pricing, and launch decisions: which action is more likely to cause the outcome you want. A stated preference, however it is collected, is not a causal effect. Reading a persona's answer as market performance, or a synthetic survey result as revealed behavior, is the substitution that ships a decision the method was never built to support. Subconscious.ai is the causal AI company: randomized experiments on a simulation of the market, validated against real human behavior, that show why people choose and which action drives the outcome. It sets up a controlled comparison between the actions and estimates which one is more likely to move a decision-specific outcome, for a defined population, with uncertainty reported where the study design supports it. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. Publishing the shortfall alongside the number is what lets a buyer check the claim against the record. It is a validation result, not a guarantee for a new market, drawn from a broader corpus of about 300 replicated studies across 9 domains (the causal fidelity paper). It can also run controlled studies against a person-level audience graph covering 800 million real people, distinct from a recruitable participant panel. Where a decision hinges on it, a team can move from a simulated experiment to real-human validation on the same causal question without redesigning the study. ## Fair comparison: where population-scale and interview-style tools fit Two other approaches sit at different ends of the synthetic-research spectrum. Aaru works at the population-simulation end: multi-agent modeling built for enterprise research functions that need statistical rigor at scale, typically with a longer implementation timeline than a self-serve tool. It fits a research team that needs population-level behavior simulation and the internal process to support an enterprise deployment. Synthetic Users sits closer to the persona-conversation category: it interviews a single synthetic respondent at a time, a narrow fit for generating hypotheses before recruiting real participants. Both answer a different question than a causal action platform: useful for representing a market or surfacing a hypothesis, but neither compares two actions on the same simulated population to estimate which one changes the outcome, the evidence a pricing, launch, or messaging decision usually needs. ## How to pick Three questions narrow the choice: - Is the research question about what people say, or which action changes what they do? A questionnaire item or an opinion belongs to a survey-replication or persona-conversation tool. "Which price, message, or feature moves the outcome" belongs to a causal experiment. - Does the decision need to move from simulation to real-human confirmation without changing the question? If yes, the platform needs to support that transition on the same causal design, not a separate re-run. - What is the cost of being wrong? A low-stakes internal brainstorm can tolerate a plausible synthetic opinion. A pricing, launch, or GTM decision that is expensive to unwind needs evidence about the action, not a stated preference. ## Limitations These limits sit next to the results in the open record, and naming them is what lets a buyer check the claim before deciding whether to trust it. Subconscious is not a conversational persona-interview tool and does not replace open-ended persona conversations. It is not a self-serve, minutes-to-insight tool for casual cross-team use outside a configured study. Current public pricing and packaging are not published for self-service comparison. Confidence intervals, segment heterogeneity, and decision-memo outputs are study-specific, not guaranteed on every study. A next step for a team weighing this decision is to look at how Subconscious runs a study, review case studies from comparable decisions, or talk to the team about the specific action under consideration. --- # How to Test Enterprise Pricing and Positioning Before Launch URL: https://subconscious.ai/blog/use-cases/how-test-enterprise-pricing-positioning-before-launch Published: 2026-08-12 A B2B product marketing or pricing team can test a proposed price, package, or position as a controlled experiment with the enterprise buyer segment. The result estimates the causal effect of each action with confidence intervals. It does not depend on treating a few sales conversations as market evidence. That distinction matters when CIOs or VP-level economic buyers are hard to reach. A weak decision can consume months of go-to-market work and budget before the team learns the target segment would not act on the offer. [figure: Four boxes in sequence: name the buyer segment, set the prices/packages/messages compared, state the behavior measured, fix the decision rule in advance. Path ends in a branch: commit, revise, or stop.] ## Test the action, not the room The central question is not whether a team can assemble an executive panel. It is which concrete commercial action the target segment is more likely to choose. A historical planning example illustrates the recruitment burden: convening eight to twelve senior buyers for a 90-minute focus group requires access and shared availability. A planning number stated without its scope is just marketing. Those figures are planning examples, not current Subconscious delivery terms or a requirement of the method. Recruiting senior B2B respondents also creates a direct research cost. CleverX treats the cost per completed B2B interview as an explicit budget item (CleverX, "Cost per completed B2B interview: what you actually pay"). When recruitment cannot answer the question before budget is committed, the alternative should not be an anecdote from a sales rep or a conference conversation. A controlled experiment can compare the actions under consideration against the segment they are meant to influence. ## Three decisions that benefit from a causal test ### How do you test enterprise pricing? Compare specific price points or commercial terms. Define the behavioral outcome in advance, such as choosing one offer over the control. The result shows the estimated causal effect for the segment and the uncertainty around that estimate. ### How do you test enterprise packaging? Compare the actual bundles a buyer would evaluate. Keep the features, terms, and alternatives concrete so the experiment tests a buying action rather than a general opinion. ### How do you test enterprise positioning? Compare complete claims or messages under the same experimental conditions. This separates the effect of the position from account-team enthusiasm, conference feedback, or other uncontrolled inputs. ## Build the test around the buyer's choice A useful enterprise-market experiment starts with four definitions: - The segment. Name the relevant enterprise buyer role and market conditions without claiming to represent a specific individual. - The alternatives. Specify the prices, packages, or messages the buyer will compare. - The behavior. State the choice or action the experiment is meant to affect. - The decision rule. Decide what effect and uncertainty would justify committing budget, revising the action, or stopping. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. A reach number means little without its limits stated next to it. That reach is distinct from a recruitable panel of named participants. When a result needs confirmation from real people, the same causal question can move from a simulated study to real-human validation without redefinition. ## Keep named executives in the right role Naming what a segment-level result cannot do is what lets a buyer check it against their own account. A segment-level causal effect is not a prediction about one CIO, one VP, or one account. Subconscious does not recruit or convene a panel of named individuals, and the experiment does not replace direct executive relationships. Direct conversations still matter when the decision depends on account history, procurement constraints, internal politics, or facts unique to a named buyer. The controlled test answers a different question: which action changes behavior across the segment, and how certain is that estimate? [figure: Two columns. Left, controlled test: answers which action the segment favors, not a prediction about one CIO. Right, direct conversation: answers account history, procurement, politics, facts unique to a named buyer.] ## Set a decision gate before committing budget This approach fits when the team has concrete alternatives, a defined enterprise segment, an observable buyer choice, and a costly launch decision. The misses sit next to the hits on the public record here. It is a poor fit when the need is a relationship with a named executive or an explanation of one account's internal process. Review the experimental process before choosing the actions and decision rule. If the question is ready to scope, book a working session around the price, package, or position the team must decide. --- # Social Listening Tools Can't Tell You How Your Audience Will React URL: https://subconscious.ai/blog/use-cases/social-listening-tools-cant-tell-how-audience-will-react Published: 2026-08-12 A social listening dashboard can tell a brand or comms leader what people have already said about a topic. It cannot tell them how those people will react to a message, concept, or crisis response that hasn't published yet. The question this raises: when is monitoring data enough, and when does a team need a forward-looking experiment before committing budget or reputation to a message? ## What do social listening tools actually measure? Social listening tools crawl public conversation on social networks, forums, blogs, and news sites, then aggregate what has already been published into volume, sentiment, and trend data (Sprinklr, "Social Listening: A Complete Guide for 2026"). A related but distinct discipline, social media monitoring, tracks direct brand mentions and engagement in near real time rather than analyzing broader conversation trends (Pulsar Platform, "What Is Social Listening? Definition, Examples & Tools (2026)"). Both jobs are retrospective by design. They can surface a spike in negative sentiment, a competitor's new claim, or a trending complaint. Neither can show how an audience would respond to a message it hasn't seen, because nothing in a crawled archive reflects a reaction to content that doesn't exist yet. [figure: Five-step horizontal path: Detect a shift, Decide on candidate responses, Test them against the audience, Publish the winner, Monitor how it lands. Test is marked as the step listening tools cannot do.] ## The decision this creates for brand and comms leaders Once a listening tool surfaces a signal, the team still has to decide what to say next, and getting it wrong carries real cost. A message, campaign, or crisis response that backfires with the target audience is usually discovered only after it publishes, once spend and reputational exposure are already committed. The question isn't whether to keep monitoring. It's whether monitoring data alone justifies publishing a response, or whether the response needs testing against the audience first. ## Why can't detection tools answer that question? A monitoring or listening platform has no mechanism to show a message to an audience and record how that audience actually responds; it's built to observe conversation that already occurred, not to run an intervention. Testing a draft statement, concept, or crisis response requires putting that content in front of a representative slice of the audience and comparing candidate versions, a different kind of tool doing a different kind of job: an experiment, not a crawl. Historically, that meant commissioning a traditional research study that could take weeks. For a fast-moving crisis or a tight timeline, that lag is often why teams end up publishing on intuition instead of evidence. ## Testing a message before publication Subconscious runs a controlled, randomized experiment on the audience segment a message is meant to reach, comparing candidate messages or response options head-to-head, before anything publishes. That produces a causal read on which option performs better with that audience before it ships. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That graph is distinct from a recruited real-human panel: it's the basis for reaching a representative segment, not a pool of people who've already agreed to be surveyed. Where the decision calls for it, Subconscious can also validate studies with real human participants, moving from a fast pre-publication read to real-human validation without changing the underlying causal question. See how the method works. ## Where this fits alongside monitoring tools A pre-publication experiment complements social listening rather than replacing it. A practical sequence: Monitoring still owns detection and post-publication tracking. Testing owns the gap in between: deciding what to say once something needs a response. ## What doesn't this replace? A published limit gives a buyer something to check before trusting the method. Subconscious does not detect real-time public conversation, brand mentions, or emerging trends. A pre-publication experiment estimates how an audience is likely to react under controlled conditions; it doesn't guarantee the outcome once a message is live, subject to real-world context, timing, and reactions the study didn't model. Teams that need representative market sizing, precise elasticity curves for final pricing decisions, or longitudinal purchasing data over months still need the research methods built for those jobs. ## Next step If a team already has a monitoring tool signaling that a response is needed, the open question is usually which response to publish. See how a pre-publication message test is structured, or look at applied examples before scheduling a walkthrough. --- # AI for Brand Strategy Research: Test Before Building URL: https://subconscious.ai/blog/use-cases/ai-brand-strategy-research-test-before-building Published: 2026-08-12 A brand or marketing strategy lead weighing a repositioning, a rebrand, or a new messaging framework has to decide something narrower first: which hypothesis is strong enough to hand to a creative team, and which claims still need a real customer to confirm. As a 2026 planning benchmark, one research provider places an agency-led positioning or concept study at $43,000 to $79,000. A weak direction can consume that budget, shape the creative work that follows, and leave a gap between the voice a company intended and the voice customers actually hear. ## Five questions a brand test has to answer Most brand strategy work reduces to a short list of decisions, and each one is testable before a dollar goes to production: Positioning. Where does the brand sit next to the alternatives, and which framing makes it the obvious choice? Naming. What does a name imply about the company, does it match the product, and does it hold up across every segment that uses it? Messaging hierarchy. Which benefit should lead, which supporting point backs it, and which proof point matters to a given audience instead of just sounding good in a deck? Brand personality. Does the intended voice read the way the team thinks it reads? "Direct and confident" can land as confident or as abrasive depending on execution. Category framing. Does the target buyer already have a mental shelf for this product, or does the company have to teach the category first? Each of these is a comparison, not an opinion poll. The useful output is not "which version do you like," it is "which version changes what a buyer believes about the company, and why." ## Why use segment comparisons instead of open-ended conversation? Subconscious tests product, pricing, messaging, and go-to-market actions with defined buyer segments before those decisions reach creative production or an agency contract. Applied to brand strategy: describe two or three positioning statements, present them under the same conditions to the same segments, and measure which one shifts category placement, comprehension, or preference. Simulated reactions and reasoning surface a hypothesis; a single "which do you like better" answer does not. The same design applies to naming, messaging hierarchy, and brand-voice checks. A name tested in isolation misses how it reads once a buyer understands the product; testing it against a defined segment produces a sharper signal about fit. A message tested against the objections a segment actually raises tells a team more than a preference score. Subconscious runs these comparisons against a person-level audience graph of 800 million real people, a targeting and modeling resource for defining each segment, not a panel scheduled for interviews. That distinction matters: recruiting real interview participants is a separate step with its own scheduling and screening requirements, one a team still takes when a finding needs it. [figure: A four-step path running left to right: a positioning hypothesis leads to a segment-level causal test, then to human-baseline validation, then to an agency-scale rollout.] ## How does a brand hypothesis earn a place in the brief? Define 4-6 segments that represent the audiences the decision depends on, and check each has the right context before comparing brand ideas. Run 3 positioning options under the same experimental conditions, record how each changes category placement or stated preference, and capture the objections and reasoning behind those effects. Compare message variants against one target behavior, separating an opening benefit that changes the outcome from language that only sounds polished in isolation. Challenge the leading combination: ask what would make a skeptical buyer doubt the claim, misread the category, or reject the pitch. The final brief should show which candidate survived, why the alternatives fell away, and what still requires human confirmation. Each stage of the research process should answer a defined decision question. ## Where the test stops and a human has to weigh in Segment comparisons surface hypotheses and the reasoning behind them. They are not a substitute for quantitative brand tracking, awareness measurement, or the sampling and validation those metrics require. A simulated buyer reasons the way that kind of buyer generally reasons, but does not know what is happening inside one specific company's category, and cannot substitute for proprietary market intelligence. A board or an executive sponsor deciding whether to fund a full rebrand often needs "we tested this with real customers," not only "we ran a segment comparison." Subconscious can test or validate studies with real human participants, letting a team move the strongest finalists into human-baseline confirmation. Save that step for claims that carry real budget or reputational risk; running every early positioning idea through human validation adds little before the field is narrowed. [figure: Four-stage path: define segments with context, test three positioning options and record what shifts preference, compare message variants against one behavior, then challenge the leading candidate as a skeptic would.] ## When this is worth running A segment-comparison test earns its place in the process when: - a rebrand or repositioning has several plausible directions the team needs to narrow before committing to one; - an agency's recommendation needs a structured check before the company signs off on the spend behind it; - internal stakeholders disagree on positioning and need outside signal to settle the argument; - a launch into a new market requires evidence about how that audience categorizes the offer; - the goal is sharper questions for a formal human study, not a replacement for one. The result a team should walk away with is a shorter list of stronger candidates, a record of why the weaker ones lost, and a clear plan for which findings still need a human to confirm before the rollout. --- # Can You Trust a Vendor's Uncertainty Intervals? A Test Case in Honest Uncertainty Modeling URL: https://subconscious.ai/blog/methods-and-validation/trust-vendors-confidence-intervals-test-case-honest-uncertainty-modeling Published: 2026-08-12 An uncertainty interval that never changes shape, never widens near a data gap, and never reacts to a structural break in the underlying process deserves a second look. Constant width alone does not prove the interval is dishonest; the real test is calibration against actual outcomes, but non-responsive width is still a reason to ask how the interval was built. Before signing off on a pricing, product, or launch decision that rests on a vendor's error bars, the real question is whether those bars come from a model built to track uncertainty or from a model that reports one anyway. ## What honest uncertainty modeling requires A worked example from the open-source PyMC ecosystem: build a time-series model of atmospheric carbon dioxide from Mauna Loa readings and ice-core records, two data sources with different precision, different sampling gaps, and a known structural break where direct atmospheric monitoring replaced indirect proxies. A generic forecasting default cannot represent any of that. The example instead uses a semiparametric Gaussian process, a custom mean and covariance function, and a dedicated changepoint kernel to represent the transition between data sources, and it treats the exact location of some inputs as itself uncertain rather than fixed (PyMC example gallery, "Gaussian Process for CO2 at Mauna Loa"). [figure: Four questions for a vendor: width changes at breaks, full posterior versus a bolted-on bound, uncertain-timing inputs modeled as uncertain, and whether the design supports a confidence interval at all.] Each choice exists because ignoring it would have made the resulting interval wrong at exactly the point where a decision-maker needed it to be right: the changepoint. A full Bayesian fit with NUTS sampling, rather than a point estimate of the GP hyperparameters, is what propagates hyperparameter uncertainty into the reported range instead of understating it (PyMC example gallery, "Example: Mauna Loa CO2 continued"). ## What failure mode does this guard against? A team that ships a pricing or launch decision on a confident-looking number, without checking whether it was ever uncertainty-tested, absorbs the cost of an action the underlying evidence did not support. The interval was cosmetic; the decision was not. One thing worth checking is structural: does the reported uncertainty change shape when the input data changes shape, or does it stay the same width regardless of what is happening underneath it? Non-responsive width is not proof of miscalibration by itself, but it is a reason to check calibration directly. ## Where does this map to Subconscious, and where does it stop? Subconscious.ai runs controlled causal experiments on the same premise: a result is only as trustworthy as the honesty of its uncertainty, though a well-calibrated interval around a poorly identified estimate can still mislead, so calibration is necessary but not sufficient. That is why experiments report confidence intervals and error bars where the study design supports them, rather than a single number standing in for a range. How that reporting works, and what it takes to design an experiment that can support it, is documented alongside the published method. Replication evidence against real-human baselines, which is a separate question from whether a model's internal uncertainty is well-calibrated, is tracked at /leaderboard. The parallel stops at the technique. Subconscious does not run Gaussian process time-series forecasting, and the Mauna Loa example above is not evidence of a Subconscious product capability. It is a methodological reference point: what it looks like to take a structural break seriously instead of averaging through it. A buyer evaluating any causal or forecasting platform, including this one, should ask the same question: does the reported interval come from a model built to represent what is uncertain, or is it attached after the fact to a number that was never in question? [figure: Five-step chain: a structural break occurs; a model averages through it; its interval keeps constant width; a team ships a decision on that number; the unsupported cost is absorbed.] ## What to check before you rely on someone else's interval - Ask whether the interval's width changes at known structural breaks in the data, or stays constant regardless of input (this test applies to time-series or forecasting components; a confidence interval on a causal parameter has no time axis to check it against). - Ask whether the estimate comes from a full posterior or sampling procedure, or from a single fitted value with a bound bolted on. - Ask whether inputs with uncertain timing or measurement error are modeled as uncertain, or treated as exact (again a test for time-series inputs, not for a causal parameter's confidence interval). - For a causal experiment specifically, ask whether the design supports a confidence interval at all, and whether that interval has been checked against real-human validation. A vendor that can answer all four concretely is reporting uncertainty. A vendor that cannot is reporting a number. --- # Pre-Testing a Webinar Registration Page Before You Spend the Media Budget URL: https://subconscious.ai/blog/use-cases/pre-testing-webinar-registration-page-before-spend-media-budget Published: 2026-08-12 A webinar registration page fails quietly. Traffic volume matches what the channel forecast, nothing looks broken on a quick read, and yet registrations still land at half the expected rate. Blame lands on the topic, the speaker, the time slot, or the algorithm instead. That is the decision a demand-gen lead who owns the webinar program and its paid-promotion budget has to make before launch: ship the headline, agenda framing, and audience segmentation as drafted, or test which variant drives registration intent first. Getting it wrong is not cheap. Thousands of euros in paid media can run against a mismatched headline or the wrong audience segment, registrations and attendance come in low, and the speaker's and sales team's follow-up time is spent chasing a campaign that never had a shot. ## Why the page is the usual suspect Registering for a webinar costs more than a click. A visitor hands over an email address, blocks time on a calendar, and often adds a job title and company on top. Each of those is a small commitment, and the decision to give it happens in about the time it takes to skim the fold. Most drafts spend that time describing what the webinar is, rather than why a specific visitor should give up ninety minutes for it. Three patterns account for most of the underperformance: - Topic-headline mismatch. The team picks a topic the audience cares about, then writes a headline that describes the topic abstractly instead of naming the specific problem it solves. - Speaker-agenda mismatch. A senior speaker is announced and featured prominently, but the agenda is written at a level that doesn't match the audience the speaker's name draws in. - The recording gap. A large share of any B2B webinar audience registers planning to watch the recording rather than attend live. Pages that don't say so explicitly leave that intent on the table. None of these show up in a dashboard until the campaign has already spent against them. ## How do you test a webinar registration page before the budget commits? The fix is to run the registration page as a causal experiment against defined buyer segments before committing paid-media spend, rather than an internal stakeholder read of a draft. Structuring the test around buyer segments common to a B2B webinar audience (a skeptical practitioner who has seen too many product pitches disguised as content, a curious manager who forwards rather than attends, a senior buyer at a target account evaluating credibility signals, and a lukewarm contact from an existing list) isolates which page element drives the registration decision for which audience, instead of an average reaction across an undifferentiated group. The test itself runs in stages that mirror how a visitor actually reads the page: Running the headline stage alone is the highest-leverage step: if a majority of segments say they wouldn't read past it, the rest of the page's quality doesn't matter yet. ## What does this test replace, and what doesn't it do? This is a pre-launch message test, not a live on-page A/B test and not a prediction of show-up or attendance rates. It estimates directional response to copy and framing across defined segments before spend commits, and it does not guarantee a specific conversion lift. Where the underlying study design supports it, results can include confidence intervals and segment-level differences, as properties of the study rather than a default output. The same test can also move from a simulated read to real-human validation without redesign: once a page has been tested against segment definitions, it can be rerun with real participants, using the same headline, agenda, and form variants. That step matters when the decision is expensive enough to warrant a real-human check before the media budget commits, and it draws on studies run against a person-level audience graph covering 800 million real people. That graph describes the population available for controlled studies, not a pool of people pre-recruited and waiting to register for a specific webinar. ## What "worth fixing" looks like in cost terms Public industry data on typical webinar economics gives a sense of scale, even without a Subconscious-specific benchmark: Zoom's 2025 webinar statistics roundup sizes the promotion and production spend that a single webinar campaign typically carries, and the Wistia webinar analytics benchmarks size typical registration-to-attendance rates. That spend and attendance run against the page regardless of whether the page was tested. A registration page that underperforms doesn't just cost registrations; it costs attendance quality, the sales team's post-event follow-up time, and the data the next campaign would have used to plan better. ## Where can you run this test? A demand-gen lead can scope a single test around one campaign's registration page (headline, agenda, and speaker framing) before the promotion budget commits. Read more on how the test design and segmentation approach works, or see the underlying experiment methodology. To scope a specific page, book time to walk through it. [figure: A five-step path: Headline, Agenda, Speaker, Form, Objection. Each names what it isolates: second-look appeal, expectation gap, teach-vs-sell tone, abandonment field, missing element that flips no to yes.] --- # When a Chat With a Simulated Target Group Is Enough, and When It Isn't URL: https://subconscious.ai/blog/use-cases/when-chat-simulated-target-group-enough-when-isnt Published: 2026-08-12 A marketing, product, or insights leader evaluating a target group before a launch, a message, or a feature faces the same question: is a fast, open-ended conversation with a simulated audience enough evidence to commit budget, or does the decision need a controlled experiment against defined alternatives? The answer depends on what the decision costs if it goes wrong, not how convincing the conversation felt. ## What an open-ended conversation with a simulated audience actually gives you A category of platforms now lets a team build a simulated version of a target group and chat with it directly, reading back a fluent, human-sounding response the same day instead of waiting three to four weeks for a recruited focus group to convene. That speed is real: for early-stage exploration, sharpening a hypothesis or drafting a message before real feedback, a directional read from that conversation is a legitimate, low-cost step. The output is still a single aggregated impression from one conversation, not a measured comparison. It shows what a simulated respondent said when asked; it does not show how a defined population's choice shifts when one specific variable, a price, a headline, a feature, changes and everything else holds constant. ## Where the directional read stops being enough This failure mode is stated plainly here so a buyer can check for it before spending against it. The cost of treating that impression as decision-grade shows up after the money is spent: a team reads a fluent, confident-sounding answer, commits launch or campaign budget on the strength of it, and only later discovers the answer was never tested against measured variation in the population, because it was never designed to be. That gap matters most where the stakes are highest: a launch decision, a pricing change, a message that will run at spend. In each case the real question is not "what does this audience say when asked," it is "which of these specific alternatives does this audience prefer, and by how much." A conversational impression cannot answer the second question, no matter how many simulated respondents take part or how natural the responses sound. ## How does Subconscious test the decision? Where an open-ended conversation produces a single reaction, Subconscious runs a controlled discrete choice experiment: it presents a precisely defined population with the actual alternatives on the table, this price against that one, this message against that one, and returns the causal effect of choosing one over the other, with confidence intervals attached. Discrete choice experiments are an established method for isolating which attribute of a decision drives the outcome, not just which one respondents mention first (Organizational Research Methods, SAGE). For a team evaluating a target group at consumer scale, that experiment can run against a person-level audience graph covering 800 million real people, rather than a smaller convenience sample built for one conversation. See the replication leaderboard for how these experiments are structured and validated. ## When the decision needs a real human, not just a defined comparison The method's edges are named here on purpose, next to what it does well, so the scope is easy to verify. A controlled experiment answers "which alternative wins and by how much" for a defined population. It does not replace early customer discovery, moderated qualitative research, or watching what happens once something ships. For a launch decision large enough to justify it, Subconscious can also test or validate a study with real human participants, the same causal question, run again against people rather than a simulation, without redesigning the experiment to do it. That step matters when the cost of being wrong in market is larger than the cost of a second round of testing; it is not needed just to sharpen a message draft. ## How do you choose between the two? Start with the open-ended conversation when the question is still soft, a first pass at a message, a hypothesis worth sharpening before anyone commits money to it. Move to a controlled experiment once real alternatives and real budget are both on the table, and add real-human validation when the decision is big enough that an in-market surprise would cost more than the extra round of testing. Teams ready to test a specific decision against defined alternatives can see how the method works on a live case at /demo or review the underlying approach at /how-we-work. [figure: Branching path: a soft question goes to an open-ended simulated conversation; real alternatives and budget move it to a controlled discrete-choice experiment; the biggest decisions add real-human validation.] --- # Will Your US Product Positioning Work in Germany? URL: https://subconscious.ai/blog/use-cases/will-us-product-positioning-work-germany Published: 2026-08-12 Test whether German buyers respond differently to the same message, price, and feature emphasis before committing the localization budget. The result should tell an international growth leader what to preserve, what to adapt, and what still needs an in-market pilot. A historical planning example put full mobile-app localization at $30,000 to $150,000 depending on complexity. It modeled the late-discovery problem as 12 percent conversion in the US, 4 percent after a German launch, and three months before the gap became visible. Those figures are inherited planning examples, not current market benchmarks or Subconscious results. [figure: Two columns, US buyers and German buyers, three rows: message, price frame, and feature emphasis. Each row shows what stays fixed, what changes, and the outcome measured per market.] ## Start with the launch decision The decision is not whether German buyers are broadly different from US buyers. It is whether a specific product action changes stated choice or conversion intent differently across the two market segments. Privacy is one reason to test instead of generalize. A BullGuard-commissioned study reported country-level differences between privacy concerns and actual behavior among consumers in the United States, the United Kingdom, and Germany (BullGuard-commissioned study). That finding supports testing privacy framing, not that every German buyer values the same message. Define the launch question in operational terms: - Which value proposition changes choice? - Which trust statement changes conversion intent? - Does the price frame work equally well in both markets? - Which feature should lead the German offer? A historical pricing example used a $9.99/month subscription to illustrate how the same price can occupy a different place in each market's value frame. Treat the amount as a test input, not a recommended price. ## Build a cross-market experiment around actions Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. The graph supports audience definition and segmentation. It is not a recruitable group of 800 million study participants. The experiment holds the product context steady, changes one action at a time, and compares the effect across US and German buyer segments. A useful design separates three decisions that often get bundled into one localization project: As a planning example, the source material split enterprise buyers into three segments: an Operations Director watching cost, an IT Manager focused on compliance, and a CIO weighing strategy. Those roles remain useful only when they match the actual buying committee, not as a universal German-market template. The output is a directional comparison across actions. Report uncertainty qualitatively unless the configured study supports a confidence interval. A country-segment difference can justify a copy revision or a pilot, not that nationality alone caused the response. ## How do you read the difference without stereotyping the market? A useful result distinguishes between a market effect and an artifact of the test. First, check whether the same intervention moves the outcome in both segments. If it does, localization may require translation and legal review without a new positioning strategy. If the effect changes direction or size, the team has a concrete adaptation to examine. Second, look for differences within each market segment. Aggregate agreement can hide disagreement between buyer roles, industries, or purchasing contexts. The study design should match the actual buyer population closely enough for the launch decision. Third, test the explanation. A weaker response to a time-saving claim does not automatically mean German buyers reject convenience. Compare it with a cost, control, quality, or risk-reduction frame while holding the rest of the offer constant. This turns three inherited categories into an actionable localization brief: - Critical gaps that could block adoption. - Cultural adaptations that require rewriting rather than literal translation. - Feature or positioning changes that alter the offer itself. ## Set the launch gate before the result Decide in advance what evidence would trigger each action: - Preserve the US positioning when the treatment effect is directionally consistent across both segments. - Adapt the German version when a specific alternative performs better for that segment. - Run an in-market pilot when the difference is material but cultural, linguistic, or operational uncertainty remains. When the decision requires human confirmation, Subconscious can test or validate studies with real human participants without changing the causal question. The simulated experiment and human study should compare the same actions and measure the same outcome. This process does not replace an in-market pilot, German legal and regulatory review, GDPR review, or a full localization quality pass. Cultural and linguistic nuance still needs human review before launch. A historical agency-planning example used 6 to 10 weeks for one round of concept research, not a current agency benchmark or a Subconscious delivery commitment. ## What should you carry into localization? The final artifact should be a decision brief, not a market stereotype. Record the winning action by segment, the uncertainty, the evidence that would reverse the choice, and the issues reserved for human review or an in-market pilot. Bring the actual US message, pricing frame, and feature alternatives to a scoped demo. The experiment can then answer the launch question before those assumptions are embedded in translated product and campaign materials. --- # How CPG Teams Should Gate Packaging Redesigns Before Production URL: https://subconscious.ai/blog/industries/how-cpg-teams-should-gate-packaging-redesigns-before-production Published: 2026-08-12 A CPG brand or innovation team should advance a packaging redesign only after it passes two different tests. The first asks whether buyers understand the brand, variant, benefit, and difference. The second asks whether the pack earns attention and holds up under physical retail conditions. Publishing what a test cannot do is what lets a buyer trust what it can do. A controlled experiment on a simulated population can answer the first question. It cannot establish the second. [figure: A left-to-right path: a pack draft enters a simulated shopper reaction test, branches to kill or refine, and the surviving design moves into in-store validation before production.] ## The retailer window makes a weak choice costly Most package redesigns do not increase sales, and some reduce them, according to Designalytics research on package redesign failure. An analysis of the U.S. CPG industry likewise found that innovative package changes can move sales in either direction rather than produce a certain lift (The Sales Effect of Innovative Package Changes: Evidence from the U.S. CPG Industry). The downside is not limited to production expense. A losing design can consume trade spend and a retailer listing window that another concept might have used. The useful decision is therefore not which design looks strongest in a review. It is which option deserves capital, a trade presentation, and physical validation. ## What's the difference between package communication and shelf performance? A pack has to communicate brand, variant, benefit, and difference in approximately two seconds of shelf attention. It can pass that test in isolation and still disappear beside competing products. It can also draw attention while leaving buyers unclear about the product, variant, or benefit. These are different failure modes and need different evidence. The first gate should remove alternatives that fail their basic communication job. The second should determine whether the survivors work on shelf. Neither gate should be described as evidence supplied by the other. ## How do you use a causal test to decide what packaging advances? Subconscious runs controlled experiments on a simulated population to compare packaging or product-concept alternatives on communication and reaction. The study should hold the buyer question steady across the alternatives: which change affects understanding or response, for whom, and in what direction? A buyer checking a study needs to know exactly who answered it. The simulated audience is constructed for simulation, not recruited from real-human respondents. For CPG decision teams, a practical test begins with these choices: - Define the production or retailer decision before evaluating creative work. - Compare the current pack and each candidate against the same communication and reaction measures. - Reject alternatives that obscure the brand, variant, benefit, or intended difference. - Send only the surviving alternatives to in-store observation, sensory work, or purchase-data validation. When the decision needs another proof point, Subconscious can test or validate the same causal question with real human participants. That step checks the simulated result without turning it into automatic proof of shelf performance or market sales. ## Read the result within its boundary The misses belong next to the hits, on the same page as the result. A simulated comparison does not reproduce actual shelf attention or eye-tracking behavior, replace sensory testing, in-store observation, or purchase-data validation, or establish that a launch will increase sales. A number without its limits functions as marketing rather than as a result. Confidence intervals, differences between buyer groups, and automated recommendation outputs depend on the study design; they are not standard outputs of every packaging comparison. The result is evidence for which alternative should receive further validation, not a substitute for it. ## Structure the next packaging decision Start with the pack or concept choice, the behavior that would distinguish a winner, and the physical evidence still required before launch. Review the research approach, then book a walkthrough to frame a controlled comparison around a specific packaging decision. --- # Automating a Consumer Research Workflow Without Losing Rigor URL: https://subconscious.ai/blog/implementation-and-procurement/automating-consumer-research-workflow-without-losing-rigor Published: 2026-08-12 Automating consumer research without losing rigor means matching each pipeline stage's automation level to its risk, letting back-end stages run automated while pricing and launch decisions still require a real human respondent. A research operations lead facing a growing backlog of ad-hoc requests has one real decision to make: which stage of the pipeline to automate first, and which decisions still require a real human respondent before the company commits budget to a price, a launch, or a message. Getting the sequence wrong costs more than time. Treating a directional, simulated result as final evidence for a high-stakes call can mean redoing a study after the decision is already public, with both the budget and the internal credibility of the research function spent. ## Break the pipeline into stages before automating any of it Consumer research is not one task. It runs through six distinct stages, and each has a different tolerance for automation: - Request intake and briefing: translating a stakeholder's ask into a testable question. - Hypothesis screening: narrowing many candidate claims, prices, or messages down to the few worth fielding. - Questionnaire pretesting: catching confusing logic or biased phrasing before a live launch. - Fielding and sample management: collecting responses and screening out low-quality ones. - Open-ended response analysis: coding and clustering free-text answers. - Reporting and synthesis: turning results into a decision memo. The stages closest to the data collection back end, pretesting, coding, and draft reporting, tolerate automation well because an error there is caught internally before it reaches a stakeholder. The stages closest to strategic framing and final validation tolerate it far less, because the cost of a wrong or overconfident answer lands on the business decision itself. ## How should you sequence automation across the pipeline? A phased rollout avoids the two most common failure modes: automating a stage stakeholders don't trust yet, and automating a stage where a hallucinated or biased result reaches a business decision unchecked. Fielding itself sits in between: the physical act of a human answering a survey does not automate, but sample-quality checks and a synthetic first pass to cut the volume of paid human sample do. ## Where a causal experiment fits, and where it doesn't Generic AI persona chat can produce a fluent-sounding read on a message or a concept, but fluency is not evidence. A research operations lead comparing candidate hypotheses needs to know which specific claim, price, or message changes a target buyer's stated choice under controlled conditions, not which one an LLM found more persuasive to summarize. Subconscious runs controlled discrete-choice experiments that compare defined alternatives, a claim against a claim, a price point against a price point, and reports which one produced a stronger response, rather than generating open-ended persona commentary. Used at the hypothesis-screening stage, this narrows a long list of candidates to the few worth fielding to a live panel. A method's misses belong on the record next to its hits. Independent review of experiments with [synthetic respondents](https://measuringu.com/review-of-experiments-with-synthetic-users/) has found that digital-persona methods vary in how closely they track real survey results depending on the population and the question asked, a caution against treating any simulated output as a stand-in for validation on its own. A study on when digital personas reliably approximate human survey findings reaches a similar conclusion: approximation quality is condition-dependent, not a fixed guarantee. ## Do not skip human validation for the decisions that matter Automation is well-suited to directional research: narrowing candidates, catching a broken survey skip pattern, clustering open-ended text. It is not a substitute for a real human respondent when a pricing decision, a regulatory submission, or a major launch is on the line. Subconscious can test or validate a study with real human participants when the decision warrants it, moving from a simulated experiment to real-human testing without changing the causal question being asked. That progression matters most at the exact point this pipeline creates the temptation to skip it: right after a synthetic screening round has already produced a clean-looking directional answer. ## What does automation in this pipeline not cover? A scope note works only when it states what is excluded. Subconscious is not a backend automation suite for this workflow. It does not perform open-ended response coding, generate draft reports, or triage intake requests. Those stages call for the natural-language-processing and workflow tools built for them, and pairing them with a causal-experimentation layer upstream is a sequencing choice, not a product substitute. [figure: Four-step path left to right: automate draft reporting and coding first, then pretesting, then hypothesis screening, ending at a required human-validation checkpoint for pricing or major launch decisions.] ## Putting the sequence to work The Subconscious research program covers how controlled experiments and human-baseline comparisons are run. The how we work page walks through the steps from a defined decision to a completed study, and the current case studies show teams that used repeated screening rounds before a live fielding commitment. A demo is the fastest way to test one real hypothesis against the sequence described here. --- # AI Content Tools vs. AI Persona Panels vs. Causal Experiments: What Each One Actually Proves URL: https://subconscious.ai/blog/comparisons/ai-content-tools-vs-ai-persona-panels-vs-causal-experiments-what-each-one Published: 2026-08-12 AI content tools prove a draft is on-brand, AI persona panels prove a simulated character can produce a plausible reaction, and only a controlled causal experiment proves a measured effect with a confidence interval on what people actually choose. A marketing or content-ops leader shopping for an AI tool right now is usually looking at one of three different products. One writes copy. One runs an open-ended chat with a simulated persona and returns a plausible-sounding reaction. One runs a controlled experiment against defined alternatives and returns a measured effect with a confidence interval. Only the last one tells you whether a message actually changes what people choose. Confusing these categories is the expensive part. Shipping content that reads well and got a favorable reaction from a chat persona is not evidence that it will move a target segment's behavior. Without a measured effect, a team can't tell whether the campaign's result was signal or noise, or catch a bad choice before the media budget is spent. ## Three tools, three different questions AI content-generation and copywriting tools answer "how do we produce more on-brand content, faster?" Tools in this category, such as neuroflash, draft blog posts, ads, social copy, and product descriptions in a trained tone of voice, often with integrated SEO and image generation. These tools sit at the production stage of a marketing workflow, and their promise is volume with brand consistency. Open-ended AI persona chat tools answer "what might this type of customer say about our idea?" A team describes a customer type, chats with a simulated persona, and gets a conversational reaction to a concept, a headline, or a positioning statement. That reaction can be useful for early ideation, but it is one plausible-sounding response, not a measurement. Nothing about the interaction controls for which alternative a defined population would actually choose, and nothing produces a confidence interval. Controlled causal experiment platforms, which is where Subconscious sits, answer a different question: "which of these specific alternatives changes what our target population actually chooses, and by how much?" Subconscious runs a controlled discrete-choice experiment that compares defined message or positioning alternatives across a precisely specified population and returns a measured causal effect with a confidence interval. Methodology and study results are tracked on the replication leaderboard. ## Comparing the three categories ## Why isn't a good reaction the same as a measured effect? Content that reads well and gets a favorable response in an open-ended chat has cleared a much lower bar than content that has been tested. A chat persona gives one answer, shaped by however that single conversation unfolded; it does not compare alternatives, sample a defined population, or report a confidence interval. A controlled discrete-choice experiment does all three. That distinction matters most when the decision is expensive to get wrong, such as a positioning change, a new category message, or a pricing frame, because the cost of shipping on the strength of a good-sounding draft or a single favorable chat is a campaign that burns budget on volume without moving the outcome it was meant to move. ## Where each tool fits Choose a content-generation tool if the team's bottleneck is producing on-brand material faster. Choose an open-ended persona chat tool for early, low-stakes ideation where a directional reaction is enough. Choose a controlled causal experiment when the decision is which specific alternative to ship, and the cost of guessing wrong is high enough to warrant a measured answer before launch. These are not mutually exclusive. A team can draft with a content-generation tool, sanity-check ideas conversationally, and then run the finalists through a controlled experiment before committing budget. Subconscious sits at that last step, not the first two. ## What are Subconscious's limitations? Subconscious does not generate on-brand copy, headlines, or content at scale; that remains a separate category of tool. A causal experiment measures which alternative changes stated choice for a defined population; it does not replace live A/B testing in market, brand voice management, or SEO tooling. When a study calls for it, Subconscious can also validate studies with real human participants, moving from a simulated experiment to real-human validation without changing the causal question being asked. Teams weighing this decision can see how a controlled study is built at /demo or read more about Subconscious's approach at /about. [figure: Three cards: content tools give drafted copy, persona chat gives one reaction, causal experiment gives a measured effect with a confidence interval.] --- # Customer Segmentation: Stated Answers or a Behavioral Test? URL: https://subconscious.ai/blog/implementation-and-procurement/customer-segmentation-stated-answers-behavioral-test Published: 2026-08-12 Building a segmentation means picking one of two ground truths: what simulated or interviewed personas say they care about, or what a controlled test shows actually changes their choice. The two produce different segment boundaries, and only one survives contact with a real launch. ## The say-do gap is the risk, not a detail Simulated persona interviews are fast to run: describe a hypothesized segment, ask it a structured set of questions, and read across the answers for patterns. That surfaces stated preference: what a respondent claims matters when asked directly. It does not test revealed preference: what actually moves their choice when price, framing, or a competing option changes. Stated intent systematically diverges from the choices people make once real trade-offs are on the table (Journal of Economic Behavior & Organization), and correction methods exist precisely because that bias is large enough to distort downstream decisions (Health Economics). A segmentation built entirely on stated answers inherits that gap: marketing builds campaigns, product reprioritizes roadmap, and sales retrains pitches around segment boundaries that were never checked against an actual behavioral response. ## Should you build a segmentation hypothesis before testing it? A segmentation hypothesis is worth writing down early. The discipline holds regardless of what tool runs the interviews: - Name segments specifically enough to act on. "Owners of independent restaurants with one to three locations who handle their own marketing" is testable; "small business owners" is not. - Write a one-paragraph persona per hypothesized segment: who they are, what they currently use, what frustrates them. - Interview each hypothesized segment with the same structured question set (jobs to be done, pain points, decision criteria, channel and source, willingness to pay) so answers are comparable across segments. - Read across the answers for three signals: convergence within a segment (it's real and tight), divergence between segments (they're worth treating separately), and surprises (segments that should merge or split). Naming a method's limit here lets a buyer check it before a budget gets committed against it. That gets a team to a defensible hypothesis about segment boundaries. It does not tell them which message, price, or feature actually changes a given segment's behavior; that needs a different kind of test. ## Where a causal test replaces a guess Subconscious runs controlled behavioral experiments where the segment or buyer definition is a direct input, and the output is a causal comparison between actions: for example, which price point or feature framing changes stated intent to purchase for one segment versus another. Subconscious can also validate a study with real human participants, without changing the underlying causal question, when a decision is high-stakes enough to justify it. Audience definition for a study can draw on a person-level audience graph covering 800 million real people (a sourcing capability, not a recruited panel), so keep the two distinct when scoping a study. ## What doesn't a controlled behavioral test replace? This section spells out the boundary of a controlled test so a buyer can scope a study correctly before committing to it. A controlled test is not a packaged segmentation, clustering, or persona-interview product; it answers one action-comparison question per study, not "here are your five customer segments." Confidence intervals, segment-level heterogeneity breakdowns, and willingness-to-pay outputs are specific to how a given study is designed, not a standard deliverable of every engagement; confirm what a specific study will produce before commissioning it. [figure: Two-path diagram: top path runs persona interviews to stated preference to a segmentation hypothesis; bottom path runs a controlled behavioral test to a causal effect with a confidence interval.] ## Next step Use the interview-based hypothesis step to narrow candidate segments and the messages or features worth testing per segment. Before betting a roadmap or a campaign budget on the result, run the highest-stakes comparisons as a controlled experiment rather than accepting stated answers as final. See how the same comparisons play out for existing customers in case studies, or book time to scope one study against a segment hypothesis already in hand. Teams shifting from stated to tested decisions can start with how a study gets designed. --- # Self-Improving AI Agents: When Is an 'It Got Better' Claim Real? URL: https://subconscious.ai/blog/use-cases/self-improving-ai-agents-when-got-better-claim-real Published: 2026-08-12 A coding agent's instructions get rewritten by an optimizer, a benchmark score goes up, and someone proposes shipping the change. Before a data science or ML engineering lead signs off on that, the real question isn't whether the score moved. It's whether one run is strong enough evidence to act on. ## What does a self-improving agent actually do? A self-improving agent measures its own performance against a benchmark and edits its own guidance (usually a skill document, a block of instructions injected into a frozen model's context), keeping only edits that beat a held-out score. An optimizer model reads a batch of scored runs, proposes bounded add, delete, or replace edits, and accepts an edit only when it strictly improves that held-out result (SkillOpt: Executive Strategy for Self-Evolving Agent Skills). Nobody hand-writes the improved instructions anymore. Somebody still has to write the benchmark that decides what "improved" means, and a benchmark with no held-out score gives the optimizer nothing to climb. ## One experiment that behaved exactly as expected In one documented run, an optimizer was pointed at a modeling skill and a synthetic benchmark of 63 tasks spanning linear regression, small hierarchical models, and time series. Scoring gated on hard checks (code executes, sampling completes, convergence diagnostics clear a threshold, zero divergences) before a continuous quality score was computed on top. The optimized skill cleared all 18 held-out tasks, up from 14, with the entire gain concentrated in the hierarchical family, which moved from 2 of 6 tasks passing to 6 of 6. The graded quality score moved alongside it, from 0.596 to 0.754. The accepted edit was a 47-line diff: wider priors to match unstandardized data, explicit integer casting an underlying library required, and a firmer sampling budget for the small-group cases. Every one of those edits maps to a known, diagnosable failure (a divergence, a type error, a prior that was too tight), visible directly in the run's trace. The pattern behind every genuine win in this kind of loop: a benchmark that returns a stable, repeatable score, and failures an optimizer can diagnose by reading output rather than recalling an exact library version. ## Where the same loop went nowhere A second attempt pointed the optimizer at a set of hand-built tasks drawn from a working practitioner's real modeling exercises rather than synthetic data. The accept gate took zero edits. The optimizer's diagnosis of the gap was correct: it identified a missing section on an unfamiliar library's API, but it wrote that section from memory, the details didn't match the installed library version, and the edit it proposed introduced a new error rather than fixing one. Two properties killed the loop here, independent of the optimizer's skill. The task set was slow to score, on the order of thirty minutes per fit, which made the validation gate noisy enough that even a real improvement would have been hard to distinguish from chance. And real data has no known ground truth to grade recovery against, so the graded signal that drove the first experiment wasn't available at all. The fix was not a smarter optimizer. It was rebuilding the same task shape as a synthetic, gradable benchmark with a stable score, which moved the same domain back into the working zone and produced measurable, trace-diagnosable wins. ## Why is a single run the wrong unit of evidence? The clearest illustration of why one favorable score is not proof came from a benchmark run across six independent seeds sharing the same test split. Three of those seeds ended the run with zero edits accepted, and yet their scores still moved, a shift traceable only to sampler and agent randomness between otherwise identical runs. Against that background, two seeds produced real gains (+0.157 and +0.169), one produced a real regression (-0.286), and the rest sat inside the noise band. Two independently-run seeds converged on the same fix, initializing an ordered model's cutpoints with a sorted starting value rather than letting the sampler choose, which is the kind of agreement that separates a transferable improvement from a lucky roll. Without seeing several seeds and the no-op baseline spread between them, a single favorable run and a single unfavorable run are indistinguishable from noise. [figure: Five-stage path: a score improves once; check if it's held out; rerun across seeds; compare the spread against a measured no-op noise floor; ship only if the gain is diagnosable and repeatable.] ## Hand-authored versus optimized: what changes and what doesn't Neither column removes the need for judgment. The optimized column just moves the judgment earlier, into the benchmark design, where it can be checked once instead of argued about after every edit. ## The decision this changes For a team deciding whether to adopt an automated skill-rewrite, a coding-agent prompt swap, or any "it scored higher" claim as grounds for a production change: demand the held-out benchmark before the edit, demand more than one seed, and demand the no-op baseline spread that tells you what noise alone looks like. A change that clears that bar is evidence. A change that doesn't is a single lucky or unlucky draw, and treating it as proof risks reversing a real decision. That is the same discipline behind testing whether a business action (a price change, a message, a positioning move) actually caused the outcome a team is claiming credit for. Subconscious's research and leaderboard methodology runs controlled, replicated experiments with confidence intervals for the same reason: one observed result isn't a causal, repeatable effect, in a benchmark or in a market test. Subconscious does not build, train, or optimize AI agent skills or prompts. The discipline shown here is the transferable part, not the specific tooling. ## What doesn't this article cover? This is a benchmark-engineering practice for coding-agent instructions, not a usability study, a clinical trial, or an automatic proof that an agent performs better in production. If a team wants to move past benchmark scores entirely and check how an actual claim performs with real people, that is a separate step: see how Subconscious moves from a simulated test to real-human validation without changing the underlying question being tested. --- # What to Demand Before You Trust a Synthetic-Respondent Vendor's Accuracy Claim URL: https://subconscious.ai/blog/methods-and-validation/what-demand-before-trust-synthetic-respondent-vendors-accuracy-claim Published: 2026-08-12 A vendor's self-reported accuracy number is a marketing claim until you can reproduce it. The question is not what number the vendor publishes, but what evidence backs it and whether you can replicate it against your own historical data. It has to survive test-retest checks, item-level scrutiny, and an independent replication before a decision leans on it. ## Why the aggregate number alone is not proof A single correlation figure between synthetic and real respondent distributions can hide a wide spread. A platform that reports one clean number might be averaging a long tail of low-agreement items against a smaller set of near-perfect ones, a different pattern than a methodology that holds steady across every item. Treat any headline accuracy figure as a starting point for questions, not the answer. ## What to ask a vendor before you believe the number A vendor that reports aggregate accuracy without test-retest reliability data is reporting half the story, but test-retest agreement alone can mislead: for LLM-based synthetic respondents, near-perfect stability is nearly free at low sampling temperature and can signal degenerate, insufficiently heterogeneous output. The defensible test is whether re-run dispersion matches human re-fielding dispersion, and whether results hold across prompt paraphrase and model version. ## What the published research actually supports The academic literature on LLM-conditioned synthetic respondents is worth reading directly, not secondhand from a vendor's own summary of it. Argyle, Busby, Fulton, Gubler, Rytting, and Wingate pulled respondent demographics from the American National Election Studies, built those into backstories for a language model, and found the resulting response distributions correlated strongly with survey distributions across political-attitude batteries, including within demographic subgroups (Argyle et al., 2023, Political Analysis). A related NBER working paper by Horton tested whether a language model conditioned on agent profiles would reproduce classic behavioral-economics results: Charness-Rabin social-preference allocations, the Kahneman-Knetsch-Thaler snow-shovel fairness scenario, Samuelson-Zeckhauser status quo bias, and a minimum-wage hiring experiment. The synthetic agents reproduced the qualitative findings, though Horton cautions against relying on the magnitudes. A 2024 Political Analysis paper by Bisbee, Clinton, Dorff, Kenkel, and Larson, on whether LLM output can stand in for human survey responses, generated synthetic ANES feeling-thermometer responses and found that synthetic response variance was drastically understated, producing artificially precise standard errors and spurious significance, that subgroup contrasts were distorted, and that results shifted with prompt wording and model version. That variance-understatement finding matters more than any headline correlation number, and none of the three studies above tested stated-preference or discrete-choice methods: the research consensus is not "synthetic respondents are accurate," it's "synthetic respondents show directional agreement on vote choice, economic games, and feeling thermometers, with precision claims requiring independent scrutiny." ## Where the accuracy gap is largest Across the published studies, the pattern is consistent: - Novel-behavior prediction. Questions about new product categories or behaviors outside what a language model has meaningful signal about show the widest gaps, though published comparisons rarely quantify the size of the gap. - Niche audiences with thin public signal. Accuracy depends on the model having seen enough data about a population, and small, specialized B2B roles or industries are where that signal thins out. - Regulatory and compliance-substantiation studies. Synthetic data is not a substitute for real-human-respondent data on the record, regardless of how strong the correlation is elsewhere. - High-stakes, real-consequence decisions. A synthetic respondent and a human survey respondent are both answering a hypothetical question; the gap between a hypothetical answer and real consequences applies to both, so a synthetic-vs-human survey comparison cannot validate against it. ## How to run the validation yourself A procurement team does not have to take any vendor's word for its accuracy number. The workflow is the same regardless of platform: - Find a past study in your own archive with a known outcome distribution. - Recreate the demographic, role, and segment specifications that defined the original sample. - Run the equivalent question battery using the same stimuli and framing as the original study. - Compare the new distribution to the original, both in aggregate and item by item. - Decide from your own numbers, not the vendor's marketing page, whether the method is accurate enough for the decision. This carries more weight than a vendor's marketing figure, but only if you also account for the human benchmark's own re-fielding variability, recognize that an archived study certifies only the conventional regime it came from, and rule out the study having been public enough for the model to have trained on it: your data, your replication, your result. ## Where a causal-experiment approach fits Subconscious's position is that validation belongs inside the experiment design, not bolted on afterward as a marketing statistic. A randomized experiment is built against a simulated population with a human baseline and holdout built in, rather than validated once and reused as a blanket claim across every future study. Subconscious can test or validate studies with real human participants, so a team can move from a simulated run to real-human validation on the same causal question, without redesigning the study. ## Limitations Simulation is not a substitute for real-human validation on high-stakes, regulatory, or commitment-context decisions. No vendor's accuracy figure should be accepted as proof for a specific question type or audience without independent replication against your own data. Current replication results are published on the leaderboard. [figure: Decision path from a vendor's aggregate accuracy claim through four evidence checks, each with a red-flag exit if the vendor can't produce it, ending in a trust decision.] ## Next step Before signing with any synthetic-respondent vendor, ask for test-retest reliability data, item-level correlation, and one study you can independently replicate against your own historical results. If a vendor can't produce all three, start a conversation about which of your decisions are ready for a controlled comparison instead. --- # Test Your First Job Description Before You Post It URL: https://subconscious.ai/blog/use-cases/test-first-job-description-before-post Published: 2026-08-12 A founder making a first hire, or a first hire in a new function, has one decision before posting: run the job description as drafted, or test its language, requirements, pay framing, and company pitch against a precisely defined target-candidate audience. Getting this wrong is expensive: a flooded pipeline of irrelevant applicants buries the handful of candidates who would have been a fit, and the hire that was supposed to unblock the business stalls instead. Job postings and candidate search are a matching problem, and matching frictions carry a measurable cost in time and quality of match (NBER: Job Postings and Labor Market Matching). ## Why job descriptions fail in predictable ways The real issue isn't writing quality. It's a founder's-eye view standing in for the candidate's. A founder writes what the company wants. A good candidate scans for what they will get. That gap explains most of the failure modes: - Buzzword overload. "Ninja," "rockstar," "wearing many hats." Strong candidates read these as "the role isn't defined yet." - Too many requirements. A list of 15 must-haves turns away exactly the senior candidates it's meant to attract; five would filter as well without the drop-off. - No sense of the company. Stage, headcount, funding, and how rough the scope is stay unstated, leaving strong candidates unable to judge fit. - Pay opacity. "Competitive salary" pushes top candidates toward whichever competitor published a number. - Mismatched tone. Copy that leans hard into startup swagger scares off operators; copy that reads corporate scares off builders. A founder cannot see these problems until the pipeline dries up, and by then the posting has turned away the candidates it needed most. ## Why are the traditional fixes slow and hard to validate? None of the standard options return a fast, falsifiable answer before the posting goes live. Labor-market conditions for job seekers and employers keep shifting, so an untested assumption about what a candidate wants can go stale quickly (SHRM: State of the Labor Market). ## How does a controlled comparison work? Subconscious runs controlled experiments that compare job-description variants (the requirements list, the pay framing, the tone, the company pitch) against a precisely defined target-candidate segment, and measures which version changes stated interest and intent to apply. That isolates which specific change moves the segment's stated intent, not general impressions of a draft. Hiring a first marketer aimed at a defined segment, say mid-career B2B SaaS marketers at a certain funding stage, gets tested against that segment, not a generic reader. Hiring a technical co-founder gets tested against senior engineers with startup experience. ## A workflow for a first hire - Write the job description you would have posted. Do not polish it first; the test should react to the real draft. - Define the target-candidate segment. Years of experience, current company type and level, pay band, and what the segment is likely to weigh most. - Test the first reaction. Present the draft as it would appear in a feed and capture initial response before asking anything else. - Isolate the dealbreakers. What in the draft would stop this segment from applying is the single most useful question in the workflow. - Test pay framing. Compare a vague framing ("competitive salary") against a stated range and measure the shift in stated interest. - Test the company pitch. The "about us" section is where strong candidates disengage. Compare variants of the pitch for what reads as compelling versus filler. - Revise and re-test. Run the updated draft against a fresh sample of the same segment to confirm the change moved the result, rather than assuming it did. ## What to watch for The "we need everything" trap. A requirements list with a dozen items usually has five that matter. A comparison test shows which ones move candidate intent and which ones narrow the pool for no reason. Senior candidates do not self-select into an impossible profile. The stage-obscurity problem. "Growing startup" reads differently from a specific, honest description of team size, funding stage, and how much ambiguity to expect. Specific framing keeps strong candidates in the funnel; vague framing loses them before they read the requirements. The mission-money imbalance. Strong candidates weigh both mission and a competitive offer. A posting that only sells mission reads, to an experienced operator, as a company that cannot afford them. A posting that only sells pay loses candidates who are evaluating the mission. A controlled comparison shows where a specific draft is out of balance. ## What are the limitations of this approach? A controlled comparison of job-description variants is not a substitute for interviewing actual candidates, observing real applicant behavior, running compensation benchmarking, or applying a recruiter's market judgment. It answers only which draft, tested against a defined segment, changes stated interest and intent to apply. Where the decision needs a real-world check, /how-we-work describes how a team moves from a simulated comparison to real-human validation without changing the underlying question being tested. See how these experiments are structured and validated at /research, or the approach applied to a specific hire at /demo. The job description is not the hire. It is the door. A founder who tests it against the candidates they want before posting spends less of that hire's runway finding out the hard way. [figure: A five-step path from an unposted draft to a revised job description: define the target candidate segment, test first reaction and pay framing, isolate what stops candidates from applying, then revise and retest.] --- # Another Survey Wave or a Causal Test? Deciding After a Metric Moves URL: https://subconscious.ai/blog/comparisons/another-survey-wave-causal-test-deciding-after-metric-moves Published: 2026-08-12 NPS drops eight points. A concept scores low in a tracking wave. Satisfaction dips a quarter after a pricing change. The instinct is to run another survey to explain it. The better question is whether the team already has the measurement it needs, and is missing a test of which specific action caused the shift, and which one would fix it. ## What a structured survey program answers Large-sample survey platforms measure at scale: tracking studies, NPS and CSAT programs, structured comparisons, and statistically significant samples across quarters. Qualtrics's experience-management platform is built around this kind of measurement across customer, employee, brand, and product research, and its survey software is embedded in how enterprise teams collect that data. That measurement answers "what changed" with precision: how many people rate something a certain way, and whether the movement is statistically significant. It does not, on its own, isolate which action caused the movement. A tracking wave can confirm satisfaction fell after a pricing change and a messaging change happened the same quarter, without saying which one did the damage, or whether a third factor did. ## Why doesn't re-surveying the same metric answer why it moved? Running another wave of the same tracking instrument mostly re-confirms the metric moved. It rarely isolates the cause, because a standard tracking survey isn't designed to hold everything else constant. Teams that re-survey to find "why" usually get a richer description of the same drop: more segments, more verbatims, more confirmation that something changed, not a controlled comparison of the actions themselves. A team ships a fix based on which explanation sounded most plausible in the verbatims, rather than which action was shown to move the outcome, and finds out at the next tracking wave whether the guess was right. ## What does a controlled causal experiment add? Subconscious is a causal behavioral platform. Instead of re-measuring the same metric, it runs a controlled experiment that holds the rest of the situation constant and tests a specific action, such as a price point, a message, or a packaging change, against alternatives. The output is a causal effect for that action, with a confidence interval where supported, rather than a description of an association. That reframes the question: not "did satisfaction change," but "does this specific action move the outcome, and by how much, before the team commits budget to shipping it." ## Comparing what each approach proves ## Proof and where it stops The misses sit on the same public record as the hits, next to the study they came from. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. When a decision warrants it, a study built this way can move from a simulated experiment into real-human validation without changing the underlying causal question: the same action is tested against the same design, with recruited participants instead of the simulated panel. Read how a study runs or the published results before deciding what evidence a specific decision needs. Publishing the boundary of a result is what stops it from being read as more than it proved. That proof point covers replication of causal experiments, not program-level tracking. ## Sequencing the two The choice isn't tracking survey versus causal experiment. It's recognizing which question is open. If the open question is "did anything change," the tracking program already answers it. If the open question is "which of these three actions would move the number back," that's the point to stop re-surveying and test the actions themselves. For a team weighing that next step, book time to walk through what a decision-specific test would look like against the current tracking data. ## Limitations Naming what this platform doesn't do is what lets a buyer check it against their own decision before they rely on it. Subconscious does not manage tracking programs, NPS or CSAT operations, or employee-experience research; those require the audit-grade, longitudinal measurement that large-sample survey platforms provide. Simulated experiments and recruited-participant validation are also distinct steps in the same causal test, not interchangeable descriptions of the same panel. [figure: Path from "metric moves" to a decision on which question is open. One branch: another tracking wave, confirms the move. Other branch: a causal test isolating one action. Both lead to shipping the tested action.] --- # AI Causal Graphs: 4 Checks Before You Act on One URL: https://subconscious.ai/blog/methods-and-validation/ai-causal-graphs-4-checks-before-act-one Published: 2026-08-12 An AI model can read a short list of variable names, say TV spend, brand awareness, website visits, sales, and return a full cause-and-effect graph without seeing a single row of your data. The output looks decisive: arrows, direction, a written explanation for every edge. It is still a hypothesis. Before it moves a budget or a strategy, it needs to clear four checks. ## What a proposed causal graph actually claims A causal graph represents each variable as a node and each cause as a directed arrow: TV spend → brand awareness, brand awareness → website visits, website visits → sales. Each arrow makes a specific, testable claim. Change the variable at the tail and the variable at the head should move. Change the variable at the head and the tail should not move back. That asymmetry, not the correlation between the two, is what makes a relationship causal. ## How an AI model builds that graph Given variable names, short descriptions, and a working definition of causality, a large language model can propose a plausible graph and attach a structured, machine-checkable explanation to each edge (Wan et al., "Large Language Models for Causal Discovery," arXiv 2402.11068). It reasons from general market knowledge, not your numbers. That makes it a low-effort way to generate a first hypothesis. It is not evidence that the pathway holds in your market. ## Four checks before an AI-proposed causal graph drives a decision ### 1. Does the direction match how the intervention would actually run? An edge like "brand awareness → website visits" implies that raising awareness would raise visits, not the reverse. Ask whether your team could run the change the arrow implies. If the intervention that would move the tail variable does not exist, or moves several variables at once, the direction is a guess dressed as a finding. ### 2. Does the edge survive without the graph's other assumptions? A graph is a set of edges that were accepted and a set that were deliberately left out. A single doubtful edge, such as routing all of TV's effect through awareness and none directly to visits, changes what the whole graph implies about where a dollar should go. Check the edges the model chose not to draw, not only the ones it kept. ### 3. Would a domain expert draw the same arrows from the same variable names? Because the model reasons from variable descriptions and general world knowledge, two plausible-sounding graphs can disagree on direction for the same variables. Recent work on LLM-guided causal discovery treats this as an active-learning problem: propose, score, and revise edges rather than accept the first pass (Wan et al., arXiv 2506.12227). Treat disagreement between the model and a domain expert as a signal to test the edge, not a tiebreaker to resolve by re-prompting. ### 4. Has the pathway been tested against real decision behavior? A graph that reads well is not the same as a graph that has been checked against how people actually respond to the intervention it describes. Until the specific pathway, TV spend to awareness to visits to sales, has been run as a controlled test, it is a hypothesis about your market, not a finding about it. ## From hypothesis to a decision-ready effect The fourth check is the one a proposed graph cannot answer on its own: does the hypothesized pathway move behavior, and by how much. Subconscious runs controlled causal experiments on simulated populations to test the total effect of a specific intervention, such as "cutting TV spend changes visits," establishing internal validity within the simulated population before it justifies a budget shift. The same causal question can move from that simulated test to real-human validation: Subconscious can test or validate studies with real human participants. Those controlled studies draw on simulated populations built from a person-level audience graph covering 800 million real people, kept distinct from the smaller set recruited for real-human validation. See how a study moves from hypothesis to replicated evidence and the kinds of decisions it has tested in past studies. ## Where this still needs care An AI-proposed graph is a hypothesis-generation step, not a validated causal effect. Multi-agent setups where several model instances propose, critique, and rank candidate graphs can widen the hypothesis space (Le et al., "Multi-Agent Causal Discovery Using Large Language Models," arXiv 2407.15073), but a wider set of plausible graphs is still a set of untested hypotheses. Treat every edge the same way regardless of how it was generated: as a claim to check against real decision behavior before it changes what your team does next. [figure: Path diagram: a proposed graph feeds a critique-and-rank step, producing more unvalidated hypotheses; a separate branch shows only a controlled test yields a decision-ready effect.] ## Next step Before reallocating budget on the strength of a proposed causal graph, pick the one edge the decision actually depends on and test that pathway directly. See current results on the leaderboard for how tested causal claims compare against untested ones. [figure: Four checks for an AI-proposed causal graph: direction matches a real intervention, edge holds alone, an expert agrees, the pathway is tested against real behavior.] --- # From Report Builder to Research Strategist: Restructuring the Research Workflow URL: https://subconscious.ai/blog/implementation-and-procurement/report-builder-research-strategist-restructuring-research-workflow Published: 2026-08-12 A research operations lead restructuring the team's workflow around AI has one decision to make: which business questions fast, directional AI-assisted exploration can answer, and which need a controlled experiment that produces a measurable causal effect with a confidence interval before anyone acts. Get that boundary wrong, and budget, pricing, or positioning decisions move on a hypothesis that was never tested against real behavior. ## What has AI actually changed in this job? AI has moved from novelty into daily research work. Drafting, summarizing, first-pass analysis, and quick directional reads are now routine. None of that removes the underlying demand for research judgment; it changes what gets rewarded. Market research analyst employment is projected to grow between 2024 and 2034 (U.S. Bureau of Labor Statistics). The mechanical parts of the job (formatting, first-draft summarizing, slide production) are getting faster and cheaper, which means the people doing that work have to move closer to the decision itself: sharper questions, better evidence choices, clearer caveats. ## The failure mode this restructuring prevents The costly mistake is not AI-assisted exploration, but mistaking it for proof. A fast, unvalidated read can surface hypotheses, objections, and comparisons. It cannot, on its own, tell a team whether the answer is safe to act on for an expensive or public decision. That judgment belongs to a human, and it must happen before the read reaches a stakeholder deck or an external claim. ## Two evidence tiers, not one blended step Two tiers, each with its own claim to make, keep fast methods safe: Naming a failure mode in advance is what lets a team catch it before it happens. The tiers are sequential. Skipping the second one to save time on a decision that warrants it is the failure this structure exists to prevent. ## Where does a controlled experiment fit? Subconscious operates at the validation tier. It runs a controlled discrete choice experiment against a modeled population and returns a causal effect for the tested action, with a confidence interval rather than a single fluent-sounding number. That distinction matters for a research operations lead because it changes what the team can defend in a room: not "this seemed compelling" but "this action moved the outcome by a measured amount, within this range of uncertainty." The same causal question can move from a simulated comparison to real-human validation when the decision is big enough to warrant recruited participants, without being rebuilt from scratch: the audience brief, the tested action, and the outcome measure carry over. ## A working process for the transition - Write the business decision in one sentence: what changes depending on which way the evidence points. - Define the audience and name the risk level: how expensive or how public is being wrong. - Use directional exploration to narrow the options worth testing. - Have a person review the audience definition, question framing, and business context before treating a directional read as an answer. - Route decisions that are expensive or will be stated externally to a controlled experiment, and to recruited real-human validation when the decision demands it. - Present every result with its evidence tier attached: what was tested, what was not, and what needs a higher tier of proof. ## What does this restructuring not do for you? A method's limits belong on the record next to what it does well. A controlled experiment does not decide the business question, does not write the audience brief, and does not decide when a decision is expensive or public enough to require full validation. Those calls stay with the research operations lead. The platform gives the team a measured effect to act on, not the judgment behind it. ## The first move this week Do not restructure the whole function at once. Take one live decision with a real deadline, write its one-sentence business question, and run it through both tiers: a fast directional pass to narrow the options, then a controlled comparison with a measured effect for the option closest to committing budget. Bring that decision into a working session once it's framed this way, and see how the two tiers connect end to end before scaling the process across the rest of the team's roadmap. [figure: A decision path: a business question feeds directional exploration, which narrows options. A branch asks if the decision is expensive or public. If not, act on the read. If yes, route to a controlled experiment first.] --- # Synthetic Consumer Research: 4 Steps to Check Before You Trust the Result URL: https://subconscious.ai/blog/methods-and-validation/synthetic-consumer-research-4-steps-check-before-trust-result Published: 2026-08-12 A synthetic-consumer result is trustworthy only as far as its weakest step. Before a pricing, concept, or message decision moves from a simulated study to real budget, a buyer needs to know which of four steps produced the number in front of them, and which step is a proxy for a person rather than an estimate of cause. ## What a synthetic consumer actually is A synthetic consumer is an AI persona built from real behavioral and demographic data, used to stand in for a human respondent in an early round of concept, pricing, or message testing. It answers questions the way a target segment might, before fieldwork starts and before launch or campaign budget is committed. The label covers a range of methods. General-purpose synthetic respondents answer social or policy survey questions. Digital twins update continuously from live data and sit closer to CX and personalization work. Synthetic consumers sit in between: purpose-built for market research, evaluating a concept, a price point, or a piece of messaging rather than modeling open-ended social behavior. None of that matters if a team cannot tell which part of the process produces a plausible-sounding persona and which part produces a causal estimate: an answer to which action moved which outcome, for which segment, with uncertainty attached where the study design supports it. Those are different claims, and the pipeline below separates them. ## The four-step pipeline ### What is the data foundation step in synthetic consumer research? For LLM-based synthetic consumers, the pretraining corpus defines the range of behavior the model can represent; survey history, purchase records, CRM data, or public datasets condition prompts or reweight personas within that range. A model trained on a narrow or stale dataset produces personas that sound plausible and answer poorly, because the population it draws from does not match the market being tested. ### 2. Persona generation A language model or probabilistic system turns the data foundation into individual profiles, each carrying attributes such as age, income, and stated motivations. Because personas are generated programmatically, a team can produce hundreds to stand in for a market segment. This step produces a population to test against, not yet a result. A well-built persona can still answer a poorly designed question. ### What happens during simulated testing? Personas are placed against a stimulus, a price, a concept, a message, and asked to respond. This is the step most synthetic-consumer vendors show off, because it produces a large volume of numbers with little setup. It is also the step most likely to be confused with a finished answer. A simulated response to a single price point is a data point, not evidence that the price causes a change in purchase intent, unless the test was built to isolate that relationship. ### 4. Benchmark validation The step that decides whether the first three were worth running: checking the simulated output against a real benchmark, whether that is a held-out human sample, a known market result, or a randomized, orthogonal factorial design that varies attributes simultaneously to isolate each causal driver. Reporting a confidence interval or an error bar here is a design choice, not a courtesy, but an interval computed across generated personas describes variation within the simulated population, not human population uncertainty, unless anchored to a human benchmark. Benchmarking here confirms the synthetic population reproduces observed responses; causal validity comes from the randomized, orthogonal design in step 3, not from this comparison. Absent this step, a synthetic-consumer result is an unverified guess with a persuasive interface. ## Where the method holds up, and where it doesn't A 2025 study using semantic similarity elicitation tested synthetic respondents against 57 real consumer surveys covering roughly 9,300 participants and found the method reproduced human purchase-intent distributions closely enough to be useful for concept work, though pricing decisions depend on the price coefficient and willingness-to-pay, an estimand intent-distribution agreement does not establish, with performance varying by task type and demographic setting. A separate methods review in Psychology & Marketing reached a more cautious conclusion: silicon-sampling studies are scattered across enough fields and methods that no single accuracy figure describes the technique, and results depend heavily on task design and evaluation choices. Both sources point to the split the four-step pipeline is built to catch: strongest at structured, rankable tasks; weakest wherever the answer depends on lived context a training set cannot supply. ## Can synthetic consumer studies be validated with real people? Step 4 does not require ending with only a synthetic benchmark. Subconscious can validate studies with real human participants, so a team can move from a simulated study to a real-human validation round without changing the underlying causal question. That matters because rebuilding the question from scratch at validation is where most of the earlier simulation's value gets lost. Real-human validation is worth adding when the decision is expensive enough that being wrong costs more than the time saved by skipping it, not as a default step on every test. ## Limitations to weigh before acting on a result - No single accuracy percentage applies across tasks, industries, or vendors. A number reported for one study's task and population does not transfer to a different concept, price range, or segment. - The validation step is a design requirement, not something a vendor demo has already solved for a given use case. Ask what the synthetic output was benchmarked against and how recently. - Structured tasks (ranking, pricing, sentiment) are where the method is best supported. Open-ended emotional, cultural, or group-dynamic questions are where it is least supported. - A simulated result is a starting hypothesis for a launch, price, or message decision, not a substitute for the fieldwork or experiment that decision's size warrants. Before committing budget on the strength of a synthetic-consumer result, a buyer can check current research on synthetic and human comparison methods or review how a study moves from design to validation. For a sense of where causal methods hold up and where they don't across published comparisons, the leaderboard tracks that record directly rather than asserting it. [figure: Four labeled stages left to right: Data foundation, Persona generation, Simulated testing, Causal validation. A break after stage 3 shows the output is unverified until stage 4 checks it against a benchmark.] --- # Recruit First or Test First: Sequencing Human Research Against a Controlled Experiment URL: https://subconscious.ai/blog/comparisons/recruit-first-test-first-sequencing-human-research-against-controlled Published: 2026-08-12 A research or insights lead deciding how to source evidence for a product, pricing, or messaging call has two starting points: commission recruited-participant interviews immediately, or run a controlled experiment first and reserve recruited sessions for validating what survives. Spend the full recruitment budget and a multi-week timeline testing variants a faster comparison would have narrowed, and both are gone before the real question gets answered. Skip human confirmation entirely and ship a decision on unvalidated output, and the team has a causal claim nobody has checked against a real person. [figure: Five-step horizontal path: define the decision, run it against the audience graph, narrow to survivors, recruit for what remains, validate with real humans for usability, sensitive, or regulator-verified questions.] ## What Does Recruited-Participant Research Actually Solve? A recruitment platform like User Interviews matches a study's target profile (job title, industry, behavior, demographics) against a pool of real people willing to be interviewed, tested, or surveyed. The researcher still designs and conducts the session; the platform's job is finding qualified humans and getting them to show up. That is a distinct capability, and it answers a different question than which of several actions is more likely to change a specific outcome. The category has been consolidating. UserTesting's acquisition of User Interviews folded a recruitment specialist into a broader user-research suite, a sign that recruitment is increasingly treated as infrastructure rather than a standalone research method. ## The Structural Cost of Recruiting Before the Question Is Narrowed Recruitment carries constraints no scheduling tool removes. As historical planning examples from the recruitment-research category, not current Subconscious figures: common participant profiles have been planned at roughly three to seven days to fill, and niche B2B targets (an enterprise SaaS engineering VP, a pharma procurement lead) have been planned at two to four weeks. Industry no-show rates in the 20-30% range are common, so paid recruiting and blocked calendar time go to sessions that don't happen. Each session is also a one-shot interaction, typically thirty to sixty minutes: if the questions should have been different, the participant is gone and re-recruiting starts the clock over. None of this is a flaw in a specific vendor; it is what happens when the unit of research is a scheduled human conversation, and it matters most before a team has narrowed which questions are worth that conversation. ## Where Does a Controlled Experiment Change the Sequence? Subconscious runs controlled experiments against a person-level audience graph covering 800 million real people. That is an experimentation surface, not a recruitable pool of participants: nobody is scheduled, screened, or paid to take part, because the graph is the substrate an experiment runs against, not a group of people waiting to be reached. Comparing several pricing, messaging, or product alternatives against that graph produces a directional read on which action is more likely to move the outcome. That changes what recruitment is for. Instead of using interviews to narrow a long list of ideas to a shortlist, a team can run the shortlist question against the audience graph first and spend the recruitment budget only on the alternatives that survive. ## What Does Real-Human Validation Add, and Where Does It Stop? Subconscious can test or validate studies with real human participants, letting a team move from a controlled experiment to real-human validation without changing the causal question: the same comparison, the same outcome definition, confirmed against real respondents rather than the audience graph alone. That is a validation step, not a replacement for what recruited, moderated research is built for. Watching someone click through a prototype and report confusion, running a study that requires regulator-verified real-respondent data, or sitting with someone through a healthcare or financial decision where emotional nuance carries the finding: those remain jobs for recruited human sessions. Real-human validation through Subconscious does not turn a controlled action test into a moderated usability session, a clinical trial, or a guarantee that a decision will perform in market. ## A Practical Sequence - Define the decision and the outcome it should move. Name the action being compared, not just the topic. - Run the comparison against the audience graph first, when the question is which of several actions is more likely to change the outcome. - Recruit real participants for what a controlled experiment can't answer: usability, emotionally sensitive topics, regulator-verified data, or the small number of ideas that survived the first pass and deserve a scheduled human conversation. - Validate the surviving hypothesis with real humans before it becomes a launch decision, keeping the same causal question throughout. Recruited interviews and a controlled experiment aren't competing for the same budget line; they answer different parts of the same decision. [figure: Two columns. Left: validation confirms the causal question with real respondents. Right: recruited research still needed for usability, regulated, and sensitive decisions.] ## Limitations Subconscious is not a participant-recruitment service and does not substitute for moderated usability testing, hands-on prototype interaction, or research contexts that require regulator-verified real-respondent data. This comparison does not include named customer results or an accuracy benchmark for this specific decision type; readers evaluating claims about experiment accuracy should look at the primary Subconscious research rather than infer a number from this article. Teams that want to see how Subconscious's experiments work in practice or want the method applied to a real decision can review case studies or book time to walk through a specific decision. --- # An AI Agent's Budget Recommendation Looks Confident. Is It Correct? URL: https://subconscious.ai/blog/use-cases/ai-agents-budget-recommendation-looks-confident-correct Published: 2026-08-12 A marketing leader gets a channel-reallocation recommendation from an AI analytics agent: shift budget toward the channel with the strongest apparent lift. The charts are clean, the language is decisive, the numbers look plausible. Should that recommendation be funded as-is, or does it need an independent causal check first? Fund it as-is only when the recommendation already comes with a confidence interval and a description of what was held constant. A single point estimate from a single agent run is not enough to act on, because a wrong answer and a right one can look identical. ## The same failure recurs across agent runs Autonomous coding agents write functioning code with high reliability. That is not the same as making a sound analytical judgment. A 2026 study by Bertran, Fogliato, and Wu gave identical datasets to autonomous agents and found the agents produced divergent conclusions across runs, with some runs reversing whether a hypothesis was even supported (Many AI Analysts, One Dataset: Navigating the Agent-Driven Data Science Multiverse). The code executed correctly every time; the analytical decision, meaning how to treat missing values, which model structure to fit, which priors to assume, which diagnostic to trust, was effectively random from one run to the next. "Across three datasets spanning distinct domains, AI analyst-produced analyses exhibit substantial dispersion in effect sizes, $p$-values, and conclusions." Bertran, Fogliato, and Wu, arXiv preprint 2602.18710 (source) That is the mechanism behind the budget scenario above. An agent deployed for three months of executive reporting can produce fabricated conclusions from flawless code, and nothing in the output signals the difference. ## A worked example: identical data, opposite recommendations One documented test isolated the effect of adding structured domain constraints to an otherwise identical agent and task. Without them, the agent found zero converged models on a stochastic volatility problem. With them, the same agent converged on two of three identical runs. Nothing changed about the model, the dataset, or the agent running it; the sole difference between the two runs was whether the agent had to stay on methodologically valid paths before it could report a result. The same pattern shows up in a marketing-mix example: given a noisy dataset with genuinely weak signal, an unconstrained agent recommended reallocating 100% of budget to television. A version of the agent built to check its own uncertainty instead reported that the signal was too weak to support a reallocation decision and recommended gathering more data first. Only one of those outputs was a decision a marketing leader should have acted on. ## Why does the recommendation still need an outside check? An agent engineered to flag its own uncertainty is a real improvement over one that isn't. It is still the same agent grading its own homework. A recommendation carries more weight when its verification comes from an independently run test, not the system that produced it. Subconscious runs controlled, randomized experiments and reports causal effects with confidence intervals. A team can move from a simulated experiment to real-human validation without changing the causal question, which means the reallocation recommendation above can be tested against real respondent behavior instead of taken on the agent's word (research). Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people, distinct from a recruitable panel of people who agree to participate in a study. Before funding a reallocation, the practical check is to run the underlying causal question (which channel actually drives the outcome, and by how much) as its own controlled test, and see whether the agent's recommendation holds up against it (demo, how we work, case studies). ## What does this analysis not cover? Subconscious does not build, audit, or certify third-party AI data-science agent frameworks, and an independent causal check does not replace the domain-specific statistical diagnostics, such as convergence checks and residual analysis, that belong inside a modeling pipeline. What an outside causal test adds is a second, independently produced answer to compare the agent's recommendation against, on the specific decision that spend depends on. [figure: A single path through four sequential judgment calls, missing data, model choice, priors, diagnostics, ending in one confident recommendation with no visible branch to the alternative calls the agent didn't take.] --- # How B2B SaaS Teams Test Product and Pricing Decisions Before Recruitment Finishes URL: https://subconscious.ai/blog/use-cases/how-b2b-saas-teams-test-product-pricing-decisions-before-recruitment Published: 2026-08-12 B2B SaaS teams under-research their buying committees, then ship positioning, pricing, and roadmap decisions built on assumptions. Causal experiments on a simulated market close that gap without waiting on a recruitment cycle the buying committee will never sit through. [figure: Four-stage path: a stalled recruitment route taking 4-8 weeks; a buyer role defined by seniority, goals, and constraints; a controlled comparison of two pricing structures; the decision landing inside the sprint.] ## Why B2B SaaS Teams Under-Research Their Buyers The reasons are structural, not cultural: three constraints keep teams from getting customer evidence on a useful timeline. ### Why Is Enterprise Buyer Recruitment Hard? Most B2B SaaS products sell into a narrow, specific audience: a head of marketing at a mid-market company, a VP of engineering at a Series B startup, a CFO at a company with 200 to 500 employees. These roles are busy, do not sit on research panels, and ignore cold outreach. Incentives that work for consumer research, such as €50 gift cards, are irrelevant to someone earning €150k or more. A typical B2B research study needs 12 to 20 interviews. Finding that many qualified participants runs 4 to 8 weeks, and recruitment fees alone add up to €5,000 to €15,000 (Drive Research's 2026 market research cost guide). Subconscious can define a target population against a person-level audience graph covering 800 million real people. That reach supports precise population definition; it does not mean 800 million people are available as recruited study participants. ### Why Do Small Customer Bases Limit Access? A SaaS product with 200 paying customers has a shallow pool to draw from. Interviewing 15 of them means talking to 7.5% of the entire customer base, and it burns goodwill for 6 to 12 months before most of those customers will agree to another interview. ### Why Doesn't Research Fit Sprint Cycles? Product teams work in two-week sprints, but traditional research needs 6 to 8 weeks to finish. Findings tend to land after the team has already shipped the feature, moved to the next priority, or changed direction. ## The Decisions That Stay Unanswered Despite the difficulty, B2B SaaS teams carry research questions that directly affect revenue, retention, and growth: - Positioning and messaging. Does the value proposition land the same way with a VP of sales as with a head of marketing? Which objections differ between enterprise and mid-market buyers? - Feature prioritization. Which features matter most to which segment, and where do user requests and buyer value diverge? - Competitive positioning. How do target buyers weigh the company against alternatives, and what triggers a switch? - Pricing and packaging. How do segments react to a pricing structure change, and where is the actual value gap? - Expansion and churn. What causes an account to expand usage, and what precedes churn? Most teams answer these with internal opinion rather than buyer evidence, because traditional recruitment costs too much relative to a two-week sprint. ## Where Causal Experiments Fit B2B Product Research Subconscious is a causal behavioral platform: it runs controlled experiments on a simulation of the market, validated against real human behavior, to estimate which action changes a buyer's decision. For B2B SaaS, that means comparing product, pricing, messaging, and go-to-market actions across defined buyer and user roles before committing roadmap capacity or sales effort to one of them. ### Define buyer roles with real depth B2B decisions depend on role, not just demographic. A useful buyer definition for this kind of test includes: This level of detail is what lets a comparison across roles reflect an actual B2B buying committee rather than a single generic response. ### Run controlled comparisons for specific decisions - Positioning. Present the value proposition to 5 buyer roles and compare what each understood, what was compelling, and whether it produced a next step. - Feature prioritization. Compare reactions to 3 candidate features and follow up on the reasoning behind the ranking. - Competitive reaction. Test how a competitor's announcement shifts perceived buying criteria. - Pricing. Compare 2 pricing structures and estimate which one a role would advocate for internally, including anticipated objections. ### Iterate inside the sprint, not around it The advantage isn't raw speed. It's fitting a controlled comparison inside the same two-week sprint where the decision is being made, instead of waiting 6 to 8 weeks for interview findings to arrive after the decision has shipped. A team can test a positioning variant before a standup or compare pricing reactions across segments before a pricing meeting. ## Where Real Buyer Contact Still Matters A causal action test does not replace every form of customer research for B2B SaaS. It works best paired with real buyer contact. Use a simulated comparison for: - Testing whether a hypothesis is worth a full research investment. - Comparing how the same message lands across 5 defined buyer roles without recruiting 5 separate cohorts. - Testing reactions to a hypothetical competitive or market change. - Exploring the question space before designing a quantitative survey. Use real buyer contact for: - Building the relationship and trust that a controlled experiment cannot substitute for. - Emotional discovery: how a buyer feels about a problem, not only how they reason about it. - Validating a high-risk decision, such as a major price change or market pivot, before it ships. - Surfacing the unexpected finding a real conversation produces and a controlled comparison cannot. Subconscious can also test or validate studies with real human participants, which lets a team move from a simulated comparison to real-human validation without changing the underlying causal question. That step matters most for the highest-stakes decisions above. ## How to Start - Start with the ideal customer profile. Define 4 to 5 roles that represent the main buyer segments. - Ground each role in real evidence: sales call recordings, support conversations, and CRM notes. - Test a live question. Pick something the product team is actively debating and run a comparison against it. - Compare the result against recent human interview data where it exists to calibrate confidence in the method. - Put the comparison inside the sprint cycle so it becomes a standard input, not a special request. A single controlled comparison does not replace a buying-committee relationship, direct sales and support evidence, or observed product use. It gives a B2B SaaS team a faster first pass at which action is worth taking. Teams can review a decision already tested or scope a comparison for a live product, pricing, or positioning choice. --- # AI Marketing Personas Can Critique Your Plan. They Can't Test It. URL: https://subconscious.ai/blog/buyers/ai-cmo-decision-support-test-marketing-actions-before-spending Published: 2026-08-12 An AI marketing persona offers an opinion on a plan, while only a causal test of buyer response reveals whether that plan actually changes what the buyer does. A senior-sounding AI marketing persona takes little effort to set up and is always available. It will tell you your positioning is vague, your channel plan looks risky, or your launch sequence is too ambitious. All of that can be accurate and still leave the real question open: will the buyer respond the way the plan assumes? An opinion, however well-informed, is a guess about what happens next. That gap, between a persona reviewing your plan and a method that shows which version of the plan a buyer would choose, determines whether the budget is well spent. [figure: One marketing plan splits into two paths to the same budget decision: persona critique goes straight to approval with no buyer evidence; buyer response test reaches the decision with evidence attached.] ## Where the gap shows up An early-stage founder team can build and sell a product with no one on staff who has scaled marketing past a small budget or built a brand from scratch, and no way to check whether a positioning line survives contact with a better-funded competitor. A growth-stage company might have five to ten marketers who run campaigns well but have never navigated a move into an enterprise segment or a shift in category strategy. At a larger company, a CMO managing a team of 50 can sit behind a three-week backlog before a product marketer gets a strategic read on a repositioning question. An AI persona tuned to sound like a seasoned marketer can fill that gap with useful triage: is the positioning specific enough, does the channel plan hold up against a rival that outspends you, is a message likely to land. That's a real service. It is not evidence that the plan works. ## Is an AI persona's opinion a stated preference? Any answer the persona gives, including "yes, launch this campaign," is what researchers call a stated preference: a description of what someone would do, without doing it. Stated preferences reliably diverge from revealed preferences, the choices people make when the decision is real (Dectech on stated versus revealed preference research). A plan approved because a persona liked it carries the same risk as one approved because an internal team liked it: nobody checked what the buyer would do. A team can commit a $200K brand campaign to a message nobody tested against the actual buyer, and learn only after the budget is spent that the message didn't land. A fluent, confident review process, human or AI, does not protect against that outcome. Only a test of the buyer's response does. ## What does a causal test add? Subconscious runs a controlled experiment on the decision itself: which positioning line, which channel and message pairing, which launch sequence. The comparison runs against a person-level audience graph covering 800 million real people, using causal experimentation and discrete-choice-style modeling rather than a single fluent opinion. When the stakes justify it, the same comparison can move from that modeled audience to real human participants without changing what's being measured. Pairing a large modeled comparison with a real-human check follows the same logic researchers have used for decades to keep a stated answer honest against what people do (Marketing Letters on combining revealed and stated preference data). ## Decisions worth testing before the money moves - Positioning. Compare a category claim against an outcome claim, or a broad audience frame against a narrower one. Test which version changes what the buyer says they would do, not which one reads better internally. - Campaign channel and message. Before a large brand budget goes out, especially into a channel where rivals already outspend you 10x, test whether a different message or a different channel produces a stronger buyer response than the default plan. - Launch sequence. A plan to enter three markets at once is a prioritization call dressed up as a launch plan. Test staged entry against simultaneous entry under matching conditions before committing the team to either. ## What a tested result does not replace This section states what the causal test does not do, so the buyer can check that limit before the budget moves. Testing a decision is not the same as running the business behind it. It doesn't build the agency relationships, media contacts, or peer network a marketing leader accumulates over a career. It doesn't execute: someone still has to write the campaign, brief the agency, and run the calendar. It doesn't carry the pattern recognition of a person who has lived through a comparable call inside a comparable company, and it doesn't sit in the room to navigate the politics between sales, product, and the executive team. A tested result gives the human decision-maker better evidence. It doesn't take the decision, or the accountability for it, off their desk. ## Where to start Pick one decision that's about to consume real budget or a real launch window: a positioning line, a campaign channel, a market sequence. Name the alternatives worth choosing between, then bring that decision to a Subconscious working session instead of another plan review. To see how a comparable test read for another team first, the published case studies show the format. --- # Aaru and EY: What a 90% Correlation Claim Actually Covers URL: https://subconscious.ai/blog/methods-and-validation/aaru-ey-what-90-correlation-means Published: 2026-08-12 A synthetic-research vendor publishes a correlation number against a Big Four partner, and the number circulates as proof the category works. Before that number changes where a research budget goes, a buyer needs to know what question type it actually covers. ## The claim: a partnership-published correlation Aaru, a multi-agent behavior simulation vendor, and EY published a correlation of approximately 90 percent between Aaru's synthetic simulation outputs and EY's real-respondent results on parallel research questions (EY, "How AI simulation accelerates growth in wealth and asset management"). EY ran studies holding both a human-respondent baseline and an Aaru synthetic result, then measured how closely the two tracked. A correlation number is only useful to a buyer when its boundaries are stated alongside it. That is a partnership-published figure, not an independently verified or peer-reviewed result. Neither Subconscious nor any other vendor has replicated it. ## What does "90 percent correlation" measure? Correlation measures co-movement: when the human result moves up, the simulated result tends to move up too. It does not by itself establish that the two match in scale or level; a synthetic series scaled or shifted relative to the human series can still produce the same correlation. A 90 percent correlation means the two cluster tightly along that pattern across the tested questions. The individual misses sit next to the aggregate hit rate for anyone checking the number. It does not mean the simulation matched any single human result exactly. Individual questions can still miss by a meaningful margin even while the overall correlation stays high, a portfolio-level statement about direction, not a per-question accuracy guarantee. ## What does the correlation not establish? - It does not establish causation. Reproducing an observed pattern is not the same as identifying which action produced it. - It does not transfer to a different question type. A correlation measured on the stated-preference and concept-reaction questions here is evidence about that question type, not a general accuracy rating for every use of synthetic research. - It does not replace a validation path for a higher-stakes decision. A single partnership case study is not the same as a vendor's own defined and sourced validation corpus that a buyer can inspect study by study. ## Where this fits against a broader validation picture Any synthetic-research or causal-simulation buyer should ask three questions before treating a headline number as sufficient: what was measured, what question type it covers, and whether there is a path from simulation to real-human validation without changing the underlying question. Subconscious approaches that third question directly. Its causal behavioral experiments move from a simulated population to real-human participants when a decision needs that added confidence. Its validation corpus is defined and sourced rather than resting on one partnership: on the causal fidelity paper, its best configuration reaches 87% of the measured human ceiling on one study, 0.832 rank correlation against a 0.959 human-to-human ceiling, with a mean of 0.73 across the 43 studies passing design filters, drawn from roughly 300 replicated studies across 9 domains. Naming exactly what a figure covers is what lets a buyer check it against their own decision. That figure describes replication of past study outcomes, not prediction accuracy on a novel decision, and is not interchangeable with the Aaru-EY correlation figure above, which measures a different comparison. Controlled studies can also draw on an 800M-person audience graph as a sampling pool, distinct from a recruitable human panel; graph size is not a measure of statistical precision, which depends on respondent count, choice tasks, and design efficiency, and it matters mainly for the breadth of population available to sample rather than everyday message or concept testing. ## A short checklist before trusting a validation claim ## What's the practical takeaway here? A correlation number like Aaru and EY's is a reasonable signal that behavior simulation can reproduce useful aggregate patterns on the question type tested. It is the start of a buyer's evaluation, not the end. Before moving budget toward any vendor's headline accuracy claim, run it through the checklist above. Subconscious's research documents that path; its case studies show it applied to buyer decisions. [figure: Four-step path before trusting a validation claim: check what was measured, what question type it covers, whether the source is one case study or a sourced corpus, and whether there is a path to real-human validation.] Teams comparing methods at this stage often also want to see how a causal experiment is structured and run before deciding which validation path fits their decision. --- # Brand Awareness Research: Turning a Recall Number Into a Causal Decision URL: https://subconscious.ai/blog/use-cases/ai-brand-awareness-research-measure-brand-recall Published: 2026-08-12 A CMO staring at a tracking-wave readout usually has one number: the share of respondents who say they have heard of the brand. That figure cannot say whether a campaign moved it, whether a competitor's launch ate into it, or whether next quarter's budget should follow the channel that produced it. Before committing spend to another wave, the real decision is whether to keep reading a single descriptive percentage or to design a test that traces a change in recall back to one cause. ## A recall score collapses several different questions into one Awareness is not one construct. Market-research practice separates it into layers, each answering a different question about where a brand sits in a buyer's head. - Unaided recall: does the brand surface when someone names the category unprompted? A brand missing from the first few unprompted answers has a different problem than one people simply forget when asked directly. - Aided recall: does the brand register once its name is shown? This is the easier bar to clear, and a wide gap between aided and unaided scores usually means the brand is visible but not top of mind. - Category fit: which category do respondents place the brand in? A premium tool filed under a generic label is not competing where its team thinks it is. - Attribute association: what comes to mind alongside the name: speed, cost, trust, confusion? This is the content behind the score, not the score itself. - Competitive standing: recall only means something relative to the other brands fighting for the same buyer attention. Standard industry guidance treats unaided and aided recall as distinct, sequential questions in a study design rather than interchangeable proxies for the same thing (SurveyMonkey; Drive Research). A single closed-ended question cannot carry all five layers at once. ## Why can't a single wave tell you what caused the change? Closed-ended surveys force awareness into a scale: on a scale of one to five, how familiar are you with this brand. That question returns a score without a reason: it cannot say why someone recognizes the name, what they remember about it, or whether the association helps or hurts the brand. Group interviews go deeper on the "why," but carry a different distortion. One participant states an opinion, the rest nod along, and the moderator records agreement that is closer to social pressure than independent recall. A brand that runs one wave before a campaign and one wave after has two snapshots and no way to separate the campaign's effect from seasonality, a competitor's move, or plain survey noise. ## How do you measure brand awareness as an experiment, not a snapshot? The fix is not a better survey question. It is a study design that isolates one variable. Instead of asking a single audience whether they have heard of a brand, expose one group to a message or campaign concept and hold a matched group back as a comparison. Ask both groups the same unaided- and aided-recall questions afterward. Any difference between the exposed group and the holdout is attributable to the thing that changed between them, not to whatever else moved in the market that week. ## Where does a causal action test fit into brand awareness research? Subconscious runs this kind of comparison as a causal action test: a message, concept, or campaign treatment is defined as an action, tested against a holdout, and the two outcomes are compared directly with uncertainty stated alongside the result, instead of a single top-line number. That structure covers the two situations a brand team faces most often with awareness work. - Pre/post campaign measurement. Capture a baseline before launch, run the same protocol on a comparable audience after launch, and read the difference as an effect of the campaign rather than an assumption. - Competitive benchmarking. Run the identical recall and association questions against audience definitions built around a competitor's buyers and the brand's own buyers, so the comparison happens in one study instead of two fielded months apart with different samples. Subconscious can run these comparisons against a person-level audience graph covering 800 million real people, supporting the specific buyer segments a competitive or pre/post study needs rather than one generic sample. When a team wants to move from that comparison to a real-human check on the same question, Subconscious can also test or validate the study with real human participants, without changing what is being asked or compared. See how this fits into a broader research program or worked comparisons from past studies. ## What this does not replace A causal action test answers one question well: did this specific message, concept, or campaign move recall, and by how much. It is not a packaged awareness-tracking product, it does not substitute open-ended interviews for a structured protocol, and it carries no data-residency or compliance guarantee. The brand team still owns the segment definitions, the questions asked, and the interpretation of category fit and attribute association described above. [figure: Five stacked layers labeled unaided recall, aided recall, category fit, attribute association, and competitive standing, converging into a single arrow pointing to one number labeled "the score a CMO sees."] ## Setting up the comparison Three decisions determine whether the test answers the real question. - Name the action. Define exactly what the exposed group sees: one message, one concept, one campaign cut. A vague "the new campaign" as the treatment makes the result hard to attribute to anything specific. - Match the holdout. The comparison group needs the same audience definition as the exposed group, differing only in exposure. An unmatched holdout reintroduces the noise the test was built to remove. - Ask the same layered questions of both groups. Unaided recall, aided recall, category fit, and attribute association, asked identically before and after exposure, so the result shows which layer moved and which did not. Once those three are fixed, a team can plan a study that answers not just whether recall moved, but which specific action moved it. Talk through a study design or see what a causal action test setup looks like before the next wave goes into the field. [figure: Chain diagram: one group is exposed to a campaign while a matched holdout is not; both are asked the same recall questions; the difference between their answers is labeled the effect attributable to the campaign.] A study that treats these layers as one number keeps producing a percentage nobody can act on. A study built as a comparison, with a matched holdout and a named action, produces a number a budget decision can rest on. --- # Customer Research Platforms vs. Sales Roleplay Tools: Picking the Right Category URL: https://subconscious.ai/blog/comparisons/customer-research-platforms-vs-sales-roleplay-tools-picking-right-category Published: 2026-08-12 Customer research platforms run causal experiments to predict buyer response before launch, while sales roleplay tools simulate prospects so reps can practice calls, and the deciding factor is whether the open question is buyer behavior or rep performance. "AI that simulates people" covers two different jobs. One category simulates prospects so sales reps can practice a call. The other simulates customers so a marketing, insights, or product team can test a message, price, or feature before it ships. Confuse the two and you buy the wrong tool. ## The question that actually decides which category you need Is the problem "our reps fumble objections and take too long to ramp," or is it "we are about to spend budget on a launch and do not know how buyers will respond"? The first is a sales-enablement question. The second is a market-research question. They require different tools, different buyers on the team, and different evidence. ## What does a sales-roleplay tool do? Tools built for rep coaching, such as Pitchbase, put a synthetic prospect on a call with a rep. The rep picks a scenario (cold call, discovery, demo, objection handling, close) and practices in real time while the system scores structured coaching feedback per call. It does not answer whether a market will respond to a given message or price. ## What does a customer-research platform do? A causal behavioral platform runs controlled experiments on synthetic populations to estimate how buyers would respond to a specific messaging, pricing, or positioning choice, before a team spends budget on it. Subconscious.ai runs this class of experiment (discrete choice experiments, mixed logit, and ICLV) and returns causal effects with confidence intervals rather than a simulated conversation. See the replication leaderboard for the methodology and the replication-accuracy evidence. ## Where the cost of picking wrong shows up A team that evaluates a rep-coaching tool to answer a go-to-market question gets conversational practice reps, not a causal read on how real buyers would respond, so the launch decision still gets made on guesswork. Naming the wrong-fit case up front is what lets a buyer check it before purchase. A sales team that tries to use a customer-research platform to drill call handling gets aggregate insight reports, not real-time per-rep coaching feedback. ## Matching the need to the category ## Where does Subconscious fit, and where does it not? Subconscious answers the customer-research side of this split: whether a message, price, or feature lands with a defined buyer population, with a causal effect and a confidence interval. It does not offer sales-rep coaching, voice-based call simulation, or per-rep performance scoring. A team running both jobs needs a tool from each category, not one stretched across both. ## Keep audience reach and validated human research distinct Reach and validation are different claims, and each gets stated on its own terms here rather than folded together. A synthetic panel's breadth (how many segments or profiles it can cover) is not the same claim as recruited real-human validation. When a launch decision is high-stakes enough to warrant it, a team can move from a simulated discrete choice experiment to real-human testing without changing the underlying causal question. See /how-we-work for how that step fits the same experimental design. Simulated output on its own does not replace validated human research; it narrows what is worth validating. [figure: Two-column comparison: sales-roleplay tools score per-call coaching feedback; customer-research platforms return causal effects with confidence intervals from synthetic-population experiments.] ## Next step If the open question is "are our reps handling objections well," that is a sales-coaching evaluation, outside what a causal research platform is built for. If the open question is "will this message, price, or feature land before we spend budget on it," book a demo to see how a controlled discrete choice experiment answers that with a causal effect and a confidence interval. --- # How PR and Comms Teams Can Pre-Test a Narrative Before It Ships URL: https://subconscious.ai/blog/buyers/how-pr-comms-teams-pre-test-narrative-before-ships Published: 2026-08-12 A PR or corporate-comms lead should pre-test a narrative when choosing the wrong version would create public damage that cannot be cleanly reversed. The test should compare specific drafts against a defined audience and a defined outcome, such as trust, perception, or support. It should not ask only whether colleagues like the copy. [figure: Four-step path: name the audience, choose the outcome, compare narrative variants against that audience and outcome, then advance the stronger draft to legal and factual review.] ## Approve the effect, not just the wording Internal review is useful for accuracy, tone, and organizational alignment. It cannot show how an audience's response changes because one narrative ships instead of another. The people closest to a client know the intended meaning, which makes them poorly placed to detect when bold sounds overhyped, accountability sounds evasive, or a provocative thesis sounds careless. Frame the decision as a causal messaging experiment comparing two or three narrative variants: - Name the audience whose response matters. - Choose the outcome the narrative is meant to move. - Estimate the relative change in that outcome across the variants. This separates the approval decision from a general copy critique and addresses the say-do gap: stated preference alone does not establish what people will trust, support, or act on when the message becomes public. ## Three narratives with different costs of error ### A launch announcement A launch narrative may need to signal category leadership without reading as inflated. Compare the assertive framing with a measured alternative for the customer or investor audience that must believe it. The useful result is not a universal score for the copy but evidence about which version better moves the outcome chosen for that audience. ### A crisis statement Crisis-communication guidance treats preparation as a plan made before an incident, not only as wording improvised once scrutiny begins (Prezly's crisis communication guide). A narrative comparison applies that preparation to the statement itself. It can test whether one draft produces more trust or support than another among customers, employees, or investors. A comparison result only means something if its edges are stated plainly. This evidence does not decide whether a statement is legally sufficient or factually complete. It helps the comms lead choose between credible alternatives before the response becomes public. ### An executive point of view A provocative thesis may be intended to win confidence from investors, peers, or enterprise buyers. The approval question is whether the sharper argument strengthens that response or causes the audience to discount the executive. Comparing the core thesis with a more restrained version makes that tradeoff explicit. ## Read the evidence at the right scale Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. A reach number means little without the limit attached to it. That number describes modeling reach. It does not mean 800 million people are available to recruit for a study. When a simulated comparison warrants another form of evidence, Subconscious can validate the study with real human participants, keeping the causal question constant while moving to real-human validation. The two forms of evidence remain distinct. This record states what the method has not proven yet. The current proof does not establish PR-specific outcomes. Subconscious does not provide a packaged crisis-response workflow, named PR case studies, or journalist-specific recruiting. A comparison estimates the effect of the drafts and audiences actually defined in the study. It does not predict how one named journalist will react or guarantee media coverage, public approval, or market performance. [figure: Three rows, each a narrative type weighing its own failure risk, converging on one shared audience-and-outcome comparison step.] ## Put the comparison into the approval room The work begins with the decision the client must make, not with a generic request to test a message. Define the cost of choosing poorly. Draft the credible alternatives. Specify the audience and the response that matters. Then use the comparison to decide which narrative advances to legal, regulatory, factual, and final editorial review. This gives an agency or in-house team a clearer approval conversation: experience still shapes the alternatives, and evidence shows how the choice between them changes the expected audience response. Teams considering this approach can review the underlying research, examine how a study is structured, or request a walkthrough for a specific narrative decision. --- # Test Pricing Assumptions Before the Real Study URL: https://subconscious.ai/blog/implementation-and-procurement/test-pricing-assumptions-before-real-study Published: 2026-08-12 Before a pricing or product research lead commits budget to a conjoint or willingness-to-pay study, the real decision is narrower: which pricing assumptions deserve that fielding spend, and which ones a cheap directional round can kill first. Sending untested attributes, levels, or value language into a fielded study risks burning a validation slot on a hypothesis a faster check would have ruled out. [figure: Four-step horizontal path: Exploration generates hypotheses; Directional testing compares options on a simulated audience; Human review checks audience, prompts, and grounding; Validation runs the fielded study.] ## Why does this pricing decision matter now? Research teams increasingly use AI for analysis, reporting, data preparation, and self-service insight, which puts pressure on the earlier stages of a pricing study, where assumptions get set before fielding budget is spent. That pressure does not remove the need for a fielded conjoint or willingness-to-pay study; it changes what happens before that study starts, moving the harder problem earlier: which attributes, price levels, and value-language options are worth fielding at all. Demand for the underlying research work is not disappearing either; the Bureau of Labor Statistics puts market research analyst and marketing specialist jobs on a growth path between 2024 and 2034. ## What changes in scoping a pricing study The old default was to field broadly and let the analysis sort out which attributes mattered. That still works, but costs more when several tested attributes or levels were never checked against a cheap directional read first. A pricing lead who scopes the study well does two things: owns the question before any exploratory tool touches it, and owns the caveat after it produces output. That means naming the decision the study answers, what would change it, the confidence level required, and where a directional read could mislead if treated as final. ## A four-layer evidence system for pricing assumptions A pricing study should not jump straight from a hypothesis to a fielded conjoint. A clearer sequence keeps the expensive step for what needs it: The value of the directional layer is not the output itself. It is a disciplined path to a safer, better-scoped fielded study, with clearer attributes, cleaner levels, and sharper hypotheses. ## Where does Subconscious fit in that sequence? Subconscious is a causal experiment layer for comparing pricing-adjacent actions, such as packaging, framing, and positioning, before the expensive fielded study runs. It is a controlled experiment on simulated buyers, not a substitute for a fielded conjoint or willingness-to-pay study, and its output is labeled directional until validated. The workflow starts with the decision, not the tool. Write down what changes depending on the answer, then define the audience precisely: segment, context, current behavior, alternatives considered, and what the buyer is trying to accomplish. Run a focused stimulus, such as a pricing story, a packaging option, or a framing choice, and ask for reactions, objections, and comparisons. Compare segments and look for contradictions before treating an early answer as settled. When the pricing decision is consequential enough to warrant it, Subconscious can test or validate studies with real human participants, moving from the simulated comparison to real-human testing without changing the underlying causal question. That step validates the narrower, better-scoped study the directional round produced, not a substitute for it. ## What is the failure mode to avoid? The mistake is treating a directional, simulated read as the answer the fielded study was supposed to produce, usually driven by a rush for speed, a confident-sounding output, and a deadline with no room to double-check it. A directional comparison and a validated willingness-to-pay result answer different questions; confusing them is the failure this workflow exists to prevent. The fix is to make the limits part of the deliverable: state the directional round's intended use, where it does not apply, and what still needs validation before the business acts on it. ## Limitations Subconscious does not produce a final price for the buyer, settle willingness to pay, or replace a fielded conjoint or WTP study. It narrows which assumptions deserve that spend. A case study is the place to look for evidence a specific method held up in a specific market; this scoping question has none attached, so treat the approach above as a process recommendation, not a proof point. ## What to do this week - Pick a real pricing decision with budget attached. - Write the pricing decision in one sentence: what changes depending on the answer. - Define the audience and how much the decision can tolerate being wrong. - Run a directional comparison on the packaging, framing, or positioning question, not the full attribute list. - Have a person flag which output is usable, which is shaky, and what shouldn't be acted on. - Decide which parts of the study still need real-human validation, and book time to scope that step. Read more about how the underlying method works in Subconscious's research. --- # Social Listening vs. Causal Simulation: When Talkwalker Data Isn't Enough URL: https://subconscious.ai/blog/comparisons/social-listening-vs-causal-simulation-when-talkwalker-data-isnt-enough Published: 2026-08-12 A brand strategy or consumer insights leader running Talkwalker already has a steady read on what customers are saying: sentiment trends, share of voice, mention volume across social, news, and broadcast. The harder question shows up the moment that data has to justify a decision that hasn't happened yet: a price change, a new claim, a repositioning. Listening data describes the conversation that already exists. It has nothing to say about a conversation that hasn't started. ## What social listening answers, and what it can't Talkwalker and platforms like it aggregate what people have already said, then structure it into dashboards: sentiment over time, spikes tied to events, share of voice against competitors (Gartner Peer Insights). It tells a team what happened and how people reacted to it. It cannot tell a team what will happen when something new ships. A price increase nobody has announced generates no conversation to listen to. A repositioned claim that hasn't gone to market has no share of voice. ## What's the gap when deciding on something that hasn't shipped? A team greenlights a price change, a new claim, or a repositioning because sentiment has been trending favorably and volume looks healthy, then finds after launch that real customer behavior in response to the new, real option doesn't match what the pre-launch conversation seemed to predict. Past sentiment wasn't measuring the thing that mattered: how people respond to a specific action once it's real and in front of them. Closing that gap requires a different kind of evidence: a controlled test of the specific change, run before it ships. ## How does Subconscious answer a different question? Subconscious runs randomized experiments on a simulation of the market to estimate what happens if a specific action is taken, a price, a claim, a feature, and reports the causal effect with confidence intervals. Where listening data is descriptive (what has been said), this is prescriptive (which action moves the outcome). The simulated results are validated against real human behavior. A fidelity number without its limits is marketing. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. When a decision calls for it, a team can move from a simulated study to real-human validation of the same causal question without changing what's being tested. ## Where each tool fits The two are complementary rather than substitutes. Listening data can surface which alternatives are worth testing, a spike in complaints about a price point, a competitor claim gaining share of voice. Simulation tests the alternatives that don't exist in the conversation yet, before a team commits budget to them. ## What doesn't Subconscious do? Naming what the tool skips is what lets a buyer check the fit before they commit budget. Subconscious does not monitor live social conversation, track sentiment trends, measure share of voice, or scan brand mentions across social platforms, news, and broadcast archives. Causal simulation is useful when there is a specific action to test, a price, a claim, a feature, not for an open-ended read on what people are currently saying. A team that needs both should expect to run both: listening for ongoing signal, simulation for the decision that hasn't shipped. ## Where this applies If the question is "what have people been saying," listening data answers it. If the question is "what will people do when this specific, unreleased thing is real," that requires a controlled experiment on the action itself, not a read on the conversation that preceded it. [figure: A two-column comparison table contrasting social listening with causal simulation across five rows: the question each answers, its data source, when it works, the evidence it produces, and how it's validated.] --- # Single Buyer Conversation vs. Segmented Causal Study URL: https://subconscious.ai/blog/comparisons/single-buyer-conversation-vs-segmented-causal-study Published: 2026-08-12 A single simulated buyer conversation is useful for exploring language, objections, and hypotheses. It is not enough evidence to approve positioning, pricing, or messaging when several buyer segments may respond differently. That decision needs controlled alternatives, segment-level effects, uncertainty, and, at final launch stakes, recruited human validation. [figure: Two-column chart: single conversation vs segmented causal study, scored on five needs. One gives impressions; the other gives segment effects with confidence intervals.] A buyer conversation opens a question. A controlled study compares the actions. ## Start with the consequence, not the interface An exploratory conversation can help a team hear a possible objection or improve a question. It cannot show whether one positioning action causes a better choice than another across a buying group. One current competitor describes its offer as a synthetic market research platform. That category can suit early exploration. The evidence threshold changes once the team is about to commit launch budget, a pricing tier, or a campaign narrative. ## What's the difference between a buyer conversation and a causal study? A conversation asks, “What might this buyer say?” A causal study asks, “Which action changes choice, for which segment, and with what uncertainty?” ## Buying groups create the failure mode A buying group often has three or five buyer types, rather than one. Treat that range as an example, not a universal benchmark. A VP of Engineering may care about implementation risk, a CFO about the commercial case, and a Head of Procurement about terms and switching constraints. Separate conversations can produce three plausible transcripts without a common comparison. The team still cannot tell whether the same message moved choice across the segments, whether an alternative worked better, or whether the apparent difference exceeds the study uncertainty. A free-form conversation can create false confidence here. The cost is not a weak transcript. It is a go-to-market commitment built around the wrong segment response. ## How does controlled choice evidence change the action? Subconscious uses discrete choice experiments and Mixed Logit to run the same positioning or messaging alternatives against multiple buyer segments in one controlled study. The output is a causal effect size with a confidence interval by segment, a comparison of actions rather than another turn-by-turn impression. A number without its limits attached is marketing. This one comes with both. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. It is a validation result, not a guarantee for a new market. A team can see which positioning action moved the target choice, where the effect differed by buyer segment, and how much uncertainty remains before committing budget. ## Population modeling belongs to a different branch Large-scale agent simulation can model broader market behavior. A published wealth and asset management example describes using multi-agent simulation to explore client decisions, market dynamics, and growth scenarios. The implementation is documented by the firm that published the work. That approach answers a population-modeling question. It is not automatically better for a bounded messaging decision, where the alternatives and buyer segments are already defined and the useful evidence is the causal comparison between those actions. ## Does the final launch decision still need human validation? Naming where a method stops is what lets a buyer check it before the decision gets made. Synthetic experiments are not a substitute for recruited real-human validation when the launch decision carries high stakes. Simulated audience reach and recruited participants are different evidence pools. Subconscious can carry the same causal question from the simulated experiment into real-human validation without changing the alternatives or outcome being tested. That continuity makes the human study a direct validation step. The misses belong on the public record next to the hits. It does not turn the work into an observed usability session, a clinical trial, or automatic proof of market performance. ## Use a three-part buying rule Choose the method by answering three questions: - Is the goal discovery or action selection? Use a conversation to form a hypothesis. Use a controlled study to compare actions. - Is there one buyer segment or a buying group? Multiple segments require a common experimental frame if their effects need to be compared. - What evidence would justify the commitment? As the cost of being wrong rises, require quantified uncertainty and recruited human validation. For a segment-spanning decision, examine the research record, see how the study process works, and review decision examples. A demo can translate a live messaging, positioning, or pricing choice into a testable causal question. --- # A Regime-Aware Risk Model: What to Validate Before You Trust the Probability URL: https://subconscious.ai/blog/methods-and-validation/regime-aware-risk-model-what-validate-before-trust-probability Published: 2026-08-12 A single risk model that averages across every market environment mispositions risk in both directions: it overstates volatility in calm months and understates it heading into a crisis. The fix: a regime-aware model that estimates a probability of being in a growth or stress regime instead of a single blended distribution. The buyer question underneath that choice is not whether the model runs. It is whether its regime-probability output can be trusted before it changes an allocation. ## Why does one distribution misprice risk? Standard risk models fit one mean vector and one covariance matrix to a long history of returns. Bull phases carry moderate drift and compressed volatility; bear phases bring negative drift, volatility spikes, and correlations that move toward one. Blending both into a single distribution understates the tail risk that shows up in a crisis and overstates the volatility priced into calm periods. A regime-switching model replaces that single distribution with one distribution per regime, plus a transition matrix that governs how the market moves between them. Instead of assigning a single label, the model outputs how likely the market is to be in each regime at a given point in time. This framework traces to Hamilton's original Markov-switching model for business-cycle analysis (Hamilton, "Regime-Switching Models," UC San Diego). ## What is a Bayesian Hidden Markov Model? A Bayesian Hidden Markov Model (HMM) with two latent states, growth and stress, treats returns as conditionally Normal given the active regime, with a first-order Markov chain governing transitions between regimes. Fitting it means sampling the joint posterior over the transition matrix, regime means, and regime covariances given observed returns. The discrete regime sequence itself is analytically marginalized via the forward algorithm, because the sampler used here, the No-U-Turn Sampler, requires a continuous, differentiable parameter space and cannot sample discrete states directly (PyMC NUTS documentation). After fitting, a forward-filter backward-sampler pass recovers the latent regime sequence as smoothed posterior probabilities, the model's best retrospective read on which regime was active. ## Validating the model before trusting its output Before any regime-probability output can inform a real allocation decision, it has to be checked against a known answer. The source demonstration does exactly that on synthetic data with known ground-truth regimes: three synthetic equity series were generated from a known two-regime process, and the fitted model recovered the true regime at 94% of months (measured with hindsight against the known generating regimes), a model-validation check rather than a live-market result. All true regime parameters, meaning the generating means, volatilities, and regime persistence, fell inside their posterior credible intervals, and the model's uncertainty concentrated exactly at the regime transition points, where a well-calibrated model should hesitate. This is the discipline any model output needs before it drives a decision: check it against a known outcome, and report the uncertainty rather than a single point estimate. Subconscious applies that same standard to its own causal outputs, testing them against real-human validation before a team relies on them for a market decision. Explore how that validation loop works. ## What does the demonstration not show? The synthetic-data result shows the modeling machinery recovers a known answer, not real-market performance. Several assumptions built into this version of the model matter to anyone considering it for live risk management: - Gaussian emissions. Returns within a regime are modeled as Normal, which captures regime-level mean and volatility shifts but not the fat tails real returns can show even conditional on regime. - A time-homogeneous transition matrix. The probability of switching regimes does not depend on observable signals such as the VIX or the yield curve slope; it is treated as constant over time. - Two regimes, fixed in advance. The choice of two regimes is a modeling decision, not something inferred from the data. - An in-sample portfolio illustration. The regime-aware allocation example in the source demonstration was fit and evaluated on the same sample, ignoring transaction costs and re-estimation risk. It is a planning illustration of how a regime probability could inform an allocation rule, not an out-of-sample backtest. Real markets are also non-stationary in ways synthetic data by construction is not: regimes, correlations, and the rules that govern transitions between them can all shift in the underlying process itself. [figure: Four open assumptions: Normal returns within a regime, a transition matrix held constant instead of reacting to signals, two regimes fixed by choice not data, and an allocation example tested on its own fitting sample.] ## The next step is proving it, not assuming it A regime-probability output is only as useful as the check behind it. Before letting one shape a live allocation, a risk team needs to see it validated against a known outcome, its uncertainty reported, not collapsed into a point estimate, and its assumptions, including the time-homogeneous transitions and Gaussian tails above, stated rather than implied. See how Subconscious structures that kind of validation for a causal claim. Teams evaluating whether this is the right layer for their own risk stack can start with a scoped conversation. [figure: Five steps: fit a Hidden Markov Model with growth/stress regimes; recover probabilities; test on synthetic data with known regimes (94% match); check uncertainty peaks at transitions; confirm assumptions before live use.] --- # How to Test a Meta Title and Description Before You Publish URL: https://subconscious.ai/blog/use-cases/how-test-meta-title-description-before-publish Published: 2026-08-12 A search-results title and description is copy a marketing team almost never tests before it ships. The decision is which variant to publish for a given page, chosen by comparing candidates against a defined searcher audience before the page goes live, rather than by publishing a change and watching organic click-through rate move afterward. Changing a title or description that is already indexed triggers a reindex and can reshuffle the page's ranking, which distorts the click signal for a stretch of time long enough that the team cannot cleanly attribute the shift to the copy change alone. A wrong choice underperforms, and re-measuring it means repeating the same disruption. [figure: Five-step path: find a weak-click page, define the searcher, draft distinct variants, score each on intent, click pull, trust, and disappointment risk, then ship the one that holds up on all four.] ## Why Is a Live Search-Results Test the Wrong Instrument? Search engines do not offer a native way to run two title-and-description variants against the same query at the same time. Any change to the live tag rewrites what searchers see immediately, and the resulting rank movement contaminates the very click-through signal the team is trying to read. The team ends up comparing a post-change period to a pre-change baseline that the change itself has made noisy. The search engine's own behavior compounds the problem: in one 2025 sample, Google changed the displayed title tag on 76% of results, rewriting it from the page owner's own source tag independent of any test the team ran (Search Engine Land). A live test result can reflect that rewrite behavior as much as it reflects the copy being tested. ## What a pre-publish comparison should score A title and description are read in context, not in isolation: alongside competing results, any AI-generated summary above the results, and other rich elements on the page. Four things matter most: - Intent match. Does the copy match what the searcher is actually looking for, given how the query is phrased and what else appears on the results page? - Click pull. How strongly does the variant pull the click relative to the other results next to it, and does it say something the neighbors don't? - Trust signal. Does the source read as credible enough to click? Brand recognition, freshness cues, and tone all play a part. - Risk of disappointment. If the searcher clicks, does the page deliver on what the copy promised? Copy that overpromises earns the click and loses the visitor, and that pattern of clicking through and immediately leaving works against the page's ranking over time. A variant that wins on click pull but carries a high risk of disappointment is a trap: it can win the comparison and still be the wrong thing to ship. ## A pre-publish workflow - Identify the page and its current position. Use whatever search-console data the team already has to find pages with strong impressions but weak click-through relative to their ranking position: the pages where copy, not ranking, is the likely bottleneck. - Define the searcher. Who actually types this query, and in what frame of mind: early research, active comparison, or already decided? A query typed by someone comparing vendors reads differently than the same words typed by someone trying to understand a category. - Draft several distinct variants, not minor rewordings of the same idea. Distinct angles (closest match to the query, the outcome the searcher wants, the specific thing this page offers that competitors don't, and a curiosity-driven framing) produce more useful spread than five versions of the same sentence. - Compare the variants against the defined searcher, in the same competitive context they will actually appear in. Score each on intent match, click pull, trust, and risk of disappointment, with a stated reason attached to each score. - Ship the variant that holds up on all four axes, not only the one with the single highest click-pull score. When two variants are close, the one with lower disappointment risk is the safer ship. - Read the actual click-through change once the page has had time to resettle after the reindex, comparing it to the pre-change baseline for the same ranking position. If the comparison's pick underperforms in the wild, that is a signal to recalibrate the method, not a reason to stop testing before shipping. ## Comparing the three ways teams choose meta copy ## What Does This Look Like With Subconscious? This is a message-testing decision, the same category as testing which offer, positioning, or launch line to use before it ships. Subconscious runs a controlled comparison of title-and-description variants against a defined audience segment and reports which variant is more likely to change click behavior, along with the uncertainty in that estimate, not a search-engine monitoring or rank-tracking tool. Subconscious does not crawl or observe live search-results pages, does not control or predict how a search engine indexes or rewrites a title tag, and does not measure actual post-publish click-through rate. It estimates relative preference among a defined audience before the page ships. What happens after publication still depends on the search engine's own ranking and rewrite behavior, the searcher's real intent at the moment of the query, and other factors a pre-publish comparison cannot see. When a page is high-stakes enough to warrant it, a team can also move the top variants from a simulated comparison to a study with recruited, real human participants without changing the underlying question being asked: which variant earns the click. See how this fits into a broader testing workflow or look at comparable studies before deciding whether to run one on your own copy. Pre-publish comparison does not replace watching the actual click-through number once a page has settled after a change. It replaces guessing which variant to ship in the first place, and it replaces treating the live results page as the only place to find out. --- # Should You Send Your First Cold Outbound Email As-Is, or Test It First? URL: https://subconscious.ai/blog/use-cases/should-send-first-cold-outbound-email-test-first Published: 2026-08-12 A hand-built prospect list gives a founder one real shot per name. Once a prospect deletes a bad first cold email, a second one from the same sender gets deleted too. What matters before the send button is pressed is not whether the draft reads well to the person who wrote it, but whether the subject line, opening line, and ask have been checked against how the actual buyer reads a cold email, because a misjudged draft does not just cost this campaign, it costs the next one on every name that already saw the first attempt. [figure: A five-step checklist for a pre-send cold email comparison: subject line, opening line, personalization, size of the ask, and predicted reply, each checked against buyer reading behavior before the email is sent.] ## The three jobs one email is asked to do A first cold email usually tries to do three things at once: describe the product, ask for a meeting, and explain why this particular prospect should care. Those jobs compete. A product description long enough to be clear crowds out the ask. An ask direct enough to get a reply can undercut the personal case for writing at all. A first-time outbound sender rarely separates the three, and the result is a single long email that handles none of the jobs well. Founders also spend their effort in the wrong place: a common pattern is roughly ten percent of drafting time going into the subject line and ninety percent into the body, even though the subject line decides whether the body gets read at all. ## Why founder feedback misses the point The usual first check on a draft is showing it to other founders. Other founders can judge whether the product description is clear, the ask is well-framed, and the story is compelling. None of that answers the question that actually decides whether the email works: will a buyer who has never heard of the company read past the first line? A senior buyer receiving on the order of twenty cold emails a week typically decides within seconds whether to keep reading, and founders showing each other drafts cannot reproduce that read, because they already know the context the buyer does not have. The only way to check before the list is spent is to test the draft against the buyer's actual reading behavior. ## What does a pre-send comparison check? A controlled comparison puts each version of the subject line, opening line, and ask in front of the same target buyer segment and reads which change moves whether the buyer opens, keeps reading, or replies. That produces a causal answer about which element is doing the work, in the order a buyer actually encounters the email. Independent research on cold-email performance supports checking the subject line first: personalized subject lines outperform generic ones on open rate (Snov.io, 2026), consistent with treating the subject line as the point where most first campaigns are won or lost rather than an afterthought. ## Patterns that repeat in untested drafts A few failure patterns show up often enough to check for directly: - Leading with the company, not the prospect. The email opens with company background and product description before reaching the prospect's problem. A single number that leaves out its limits functions as marketing. As a planning example, one prior case that moved the prospect's problem to the first line saw open-to-reply rates increase by a factor of three or more; that describes a single instance, not a guaranteed result. - A soft-sounding ask that is not soft. "Would love to share a quick demo" reads to a buyer as a request for real meeting time, not the low-commitment favor it is meant to sound like. A genuinely soft ask is a yes-or-no question about whether a problem exists right now, closer to a two-line reply or a fifteen-minute conversation than a thirty-minute call. - No proof at all. An email with no customer name, no metric, and no concrete detail reads as one more unverified pitch. Naming the actual stage honestly, including an early one, reads better than a vague claim of traction the buyer cannot check. - The wrong channel for the message. Some prospects on a hand-built list of, say, 200 to 800 names read more like a warm introduction or a direct message than a cold email. Sending everyone the same email regardless of channel fit lowers the response rate for the list as a whole. ## Is the list being tested too? The same comparison that evaluates the email also evaluates the list. If a draft reads well for one part of the target segment and poorly for another, that gap says as much about who is on the list as it does about the message. This matters most for a first campaign, where the list and the message are usually built by the same person, in the same sitting, on the same assumptions. Testing the message against the segment separates those two variables, something a single reply rate from the live send cannot do. ## How is the comparison run as a causal experiment? Subconscious can run this comparison as a randomized experiment: the same target buyer segment sees each version of the subject line, opening line, and ask, and the result isolates which specific change moves open and reply behavior, rather than relying on founder or peer opinion about which draft feels right. Where the decision benefits from checking a simulated read against real people, Subconscious can also test or validate the same study with real human participants without changing the underlying question. See how this fits into a broader research workflow. ## What this does not cover Naming what a method cannot do is what lets a buyer check the claim against reality. Testing a draft before it sends does not replace deliverability engineering, list-building judgment, or a founder's read of an individual prospect relationship. It tests a message and targeting hypothesis before the list is spent. It does not guarantee a reply rate, and it does not substitute for running the actual campaign and reading what comes back. The email is going to send either way. The question worth answering first is whether the draft in front of you is the one that deserves the list's one shot, or whether a specific line in it is quietly costing the reply. Book time to run the comparison before the next send, or see the underlying research approach behind it. --- # AI Pricing Research Tools: How to Choose One for a Price Decision URL: https://subconscious.ai/blog/use-cases/ai-pricing-research-tools-how-choose-one-price-decision Published: 2026-08-12 A pricing study once meant a six-figure line item and a full quarter: staff a panel, commission a van Westendorp or Gabor-Granger exercise, then sit on the answer for three weeks before a price ladder reaches a decision-maker. In 2026, synthetic-audience pricing research compresses that loop to hours: build an audience calibrated to a real ideal customer profile, test price points and packaging, and get a directional read the same day. The decision this article answers: which price point, tier structure, or discount framing should ship before a launch or renewal cycle locks it in, and which method actually tests that question rather than just describing it. [figure: A branching diagram: one pricing question splits into four method branches (price sensitivity, elasticity, packaging, discount framing), all converging on validating against real behavior before shipping.] ## Why the Decision Matters A mispriced launch leaves revenue on the table or suppresses demand outright. A pricing study built on the wrong method wastes the research cycle along with it. A van Westendorp ladder answers a different question than a packaging test, and neither answers what happens when a price changes inside a live market with competitors reacting. ## What Causes the Outcome The tools in this space cover the same core question set that traditional pricing research answered by hand: - Willingness to pay (van Westendorp). A price-sensitivity ladder built from four questions. - Gabor-Granger. Direct price elasticity, walking a respondent through a price ladder. - Conjoint and choice-based conjoint. Tradeoff modeling across price and product attributes; accuracy gets shakier once the category has no close precedent. - Packaging tests. What a tier is called, how features get split across it, and where the anchor price sits. - Discount and promotion testing. How big the discount is, how it's framed, and how much urgency the expiry date creates. - Competitive pricing benchmarking. How an audience reacts to a price against a competitive set, in the audience's own words. Where this approach breaks down: novel categories with no analog in the underlying training data, very small-segment elasticity (B2B categories with under 1,000 global buyers), and ultra-luxury categories where stated preference diverges from actual purchase behavior. ## Evidence The methods above elicit a stated preference from a simulated respondent rather than observing a real purchase decision, so accuracy depends on how the elicitation is designed and calibrated. Directly asking a model for a numeric price-sensitivity answer tends to produce unrealistic distributions; better methods elicit a fuller response and calibrate against a human baseline. Aggregate patterns (how a segment as a whole responds to a price) are easier to recover than individual-level fidelity. Known failure modes include variance collapse (every respondent converging on the same answer), demographic flattening, and over-rationality relative to how people actually behave. Vendors in this category report accuracy in the range of 80 to 95 percent against historical research benchmarks on directional pricing tasks. That figure describes the category generally, not a specific tested claim; treat it as a planning reference rather than a guarantee for any single study. ## Nine AI Pricing Research Tools Compared ### 1. Aaru: Behavioral Price-Dynamics Modeling A multi-agent simulation approach, validated by EY at around 90 percent correlation (EY, "How AI simulation accelerates growth in wealth and asset management"). Models how a price change cascades through an audience, including referral effects and competitive response. Best for: modeling price like a system, not one fixed number. Pricing: enterprise, high ACV. ### 2. Evidenza: B2B Pricing Research Started by people who ran the LinkedIn B2B Institute. Its synthetic respondents are built to hold up under procurement scrutiny: CFOs and procurement roles, not a generic consumer panel. Best for: enterprise procurement cycles. Pricing: enterprise, on request. ### 3. Conjointly: Specialist Conjoint Research A conjoint-only platform whose methodology predates AI and is now adding synthetic-respondent capability. Best for: a conjoint specialist with a track record. Pricing: per-study and subscription. ### 4. Synthetic Users: Pricing UX Research Simulated respondents that react in their own words to pricing copy, plan names, and how a package is laid out. Best for: refining pricing-page language or tier framing. Pricing: self-service subscription. ### 5. OpinioAI: Budget Pricing Research Runs AI-moderated synthetic focus groups to gauge price reactions. Best for: early-stage teams or a solo consultant. Pricing: from $99 per month. ### 6. Electric Twin: Large Consumer Pricing Studies Has built synthetic crowds for major media brands, backed by $14M in funding. Best for: a consumer brand scaling pricing studies. Pricing: enterprise, on request. ### 7. Lakmoos: Regulated-Industry Pricing Research German neuro-symbolic AI with an audit trail, relevant when a price change needs to be defensible, such as in regulated financial products, insurance, or healthcare. Best for: pricing under real compliance pressure. Pricing: enterprise, on request. ### 8. Qualtrics XM: Enterprise Standard With Pricing Modules Qualtrics now ships dedicated pricing research modules, including van Westendorp and conjoint. It runs slower and costs more than AI-native alternatives, but the platform is already sitting inside most large enterprises' stacks. Best for: enterprises already running Qualtrics. Pricing: enterprise, high ACV. ### 9. Sanctum: Pre-Launch Pricing Validation Runs pricing options past simulated users before a public launch. Best for: gating a pricing decision before launch. Pricing: self-service. ## A 15-Minute Workflow for Testing a Price - Build the audience. Anchor 5 to 10 personas, spanning the segments sold into, to a real ideal-customer-profile. - Run van Westendorp. Put the four standard price-sensitivity questions to each persona, then aggregate the answers. - Run Gabor-Granger. Take each persona through a price ladder, step by step, and aggregate the result. - Test packaging. Show 2 or 3 packaging options and ask which would be bought, and why. - Cross-validate qualitatively. Take the resulting recommended price to a 1:1 conversation with a key persona, present the price, and capture the objections. In under 20 minutes, that produces a defensible price recommendation, a packaging recommendation, and the likely objections. The traditional version takes 3 to 4 weeks and runs $30k to $80k. ## How to Pick a Method for the Decision - Behavioral dynamics on a price change: a system-modeling tool like Aaru. - B2B procurement realism: a specialist like Evidenza. - Conjoint specialist with an established methodology: Conjointly. - Lowest entry cost: OpinioAI. - Regulated pricing: Lakmoos. None of these tools tests what happens when the price actually changes in front of a defined audience under controlled conditions. They describe stated preference. A causal experiment fills that gap. ## Where Does Subconscious Fit Into Pricing Research? Subconscious runs a controlled, randomized experiment comparing defined price and packaging alternatives across precisely defined buyer segments, so the price decision is backed by a measured behavioral comparison rather than a single ranked number from a stated-preference survey. See current research methodology for how the experiments are structured, and case evidence for how the results have held up. Subconscious can also validate a study with real human participants without changing the underlying causal question. That step matters most when the price decision is large enough, or novel enough, that a stated-preference read alone is not sufficient evidence to act on. ## What Are the Limitations of AI Pricing Research? A pre-launch price experiment does not replace a live A/B pricing test, sales and renewal negotiation evidence, or judgment calls in novel categories and very small B2B segments where no comparable behavioral data exists. Treat a synthetic-audience pricing read as evidence that narrows the decision, not the final word. Book a working session to scope which stage of that process fits the current decision. --- # Bechtel Carbon Tax Policy Study: Simulated vs. Published Results URL: https://subconscious.ai/blog/case-studies/bechtel-carbon-tax-policy-study-simulated-vs-published-results Published: 2026-08-12 Policy and public-affairs teams weighing a costly, slow fielded study on a carbon tax package need to know whether a simulated discrete-choice experiment gets close enough to trust for an early pass. ## The comparison Bechtel, Scheve, and van Lieshout fielded a discrete choice experiment across four countries, the U.S., U.K., Germany, and France, asking what international climate agreement design features people there favor, among them participation breadth, cost distribution, and enforcement (Improving public support for climate action through multilateralism, Nature Communications, 2022). Subconscious ran the same discrete-choice design on a simulated U.S. population and compared the resulting preference ranking against the paper's pooled U.S. results. The two rankings correlate at rs = .6711. [figure: Two ranked lists of carbon tax package features, fielded U.S. survey and simulated study, linked by a moderate rs = .67 correlation showing directional agreement but feature-level divergence.] ## What does a correlation of .67 mean for this decision? A number without its limits reads like marketing. A Spearman correlation of .67 is a moderate, positive relationship: the simulated and fielded studies tend to agree on which carbon tax features respondents prefer more or less, but not closely enough to treat the simulated ranking as a stand-in for the fielded one on every feature. Treat it as one data point on whether the method is directionally reliable for this kind of policy-preference question, not proof that a simulated ranking will match a real vote or survey. ## What are the limitations of this comparison? The misses appear on this page next to the hits. This comparison covers one replication, on one country's pooled results, against one published study. It does not extend to the U.K., Germany, or France samples in the original paper, and one correlation coefficient does not establish a fixed accuracy rate for policy-preference studies generally. A moderate correlation also means feature-level orderings can still diverge even where the overall ranking pattern agrees. [figure: Two-column comparison. Left, tested: U.S. pooled results. Right, not tested: U.K. sample, Germany sample, France sample, and a general accuracy rate for policy-preference studies.] ## What's the next step? For a policy or public-affairs question where getting the tradeoff wrong is expensive, use a simulated run like this one to sharpen the design and flag which features carry the most disagreement risk before committing to a fielded study. See how the underlying method works on /how-we-work, or review more replication comparisons on /case-studies. --- # What a Validation Gate Is, and Why It Should Decide What an AI Agent Ships URL: https://subconscious.ai/blog/methods-and-validation/what-validation-gate-why-should-decide-what-ai-agent-ships Published: 2026-08-13 An AI agent's output looks finished the moment it stops generating text. Whether it matches something real is a separate question, and most AI-agent systems never force an answer before the output ships. A research team behind the open-source tool alchemize built a system that forces the answer, and the architecture is a useful model for any buyer deciding whether to trust an AI agent's output for a business decision. ## The problem: code translation an AI agent could not simply generate once and ship alchemize converts statistical models written in PyMC, a Python library for Bayesian modeling, into Rust code that runs faster. The naive approach, asking an AI agent to write the Rust translation once, fails for the same reason any one-shot AI generation fails: nothing checks whether the output is correct before it reaches production. The team's fix was architectural, not a smarter prompt. They put the agent inside a loop with four tools: - generate or update the Rust implementation - compile it and surface the error messages - compare its numerical output against the original PyMC model's exact reference values - read code or error logs for diagnosis The agent cannot stop until the compiled Rust output matches the source model's numbers. If a gradient is off, the agent reads the mismatch, finds the missing term, and rewrites: a hierarchical model with a zero-sum constraint took three to four iterations before its gradients matched. That validation step, not the code generation step, is what makes the system behave like a compiler instead of a generator: it produces output that matches the reference model's outputs at the tested points, and the agent only finishes once it clears that bar. ## How much iteration does a validation gate actually take? The team logged convergence across five model types of increasing complexity. A two-parameter Normal model cleared the gate in 4 tool calls on 40K tokens, passing the first time it ran. A three-parameter linear regression model matched that: 4 tool calls, 54K tokens, no retries needed. A twelve-parameter hierarchical model took 8 tool calls and 153K tokens, and needed one retry before its gradients matched. A three-parameter GP model built on an ExpQuad kernel was the most expensive of the five, spending 11 tool calls and 467K tokens across three retries. A 142-parameter ZeroSumNormal model, despite its much larger parameter count, converged in 9 tool calls and 484K tokens after two retries. Simple models converge on the first pass. Structural unusualness, not parameter count, seems to drive extra iterations here, though five models is too few to establish a general relationship; the gate is clearly doing real work, not rubber-stamping. The team separately benchmarked the resulting Rust output's runtime against an already-optimized Numba backend across those same model types, with per-model speedups ranging roughly from 3x to 7x depending on model complexity. One maintainer noted part of that gap reflects headroom in the traditional backend's compilation pipeline, not a hard ceiling on non-AI approaches, worth reading as a model-specific figure, not a fixed multiplier. ## Where the validation gate breaks down The team was direct about the limits, and they matter for anyone evaluating a similar architecture: - Context limits on large models. A portfolio-optimization model with thousands of parameters exceeded the agent's context window; the full computation graph has to fit for the loop to work at all. - No formal convergence guarantee. Every tested model converged within a handful of iterations, but that is an empirical observation from testing, not a proof that every model will. - Spot-check validation has blind spots. The gate compares output at a single reference point. Subtler numerical issues, such as cancellation, underflow, or precision loss over long runs, can survive a spot check. - Toolchain dependency. The approach requires a working Rust build pipeline in whatever environment runs it. None of this makes the architecture unsound; it marks where the guarantee stops, information a buyer needs before extending trust to the pattern. ## Why does this generalize past one transpiler? The team's bigger claim isn't about Rust or PyMC. It's that the pattern (thin scaffolding, an agent that reasons about each case individually, and a validation step the agent cannot bypass) applies to any tool whose core job is transformation. The same team used the loop to convert the large majority of a public benchmark set of Stan models into PyMC and Rust, with both gradient and log-probability values validated at each conversion, and also applied it to translation between two other modeling frameworks. That is the architectural takeaway for anyone evaluating an AI-agent system before trusting its output for a business decision: the question is never whether the agent looks confident. It's what reference value the output was checked against, and whether the agent was mechanically blocked from shipping until it matched. ## The same discipline, applied to a decision instead of a code path The transpiler validates a translated program against the exact number the original model produced. Subconscious checks a different kind of output, a causal experiment run on a simulated population, against a different kind of reference: real human behavior, within the confidence interval a finite human sample allows, not as an unambiguous match. Subconscious can test or validate studies with real human participants, moving from simulation to real-human validation without changing the causal question being asked, though matching observed choice shares does not by itself validate the causal quantities reported, such as marginal effects, WTP, or substitution patterns, since distinct utility specifications can fit the same aggregate shares while implying different effects. Subconscious does not build compilers or transpile code; the comparison is architectural, not a claim about what the product does. Not every study is checked this way; the practice fits when the decision depends on it, the same way the transpiler's gate exists because a silently wrong translation is unacceptable. If you're deciding whether to trust an AI-generated study result the way you'd trust a compiled binary, see how a study moves through that process before it reaches a decision. [figure: A cycle of four stages: generate code, compile it, compare its output to reference values, diagnose any mismatch. Diagnosis loops back to generation. Only a match exits the loop, into a final "ships" box.] [figure: Four limits: models too large for context fail outright; convergence is observed, not proven; the gate checks one reference point and can miss precision loss elsewhere; the approach needs a working build pipeline.] ## Sources - pymc-labs/alchemize: LLM-based, self-correcting transpiler, GitHub. - Agent SDK overview, Anthropic. --- # AI Audience Simulation Platforms: What the 2026 Buying Wave Actually Tests URL: https://subconscious.ai/blog/use-cases/ai-audience-simulation-platforms-what-2026-buying-wave-actually-tests Published: 2026-08-13 [figure: Two-column table: vendor self-reported accuracy vs. causal experiment with human-baseline validation, compared across what each measures, who can check it, and what it proves.] ## The number on the homepage is not the test that matters By 2026, AI audience simulation split into a real market: synthetic panels, simulated target groups, AI focus groups, and behavioral audience simulation each solve a different buyer problem, and no platform wins every row. Several vendors now publish simulated-agreement figures ranging from roughly 80 percent up to the mid-90s against historical research benchmarks, not a Subconscious result and not an independently verified measurement of the study you are about to run. The buying decision is narrower than "which platform has the highest accuracy number." It is: does this vendor prove its simulated output against a real human baseline before you commit budget to whatever action the simulation favored? A Tech Xplore report on AI survey respondents found that simulated opinions can diverge from public opinion in ways a self-reported accuracy score does not surface. ## What Does a Self-Reported Accuracy Figure Actually Tell You? A vendor benchmark answers one question: how often did this simulation's output match a past study the vendor had answers for. It does not answer whether the simulation will hold on the next question you ask it, because that requires holding out real human data you have not seen and checking the simulation against it. Two vendor-published figures illustrate the gap between a marketing number and a validated one. Some platforms cite a correlation of roughly 90 percent to real research from a single partnership case. Others advertise a headline agreement rate stretching from about 80 percent to the mid-90s without stating the benchmark set, sample size, or whether the comparison used a holdout. Neither is proof that a specific study you plan to run will replicate. ## What Is the Real Cost of Picking the Wrong Platform? The cost of picking wrong is not "the simulation was inaccurate." It is committing a launch, price, or message decision to whatever a simulated panel favored, then discovering real customers respond differently once the campaign is live. A simulated agreement score does not tell you which specific choice caused a specific outcome. Subconscious is the causal AI company. Randomized experiments on a simulation of your market, validated against real human behavior, tell you why people choose and which action drives the outcome. A number about scale is marketing until its limits are published alongside it. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That audience graph describes modeling scope, not a recruitable panel of 800 million people available to survey directly. Subconscious can also test or validate studies with real human participants, moving from a simulated experiment to real-human validation without changing the causal question. ## A comparison worth making before you buy ## Simulated reach is not recruited reach This line is drawn here so a buyer can hold any vendor's claim, including ours, to the same check. Some vendors describe large-scale simulation as if it substitutes for a recruited panel. A simulation covering a large modeled population estimates how that population would respond; it is not evidence that a specific number of real people were asked and responded that way. ## Where a simulation still needs a human check Naming where a method stops is what lets a buyer check it before committing budget to it. Causal experiments on a simulated market do not replace real focus groups, in-person usability sessions, or lived-experience research where the value is qualitative depth rather than a directional answer. A simulation is not a clinical trial and is not automatic proof of market performance once a product ships. It narrows which action to test next and confirms that a causal answer holds before a team commits budget to it. ## Next step Before selecting a vendor on the strength of a self-reported accuracy figure, ask what the number was benchmarked against, whether an independent holdout of real human responses confirmed it, and whether the vendor can test the same causal question with real participants once the simulation points to an action. Review completed comparisons on the leaderboard, read the underlying research, or book time to see how a study moves from simulated experiment to human-validated answer. --- # AI Persona Tool or Causal Decision Platform: Which One Does Your Buying Decision Need? URL: https://subconscious.ai/blog/comparisons/ai-persona-tool-causal-decision-platform-which-one-buying-decision-need Published: 2026-08-13 A team gets pitched an "AI persona" tool and has to decide fast whether it fits the job in front of them. "AI persona" describes two very different products doing two very different jobs, and confusing them costs real budget. [figure: Split diagram: left, a single persona reviews one document with no comparison group; right, a causal platform compares two or more actions on a simulated market and reports a confidence estimate.] One end of that spectrum is a narrow, open-source documentation-review tool such as Impersonaid, which runs a docs page past simulated personas to surface confusing passages. That is a useful, focused job, and it is not the same job as testing a pricing, positioning, or launch decision. ## Why this decision matters One job is a documentation and content clarity checker: run a page of text past a simulated reader, see where it confuses them, fix the copy. The other job is a causal decision platform: run a controlled experiment on a simulated market to estimate which product, pricing, or messaging action actually changes buyer behavior. Using the first to justify the second is the expensive mistake. A single-pass LLM opinion on whether a paragraph reads clearly is not a controlled comparison, carries no causal estimate, and reports no uncertainty. Committing a pricing, positioning, or launch decision to that kind of output means shipping on a plausible-sounding guess, the same say-do gap problem every simulated-opinion tool inherits. The reverse mistake also happens: buying a full market-simulation platform to catch confusing sentences in a docs page is over-scoped spend for a job a lightweight, linting-style checker already does well. ## What causes the outcome? The difference is architectural, not cosmetic. A documentation-clarity checker takes one piece of text, applies one simulated persona, and returns a reaction: confused, clear, missing a step. It is a single-shot evaluation with no comparison group and no measurement of what a real audience would do differently. A causal decision platform is built around comparison. It defines two or more actions (a price, a message, a product concept), exposes a simulated population to each one under controlled conditions, and estimates which action is more likely to move a specific behavioral outcome. That structure is what makes an estimate causal rather than descriptive: something is held constant, something is varied, and the difference in outcome is attributed to the variation. ## What evidence supports this? Subconscious runs randomized experiments on a simulation of a buyer's market, validated against real human behavior, and can extend into real-human validation studies without changing the underlying causal question. Neither job is inherently wrong. ## Options and comparison ## Recommended decision process - Name the actual decision. If it is "will readers understand this doc," that is a content-clarity question. If it is "will this price, message, or launch choice change what buyers do," that is a causal decision question. - Check whether the job requires a comparison. A clarity check evaluates one artifact. A decision that commits budget or roadmap capacity needs at least two alternatives compared under controlled conditions. - Match the tool's evidence model to the stakes. A single-pass opinion is fine for catching a confusing sentence. It is not sufficient grounding for a pricing, positioning, or launch commitment. - Ask what happens after the answer comes back. A clarity checker's output is a fix list for the same document. A causal platform's output should change which action a team takes next. ## Where does Subconscious fit? Subconscious is built for the second job. It is not a documentation-testing or content-linting tool and does not claim to be one. A team can reasonably run both kinds of tools in the same stack without conflict. A documentation-clarity checker can sit in a CI pipeline, testing docs before they publish. A causal decision platform sits earlier in the stack, testing the market-facing actions (pricing, positioning, launch messaging) before they ship. ## Limitations and failure conditions A platform's published scope is what a buyer checks against their own job before they buy. Subconscious is not built for documentation UX testing, content-clarity linting, or CI-integrated docs QA. It does not ship a packaged product-catalog simulator, an automatic price or promotion optimizer, or standard substitution and cannibalization output as universal deliverables. Confidence-interval and segment-heterogeneity outputs are specific to how a given study is designed, not a guarantee attached to every result. If the actual job is checking whether documentation reads clearly to a technical audience, a narrow, engineering-facing clarity tool remains the better-scoped choice. ## Buyer questions Is a single AI-generated opinion the same as a causal estimate? No. An opinion describes what one simulated persona thinks about one piece of content. A causal estimate compares at least two actions under controlled conditions and attributes the difference in outcome to what changed. Can the same team use both a documentation-clarity tool and a causal decision platform? Yes. They sit at different points in the workflow: one checks whether published content is clear, the other checks whether a market-facing decision is likely to change buyer behavior before it ships. What should change my mind about which one to use? The question being asked. "Is this text confusing" needs a clarity checker. "Will this action change what buyers do" needs a controlled experiment with a validated causal estimate. Explore how Subconscious designs these experiments, see the research behind the causal method, or book a walkthrough of a decision-specific study. Learn more about the company. --- # Research Agency, Self-Serve AI Panel, or Causal Experiment: Choosing by Decision, Not by Vendor URL: https://subconscious.ai/blog/comparisons/research-agency-self-serve-ai-panel-causal-experiment-choosing-decision-not Published: 2026-08-13 A team deciding how to test a pricing, product, or messaging question usually reaches for one of two defaults: hire a managed research agency, or run a fast self-serve exploratory tool. Neither answers the question a lot of buyers actually have, which is not "how fast" or "how polished," but which action will change what customers do. That gap is where a third option, causal experimentation, fits. ## The two defaults, and what each is built for A managed research agency scopes a project, designs and runs the study, and delivers a report. RAD Research illustrates the model: brand and product teams bring it a question, and it designs custom qualitative and quantitative studies to answer them. Buyers who choose this route usually have an existing research workflow the agency plugs into. The strength is depth and defensibility for a single, high-stakes study. Publishing where a method runs out is what lets a buyer check it against their own timeline. The limitation is that the process runs on a project timeline and a project budget, so it does not scale to many small, fast questions. A self-serve exploratory tool lets an operator brief a panel of AI-simulated respondents directly and get a directional read back quickly, without a project manager in the loop. This model trades rigor for iteration speed: a follow-up question against the same panel costs little, so a team can explore many more angles than a single commissioned study allows. The tradeoff is that a directional read is calibrated against historical patterns, not validated against a new set of real respondents for the specific question being asked. ## What do both approaches miss? The buyer's real decision is rarely "what do customers think of this." It is "if we change the price, the message, or the feature, which version moves the behavior we care about, and by how much." An agency study can measure stated preference well. A self-serve panel can generate a fast directional signal. Naming what a method doesn't measure is what makes its measurements trustworthy. Neither is designed as a controlled comparison between specific alternatives with an estimate of the effect size and its uncertainty. This is the same limit the broader research on AI-simulated respondents has flagged: models can often recover plausible aggregate patterns, but elicitation design, calibration against a real baseline, and the risk of mistaking correlation for causation remain open problems in the field, not solved ones. ## Where does causal testing fit? Subconscious is a causal behavioral platform. It runs controlled discrete-choice experiments against a defined population and returns causal effects with confidence intervals where the study design supports them, rather than a single predicted answer or a report built from stated preference. That makes it a third lane, not a faster agency and not a more accurate self-serve panel. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, which is a reach claim, not a recruitable panel. Separately, when a team needs to check a simulated result against real respondents, Subconscious can validate the same study with real human participants, without changing the underlying causal question. A team can start with a simulated experiment and move to real-human validation on the same design when the decision warrants it. ## What does causal testing not replace? A platform that lists only what it covers is marketing. This is what it does not cover: Subconscious does not offer full-service agency deliverables. Dedicated project management, a delivered agency report, and bespoke qualitative moderation are not part of the platform. Nor does it publish a directional accuracy percentage the way a self-serve exploratory product might, because the platform is built around estimating causal effects for a specific comparison rather than scoring how closely a persona matches a historical benchmark. A team that wants a fully managed engagement with a written deliverable, or a lightweight always-on chat panel for open-ended exploration, is better served by one of the two defaults above. ## Matching the tool to the decision Historically, source material for this kind of comparison framed the agency lane as reserved for decisions defending roughly $10 million or more in business impact. Treat that threshold as a planning example rather than a fixed rule: the real filter is whether the decision needs a defensible, delivered report (agency), fast open-ended exploration (self-serve), or an estimate of which action causes the outcome (causal experiment). ## Limitations and failure conditions These failure conditions sit on the record next to the results, so a buyer can check the method before relying on it. A causal experiment is only as good as the alternatives and the population it is built against. It does not replace real-world confirmation, discovery-stage qualitative work, or regulatory-grade fieldwork where those are required. A controlled comparison answers the specific question it was designed to test, not a general-purpose substitute for the other two lanes. ## Next step Teams evaluating which lane fits a specific decision can review how the platform runs a study, see worked examples, read the underlying research, or book time to scope a comparison against a real pending decision. [figure: Four columns compare what each approach delivers: agency report, self-serve directional read, causal effect estimate with confidence intervals, and that same experiment validated against real respondents.] --- # Does a Simulated Population Reproduce a Published Refugee-Preference Study? The Adida Replication URL: https://subconscious.ai/blog/case-studies/simulated-population-reproduce-published-refugee-preference-study-adida Published: 2026-08-13 A policy researcher, DEI research lead, or research methodologist weighing a simulated discrete-choice experiment for a sensitive, non-commercial topic needs more than a commercial pricing test to trust it. The check that matters: whether the simulated population reproduces a preference pattern a published, peer-reviewed human study already established, before that method is applied to a new question on similarly sensitive ground. [figure: A comparison chart lining up two columns, published human study and simulated run, against the same three attributes: gender, language, and religion, showing matching preference order on each.] ## What Did the Published Study Find? Adida, Lo, and Platas (2019) ran a conjoint analysis in 2016 asking Americans to choose between hypothetical Syrian refugees who varied by attributes including gender, language, and religion. Their published result: Americans preferred refugees who were female, English-speaking, and Christian. "The most consistent and substantive determinant is religion: Muslim profiles rate on average 0.5 points lower than do Christian profiles, a substantive difference for a scale that runs from 1 (the respondent believes the United States should absolutely not admit the refugee) to 7 (the respondent believes the United States should definitely admit the refugee)." Adida, Lo, and Platas, PLOS ONE (2019) (source) ## What Did the Replication Compare? A matched run against a simulated population used the same attribute set (gender, language, and religion) and produced its own preference ordering over refugee profiles. The comparison of interest: whether the simulated ordering lines up with the published human ordering on these shared attributes, not whether the simulation invents a new result. The simulated preference order tracked the published order across these attributes. A preference match without its scope stated reads as marketing. That is the evidence reported here, not a claim that the simulation reproduces Adida, Lo, and Platas's underlying respondent-level data, nor evidence about a different topic, population, or experiment type. ## Why Does a Sensitive Topic Change the Stakes? A commercial product test that misses the mark wastes a study budget. A misread preference on immigration, refugee policy, or discrimination carries reputational and ethical stakes beyond that: a recommendation built on a simulated population that quietly diverges from real human attitudes can do harm before anyone catches the gap. Checking a simulated run against a published academic result on this kind of topic, before extending the method to a new sensitive question, catches that divergence early at lower cost. ## Where a validation path fits A team does not need to commit to full human fielding before learning whether its simulated design is pointed in the right direction. Subconscious can move a study from a simulated experiment to real-human testing or validation without changing the underlying causal question, so a researcher can use a replication check like this one as a first-pass filter before deciding whether a new sensitive-topic study needs its own fielded validation. Related replication comparisons are collected in case studies; the underlying method is documented on the research page. ## What this replication does not establish Stating the scope of a single result is what lets a buyer check it before relying on it. This is one historical replication against a single study fielded in 2016 and published in 2019, not a live customer engagement, a benchmark run, or evidence about commercial pricing or product decisions. It does not establish accuracy across other topics, populations, or experiment types, and should not be read as a named customer outcome or a dollar-figure result. A team applying this approach to a new sensitive-topic question needs its own matched comparison before treating a simulated ordering as a substitute for fielded research. --- # Marketing Mix Models, Attribution, or Experiments: Which One Should Decide Your Next Budget Move? URL: https://subconscious.ai/blog/use-cases/marketing-mix-models-attribution-experiments-which-one-should-decide-next Published: 2026-08-13 A CMO deciding where to move next quarter's media budget usually has three kinds of evidence on the table: a marketing mix model, an attribution report, and maybe a handful of A/B tests. They rarely agree, and only one of the three actually tells you what happens if you change something. ## Three ways teams measure marketing effectiveness Attribution and mix modeling both describe the past. Controlled experiments are treated as the reference standard for marketing measurement because randomization is what separates correlation from cause, a distinction that isn't specific to any one vendor or method (Forbes: Measuring The ROI Of Marketing). [figure: Three methods feed one budget decision. MMM shows spend history, not cause. Attribution shows touchpoint order, not cause. Controlled experiments test a change and show if it causes the outcome, linked to the decision.] ## What does it cost to pick the wrong measurement method? Reallocating spend based on a channel that only correlates with results, rather than one that causes them, wastes the budget moved into it and can starve the channel actually driving performance. Teams commonly earmark around 10% of total media budget for measurement and analytics work, a starting point that shifts with business size and market characteristics. ## What can controlled experiments do that MMM and attribution can't? MMM and attribution both work from data that already exists. Neither one can run a change and observe the result before you commit budget to it. A controlled experiment does exactly that, and Subconscious runs that layer on top of MMM and attribution rather than in place of them: randomized comparisons against a simulated population that estimate whether a specific action (a message, a channel shift, a budget reallocation) changes the outcome. Subconscious can test or validate the same studies with real human participants, which lets a team move from a simulated read to a real-human check without changing the underlying causal question. ## What does this approach not replace? Controlled experiments on a simulated population are not a substitute for MMM's macro, exogenous market modeling, and they don't replace attribution's touchpoint tracking. Packaged pricing and budget-optimization outputs, confidence intervals as a standard deliverable, and ready-made decision memos remain capabilities the product is still confirming, not yet available. ## Choosing a method for the decision in front of you If the question is "what happened across our channels last year," MMM is built for that. If the question is "which touchpoints preceded conversion," attribution answers it. If the question is "will this specific change move the outcome," that's the question a controlled experiment is built to answer. Recent case studies show how that plays out across categories; read how the process works before running one against your own budget decision. --- # How to Build Synthetic Customer Panels for Research URL: https://subconscious.ai/blog/implementation-and-procurement/how-build-synthetic-customer-panels-research Published: 2026-08-13 A synthetic customer panel is a standing set of AI-simulated respondents, calibrated to represent real customers, that a team can query on demand instead of recruiting participants for every research question. The decision that actually matters isn't whether to build one. It's whether the panel's answers get checked against real outcomes before they inform a launch, price change, or message, or whether the team trusts them on persona consistency alone. Skip that check and a panel can return a confident, wrong answer with no warning before the decision ships. ## Building the panel ### How do you define the panel architecture? Deciding what the panel needs to represent is a segmentation exercise, not a respondent exercise. Most B2B companies land on 3-5 segments that actually matter for decisions. A SaaS company's split might run along company size, industry, role, and buying stage, while a consumer brand's split is more likely built from demographics, purchase behavior, and brand relationship. For each segment, map the dimensions that matter to the research question: functional needs, decision criteria, information sources, competitive context, and emotional drivers. A workable starting panel runs 8-15 simulated respondents: go below 8 and real variation disappears, and past 15 the extra respondents go unused. ### How do you build the individual respondents? Each simulated respondent needs a profile (demographic and firmographic basics), calibration data (interview transcripts, CRM notes, survey responses, support tickets, and behavioral data from real customers in that segment), and personality variation. Panels that vary the analytical decision-maker against the intuitive one, or the early adopter against the skeptic, produce more realistic spread than a panel built from a single archetype per segment. Depth accumulates with use: a respondent that has been through 20 sessions carries richer, more specific context than a fresh one. ### Establish research protocols and maintenance A panel without protocols becomes a toy. Standard question formats for reaction testing, competitive probing, and journey mapping, plus documentation standards (key themes, segment-level patterns, recommended actions), keep sessions and findings comparable over time. Panels also need upkeep: quarterly reviews to check the panel still reflects current segmentation, data refreshes as new customer interactions come in, and retirement of respondents built for segments that no longer exist. ## What step decides whether any of it is real research? Architecture, respondent design, and protocol all matter, but none of them determine whether the panel's answers are true. That's calibration: testing the panel against known realities and adjusting until it matches. In practice this means historical validation (presenting a scenario with a known outcome and checking the match), known-answer testing (asking questions with an answer already known from prior research), and blind comparison (having someone who works with real customers review panel output without being told it's synthetic). Skip this step and the risk isn't a bad answer. It's a good-sounding one. Independent UX research documents the same failure mode in AI-generated research more broadly: simulated respondents can produce plausible, internally consistent answers that diverge from what real people would say, with no signal in the output itself that marks the divergence (ACM Interactions, "The Synthetic Persona Fallacy: How AI-Generated Research Undermines UX Research"). [figure: Five-stage path: define panel architecture, build respondents, establish protocols, validate against real outcomes, maintain the panel. Validation is marked as where trust is earned or lost.] ## What "calibrated" needs to mean Historical validation and blind comparison are useful spot checks, but they're manual and only as good as the known answers on hand. The underlying question is measurable: how closely does a simulated respondent's answer track what a real person would actually say. Independent research has measured this directly, testing how accurately AI agents reproduce real individuals' survey responses across a large sample (Stanford HAI, "AI Agents Simulate 1,052 Individuals' Personalities with Impressive Accuracy"). That's the same class of question a calibrated panel has to keep answering, on an ongoing basis, not just at setup. Subconscious approaches this from the causal-testing side rather than the persona side: it runs controlled experiments comparing specific actions, such as one price or one message against another, on a simulated population, and reports the directional comparison with the uncertainty the study design supports. Subconscious can also test or validate a study with real human participants, so a team can move from a simulated experiment to real-human validation without changing the underlying question being tested. Read how a study moves from a simulated experiment to a validated result. ## What this doesn't replace Subconscious does not offer a self-service builder that replaces the five-stage process above, and it isn't a source of automated next-step recommendations as a standard study output. It answers one narrower question well: given a defined set of actions, which one moves the outcome, and does that answer hold up against real human behavior. A team still has to decide what to test, define the population, and interpret the result inside its own decision. For research questions that are exploratory or discovery-stage rather than a comparison between defined actions, recruited real-participant research still does that job better than any simulated panel, calibrated or not. [figure: Comparison: three manual spot checks (historical validation, known-answer testing, blind comparison) against continuous measurement of how closely simulated answers track real respondents.] ## Next step If a panel is already running and the open question is whether Step 3 is solid, the useful next move is testing one real decision, such as a pricing change or a messaging choice, as a controlled comparison rather than a persona reaction. Review the published evidence behind Subconscious's method, or book time to walk through a specific calibration question. --- # Fast Persona Tools vs. Enterprise Simulation: The Comparison That Actually Matters URL: https://subconscious.ai/blog/comparisons/fast-persona-tools-vs-enterprise-simulation-comparison-actually-matters Published: 2026-08-13 When a team compares a fast, self-serve persona tool against a slower, enterprise simulation platform trained on real interviews, the question that decides the outcome isn't speed or the higher accuracy number. It's whether either tool's output came from a controlled experiment that isolates cause from correlation. A directional read that can't be replicated is not evidence a pricing or launch decision should rest on. ## Two default architectures, one shared gap Synthetic research tools today mostly split into two families. Neither column answers the question that should come first: was the study designed as a controlled experiment that can isolate which action changed the outcome, or is it a plausible read that happens to be fast or expensive? ## What does the interview-trained approach get right, and where does it stop? Simile trains synthetic respondents on real qualitative interviews and validates them by comparing synthetic answers against the original human transcripts (Simile). That gives it real fidelity to the specific population it has already interviewed. Naming this ceiling is what lets a buyer check fit before signing a contract. The trade-off is structural, not a matter of execution. Without new interview data, an interview-trained system has no way to extend into an audience it hasn't studied yet (Simile). A team researching a new market, a new segment, or an early-stage concept has nothing to calibrate against until it runs the qualitative work first. The synthetic layer amplifies research that has already happened. It doesn't replace the research that hasn't. Fast, self-serve persona tools solve the opposite problem: no training data required, any audience addressable immediately. Speed on its own is a marketing number until the failure mode sits next to it. But that speed doesn't answer the underlying question. Generating a plausible persona and generating a causal answer are different claims, and the industry's own research literature is explicit about where description-based simulation tends to fail: aggregate pattern matching is easier to get right than person-level fidelity, and naive elicitation can produce distributions that look confident and aren't. ## The question a speed-vs-rigor comparison skips Neither architecture, on its own, tells a buyer whether the result would survive a real intervention. A model can recover a plausible answer without ever testing whether changing the price, the message, or the offer actually changes the choice. This is the decision that should come before setup time or a self-reported accuracy score: does the study randomize the alternatives under test, hold conditions constant except for the thing being changed, and report the result against a human baseline where one exists? ## Where does a causal platform sit above this split? Subconscious runs controlled, randomized experiments on a simulated population rather than generating personas from a description or training exclusively on a fixed set of past interviews. It can run those studies against a person-level audience graph covering 800 million real people, which keeps the addressable-audience problem of interview-trained systems from applying. It can also test or validate the same study with real human participants, without redesigning the causal question. Audience reach and recruited human validation are two distinct claims: reach describes who the simulation can model, not a panel of people available to recruit. That combination is why the buyer question isn't "self-serve or enterprise." It's whether the study was built to isolate an action's effect in the first place. See how a study is structured. ## What this costs a team that skips the question Stating this cost is what lets a buyer weigh it before the budget is committed. Committing launch or pricing budget to a directional read from an uncalibrated persona tool, or paying an enterprise contract for interview-trained simulation, without confirming the study was designed to isolate cause from correlation, means shipping a decision on a number that nobody can reproduce. The cost shows up later: in a launch that underperforms a synthetic read, or in a pricing change that moves the wrong segment, with no experiment on record to explain why. ## Before you sign a contract with any vendor Ask three questions regardless of which architecture a vendor sells: - Was the alternative under test randomized against a control, or is the output a single plausible answer? - Does the vendor report the result against a real human baseline, and can they show where the method fails, not just where it works? - If the decision is expensive enough to justify it, can the same study move from a simulated population to recruited human participants without changing what's being tested? A vendor that can't answer the third question with a concrete workflow is selling a persona demo, not a decision tool. Compare methodologies on the causal-effect leaderboard and see examples of tested decisions before the budget is already spent. [figure: Two columns, fast persona tool and enterprise simulation, each with a strength and limit, both pointing to a shared gap: no controlled experiment, resolved by a causal experiment box.] --- # AI for Consumer Insights Analysts URL: https://subconscious.ai/blog/use-cases/ai-consumer-insights-analysts Published: 2026-08-13 Consumer insights teams face ad-hoc questions while panel recruitment and fieldwork take weeks. AI can accelerate parts of the workflow. It cannot replace human empathy, statistically representative measurement, or evidence required for a consequential decision. The analyst's job is to know where simulation helps, where it fails, and how to sequence it with human research. [figure: Three boxes in sequence: explore with simulation, optimize instrument, validate with people. Branch after third asks if decision is consequential; if yes, route to human evidence first.] ## Silicon sampling and its limits Silicon sampling conditions a language model on a defined background, demographics, or psychographics, then records the responses. Political scientist Lisa Argyle and coauthors gave the technique its name in a 2023 study that conditioned models on real respondents' backstories and checked the output against benchmark survey data (Political Analysis, Cambridge University Press). A published limit is what separates a real result from marketing. The method approximates some aggregate opinion patterns but stays sensitive to elicitation, calibration, model choice, and prompt design. Aggregate prediction is generally easier than individual simulation. Naming the failure mode is what lets a buyer check a number before they trust it. Commercial claims of 80 to 90 percent correlation, or 80 to 95 percent against historical benchmarks, need the exact question type, population, baseline, and validation method. Do not transfer one range to another study. ## Four areas for an AI-assisted workflow ### What is questionnaire pretesting? Run a draft instrument before fieldwork to find ambiguous wording, missing options, logical dead ends, and high cognitive load. Treat the output as a review aid; a qualified researcher still owns the questionnaire. ### How can AI help analyze open-end responses? Models can cluster large sets of text into provisional themes. A first pass may turn thousands of responses into a draft taxonomy in minutes. A researcher must inspect the assignments, preserve minority themes, and interpret the result. ### What can simulation do between tracking waves? Quarterly or bi-annual tracking can show a change without explaining it. If the next wave is three months away, simulation can screen possible explanations and improve the questions used in the real study. It cannot establish the cause of the change by itself. ### 4. Segment interrogation Static segment profiles can become structured study definitions. Compare reactions to the same product concept, package, or claim across those definitions. Treat differences between a working parent in Munich and a young professional in Berlin as hypotheses until people and market behavior validate them. ## Hard limits A number without its limits is marketing. Simulation does not provide statistically projectable market sizing. If a decision depends on exactly 34 percent of a market buying at a price, use representative human evidence. The misses belong on the public record next to the hits. Do not rely on simulated respondents alone for final pricing, financial commitments, health claims, legal defenses, or regulatory submissions. Simulated participants have no bank accounts, sensory experience, or legal standing. Novel products and unprecedented events are also difficult because historical patterns may not cover the behavior that matters. ## A three-phase hybrid framework Phase 1: Exploration with simulation. Screen dozens of hypotheses, compare defined segments, and refine concepts. Phase 2: Instrument optimization with simulation. Pretest questions and remove confusing language. Phase 3: Validation with people. Field targeted research and confirm the selected options with evidence appropriate to the decision. This sequence puts human attention on the questions that matter most. ## Compare workflows with explicit examples A traditional concept screen might draft 10 concepts, wait two weeks, and find that 8 were weak. A simulated-first planning example might run 50 variations, identify the top 3, then validate those options with people. The quantities are examples, not promised throughput. For open-end analysis, an agency or analyst may spend days coding responses. A model may cluster them in minutes. Measure the actual review time and error rate before claiming a saving. For an ad-hoc request, a simulation may return directional material within hours. Label it as hypothesis work, not representative research. [figure: Four labeled boxes: questionnaire pretesting, open-end exploration, between-wave hypothesis work, segment interrogation. Each names the human check it still needs.] ## Protect research quality Document the model, audience definition, prompt, stimulus, and version. Keep private customer inputs out of public models unless the organization's approved data process permits them. Review security, data location, retention, and sub-processors through procurement rather than relying on marketing claims. Start with one low-risk project. Use the result to improve the next human study. The value is a more deliberate research sequence, not a replaced researcher. --- # Why Causal Effects Come With a Spread, Not a Single Number URL: https://subconscious.ai/blog/methods-and-validation/why-causal-effects-come-spread-not-single-number Published: 2026-08-13 A vendor hands you a causal effect with a confidence interval attached. Before a pilot budget moves, it helps to know where that interval comes from and why it isn't a single, exact number. ## The posterior is usually too complex to solve directly Bayesian inference updates a belief about an unknown quantity, say the effect of a price change, using observed data. The updated belief is called the posterior distribution. For simple problems, that distribution has a closed-form solution: plug in numbers, get an exact curve. Most real problems aren't simple. Once a model has more than a handful of interacting parameters, the posterior has no closed form, only a way to describe its shape. ## How does MCMC sampling work? The common workaround is to build the posterior out of samples instead of solving for it in one step. A family of methods, generally called Markov Chain Monte Carlo, does this by taking a long random walk through the space of possible parameter values. At each step, the walk proposes a nearby point, compares how well that point explains the observed data against the current point, and then keeps the proposal or rejects it and stays put. Repeated many thousands of times, the sequence of kept points is not a scatter of noise: it spends more time in regions that fit the data better and less time in regions that fit poorly. The random-walk Metropolis algorithm is the simplest version of this idea, and it explains the general mechanism most current samplers refine (Toptal, "Metropolis-Hastings and Bayesian Inference"). [figure: Five-stage chain: complex posterior, random walk proposes values, each kept or rejected against the data, kept points trace the posterior's shape, spread reported as the interval.] ## What do the resulting samples actually represent? After enough steps, the collection of kept points approximates the posterior distribution well enough to use. No single sample is "the answer." The samples, taken together, describe a range of plausible values and how likely each one is relative to the others (QuantStart, "Markov Chain Monte Carlo for Bayesian Inference"). That is why a credible interval or error bar sits next to a reported effect: it is a direct readout of how spread out those samples are, not a cosmetic addition. A tight spread means the samples agree closely. A wide spread means the data supports a broader range of possible effects, and a decision built on the point estimate alone would be overconfident. ## Reading a sampled result as a buyer Subconscious reports causal effects on business actions with confidence intervals rather than a single number, estimated with DCE, mixed logit, and ICLV, where causality comes from the randomized experimental design rather than from any sampler: the interval reflects how much the data constrains the answer, following the same general logic as a sampled posterior's spread. It is not a claim that any particular sampling algorithm runs underneath Subconscious's experiments, but the general statistical logic that makes an interval meaningful, applying whether the underlying inference runs on a market simulation or a public-health dataset. For a technical evaluator, the practical test is simple: ask what the interval would look like if the number of respondents were doubled, or the design were changed. A tool that can only produce a point estimate, with no visibility into how that estimate was assembled, does not let you ask that question. [figure: Branching path: reported effect and interval, then a check on doubled sample size, splitting into a check on what narrowed the interval or real-participant validation for a wide one.] ## Where does this reasoning stop applying? This is general Bayesian-statistics intuition, not a description of any specific inference engine, benchmark, or customer result. A wide interval on its own does not tell you whether a model's assumptions are correct; it only shows how much the data constrains the answer given those assumptions. When a decision depends on more than statistical mechanics, the same causal question can move from a simulated study to real-human participants without changing what is being tested. Subconscious can test or validate studies with real human participants, which is the check worth asking for before a pilot converts to a production commitment. The research program and replication leaderboard describe how that check is run, and how Subconscious's process works covers the steps between a defined decision and a reported effect. --- # Structured Research Platform or Fast Causal Experiment: How to Route the Decision URL: https://subconscious.ai/blog/comparisons/structured-research-platform-fast-causal-experiment-how-route-decision Published: 2026-08-13 A CMO or insights lead already running, or evaluating, a structured research program faces a recurring routing problem: some decisions belong in that program, and some will ship or die before the program can answer them. The question is not which vendor is better, but which path fits the decision in front of you: a multi-week, dashboard-integrated research cycle, or a fast, causal experiment built to test one action before the window closes. ## What is a structured predictive-modeling program built for? In the market research and consumer-data space, Civis Analytics is known for predictive modeling from aggregated consumer and voter data, delivered through dashboards and reports that plug into an existing analytics stack. That design fits organizations with a dedicated research or insights function, an established workflow, and budget allocated specifically to research tooling. The tradeoff is cycle time: defining the question, designing the methodology, collecting data, and analyzing results is a multi-week undertaking even under favorable conditions, and updating the answer means running the cycle again. That cost buys segment-level modeling, integration with an existing reporting stack, and a methodology built for ongoing, institutional use rather than a single decision. ## What is a fast causal experiment built for? The alternative is not a faster dashboard. It is a different question shape: instead of building a durable model of the market, a causal experiment tests one specific action, such as a price point, a message, or a feature description, against a defined outcome, on a timeline set by the decision rather than by a research cycle. Subconscious is a causal behavioral platform: randomized experiments run on a simulation of the market estimate which action moves which outcome, with confidence where the evidence supports it. The method is closer to a designed experiment than to a survey or a dashboard refresh. ## Deciding which path fits the decision Most real decisions are not purely one or the other. A launch pricing call might need a structured program for the long-run pricing architecture and a fast causal test to check one price point against a deadline. The routing question is which need is binding this week. ## Where a causal experiment adds a distinct capability Where the decision genuinely needs corroboration beyond a simulated experiment, Subconscious can test or validate the same study with real human participants without changing what is being tested. That keeps the simulated and human-validated results answering the same question, rather than switching methods and losing comparability. Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people, which supports targeting a specific buyer segment rather than a generic panel. That is a targeting capability, not a claim that all 800 million are recruited study participants: the audience graph and a recruited human sample are different things, and only the latter is a live validation study. ## When is a fast causal experiment the wrong tool? A single causal experiment does not replace a dedicated research or insights team, a dashboard-integrated analytics program, or a long-running segmentation study. If the real need is an ongoing feed of consumer data into an existing reporting stack, or statistically rigorous modeling maintained by a research function over time, a structured program remains the right infrastructure. Timeline, pricing, and accuracy comparisons between the two are not included here, since no sourced benchmark supports a direct comparison. The distinction that matters is methodological, not a speed or cost claim. ## Making the call Start with the decision, not the vendor. If a team already has budget and a mandate for a structured, dashboard-integrated research program, and the decision has weeks of runway, that program is infrastructure worth keeping. If a specific action needs a causal answer before a deadline the research cycle cannot meet, a fast, targeted experiment is the better-fitted tool, and it can graduate to real-human validation without changing the question asked. Reviewing current causal research methods or how a causal experiment moves from design to evidence is a reasonable next step. [figure: Branching path from "what the decision needs." Weeks of runway routes to a structured program. Days of runway or one-action question routes to a causal experiment, then optional human validation.] --- # Testing Onboarding Flow Decisions Before Build URL: https://subconscious.ai/blog/use-cases/testing-onboarding-flow-decisions-before-build Published: 2026-08-13 Test an onboarding flow before build by treating each version as a competing action, not a finished screen. A historical industry benchmark reported that 40% to 60% of new signups did not complete onboarding (Userpilot). A number without its limits is marketing copy. That range is a planning example, not a current Subconscious benchmark. The product decision comes before the dashboard can show where a live flow lost people. [figure: Branching diagram: define audience and alternatives, then run a causal action comparison against a named activation behavior, splitting into reject before build, revise before release, or monitor in production.] ## Decide Which Behavior to Ask For First An onboarding test should answer a build decision: which sequence should the team ship? The alternatives might be an email-only signup against a form that also asks for role and company, or guided setup against immediate product exploration. The comparison needs three explicit parts: - A target signup audience. Define whose activation behavior matters for this decision. - Two or three credible alternatives. Compare only versions the team could actually build and support. - A behavioral outcome. Name the completion or activation action each version is intended to move. Subconscious supports this decision through causal action testing and discrete-choice-style experiments, comparing alternatives against the same behavioral question before engineering commits to a single design. ## Which Method Should You Use for Which Question? Pre-build comparison, observed usability, and product analytics answer different questions. Keeping those roles separate prevents an early directional result from being treated as proof of live performance. A team can use the action comparison to choose what deserves engineering investment, usability work to diagnose interaction friction, and live analytics to monitor the released experience. ## Stress the Paths That Threaten Activation The happy path is rarely the whole decision. Compare each alternative against the moments that can change behavior: - Confusion. If a step is misunderstood, can the person recover? - Deferral. If setup is optional, does skipping still lead to a useful first experience? - Expectation mismatch. Does the flow correct a mistaken expectation before it becomes abandonment? They are competing behavioral hypotheses, not interface observations, and should be stated before the study. This method's failure mode is published here for the buyer to check. Direct observation remains necessary when the question is exactly where someone looks, clicks, hesitates, or gets stuck. ## Can You Validate Results With Real Human Participants? Subconscious can test or validate studies with real human participants, carrying the same causal question from a simulated comparison into real-human validation of the preferred flow. Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people. Naming what a number can't do is what lets a buyer check it. The graph supports audience definition, not a pool of recruited research participants. The misses sit on the same page as the hits. Neither form of pre-build evidence guarantees activation after launch. It does not replace direct usability observation, real-user session data, or live product analytics. It narrows the build decision and makes the remaining uncertainty explicit. ## Convert the Result Into a Release Plan Give every finding one operational destination: - Reject before build when an alternative presents a clear abandonment risk. - Revise before release when the comparison identifies a plausible weakness that can be addressed in the first implementation. - Monitor in production when the result remains ambiguous and needs live behavioral evidence. Start the brief with the audience, the two or three alternatives, and the activation behavior that should move. Subconscious research explains the causal approach, case studies show applied work, and a walkthrough can help turn the decision into a study. --- # Subconscious.ai FAQ: What a Causal Behavioral Experiment Can and Can't Tell You URL: https://subconscious.ai/blog/company-and-trust/subconscious-ai-faq-what-causal-behavioral-experiment-cant-tell Published: 2026-08-13 Subconscious.ai runs controlled causal experiments on simulated populations so a team can estimate which product, pricing, or messaging action is likely to change customer behavior before it ships. This page answers the questions a buyer usually asks before trusting that kind of result for a real decision: how the method works, where it breaks down, how consent and bias questions apply to simulated respondents, and when to bring in real human participants. [figure: A four-step decision path from defining the decision through running a causal experiment, checking method and replication evidence, and validating with real humans when the decision is high-stakes.] ## What does Subconscious.ai actually test? Causal experiments compare alternatives under controlled conditions to estimate which one changes an outcome, rather than describing what already happened or predicting a single likely answer. Subconscious.ai applies the same logic to simulated populations: it defines a decision, an audience, and a set of alternatives, then reports which alternative is more likely to move the outcome the team cares about. ## How does a team use it? A typical workflow moves through five steps: - State the question about behavior the team needs answered. - Choose the outcome the decision should move, such as adoption, conversion, or reduced risk. - Design the experiment: which attributes and levels to compare. - Run the experiment against a simulated population and get the causal comparison between alternatives. - Look at how population or segment characteristics change the result. This is closer to running a study than filing a research request. ## Where does the method have real limits? Limits specific to how an experiment is built. A single experiment can only examine a bounded number of attributes and levels before it strains both compute cost and the amount of a respondent's attention that survey-style methods can reasonably ask for. That is a design constraint on any one study, not a hard ceiling on what causal experimentation can eventually cover. Standard analysis uses named regression and choice-modeling methods; a team that wants a different analysis can work from the exported experiment data directly. Limits that apply to any simulation built on a trained model. A simulated population is only as reliable as the data the underlying model was trained on. If that training data carries bias, the bias can surface in the simulated responses. Subconscious.ai's the causal fidelity paper reports that our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. That comparison is corpus-level evidence, not a guarantee for a question type the corpus hasn't covered. Complex emotional or social decision-making in particular may still call for a recruited real-human study rather than simulated respondents alone. ## Does using simulated respondents raise the same ethical questions as human research? Simulated respondents have no personal feelings or privacy to protect, so a team running an experiment on them does not need informed consent or participant-privacy safeguards for the respondents themselves, and the experiment can explore sensitive topics without risking psychological or emotional harm to a real person. Because no recruitment, training, or compensation of human participants is required, resources that would have gone toward panel logistics can go toward the analysis itself, and studies that would be logistically difficult or impossible to run with human subjects become possible to explore first. When a decision hinges on complex human emotion, social dynamics, or lived experience, real human subjects remain the more accurate source. And the data used to build or calibrate a simulated population still has to be handled ethically: it must be designed and sourced in a way that does not encode or amplify bias, and any consent or privacy obligations attached to that underlying data still apply to the team that collected it. ## When should a team add real-human validation? Add it when the decision is high-stakes and the simulated result would be acted on directly, not merely used to narrow options. Subconscious.ai can test or validate studies with real human participants, and a team can move from a simulated experiment to real-human validation without changing the causal question it's asking. That matters most for pricing, launch, and other decisions where the cost of acting on a wrong result outweighs the time saved by skipping the check. For most exploratory or hypothesis-narrowing work, the simulated result stands on its own. ## Where to look next - Research and replication evidence for the current corpus and its scope. - How Subconscious.ai works for the experiment workflow in more detail. - Case studies for how teams have applied a specific result to a specific decision. - About Subconscious.ai for the company behind the method. [figure: Two columns. Design limits: bounded attributes, a compute/attention constraint, fixed by redesign. Model limits: capped by training data, bias can surface, corpus-level evidence, fixed by human validation.] ## Limitations This page does not cover pricing, security certifications, or team composition, and does not replace reading the study design behind any specific result before acting on it. --- # Why a Confounder Can Make a Marketing Channel Look Effective When It Isn't URL: https://subconscious.ai/blog/methods-and-validation/why-confounder-make-marketing-channel-look-effective-when-isnt Published: 2026-08-13 A marketing team sees sales rise whenever Google Ads run. The obvious read is that the ads work. The obvious read can be wrong, and a worked Bayesian example shows exactly how. ## The setup: two channels that move together The team runs TV campaigns and Google Ads together. When TV is live, Google Ads are usually switched on; when TV is off, Google Ads are usually switched off to conserve budget, though the coordination isn't perfect. That coordination is the problem. If TV independently drives sales, then Google Ads will appear to drive sales too, even if Google Ads contribute nothing on their own. This is a confounder: a variable that causally influences both the treatment (Google Ads) and the outcome (sales), producing a correlation between the two that has nothing to do with a direct causal link. [figure: Diagram: TV branches to Google Ads and Sales, causing both. A second path cuts TV's arrow into Google Ads via the do-operator, measuring Google Ads' isolated effect, ending in an interval spanning zero.] ## How is the causal structure built? The example maps the scenario to three variables: C (TV, the confounder), Z (Google Ads on/off), and Y (sales). TV influences both Z and Y. The analysis isolates how Z acts on Y on its own, stripped of the raw correlation between Google Ads and sales. Two regressions encode this structure: a logistic regression of Google Ads on TV status, and a linear regression of sales on TV and Google Ads. This directed acyclic graph is the model's explicit statement of what causes what, the only thing that makes the later estimate interpretable as causal rather than merely predictive. ## How is the model tested against a known answer? Before trusting any real-world estimate, the analysis simulates data from the same structure with the true causal effect of Google Ads on sales set to exactly zero. This is a parameter-recovery check: if a model can't recover a known answer on data built to match its own assumptions, it has no business estimating an unknown one. A naive look at the simulated data, comparing sales when Google Ads were on versus off, shows a large, visually convincing gap between the two groups. The true causal effect is zero. The entire apparent gap is TV's doing. PyMC's new do operator makes the counterfactual comparison possible inside a standard probabilistic model. It fixes a variable to a specific value and severs it from the variables that would normally influence it, the mechanics behind asking "what would sales have looked like if Google Ads had been forced on, or forced off, independent of TV?" Applied here, do(z=0) and do(z=1) generate two counterfactual sales distributions with TV pinned to its observed values. The Average Treatment Effect (ATE) is the difference between them, and in the simulation the model's posterior concentrates near the true value of zero, with the interval covering it. For the code walkthrough, covering the generative model, the do() calls, and the posterior sampling, see the source do-operator example and the related excess-deaths counterfactual example from PyMC. ## What does the estimate look like once it's real? With the model validated on known data, the same procedure runs on the (still simulated, but now "observed") dataset without knowing the true effect in advance. The estimated ATE comes out at 0.06, small and positive. A point estimate alone would tempt a naive analyst to credit Google Ads with a real, if modest, lift. The full posterior tells a different story: the credible interval for the ATE spans zero. Scaled against TV's own effect, Google Ads shows a posterior median of 6.2% relative to TV's effect, with a credible interval of -4.1% to 16.5% that includes zero, so no directional effect is supported. The practical argument here is for reporting full uncertainty rather than collapsing it into a single threshold verdict. A 2019 commentary in Nature made a related point about statistical-significance thresholds more broadly: dichotomizing results at a significance cutoff encourages false confidence in exactly this kind of ambiguous result. ## What this generalizes to, and what it doesn't Real marketing budgets span more than two channels, and the same logic scales: higher-funnel spend like TV builds awareness over time, lower-funnel spend like paid search acts closer to the point of purchase, and confounders can sit at multiple levels of that funnel. Media Mix Models formalize this at scale, adding saturation and time-delayed adstock effects on top of the same causal reasoning. But the method has a hard limit: a posterior estimate like this one is only as trustworthy as the DAG it's built on. Get the confounder structure wrong, miss a variable that influences both treatment and outcome, and the model will produce a confidently wrong ATE with a narrow, reassuring-looking credible interval. This single simulated example demonstrates the mechanism; it doesn't establish that any particular real-world estimate built this way is accurate. Parameter recovery on simulated data validates that the model works as intended, not that the assumed causal graph matches the real world. [figure: Five stages: a point estimate of 0.06 for the Google Ads effect, the full posterior sampled, a credible interval of -4.1% to 16.5% relative to TV's effect, flagged as crossing zero, ending in no confident causal claim.] ## Beyond one model per question The do-operator example is a single structural causal model, hand-built around one specific DAG, answering one specific question about two channels. That is real, careful work, and it's also the pattern's limit: every new causal question needs its own model, its own DAG, and its own parameter-recovery check. Subconscious sits a step above that per-question modeling work. Once a team has identified the causal question, does this action change this outcome for this population, Subconscious runs a randomized controlled experiment, using McFadden MNL, mixed logit, or ICLV, that compares the specific actions directly, on a simulated population, and reports the effect with quantified uncertainty, without requiring a bespoke structural causal model like the DAG above to be hand-built for each new question. The research program and replication leaderboard document how those effect estimates get checked against outcomes. Where a question benefits from moving past simulation, the how-we-work overview covers testing against real human participants without changing the causal question being asked. The lesson from the confounder example holds either way: a number that moves in the direction you hoped for is not evidence until you've asked what else could have produced it. --- # AI Decision Support for Startup Founders URL: https://subconscious.ai/blog/buyers/ai-decision-support-startup-founders Published: 2026-08-13 Founders make decisions across every function, every day. The risk is treating one AI response as an expert network. Use simulation to rehearse questions and test buyer-facing actions. Use accountable people for finance, law, security, architecture, and other decisions where credentials and current evidence matter. [figure: A path diagram: a decision enters, splits on whether it needs credentials or evidence, runs through simulated perspectives, then routes to either an accountable expert or a real buyer test.] ## Sort the week's decisions A founder's week may include pricing on Monday, investor preparation on Tuesday, architecture on Wednesday, compliance on Thursday, and a choice between two go-to-market approaches on Friday. All five decisions need different evidence. Product, pricing, messaging, and go-to-market alternatives can support decision-specific experiments. A real finance professional should review the model. Engineers should own architecture. Qualified security and legal staff should own compliance. ## Use perspectives as rehearsal ### An investor perspective A capability without its limits stated is marketing copy. A prompt such as, "I'm raising a Series A at $15M pre-money. What are the three biggest concerns?" can surface questions. It cannot know a current investor's portfolio, appetite, or valuation view. ### What is a target-customer perspective? A study might define a vice president at a company with 200 employees and compare buying messages or product concepts. The audience definition is an experimental input, not a living copy of a real buyer. ### A domain-expert perspective Naming what a tool cannot do is what lets a buyer check the claim before relying on it. AI can prepare a list of questions for a real regulatory, data, or supply-chain expert. It should not give final compliance or technical advice. ### A competitor perspective This tool's misses appear in the open, next to its hits. A simulated competitor lens can challenge positioning. It cannot reveal a competitor's private plan or predict its next action. ### What is a senior-operator perspective? A founder building a first sales team with two account executives might use a 90-day plan as a review artifact. An experienced operator still needs to assess the company's actual team, pipeline, and market. ## Five decision rehearsals ### Monday: pricing Compare a per-user and usage-based model with three perspectives: the target buyer, an investor, and an operator. Do not let the exercise claim cannibalization, spending changes, or financing effects without real data. A thirty-minute review can produce three sets of questions. It is not three consulting projects. ### Tuesday: pitch preparation If a Series A meeting is on Thursday, rehearse objections, revise the deck, and ask which evidence would change the critique. ### Wednesday: architecture When engineers are comparing two database approaches, use AI to organize the tradeoffs and failure modes. The technical team must inspect both options and own the decision. ### Thursday: enterprise sales For a buyer with 5,000 employees, rehearse security-review questions. Answer them from approved security evidence, not generated text. ### Friday: go-to-market Compare two approaches, such as self-serve and sales-led, through three defined frameworks. Then test the buyer-facing action with the intended audience and outcome. [figure: Five-row list pairing each weekday decision with its accountable owner: pricing to finance, the pitch to a real investor, architecture to engineers, compliance to security/legal, go-to-market to a buyer test.] ## What can't simulation replace? Publishing what the tool cannot do belongs in this record too. Real networks take years to build. Advisers provide social capital, emotional support, accountability, current information, and unexpected connections. AI does not introduce a founder to a customer, accept professional responsibility, or know what changed inside a market yesterday (Greenberg and Mollick, "Sole Survivors: Solo Ventures Versus Founding Teams," SSRN). Start with the three people you would call most often if access were unlimited. For many founders, that means an investor, a target customer, and a domain expert. Use three simulated perspectives to improve the questions you bring to those people this week. The goal is not to replace the relationship. It is to arrive prepared and test the decisions that can be tested. --- # Pressure-Test a Startup Fundraise Before Investor Meetings URL: https://subconscious.ai/blog/use-cases/pressure-test-startup-fundraise-before-investor-meetings Published: 2026-08-13 A pitch deck, a model built at 2 AM, and a valuation borrowed from a comparable company do not make a defensible fundraise. An investor may challenge the assumptions in the first five minutes. A tool that only lists what it can do is marketing copy. Here is what this one cannot do: AI can help organize an adversarial review. It is not a financial adviser, valuation opinion, or substitute for a CFO, lawyer, accountant, or investor with current market knowledge. [figure: Four-step path: review strategy and find gaps; fix the model and replace weak assumptions; rehearse investor questions and tighten evidence; begin real meetings, gated on readiness not the calendar.] ## Where fundraising plans break Common problems include revenue projections without a link to observed growth, valuation expectations without useful comparables, a raise amount that does not match the operating plan, weak unit economics, and an undefined use of funds. Running out of cash before the next raise closes is among the reasons CB Insights lists most often when startups fail (CB Insights, Top Reasons Startups Fail). A deliberately sharp example: asking for a $15M valuation with $2K MRR, raising $3M when the plan needs $800K, or raising $500K when the plan requires $2M. ## Build an adversarial review Give reviewers the same inputs: - monthly revenue for the past 6-12 months, or an explicit pre-revenue model; - how fast the company is spending and how many months of cash remain at that pace; - the size of the raise being sought and the valuation attached to it; - where the new capital will actually go; - growth, churn, and margin assumptions; - comparable companies and the reason each one belongs. Use several review frames. A finance leader can inspect model consistency. An investor can challenge the financing story. An operator can test whether the plan and hiring sequence fit the capital requested. Publishing the limit next to the capability is what lets a reader check the claim. AI-generated perspectives can help assemble questions, but qualified people must verify the model. ## Five questions that expose weak assumptions ### 1. Is the valuation defensible? List the evidence behind the proposed valuation. Separate company facts from market comparables and negotiation goals. ### 2. Does the model have holes? Trace each projection back to an assumption. Identify which assumptions have observed support and which are still guesses. ### 3. Is the raise amount tied to the plan? Too little may force another raise in 6 months. Too much may cause unnecessary dilution. Explain how the amount supports the stated 18-month plan and what changes if the raise is smaller. ### 4. What will investors challenge first? Rehearse the likely objection before the meeting. A good response names the uncertainty and the evidence that would resolve it. ### 5. What must be true before raising? Possible planning thresholds include $10K MRR, churn below 5%, or three enterprise contracts. Choose milestones that follow from the company's actual model and financing market. ## A four-month planning example Month 1: Review the strategy and identify gaps. Month 2: Fix the model, replace weak assumptions, and review it again. Month 3: Rehearse investor questions and tighten the evidence behind each answer. Month 4: Begin real meetings only if the company, advisers, and financing plan are ready. [figure: A list of five questions used to pressure-test a fundraise: valuation evidence, model assumptions, raise amount versus the 18-month plan, likely investor objections, and readiness milestones.] ## Use experiments for claims about buyers Subconscious can help teams test product, pricing, messaging, and go-to-market actions before committing capital; see how a study moves through the process. A decision-specific experiment can compare how defined buyer segments respond to pitch or positioning alternatives. The scope of an experiment means nothing without its edges stated plainly. It cannot validate a cap table, calculate a correct valuation, guarantee a raise, or replace financial diligence. The practical division is clear. Use structured critique to find weak assumptions. Use causal experiments for testable buyer-response questions, with examples in the case studies. Use accountable finance and legal professionals for the financial decision. --- # Testing a Founder Story Before It Reaches Investors URL: https://subconscious.ai/blog/buyers/testing-founder-story-before-reaches-investors Published: 2026-08-13 A founder story is usually written once, in twenty minutes, and never touched again. It is also the first thing an investor reads after the deck, the first thing a journalist skims before a story, and the first thing a strong hire reads before deciding whether to leave a stable job. Publishing it untested means finding out which line cost you the meeting only after the meeting never happens. ## Why does a founder story carry so much weight? When there's no revenue, no customers, and no track record yet, the founder story has to carry the weight a product would otherwise carry. It has to do several jobs at once: establish credibility, establish motivation, establish insight, and sound like a person rather than a press release. Most drafts fail at least one of these jobs, and readers pick up on the failure in seconds. The conclusion they draw is blunt: wrong founder for this idea, or wrong idea altogether. The founder is rarely the person who can catch this: having repeated the story internally so many times, their ear for it is gone. Research on early-stage investor evaluation points to first impressions as a persistent driver of how pitches get judged, which is why a story needs an outside read before it goes live, not after. ## Five readers, five different filters An About page or founder story is read by at least five distinct audiences, each applying a different filter: A strong story survives scrutiny from all five. A weak one usually survives only the last, most forgiving reader. ## What does a pre-publish test actually measure? The buyer decision here is narrow: which version of the narrative goes live, before it reaches an investor, a candidate, or a reporter. Guessing wrong doesn't produce an error message: it produces an investor who passes, a candidate who declines, or a journalist who skips the story, with no record of which line caused it. A controlled comparison answers this before publication instead of after. It sets up two or more candidate narratives or opening paragraphs, defines respondent profiles matched to the intended reader (investor-type, operator-type, candidate-type) and measures which version moves stated interest or intent, with a result that can be rerun rather than a single subjective read from a friend. Subconscious runs this kind of comparison against a person-level audience graph covering 800 million real people, and can extend a study from a simulated read to real human participants without changing the underlying question being tested. ## Questions worth asking before a rewrite The value of a pre-publish test is less in the verdict and more in what it forces you to ask. Four questions repeatedly separate a story that lands from one that doesn't: - First reaction. What is the one-sentence takeaway, and does it converge across readers or scatter into five different interpretations? - The missing piece. What question does a reader want to ask the founder that the page doesn't answer? The most common gap is "why you": the problem and the solution are established, but not why this founder is the one to solve it. - Tone. Does the founder read as competent, arrogant, humble, or out of their depth? Tone is the hardest thing to self-evaluate and often the deciding factor in whether a reader keeps going. - The weak line. Is there one sentence that would make a reader lose interest, a claim that feels unearned or a detail that rings false? Cutting that single line often improves the page more than any amount of additive editing. Testing two or three opening paragraphs against each other, rather than shipping the first draft, tends to surface a different winner than the one the founder started with, usually because founders open with backstory when the reader wants the insight first. ## What patterns repeat across founder stories? A few failure modes show up often enough to check for directly: backstories that run long when readers want insight over biography; "we" language that reads as inflated when the company is still one person; category labels chosen for safety that end up making the company sound interchangeable with competitors; a team section that looks thin and undercuts credibility rather than building it; and a closing call to action that is usually the weakest paragraph on the page. ## What this does not replace A controlled narrative comparison does not write or edit the story; it tells you which version is working and where it isn't. It also does not substitute for an actual investor meeting, interview, or press conversation; it reduces the chance that a preventable line costs you the meeting before you get there. ## Where to start If an About page or founder story is already live, the same test applies before the next revision: define the reader archetypes actually reading the page, run the comparison against the current draft and one or two alternatives, and ship the version that converges best. See how a study like this runs or book time to scope one against a specific investor, hire, or press narrative you're about to publish. [figure: Five labeled reader types arranged around a single founder-story paragraph, each with its own filter question, showing the same text screened differently by each audience.] --- # When a Yield Difference Isn't the Treatment: A Field-Trial Case Study in Spatial Confounding URL: https://subconscious.ai/blog/case-studies/when-yield-difference-isnt-treatment-field-trial-case-study-spatial Published: 2026-08-13 A field trial testing a microbial treatment's effect on plant yield found a difference between treated and untreated plots. Before that difference could be trusted as a treatment effect, a Bayesian consultancy had to answer a prior question: how much of it was actually caused by where each plot sat in the field. ## What confound hides in a field trial? Soil quality, moisture, and drainage vary across a physical plot, and that variation does not respect treatment boundaries. A plot near a wetter corner of the field can outperform a drier one regardless of what was applied to it. Field-trial statisticians have documented this spatial-correlation problem for decades: in trials without randomized, replicated block assignment of treatment to plots, treatment comparisons that ignore the spatial layout of plots can attribute yield variation to the treatment when the real driver is location. For a biotech or agtech R&D team, that mistake produces a false-positive product claim or a rollout built on noise. ## How do you isolate the spatial pattern before reading the treatment effect? The consulting team's approach, described in the project's blog post write-up, was to model the spatial pattern rather than let it hide inside the treatment estimate. Using a Gaussian process over each plot's location, they estimated a smooth spatial surface across the field, then subtracted that surface from the raw yield readings before comparing treated and untreated plots. What remained after removing the spatial estimate is what they read as the treatment effect. This two-step separation (estimate the confound, then remove it) is a weaker relative of the joint-estimation approach in Bayesian spatial modeling in three-dimensional agricultural trials, where a spatial Gaussian process is fit alongside the treatment term in a single model so the treatment coefficient cannot be contaminated by unmodeled position effects. ## Why this generalizes past one field The project had limited data: a small number of plots, a single field layout, and one microbial treatment. That constraint is common in on-farm precision experimentation, where spatial variability in crop response to agronomic inputs is large enough that a treatment comparison without a spatial term can point the wrong direction even when the sample looks adequate on paper. The decision this case illustrates is not specific to microbes or yogurt cultures: any team drawing a causal conclusion from physical trial data has to ask whether the effect it sees would survive a spatial control, before it survives a launch decision. [figure: A causal chain diagram showing raw yield readings split by a Gaussian-process spatial pattern estimate; that estimate is subtracted from the readings, leaving an isolated treatment effect as the final output.] ## The same discipline, a different population Subconscious's causal testing method rests on the same core move as this field trial: isolate which action actually drives an outcome, under quantified uncertainty, rather than accept an observed difference at face value. In a simulated population the confound is usually an unbalanced design or an unmodeled interaction rather than field geography, but the discipline is identical: name the candidate confound, model it explicitly, and read the treatment effect only after it is accounted for. Teams that want to move a causal question from a simulated population to real-human testing without changing what's being asked can see how that transition works on how we work and in the broader research documentation. ## What does this case not show? This is a third-party consulting case study on physical agricultural field-trial data. It is not a Subconscious customer engagement, and Subconscious did not perform or replicate this analysis. The write-up did not report an accuracy figure, an effect size, or a named outcome beyond describing the modeling approach, so none is asserted here. A biotech or agtech team applying this same discipline to its own trials, physical or simulated, still needs its own spatial or design-based check; the lesson generalizes, the numbers do not. [figure: A four-step path: name the candidate confound (field geography or an unbalanced design), model it explicitly, read the treatment effect only after that model is subtracted out, then run your own check before deciding.] ## Sources - Treatment comparisons in agricultural field trials accounting for spatial correlation, Journal of Agricultural Science (Cambridge Core) - A Bayesian analysis of an agricultural field trial with three spatial dimensions, ScienceDirect - Spatial variability of crop responses to agronomic inputs in on-farm precision experimentation, Precision Agriculture (Springer) --- # Pre-Testing a Rebrand Announcement Before It Goes Public URL: https://subconscious.ai/blog/use-cases/pre-testing-rebrand-announcement-before-goes-public Published: 2026-08-13 A rebrand announcement is a single, public, largely irreversible event. The name either lands or it doesn't. The "why now" story either reassures the room or invites the comments section to write it instead. A version that misreads even one audience can unsettle the brand recognition the rebrand was meant to strengthen. The people who decide whether that story lands are not the marketing team. They are existing customers, prospects mid-evaluation, former customers, employees, and press or analysts, each reading with different anxieties. Most teams write one draft optimized for one of those groups, usually prospects, and are surprised when employees feel blindsided or customers read the news as a warning sign. [figure: Sequence diagram: a rebrand narrative moves from drafted candidates through testing against five audience segments (customer, prospect, former customer, employee, press), then comparison, selection, and publication.] ## The buyer's decision The person accountable for this is typically the CMO or head of brand and communications running the launch sequence: the name, the narrative, the founder letter, the press release, the social rollout, and internal comms. By the time an announcement is being drafted, the decision to rebrand has already been made. What remains open is which version of the "why now" story to commit to, and whether the tone and sequencing hold up across every audience that reads it. Research on corporate rebranding backlash finds that even changes made for defensible strategic reasons can still trigger public pushback when the reasoning behind them isn't made legible to the outside audience (University of Miami News). Research into the internal side of rebranding points to the same root cause from the other direction: launches misfire when the people inside the company, and the customers who feel like insiders, are not treated as part of the story the rebrand is telling (Journal of Business Research). Neither failure is about the logo. Both are about who the announcement was written for. "Townsend said research shows that millennials—with their considerable purchasing power—are exceptionally cause-driven and ready to financially reward companies they respect." Claudia Townsend, University of Miami Herbert Business School, News@TheU (source) ## Why this is a reader-modeling problem, not a design problem Three patterns send rebrand announcements sideways: - The "why now" gap. Nearly every reader lands on the same unspoken worry first: has something gone wrong internally, and is this rebrand just a cover story? An announcement that doesn't answer that in the first two sentences leaves the audience to answer it themselves, publicly. - Audience mismatch. Optimizing the copy for one audience, and assuming the others will follow along, is the most common single mistake. - Tone contradiction. How the announcement sounds carries as much brand meaning as what it says. A rebrand built around maturity announced in an irreverent voice, or the reverse, undercuts the strategic reason for the change before anyone reads past the first paragraph. All three are testable before publication: each is about how a defined audience reacts to specific copy, not about taste. ## Who reads a rebrand announcement in the first 72 hours? A rebrand test is broader than a normal message comparison: it covers five distinct personas encountering the announcement in the first 72 hours, not one buyer segment. ## How does Subconscious test a rebrand narrative? Subconscious runs controlled experiments comparing narrative and message alternatives against simulated audience segments defined by role and relationship to the brand: existing customer, prospect, former customer, employee. Given a small number of candidate narratives, such as a growth story, a maturity story, or a customer-obsession story, the platform estimates which version is more likely to land with each segment before publication. This is decision-specific message and positioning testing against a defined population and a defined outcome, not a general opinion generator reacting to a logo. Subconscious can run these comparisons against a person-level audience graph covering 800 million real people, moving from a simulated comparison to real-human validation without changing the underlying causal question once a simulated round has narrowed the field. ## A pre-launch cadence, as a planning example A team running this discipline against a launch might sequence it across six weeks of lead time: test the core narrative six weeks out, test the name and tagline in isolation four weeks out, test the full announcement draft three weeks out, run a red-team pass on likely criticism two weeks out, and test the complete sequence together one week out. The cadence is a scheduling choice; the order matters. The announcement itself is rarely one piece of copy. It's a sequence: teaser, hero post, founder letter, customer email, social rollout, press embargo lift. Sequences fail in ways a single draft never reveals, since a later touch can quietly contradict an earlier one. Testing the narrative early and the assembled sequence separately catches two different failure modes. Early testing surfaces which "why now" story resonates before a single asset is written; late testing surfaces whether the assets built on it still agree with each other by the time the last one ships. ## What are the limitations of this testing? Subconscious does not offer a packaged "rebrand testing" product or a workflow that recruits a company's actual former customers, employees, or press contacts as a live panel. The audience segments are simulated populations built from defined characteristics, not a reconstruction of a specific company's actual customer list. Reproducing a precise real-world audience needs a configured study, not a standard output. Confidence-interval reporting, automatic segment ranking, and decision-memo generation are not standard current capabilities. Testing a narrative or a tone is not the same as predicting whether a specific announcement will trigger press coverage or social backlash, and it doesn't substitute for legal, HR, or executive judgment about rebrand timing and disclosure. What it changes is whether the team is arguing from opinion or from a comparison of tested alternatives before the launch goes public. ## Next step Before a rebrand announcement goes out, the useful question is narrower than "will people like this": which of a small number of concrete narrative and tone alternatives holds up across the audiences who will actually read it. Comparing those alternatives against a defined population, before the copy is final, turns "we think this will land" into a testable claim. See how Subconscious runs these comparisons or book time to scope one against an upcoming launch. --- # AI Message Testing: Compare Copy Before Launch URL: https://subconscious.ai/blog/use-cases/ai-message-testing-compare-copy-before-launch Published: 2026-08-13 Marketing teams often learn whether a message works after production and media spending are committed. A campaign can take weeks to develop, then reveal three weeks later that the tagline confused buyers or the call to action missed. Traditional message-testing studies are often planned at 4-6 weeks and $15,000-40,000 per study. Those are example planning ranges, not current Subconscious pricing or delivery commitments. Decision-specific simulation creates an earlier comparison point. [figure: Four columns, one per method. A/B test spends real traffic. Focus group is eight people swayed by conformity. Survey rates preference without the mechanism. Simulation compares versions fast, as signal not proof.] ## Start with the behavior Define the action the message should change: attention, comprehension, preference, sign-up, or purchase. Compare concrete alternatives under the same audience and stimulus conditions. ### Taglines and ad copy Ask what each version communicates, who it appears to address, and what is confusing. A conversational response can explain why a phrase succeeds or fails, but it does not establish a conversion rate. ### Subject lines Compare a subject line across ten audience definitions and inspect differences. The result is a hypothesis for a real email experiment, not proof that someone will open. ### Campaign concepts A week before launch, test the core idea, emotional angle, and call to action while changes are still possible. A planning sprint might compare five emotional angles, then take the strongest alternatives into human validation. ## Why common methods answer different questions An in-market A/B test can show that Version B performed 15% better than Version A. It measures live behavior but spends real traffic or media to learn. A focus group can explore reactions from eight people in one session. Group dynamics can distort what participants say, as Solomon Asch's conformity experiments documented: people will give an answer they know is wrong to match a group. A survey can report a rating on a 1-5 scale. It may quantify preference without explaining the mechanism. A result such as 62% calling Tagline A “appealing” still needs interpretation and a link to behavior. [figure: Four-step path: compare many message versions fast by simulation, narrow to strongest candidates, validate the shortlist with human research or a live test, then launch the confirmed winner.] ## Use iteration without claiming certainty A team can compare Version 1, revise, compare Version 2, then compare Version 3 in a single afternoon. Testing ten versions in a week contrasts with two versions in two months. Speed makes exploration broader, but it does not make simulated reactions equivalent to click-through or conversion data. Use causal behavioral experiments to compare messages before launch. Use human research and live experiments to validate the winner, or book a walkthrough to test a specific message. Simulated intent is a signal, not a guarantee. --- # Why a Single-Number Forecast Hides the Decision You're Actually Making URL: https://subconscious.ai/blog/methods-and-validation/why-single-number-forecast-hides-decision-youre-actually-making Published: 2026-08-13 A demand forecast that returns one number is answering a question nobody asked. Inventory, pricing, and budget decisions depend on a range of plausible outcomes and how likely each one is, not a single point that erases the uncertainty around it. Probabilistic forecasting methods model the full outcome distribution instead, and five common data problems show why that distinction changes the decision. [figure: Five data problems hidden behind a single-number forecast, each linked to the probabilistic technique that recovers the true signal.] ## Why does sparse per-region data understate the trend? When a business forecasts hundreds of related series at once, such as sales by state, region, and product category, the smaller groups often carry too little history to estimate a trend on their own. Fitting each series independently treats that sparsity as noise rather than as a signal that can be borrowed from related series. Hierarchical models address this by sharing information across the group structure, a regularizing effect known as shrinkage. In a tourism-demand example spanning 308 combinations of state, region, and travel purpose from 1998 to 2016, an independently fit model missed a mild trend component in the smaller regions that a hierarchical model recovered from the shared structure (Forecasting: Principles and Practice). The result is not automatic: strong single-series baselines are hard to beat, so establishing one before layering in hierarchy remains good practice. ## Why do stockout-censored sales understate true demand? Retail sales data only records demand while a product is in stock. Once inventory runs out, the recorded sales figure is capped, or censored, at whatever was available to sell, not the true unconstrained demand for the rest of that period. A model that treats observed sales as the complete signal will systematically underestimate future demand, because it never learns what would have sold absent the stockout. A censored-likelihood model treats observed sales as a lower bound rather than the full truth, which lets it reconstruct the unconstrained demand curve through periods with stockouts or capacity limits, keeping undercounted demand from compounding into the next reorder. ## Why do unavailability-driven zeros get mistaken for no demand? Most items in a retail catalog do not sell every day. For products with sporadic demand, a standard intermittent-demand model such as Teunter-Syntetos-Babai (TSB) cannot tell the difference between a true zero (no customer wanted it) and a zero caused by the product being unavailable to buy. Extending the model with an availability mask corrects this: it keeps the estimated probability of demand from collapsing just because recent days happened to be out of stock. A product with two consecutive zero-sales days due to unavailability still gets a non-zero forecast once availability returns, instead of the more sharply depressed forecast a standard model would produce by treating unavailability zeros as true zeros. ## Missing extreme-case data leaves edge cases unconstrained Historical data rarely covers every condition a forecast needs to handle. An electricity-demand model built only on observed temperatures, for example, has little information about how demand behaves at extremes the data barely touched. Calibrating the model with domain knowledge, such as the physical expectation that demand rises sharply above roughly 32°C because of air-conditioning load, constrains the estimated relationship in those thin-data regions instead of leaving it to the sparse extreme observations alone. The corrected model shows demand increasing at both ends of the temperature range, consistent with heating and cooling effects outside roughly 15°C to 25°C. The same logic extends to price elasticity: with more than 5,000 SKUs across roughly 200 categories, individual products can have too little price variation to estimate elasticity reliably on their own, and a hierarchical model reduces the variance of those estimates by pooling toward category-level values, though it does not correct the price-endogeneity bias that can produce implausible positive elasticities in noisy independent fits. ## Opaque trend and seasonality resist explanation A forecast that is accurate but unexplainable is a hard sell to a stakeholder who has to act on it. State-space models decompose a series into separate, interpretable components, such as trend, seasonality, and the effect of external regressors, so a team can see which component is driving a change rather than treating the whole model as a black box. ## Where this connects to the decision after the forecast These are forecasting techniques for characterizing uncertainty in a time series or a price-response curve. Subconscious.ai does not perform demand forecasting, elasticity fitting, or state-space modeling. The shared idea worth carrying over is the discipline itself: reject a single confident number in favor of a distribution and a stated uncertainty range before committing budget. Subconscious applies that discipline to a different question: not what future demand will be, but which product, pricing, or messaging action to take next. It runs controlled experiments on a simulated market and reports the causal effect within that simulated population, with a confidence interval that reflects estimation uncertainty rather than simulator-to-human transfer error, benchmarked against a public replication study. Current accuracy figures live on the research and leaderboard pages rather than being restated here, since those numbers change as the benchmark is rerun. ## Limitations The hierarchical shrinkage, censored likelihoods, availability-aware intermittent models, calibrated Gaussian processes, and state-space methods above quantify uncertainty in what will happen. They can support counterfactual questions about a specific action only under untestable exogeneity assumptions; a randomized choice experiment gets there by design instead. [figure: Comparison diagram: capped sales feed two models. One reads the cap as complete demand and understates the forecast. The other reads it as a lower bound and reconstructs demand beneath the stockout.] ## Moving from a simulated test to real-human validation When a decision is large enough to warrant it, a team can move from a simulated experiment to real-human validation without changing the underlying causal question being tested. See how we work for what that validation step involves, or book a walkthrough to discuss a specific forecasting-to-decision handoff. --- # How to define the population for a discrete-choice experiment URL: https://subconscious.ai/blog/implementation-and-procurement/how-define-population-discrete-choice-experiment Published: 2026-08-13 ## The decision this step makes Once a research or insights lead has a causal question, the next decision is who the study represents. Discrete-choice and conjoint-style experiments compare how people trade off attributes, and the answer only means something for the population the study was actually run against (Sawtooth Software, What Is a Conjoint Analysis). A number without its limits is marketing copy. Get the population wrong and the preference results won't map back to the real buyers the decision was supposed to inform. ## Why does population matter more than sample size? Cross-tabulating population traits against which attributes matter to which respondents is what makes a study useful, because different segments trade off price, features, and framing differently (Qualtrics, Conjoint Analysis Technical Overview). This failure mode is published here so a buyer can check for it before trusting the result. A population defined too broadly averages those differences away, and one defined on the wrong traits measures preferences that don't belong to the buyers the recommendation is about. ## What traits define the population? For a synthetic-respondent study, defining the population means selecting the traits (profession, income, age, and similar characteristics) a respondent needs before they're relevant to the comparison. Between 2 and 10 traits is a workable range. Fewer than that leaves the population too broad to be distinct from the general public. More than that starts to over-specify a niche population the real market doesn't resemble. Traits should follow from the decision, not from a list of available fields. Three questions narrow the choice: - Who actually makes or influences this decision? If the study is about a purchase, the relevant trait might be role or buying authority, not just demographics. - Which traits would plausibly change the preference being tested? A trait only belongs in the population if respondents who have it would answer differently than respondents who don't. - Would a domain expert recognize this group as the real buyer? A population that looks reasonable on paper but doesn't match anyone a sales or product team actually talks to won't generalize. ## Reviewing traits before committing the design Target audience and buyer or segment definitions are supported inputs to a Subconscious causal action test: a team specifies population traits, and Subconscious runs the comparison across actions for that population. [figure: Decision path: select 2-10 traits, test each against three questions, review the traits against real buyers, then carry the population into the causal comparison.] ## What this step doesn't decide Trait selection defines a simulated population for a synthetic-respondent study. This limit sits on the record next to what the step does do. It does not field a recruited real-human sample: when a decision needs recruited real participants, the same traits guide who to recruit but don't substitute for fielding them. A team can move from a simulated experiment to real-human validation without changing the underlying causal question, but that step is separate from defining the population here. [figure: Two columns: simulated population feeds the causal action test directly; recruited real-human sample uses the same traits only as a recruiting brief. An arrow marked "same causal question" links them.] ## Where to go from here With traits named and reviewed, the next design decisions are what's being compared and how results get interpreted. Research covers how Subconscious structures causal action tests once a population is defined, and How we work walks through the process end to end. Teams ready to scope a specific population can book time to talk it through. --- # When to Simulate First: Seven Decisions That Need a Triage Pass Before Real-Human Research URL: https://subconscious.ai/blog/case-studies/when-simulate-first-seven-decisions-need-triage-pass-before-real-human Published: 2026-08-13 A marketing, product, or growth leader rarely gets to test every option with real people. Budget and time force a cut before the research starts. Which options get tested, and who decides, is the question worth asking before that cut. Fielding full human research on every candidate (every name, price structure, audience segment) burns weeks and budget on options a faster pass would have eliminated. Skipping validation and shipping on a hunch risks a mispriced package, an alienating pricing change, or a message aimed at the wrong buyer. Neither failure mode is a methodology footnote; it is a resourcing decision made with incomplete information. A middle path: run a controlled synthetic experiment first to cull a large option set to a short list, then commit real-human validation budget only to that short list. The pattern shows up across decisions a growth or product team makes in a normal week. The seven below describe the class of decision each represents, not any single vendor's results. ## Seven decisions that benefit from a triage pass Naming and positioning for a launch. A team with more candidate names and angles than it can afford to test chooses which few reach a full study. A fast simulated pass, followed by a focused human study on the top performers, concentrates research spend on the options worth it. Mapping objections across a buying committee. Recruiting a full B2B buying committee (CTO, CISO, budget owner, procurement) for a traditional panel is slow and expensive by role. Simulating role-specific reactions to a discovery, demo, and pricing conversation surfaces likely objections fast enough to prepare sales-engineering responses before the next real deal, then check those objections against actual win/loss conversations. Choosing among several pricing structures. Moving pricing models is a one-way door for existing customers. Testing multiple structures across segments in simulation narrows the field before committing a real pilot cohort to the strongest candidate. Localizing a message across markets. Testing every message variant in every target market at a meaningful sample size is one of the most expensive research workflows, because cost multiplies by market. A simulated pass across markets and variants flags which messages carry and which trip on culture-specific friction before the paid campaign spends against them. Pressure-testing a feature the team is already sure about. Confidence built from inbound feedback is a biased sample: whoever bothers to file a complaint is not a stand-in for the quiet segments who stop using a product without saying why. Walking simulated segments through a feature spec before launch surfaces workflow-disruption or trust concerns that inbound channels miss. Tracking brand perception between the years a full wave affords. A traditional brand tracker often runs annually because of its cost. A lighter simulated read run more often can catch a perception shift or a competitor-driven swing early enough to act, with the traditional wave serving as the periodic ground-truth check. Narrowing an ideal customer profile with no research budget. A seed-stage team choosing among candidate ICPs without budget for customer research can walk simulated respondents from each candidate profile through the same sales narrative and compare intent, objections, and willingness to pay before committing scarce outbound effort. ## What do these seven decisions have in common? None of these seven replaces the final validation step. Each operates at the triage layer: a fast, broad pass that culls a long list, surfaces likely objections, or points a directional thesis before anyone commits the larger budget. The decision is not "simulation or humans" but which options get pushed to the top of the queue for human validation. ## Where does Subconscious fit in this process? Subconscious runs controlled discrete-choice experiments to estimate causal effects across pricing structures, messages, positioning angles, and audience segments. A fit claim without its boundaries is a sales pitch, so this article points to the study that draws them. Independent research on when digital personas reliably approximate human survey findings finds that the fit varies by domain and question type, which a vendor's own case narrative cannot establish alone. A separate uncertainty-quantification study asks how many human respondents a language model's simulated response is worth for a given estimate, a way to reason about where simulation substitutes for humans and where it does not. Subconscious can test or validate studies with real human participants. The advantage: a team can move from a simulated experiment to real-human validation on the same causal question, without redesigning the study or waiting on a separate vendor relationship to catch up. ## What does a triage pass not prove? Simulated screening narrows options. It does not replace validation for a final, high-stakes decision, and no single accuracy number travels across use cases. Naming where a study's participants actually come from is what lets a buyer check the claim. Three things stay distinct in this pattern, and blending them is where a team gets misled: the audience a platform can reach, the participants in a simulated experiment, and the participants recruited for real-human validation. A large reachable audience is not a validated finding, and a simulated panel is not a recruited human sample. ## Next step A team weighing this sequence for its own decision, whether a launch name, a pricing change, or an ICP bet, can see how the causal effects methodology works, review other case examples, or read how a study moves from simulation to human validation. For a specific option set, booking a working session is the fastest way to find out. [figure: Four-step path: a large option set feeds a simulated triage pass, narrowing to a shortlist, which goes to human validation. Notes mark two failure modes: testing everything with humans, and skipping validation.] --- # How Market Researchers Become Strategic Advisors URL: https://subconscious.ai/blog/buyers/how-market-researchers-become-strategic-advisors Published: 2026-08-13 Market researchers become strategic advisors by owning the decision, not only the deliverable. They define the action under consideration, the alternatives, the relevant population, the evidence threshold, and the uncertainty that remains. A capability claim printed without its limits functions as marketing. Automated drafting, summarization, and open-ended model output can support exploration. They can also produce a fluent answer before the underlying decision is clear. Presenting that answer as validated evidence can damage the researcher's credibility and direct budget toward the wrong action. ## Strategic Value Moves Closer to the Decision In 2026, production skill still matters. Strategic value sits in choosing which evidence a decision requires and explaining why that evidence deserves trust. Federal labor projections through 2034 show continued growth in market research analyst and marketing specialist jobs, according to the U.S. Bureau of Labor Statistics (U.S. Bureau of Labor Statistics). The pressure is on how the role is performed, not on whether it exists. The researcher should answer these questions before selecting a method: - What action will change because of this work? - Which alternatives are actually under consideration? - Which buyer or market population does the decision depend on? - What evidence would justify acting, and what would still require validation? The researcher is not defending a process from automation. The researcher is preventing the business from confusing a plausible output with evidence for a specific action. ## Choose the Right Read for Each Stage The method should match the decision and the cost of choosing poorly. Subconscious fits the middle stage. Its research approach uses controlled, discrete-choice-style experiments to compare defined alternatives across a defined population. A study can estimate causal differences and report a confidence interval when its design supports one. This gives the researcher a defensible layer between open-ended exploration and fielded human research. One transition matters when the decision requires stronger validation: the team can move from a simulated experiment to real-human validation without changing the causal question. The working method keeps the alternatives and outcome aligned while the evidence source changes. Audience definition for a simulated experiment is not participant recruitment; recruited people belong to the real-human validation stage. ## What Is an Evidence Contract? A planning framework has four layers: exploration, controlled causal testing, human review, and validation. A strategic researcher makes the evidence boundary visible across all four before presenting a recommendation. The contract can be short: - Decision: State the action the executive will approve, reject, or revise. - Alternatives: Name the options compared. Open-ended themes are not alternatives. - Population: Define the buyers or market participants whose behavior matters. - Method: State whether the work is exploration, a controlled causal experiment, or real-human validation. - Estimate: Report the causal comparison and uncertainty only to the extent supported by the study design. - Limit: Name what the work does not establish and what evidence should come next. Instead of presenting a collection of findings, the researcher presents an action, the evidence for choosing it, and the conditions under which the recommendation could fail. ## Why Should Method Boundaries Stay Visible? A method's failure modes belong on the record next to its successes, where a buyer can check both. A controlled causal experiment does not replace the researcher's judgment, stakeholder framing, fielded human research, or observation of real market behavior. It does not guarantee revenue, adoption, or market performance. It does not make someone a strategic advisor by itself. The human skill is deciding which question deserves a controlled test, defining meaningful alternatives, identifying the relevant population, and refusing to present an exploratory output as conclusive evidence. The method strengthens that judgment. It does not substitute for it. ## Present the Recommendation as a Decision An executive-ready research recommendation should answer: - Which action should the business consider? - What alternative did the test compare it with? - Which population and outcome does the estimate cover? - How uncertain is the estimate? - Which assumptions or failure conditions matter? - What validation is required before an expensive or public commitment? These questions keep the recommendation connected to behavior and make scrutiny useful. A stakeholder can challenge the population, alternative, outcome, or evidence threshold instead of debating a polished narrative. ## What to Do This Week Start with one visible project that has a live decision behind it. - Write the business decision in one sentence. - Define the alternatives and the population. - Use open-ended exploration to frame the question, not to settle it. - Use a controlled causal experiment when the alternatives and cost of choosing poorly are clear. - Present the recommendation with its uncertainty, limits, and next validation step. Add a decision-impact line to the next report before adding another data table. Review one decision-impact line once a week for a month. The point is to build a repeatable evidence practice, not to promise a delivery schedule. ## Where Does This Leave the Researcher's Role? Naming what a method cannot decide is what lets a buyer trust what it can decide. Automation changes who can produce a draft, summary, or plausible answer. It does not decide which action the business should take or which evidence justifies that action. The strategic advisor owns that boundary. They connect the buyer decision to the right method, state what the evidence supports, and name what still needs validation. Talk to Subconscious about where a controlled causal experiment fits in a research workflow, or review past studies. [figure: Four-step path: frame the decision, then exploration for undefined problems, a controlled experiment once alternatives and population are named, and real-human validation for expensive or public decisions.] The evidence boundary should be visible at every stage, not just in the final deck. --- # AI Consumer Insights for Faster Customer Decisions URL: https://subconscious.ai/blog/use-cases/ai-consumer-insights-faster-customer-decisions Published: 2026-08-13 Consumer insight should shape product, marketing, and brand decisions. Traditional research arrives in weeks while product and campaign choices happen daily. AI can shorten exploratory work to minutes, but speed does not remove the need for calibration or validation. [figure: Four connected steps left to right: define the audience by role and context, run a focused session on one action, compare responses across audience segments, then synthesize a hypothesis with assumptions attached.] ## What AI consumer insights include Methods include: - Synthetic audience research that probes how a defined customer type may respond. - Automated analysis of interviews, support tickets, reviews, social comments, and feedback. - Predictive models that estimate how segments may react to a product, price, or event. - Open conversation that explores the reasoning behind an attitude or preference. These methods answer different questions. Synthesis finds patterns in existing evidence. Simulation generates hypotheses. A causal experiment compares actions. ## Why does the traditional research cycle create an insight gap? A typical study moves through three stages: about two weeks to write the brief and lock the design, then two to four more weeks to recruit participants and run fieldwork, then one to two weeks to analyze results and write the report. The decision can pass before the report arrives. A well-designed qualitative study may cost $15,000 to $40,000. A serious quantitative study can cost more. A mid-market team may fund only a handful of studies per year, leaving many decisions without direct customer evidence. AI-assisted exploration can help a team arrive at better questions before paying for validation. It should not be used to label an unsupported prediction as customer truth. ## How the workflow works ### Define the audience Specify the role, industry, company size, behavior, motivation, and category context relevant to the decision. Broad demographic labels rarely produce useful distinctions. ### Run a focused session Present a product, message, price, or go-to-market action. Ask follow-up questions about the tradeoff and expected behavior. Avoid a generic request for opinions. ### Compare audience segments Run the same protocol across several customer types. Record where responses align and where they diverge. ### Synthesize a decision Translate the result into a clear hypothesis, experiment, or human-research question. Keep the assumptions attached. ## What should you look for in a platform? Persona specificity matters because the audience definition constrains the result. Conversational depth matters when the team needs an explanation rather than a score. The platform should fit the range of research jobs the team runs, expose enough method detail for review, protect approved customer context, and distinguish simulated evidence from observed customer data. Self-serve access can reduce operational friction, but it does not make every employee a trained researcher. High-stakes studies still need careful design. ## What are the common uses? Product teams can compare concepts before development and identify objections or missing requirements. Brand teams can study how different audience types describe a category. Marketing teams can compare messages and creative directions. Competitive work can model how an alternative's customers may evaluate a switch, using public evidence as a constraint. Market-entry work can explore a new geography or segment before commissioning local fieldwork. Customer-experience teams can map friction across a journey. [figure: Two tracks compared: top is a traditional study over five to eight weeks ending in a validated result; bottom is an AI-assisted session of about one hour ending in a hypothesis with assumptions, not a final answer.] ## Start with a live decision An example self-serve session can produce an initial result within an hour, including 30 minutes of focused conversation. Treat those durations as planning examples, not delivery guarantees. Choose the customer type closest to a decision already on the calendar. Define the alternatives and outcome. Use the simulation to narrow the question, then use observed behavior or human research to validate the choice. Subconscious supports decision-specific causal experiments on product, pricing, messaging, and go-to-market actions. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, documented in the causal fidelity paper. See how the process runs end to end. It should not be described as automatically replacing every qualitative or quantitative research method. --- # Testing Healthcare Launch Decisions Before the Qualitative Study Starts URL: https://subconscious.ai/blog/industries/testing-healthcare-launch-decisions-before-qualitative-study-starts Published: 2026-08-13 A medtech or pharma commercial team planning a device or program launch usually has to lock positioning, HCP messaging, or a patient support program design before qualitative research can confirm any of it. IRB approval alone can take months to clear, and patient, HCP, and caregiver recruitment is a separate bottleneck. Causal action testing gives that team a way to compare positioning or messaging alternatives against defined patient, HCP, and caregiver populations before the traditional research cycle finishes. ## Why is healthcare research access the real bottleneck? Traditional healthcare research is slow for structural reasons, not process failures. Finding patients who match a specific condition, then persuading enough to enroll, costs real time and budget. HCP advisory boards are difficult to convene on a launch timeline. Payer committees are unreachable outside formal submission cycles. Caregivers are worn down by the caregiving role itself and rarely respond to research invitations. The practical result: most launch positioning, patient journey design, and HCP messaging decisions get made on outdated qualitative data or internal assumptions, because rigorous research arrives after the decision is locked. ## What can a causal test compare before qualitative research happens? Causal action testing reports which alternative is more likely to change engagement or preference, tested against defined patient, HCP, or caregiver populations. Caregivers in particular are chronically underrepresented in healthcare research, which is one reason their perspective is often missing from launch decisions entirely. ## A device launch example A medtech company launching a new glucose monitoring device must know, before launch: what patients think about giving up the device they already use, which features actually drive that decision versus which ones patients would shrug off, whether endocrinologists find the clinical data package convincing enough to prescribe, and whether the onboarding flow makes sense to someone who isn't a diabetes educator. A traditional qualitative research cycle for questions like these runs for months, involves multiple vendor relationships, and returns results after the key decisions are locked. Running a causal test first, comparing patient personas across diagnosis stage, device familiarity, and age alongside HCP personas across prescribing style, produces a ranked read on messaging and positioning before that cycle completes. That output does not replace the qualitative study that follows. It sharpens which questions that study needs to answer, so the real qualitative study starts pointed at the open question rather than the obvious one. ## What this does not do Three limits matter more than any efficiency claim: - It is not a substitute for clinical evidence. It does not generate clinical evidence, and no regulator or promotional reviewer should see it framed that way. - It does not use real patient data. No PHI is required to define a patient population for a causal test. The population is built from stated attributes like diagnosis stage, treatment history, and demographic factors, not from identified patient records. - It is for commercial strategy, not regulatory submission. Messaging, positioning, and program design are in scope. Anything destined for a regulatory dossier, an IRB-governed protocol, or a payer submission runs through the qualifying human process for that decision. ## Where this fits in a launch research plan The highest-value use isn't a one-off pre-launch check. Teams get the most out of causal testing for healthcare launches when they run it before a qualitative study to sharpen its question set, between studies to test a new hypothesis without restarting the full research process, and after a study to check edge cases or adjacent segments the original study didn't have time to cover. In every case, the causal test moves a team from a locked assumption to a testable question, and the qualitative or clinical study that follows has the final word before a demo or launch commitment gets made. [figure: Five-stage horizontal path: locked launch assumption, causal test comparing patient, HCP, and caregiver personas, ranked read on messaging, qualitative or clinical study, demo or launch commitment.] --- # 8 Procurement Gates for Enterprise AI Audience-Simulation Pilots in 2026 URL: https://subconscious.ai/blog/use-cases/8-procurement-gates-enterprise-ai-audience-simulation-pilots-2026 Published: 2026-08-13 Before an enterprise team lets any AI audience-simulation vendor near a pilot, it should require written answers on eight points, in this order: data residency, a Data Processing Agreement, SSO and role-based access control, an audit trail, vendor stability, evidence behind any accuracy claim, a contract structure built for enterprise use, and a pilot with a benchmark set before results exist. A promising method does not compensate for a failed security or legal review. Discovering a gap after the contract is signed forces a new procurement cycle, stalls the pilot, and costs credibility with the teams that planned around it. [figure: A five-step path from data residency through a Data Processing Agreement, SSO and audit trail, vendor stability and accuracy evidence, ending at a pilot with its benchmark written down before the vendor runs a result.] ## The procurement filter, in order Most vendors are eliminated on procurement mechanics before anyone evaluates research quality. Work through these eight gates in sequence rather than jumping to a demo. ### 1. Data residency Ask where the platform stores study prompts, audience definitions, results, logs, and backups, and which regions and subprocessors touch that data. EU residency is a hard requirement for most buyers under European data-protection law, and a vendor that cannot answer clearly should not advance. A GDPR compliance guide for SaaS vendors walks through the residency and transfer questions procurement teams are expected to ask. ### 2. A Data Processing Agreement on day one A standard DPA should be available for review without a multi-week legal back-and-forth. It should name controller and processor roles, retention and deletion terms, subprocessors, and incident-notification duties. A SaaS compliance checklist lists the DPA terms that belong in this review before signature, per Article 28 and Article 32 of the GDPR. ### 3. SSO and role-based access control Single sign-on against the buyer's identity provider and role-based access across teams are not optional at enterprise scale. Test provisioning, de-provisioning, and least-privilege roles directly in the vendor's environment rather than accepting a feature name on a slide. ### What should an audit trail include? Every study configuration, every persona or audience definition, and every result should be logged and exportable. Research operations and compliance need a reviewable chain from study setup to result, not a promise that logging exists somewhere in the product. ### 5. Vendor stability Ask who owns support, security incident response, business continuity, and data export if the relationship ends. Team size and funding history matter only as inputs to that question, not as a standalone score. A small team can still pass this gate with a documented plan for key-person risk, support coverage, and an exit path. ### What evidence backs an accuracy claim? An accuracy number is only useful once the vendor states what it predicted, against what comparison, and where the method fails. Ask whether the claim describes an aggregate pattern, the direction of an effect, or individual-level behavior, and ask for the benchmark population and holdout procedure that produced it. ### What contract structure fits enterprise use? Pricing and contract terms should match enterprise procurement, not a self-serve plan relabeled for a larger buyer. Confirm the agreement covers the security and legal terms already reviewed in gates one through four, not just seats or usage volume. ### 8. A pilot with the benchmark set in advance The pilot should run against a business decision the buyer already owns, with the alternatives, target population, outcome measure, and a defined failure condition written down before the vendor produces a result. This checklist puts a pilot's misses on record next to its hits. A retrospective example can illustrate the method. It cannot substitute for a pre-defined benchmark, and most failed pilots fail on workflow fit rather than on accuracy. ## Keep audience reach and recruited participants distinct A platform's stated reach describes the scale of the simulated population it can draw on, not a pool of people available for interview. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people; that describes simulation scale, not a recruitable panel. When a decision calls for it, Subconscious can also test or validate a study with real human participants, without changing the causal question the study was built to answer. Buyers evaluating any vendor's reach claim should ask the same question: does this number describe simulation scale, recruited participants, or both. ## Hold the causal question stable through validation A pilot is easiest to defend when the team names the action, the alternative it is compared against, the buyer or market population, and the decision outcome before any result exists. If a later real-human study changes any of those four elements, it answers a different question rather than confirming the first one. Subconscious frames its fit as a causal behavioral platform: it structures the experiment around the action and its alternatives, then lets a team move from a simulated experiment to real-human validation without re-defining the question. Buyers can review the research and validation approach and how a study moves through the process before deciding whether that structure fits their pilot. ## What this checklist does not prove Naming a failure mode here is what lets a buyer go check it directly. Passing these eight gates does not confirm that a vendor's DPA, SSO, RBAC, or audit trail work correctly in the buyer's environment, and it replaces neither legal sign-off nor implementation testing. A simulated experiment is not automatic proof of market performance, and aggregate agreement in a benchmark does not establish accuracy at the individual level. Real-human validation does not turn a causal action test into a usability session, a clinical trial, or a field result; the conclusion stays bounded by the audience, alternatives, outcome, and benchmark used in that specific study. [figure: Two columns compare a reach claim. Left: simulation scale, the size of a simulated population for a study. Right: recruited participants, real people available for interview or validation.] ## Bring procurement and research into one evaluation Run procurement and research evaluation together rather than in sequence: the decision, target population, alternatives, outcome measure, and benchmark belong in the same review as the data-residency, DPA, SSO, and audit-trail questions. Once both sides are answered, discuss a specific pilot with Subconscious. --- # Is Your Marketing Model's Answer Data-Driven, or Just Your Priors Talking Back? URL: https://subconscious.ai/blog/methods-and-validation/marketing-models-answer-data-driven-just-priors-talking-back Published: 2026-08-13 A statistical model hands a marketing team a channel-attribution number, and the team has to decide: act on it now, or check it first. A model can produce a confident-looking estimate that is really just an echo of its own assumptions. ## The decision this creates Before shifting spend across channels on a model output, a measurement lead has to answer one question: is this estimate supported by the data, or mostly a reflection of the priors that went into the model? Reallocating budget on a prior-driven estimate, or one a handful of outlier weeks distorted, costs real money in the wrong channels. ## What does this check look like in a real marketing-mix project? One documented Bayesian media-mix-modeling engagement illustrates the check. The setup: a model estimating how paid channels convert into new customers, refit repeatedly as new spend and outcome data arrived. Two changes addressed a common failure mode in that setup, where a small number of unusually large observations were pulling the fitted curve away from the typical pattern: - The outcome was modeled on a log scale instead of a raw count, so the fit responded to proportional changes rather than raw spikes. - A Student-T likelihood replaced a Normal likelihood, because a Student-T's heavier tails discount extreme points instead of letting them dominate the fit. (Setting Priors in Bayesian Marketing Mix Modeling (MMM)) Neither change is specific to marketing data; both are standard responses to outlier-sensitive likelihoods in Bayesian modeling generally. ## How was the reach function rewritten so people can read it? The project also rewrote the function describing how a channel's reach saturates as spend increases, in terms two marketers could argue about directly: the number of users at saturation, and the initial cost to reach one user. Here x is spend, β is the saturation user count, and c_0 is the initial cost per user. Written this way, a domain expert can set a reasonable prior on β or c_0 from what they know about the channel, instead of guessing at an abstract curve-fitting parameter. Priors are only useful when a human can say whether they are plausible. (Understanding Bayesian Marketing Mix Modeling: Prior Specifications) ## What is the actual check, and how do you run it under different priors? Readable priors make the next step possible: refit the model under a range of plausible prior settings and watch which parameter estimates hold steady and which ones swing. - An estimate that stays roughly put across reasonable prior choices means the likelihood is dominating the prior, not that the estimate is free of confounding: MMM spend is chosen, not randomized, and can still track seasonality, demand shocks, or promo calendars. - An estimate that swings whenever the prior changes is telling you mostly about the prior, not yet about customer behavior. A model output that has not been run through this check is not yet a decision-ready number, no matter how precise it looks in the first pass. Passing the check is necessary, not sufficient: reallocation decisions hinge on the marginal response at current spend, which is more sensitive to the adstock and saturation functional form than the fitted curve itself. [figure: A causal chain diagram showing a raw model estimate passing through a sensitivity check, splitting into a data-driven signal or a prior-driven signal, with only the data-driven signal reaching a decision-ready estimate.] ## Where this fits with causal testing Subconscious does not build or sell media mix models, adstock functions, or marketing-channel attribution infrastructure. It tests a specific decision, such as a price, a message, or a launch action, through a randomized causal experiment rather than an econometric attribution model, and it does not replace channel-level MMM work. The discipline in the check above still applies. Causal experiments produce an estimated effect with a confidence interval, and that estimate should also survive scrutiny, not get treated as ground truth on the first run. Subconscious can test or validate studies with real human participants, which lets a team move from a simulated experiment to real-human validation without changing the causal question being asked. See the research behind that approach. ## Before the next reallocation Before a model's channel or budget number changes real spend: has anyone checked whether it holds up under different reasonable assumptions, or is it running on its first pass? If not, that check is the next step, not the budget change itself. --- # Quantify the Uncertainty Before You Pick a Risk Policy URL: https://subconscious.ai/blog/methods-and-validation/quantify-uncertainty-before-pick-risk-policy Published: 2026-08-13 A planner who commits capacity, budget, or inventory against a single-number forecast is committing to a guess about the future. The forecast is usually close. The cost shows up when it isn't, and the plan has no built-in answer for how to absorb the miss. ## The forecast that hides a decision A point forecast (one number for expected demand) throws away the shape of the uncertainty around it. A Bayesian model instead produces a full distribution of plausible outcomes: a range of scenarios, each with a probability attached, rather than one estimate treated as fact. PyMC is one open-source tool built for this: it samples a distribution of parameters and forecasts, using an MCMC algorithm to explore the range of outcomes consistent with the data. That distribution only pays off once it feeds a decision. A worked example: a planner forecasting daily demand for a resource over a multi-month horizon can generate thousands of posterior scenarios per period, then hand that full set of scenarios, not just a mean, into an optimization model. A commercial solver such as FICO Xpress is one tool built to take that scenario set and turn it into a generation or allocation plan under an explicit risk policy, rather than a plan built against a single expected value. ## Two ways to turn a distribution into a policy Once uncertainty is quantified, a planner still has to choose how to use it. Two approaches are common, and they answer different questions: - A chance-constrained policy caps how often the plan is allowed to fall short of demand. In the worked example, the plan is required to cover the 95th percentile of forecast demand each day, meaning the plan is undersized in roughly 5% of scenarios. - A conditional value-at-risk (CVaR) policy instead caps how severe the worst shortfalls are allowed to be. It works directly with the full scenario set, identifies the worst-case tail (the paper below uses the worst 5% of outcomes), and penalizes the average severity of shortfalls within that tail. The formulation used in this example is the Rockafellar-Uryasev model, published as "Optimization of Conditional Value-at-Risk" in The Journal of Risk (2000). It stays popular in practice because it is convex and often reducible to a linear program. Neither policy is more correct in the abstract. A regulator that requires demand to be met 99% of the time is describing a chance constraint. A planner reasoning about the dollar cost of a bad tail outcome is reasoning in CVaR terms. ## What does a stricter guarantee cost? Moving from one policy to the other has a price in the worked example, though the two runs also differ in risk stringency (a 5% chance constraint versus a 0.35% realized shortage), so the comparison doesn't isolate the cost of the CVaR formulation alone. Cutting shortage risk from 5% to 0.35% required roughly 8.5% more average generation and raised total cost by about 22%, though that delta reflects both the change in policy form and the tighter risk target together. In a related sweep of the same model, varying the risk weighting and a separate clean-energy policy from 80% to 90% of generation, tightening the policy floored the best achievable shortage rate at 5.36% and added roughly $440M in cost, because a stricter side constraint became the binding limit on how much tail risk could be removed at any price. [figure: Comparison: chance-constrained policy holds 5.00% shortage risk at baseline cost; CVaR policy cuts shortage risk to 0.35% but raises generation and cost by about 22%.] The tradeoff was visible before a dollar was committed, because the forecast carried its own uncertainty into the optimization step instead of being collapsed into one number first. ## Does this same pattern apply to go-to-market spending? The decision pattern generalizes past power generation: quantify the range of plausible outcomes for an action, then choose an explicit policy for how much risk that range implies, before committing budget. Subconscious applies that same discipline to product, pricing, messaging, and go-to-market decisions. It tests an action with causal experimentation and discrete-choice-style modeling, producing parameter uncertainty on choice probabilities and stated WTP conditional on the experimental design, attribute set, and respondent pool, rather than committing capital on a single-point estimate of buyer response or relying on generic synthetic-user roleplay. Those intervals aren't a calibrated predictive distribution over realized market outcomes: stated WTP runs high before accounting for hypothetical bias, and applying a chance-constraint or CVaR policy directly to them would understate the true uncertainty. That fit has a boundary. Subconscious does not provide mathematical optimization solvers, does not solve linear or mixed-integer programs, and does not run power-grid, supply-chain, or capacity-planning optimization. Teams that need the solver step still need a tool built for that job. How Subconscious works covers the part of the pipeline it does own. ## From a modeled test to a real-human check Subconscious can test or validate studies with real human participants. When a decision is close enough that the answer would change under real-world response, a team can move from a simulated test to real-human validation without changing the underlying causal question being asked. ## Where to start The worked example above is a demand-and-generation problem, but the underlying question, what does our forecast uncertainty actually cost us, and which risk policy fits our tolerance, applies to any budget commitment made before an outcome is known. Book time to walk through what quantifying that uncertainty would look like for a specific action under consideration. --- # What to Run Before a Qualtrics Survey When You Need the Why URL: https://subconscious.ai/blog/comparisons/what-run-before-qualtrics-survey-when-need-why Published: 2026-08-13 A CMO or VP of Consumer Insights fielding a large Qualtrics wave has one decision to make before launch: whether the concept, message, or driver about to be measured is the right one to test. Skipping that check risks a clean, statistically valid survey that answers a question the business didn't need answered. ## Why a survey-first sequence backfires Qualtrics Core XM is built for structured data collection: fixed questions, closed-ended or Likert responses, and large respondent counts. That makes it a strong choice for employee-experience surveys, customer-satisfaction tracking, and large academic-style studies. A tool's blind spots belong in the open, next to what it does well. Qualtrics Core XM also can't explain why a score moved. A "somewhat dissatisfied" rating flags a problem area. It doesn't reveal the underlying driver, the alternative the customer considered, or whether the issue is a deal-breaker or a minor annoyance. Committing budget and a multi-week fielding cycle before that driver is known risks discovering the concept or question set was misspecified only after the field window has closed. ## The exploratory step, and where it can go wrong The usual fix is an exploratory step before the quantitative wave: something that generates a hypothesis about why, so the survey can be built to confirm or reject it. Three approaches are common, and each answers a different question. Asynchronous qualitative and live-interview tools are strong at what they're built for: contextual habits, workflow understanding, and screen-level usability. The risk is asking either one a causal question they weren't designed to answer. What someone says about a concept in an interview isn't the same as which concept actually changes their choice. ## Where does a causal experiment fit before the field window opens? Subconscious is a causal behavioral platform: randomized experiments on a simulation of a target market, validated against real human behavior, test which concept, message, or action drives an outcome for a defined audience before a team commits to a full quantitative fielding plan. This is a controlled experiment design, not open-ended conversation with a generated character. The value is comparing alternatives against each other and estimating which one moves the outcome, so the resulting hypothesis gets built into the survey instrument instead of guessed at. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That audience graph describes reach, not an on-demand recruitable pool for a given study. When a finding needs confirmation with recruited participants, Subconscious can test or validate studies with real human participants without changing the underlying causal question, so the exploratory hypothesis and the quantitative confirmation stay comparable. ## What does a causal experiment not replace? Naming where a method stops is what lets a buyer check it against their own use case. A causal experiment on a simulated market is not a diary study or an asynchronous ethnographic tool, and it does not replace live moderated interview or usability-testing sessions for screen-level observation of an interface. It does not price, schedule, or guarantee delivery time. The exploratory step is decision-specific experiment design, not an automated optimizer. ## A practical sequence - Run a randomized experiment on the concepts, messages, or actions in question, on a simulated population matched to the target audience. - Use the result to decide which concept or driver deserves a full Qualtrics wave, and write the survey instrument around that hypothesis instead of an open-ended one. - Where the decision is high-stakes enough to need it, confirm the finding with real human participants before committing further budget. - Reserve moderated interviews or asynchronous qualitative platforms for the workflow- and screen-level questions a causal experiment isn't built to answer. Read how these experiments are designed, see how the process fits an existing research stack, or book time to walk through a specific decision. Related decision write-ups are collected in case evidence. [figure: Four-step path: run a randomized experiment on candidate concepts; pick the one that moves the outcome; confirm with real participants if high-stakes; field the Qualtrics survey around that confirmed hypothesis.] --- # When to Run a Simulated Study Before Recruiting Real Research Participants URL: https://subconscious.ai/blog/use-cases/when-run-simulated-study-before-recruiting-real-research-participants Published: 2026-08-13 A head of product or research lead sequencing a roadmap or design decision faces two bad defaults: skip any directional check and ship on an untested assumption, or recruit real participants for a question a faster experiment could have narrowed first. Run a controlled simulated study before recruiting when the goal is to sharpen the question, not to answer it. ## The cost of getting the sequence wrong Recruiting, scheduling, and running a study with real participants is slow. Teams that skip a directional check first often build a feature or flow around an assumption nobody tested, then find the problem after engineering and design time is spent. Teams that recruit before sharpening the question often spend that time on a broad or poorly framed study, then have to recruit again once the real question becomes clear. Neither failure is about research effort; both are about sequencing: which check happens first, and whether it changes what gets asked next. ## What can a simulated study do and not do? A simulated study runs a controlled experiment on a decision, using a modeled population instead of recruited humans. It can compare which of several concepts or flows changes stated intent for a target segment, and surface where that response is likely to diverge across segments. What people say they would do and what they actually do under real conditions are different questions; a simulated study answers the first one. Independent reviewers of AI-generated research participants draw the same line: participant simulation can approximate a stated response, but it does not substitute for watching how someone actually uses a product (Nielsen Norman Group, "Synthetic Users: AI 'Participants'"). ## Where each method fits Running a simulated study first does not remove the need for the other two methods. It changes what the team asks once real participants are in the room. ## A sequence that keeps the causal question intact - Define the decision: which action or flow, for which target segment, is being compared. - Run a controlled experiment on that decision in a simulation before recruiting anyone. - Use the result to sharpen, not replace, the study plan for real participants: which segments to prioritize, which alternatives are worth a moderated session, and which questions are already answered. - Move the same causal question into real-human validation. Subconscious tests or validates studies with real participants, so the team carries the decision forward instead of redesigning the study from scratch. This sequence matters most when the target segment is expensive or slow to reach: a narrow B2B buyer role, a regulated population, or a segment spread across geographies. In those cases, a controlled study against a person-level audience graph covering 800 million real people can narrow the question before a recruiting budget is spent. ## What are a simulated study's limitations and failure conditions? A simulated study estimates causal effects on stated choices and intent; it is not a substitute for usability testing or ethnographic interviews. Treat results as directional until validated with real participants, especially where the gap between what someone says and what they do is the entire risk being managed. Real-human validation confirms or corrects the simulated answer; it does not turn a stated-intent test into an observed usability session, a clinical trial, or automatic proof of market performance. ## Practical next step If the target segment is reachable and the decision is narrow, a moderated study alone may be enough. If the segment is hard to reach, the question is still broad, or the team cannot afford to recruit twice, run the controlled simulated study first, then carry the same question into real-human validation. Teams weighing this sequence against a specific roadmap or launch decision can review prior case evidence, read the research methodology, or book time to walk through a live decision. [figure: Four-step sequence: define the decision and segment, run a controlled simulated study, sharpen which segments and alternatives need a real session, then validate with real participants.] --- # How to Test a Pricing Decision Before You Commit to It URL: https://subconscious.ai/blog/implementation-and-procurement/how-test-pricing-decision-before-commit Published: 2026-08-13 Set product pricing by testing a defined choice, not by searching for one perfect number. Compare concrete price points, packaging structures, or free-tier framing for one buyer segment. Measure which action changes stated purchase intent or preference. Then decide what deserves real-market validation. Setting or resetting a first real price is expensive to reverse. Underpricing can cap revenue per account and signal low value to enterprise buyers. Overpricing can stop conversion before the sales and marketing motion pays off. Existing customers also notice price changes, so a poor entry price can compound instead of correcting itself. McKinsey found that a 1% improvement in price can produce a roughly 6-11% improvement in operating profit, depending on the study cohort (McKinsey & Company, "The power of pricing"). An older planning heuristic makes the same point more bluntly: moving price by 10% does more for the business than moving customer acquisition by the same 10%. [figure: Four boxes left to right: define segment/alternatives/outcome; compare tested price points or packages; gate into advance, revise, or validate; retest on a new tier, competitor, or cost change, looping to step one.] ## What should you define before testing a pricing choice? Name the buyer segment, the alternatives, the outcome, and the commercial constraints before comparing anything. Bring these inputs: - The product type, core features, and delivery model. - The customer profile, company size, and relevant budget range. - Current pricing, if any, and how it was set. - The realistic alternatives a buyer would compare. - COGS, CAC, operating expenses, and other unit-economics boundaries. - The business objective, such as penetration, growth, profitability, or premium positioning. The experiment tests buyer response to controlled alternatives; the commercial model determines whether a preferred alternative is viable. ## Turn five questions into controlled comparisons The most useful pricing questions become explicit actions and outcomes: Keep the product, buyer definition, and outcome constant where possible. Otherwise a team cannot tell whether price, packaging, or a second change caused the difference. ## What can Subconscious test? Subconscious is the causal AI company. It runs randomized experiments on a simulation of the market and can validate studies against real human behavior. For pricing, the supported fit is decision-specific scenario testing. A number without its limits is marketing. Uncertainty should be reported only when the study design supports it. The comparison does not calculate a complete commercial answer on its own. The practical advantage is continuity. A team can move from a simulated experiment to real-human validation without changing the causal question. The intervention, alternatives, buyer population, and outcome remain aligned. ## What can the evidence settle? The misses sit on record next to the hits. The result can support a choice among the alternatives actually tested. It cannot establish a universally correct price. Naming the failure mode here lets a buyer check the study against it. Do not infer a demand curve, portfolio switching effects, or a profit-maximizing price unless the study was explicitly designed to estimate that output. Do not treat stated preference as guaranteed market performance. Unit economics, contractual constraints, sales execution, and actual post-launch behavior still belong in the decision. Use the evidence as a gate: - Advance an alternative when the directional result is clear and the economics are viable. - Revise the alternatives when the comparison does not separate them. - Validate with real participants when the consequence of being wrong demands stronger confirmation. - Monitor actual conversion, retention, and buyer response after launch. ## Retest when the decision changes Do not rerun the same study on stale assumptions. Retest when: - A new tier or add-on changes the offer. - A competitor changes the realistic choice set. - Costs change the viable price range. - The product enters a new buyer segment. - Conversion changes without a clear cause. Update the alternatives, buyer definition, and commercial inputs while preserving the outcome that matters. ## Bring one pricing decision, not an entire price book Start with the price, tier, or packaging choice that has the highest cost of being wrong. Review how a study moves from decision to evidence, see completed decision studies, or read the research behind the method. When the segment, alternatives, and outcome are clear, set up the pricing study. --- # Market Research Automation: What to Automate and What Still Needs a Causal Test URL: https://subconscious.ai/blog/use-cases/market-research-automation-what-automate-what-still-needs-causal-test Published: 2026-08-13 A VP or Director of Consumer Insights weighing AI-driven research automation faces one real decision: which parts of the workflow are safe to automate, and which decisions require a controlled experiment before the team commits capital. Automating logistics and validating an answer are not the same problem; treating them as interchangeable is where the risk lives. [figure: Two-column comparison: recruiting/transcription and coding/drafting fall under automation tools; which message or price wins and real-human validation require a controlled experiment, no automation substitute.] ## What automation actually removes from the workflow These logistics tasks now run through AI tools competently: - Transcription and speaker separation. Automated tools produce a searchable transcript from a recorded interview without manual typing. - First-pass qualitative coding. AI can tag transcripts against predefined codes or surface emergent themes, giving an analyst a starting point. - Survey drafting. AI can draft a survey's question types, wording, and skip logic from a research question and target audience, for researcher review. - Report drafting. AI can synthesize findings from multiple sources into a structured first draft for a researcher to edit. - Screener and recruitment messaging. AI can draft participant screeners and scheduling messages, cutting fieldwork's administrative overhead. Listing what a tool does without its ceiling is marketing copy. None of this changes what the research found. It changes how much labor it took to produce the write-up. ## What automation does not remove A tool's task list means little without the tasks it skips listed too. Four things stay with the researcher regardless of tooling: - Strategic question formulation. Knowing what to research, which hypotheses matter, and what decision it needs to inform requires organizational context automation doesn't have. - Insight interpretation. Software can surface a pattern; deciding whether it's meaningful and what to do about it is a judgment call. - Stakeholder buy-in. Getting a finding to change a decision requires navigating organizational politics, not just producing an accurate report. - Direct behavioral observation. Ethnographic and in-context usability work requires physical presence and human perception. It cannot be simulated. ## What gap does automation not close? Automating logistics answers how fast a project moves. It does not answer whether the underlying method produces a causally valid answer about what customers will do. A team that treats every research question as safe to automate can ship a pricing, launch, or messaging decision a controlled experiment against real behavioral outcomes would have contradicted. The cost is the wrong action taken, not the hours saved on transcription. The 2026 GRIT Insights Practice Report and Greenbook's coverage of AI adoption in qualitative research both track this split: research teams are automating operational overhead faster than they're changing how they validate a decision before it ships. ## When is a controlled experiment the better tool? For decisions with real cost if the team is wrong, such as a pricing change, launch message, or positioning shift, the question isn't "can we produce an answer quickly" but "does the answer describe what a real audience would actually do." Subconscious runs controlled experiments on a simulated market to estimate which action moves a specific behavioral outcome, then validates that study with real human participants when the decision warrants it. Subconscious also runs controlled studies against a person-level audience graph covering 800 million real people; that reach describes the scale for comparison, not a recruitable panel. This is a different layer from logistics automation: it addresses whether the answer itself would hold up against real behavior before the team acts on it. ## Limitations A method's reach means little without its limits attached, and naming them here is what lets a buyer check the method before staking a decision on it. An 800-million-person audience graph is not a recruitable participant pool. A simulated experiment is a first pass on a causal question, not a replacement for real-human validation on a high-stakes decision. Subconscious does not automate strategic question formulation, insight interpretation, or stakeholder buy-in. Those stay human judgment calls regardless of method. ## Where should a team start? Sort the research backlog into two piles: logistics work automation can absorb today, and decisions that carry real cost if the team gets the causal question wrong. For the second pile, see how Subconscious runs a study or book a walkthrough against a specific pricing, launch, or messaging decision. --- # AI Persona Panels: What They Are and When to Trust Them URL: https://subconscious.ai/blog/use-cases/ai-persona-panels-what-they-when-trust-them Published: 2026-08-13 An AI persona panel is a set of grounded synthetic personas built from demographic and psychographic detail, queried through a language model, and used to test a question before a team commits budget to fielded research. The persona is not a live capability that replaces real people. It is a controlled first pass that tells a team which questions are worth escalating. The decision this page answers: when should a research or marketing lead trust a synthetic-panel read on its own, and when does the same question need a real-human validation step before it drives a launch, pricing, or messaging call? ## Why isn't a synthetic panel just a stateless prompt? Asking a language model to "imagine you are a 42-year-old marketing director" produces a stateless, surface-level answer with no way to check whether it represents the segment a team cares about. A grounded persona differs in three ways: Grounding depth. The persona carries a consistent professional history, category-specific knowledge, and behavioral pattern, not a one-line demographic tag. Internal consistency. Values, priorities, and decision style stay coherent, part of a stored backstory rather than reinvented on each query. Panel structure. Many personas queried together produce segment cross-tabs and aggregate distributions instead of one isolated answer. ## Questions suited to synthetic triage Useful for an early read before a team commits to a consequential market decision: ## Questions that must leave the synthetic layer - Sensory testing is out of reach: a language model has no channel for taste, smell, touch, or fit. - Fully novel categories strain the grounding. A category with no public precedent gives the model nothing to condition on, and reliability drops. - Precise purchase-rate prediction is unreliable. A synthetic panel is a reasonable guide to which of two segments responds more, and a poor guide to an absolute conversion number. - Regulatory and legal substantiation stays out of scope. Synthetic data does not satisfy marketing-claim substantiation or formal research deliverables in most jurisdictions. - Recent events past a model's training window return a guess, not the audience's real reaction. - Minority-opinion tails compress toward the average, harder to surface synthetically than through well-recruited human research. ## A four-stage path from question to evidence Teams that get the most out of synthetic testing use it as a triage layer, not a replacement: - Question intake. Every research question enters the stack here, before a channel is chosen. - Synthetic triage. Run the question through a persona panel first. Most questions get answered at this resolution. - Decision-validation layer. The small set of high-stakes, novel, or regulated questions per quarter go to real-human research, briefed sharper because the synthetic pass already did the triage. - Periodic calibration. Run a real-human study alongside a synthetic panel on the same question once or twice a year, and adjust the synthetic setup if the two have drifted apart. Skipping the decision-validation layer on a high-stakes call is the failure mode this stack exists to prevent: a pricing, message, or launch decision that looks confirmed synthetically but does not hold once real buyers respond. A 2024 Political Analysis study on simulating human samples with language models found the same pattern from the research side: language-model samples can track certain human response distributions but should not be treated as a substitute for the underlying population without checking that fit. ## How do you keep the causal question intact during validation? Subconscious runs controlled, discrete-choice-style experiments that estimate which tested action moves which outcome, rather than open-ended persona chat. It draws on a person-level audience graph covering 800 million real people, a modeled reach figure distinct from a recruitable panel of consenting participants. When a question clears the triage layer and needs the decision-validation step, Subconscious moves the same causal question to fielded participants instead of switching to a different method. ## What checks should a team make before it acts? Confidence intervals, segment-heterogeneity breakdowns, and full decision memos are not standard on every study; treat them as something to confirm for the specific study a team is planning. Pricing optimization, substitution and cannibalization matrices, and full catalog simulation sit on the roadmap rather than as a live, ready-to-use capability today. ## Start with one live decision Pick one real, current question a team is already wrestling with. Build a small panel calibrated to the audience the question is about. Compare the panel's answer to what the team would have predicted: a meaningful gap is a signal worth digging into, and a match is a useful confirmation of the team's existing read. From there, route only the questions that clear the triage bar (high stakes, novel behavior, or regulatory exposure) into a real-human validation study, and keep the rest at the synthetic layer. Teams weighing this decision in more detail can see how the stack plays out in the Subconscious case studies, review the research program, or book time to scope a study. [figure: Four-step path: Question intake feeds Synthetic triage, which answers most questions. High-stakes ones branch to Decision-validation layer for human research. Periodic calibration loops back yearly to check drift.] --- # Continuous Discovery Finds Opportunities. Causal Tests Decide What to Build. URL: https://subconscious.ai/blog/implementation-and-procurement/continuous-discovery-finds-opportunities-causal-tests-decide-what-build Published: 2026-08-13 A recurring discovery conversation reveals where a product team should look next. It cannot show which intervention the market would choose or how large that preference is. Before committing engineering or launch resources, the buyer must decide whether the finding is an opportunity to explore or a defined choice ready for causal testing. Continuous discovery is commonly framed as regular customer contact that informs product decisions while work is in progress. This overview describes the practice as weekly touchpoints centered on desired outcomes, opportunities, and solutions. The useful principle is sustained contact with customer problems. [figure: A finding splits: open opportunity loops back to the map; defined choice moves through alternatives, population, outcome, then a causal experiment returning an effect with a confidence interval.] ## The decision boundary is more important than the cadence Open-ended discovery generates candidate explanations. A conversation might surface a workaround, an onboarding confusion, a segment-specific expectation, or an assumption the team has not examined. Those findings help map the opportunity space and decide what deserves a sharper question. A historical planning example used three reusable synthetic groups: 8 to 10 core profiles, 6 to 8 expansion profiles, and 6 to 8 churned or at-risk profiles, not a current Subconscious configuration, audience requirement, or vendor benchmark. The grouping helps a team look for competing hypotheses; it does not measure which alternative a defined population would choose. The boundary appears when a finding begins to justify an action. "Customers struggle with exports" is an opportunity signal. "We should build guided export presets instead of improving the existing manual flow" is a choice between interventions. The second statement needs evidence designed around that comparison. ## Two methods answer two different buyer questions Confusing these methods creates a procurement problem and a research problem. A team can buy a continuous stream of plausible observations and still lack evidence for the build decision it faces. The question is not whether discovery produced signal, but whether the evidence matches the action the team is about to fund. ## Preserve the habit, then hand off the decision One historical planning format allocated 45 minutes across four blocks: - Minutes 1-5: state one opportunity or assumption. - Minutes 5-25: explore it with the core perspective. - Minutes 25-35: check it against a second segment. - Minutes 35-45: record what changed and what remains uncertain. This agenda is a historical planning example, not a Subconscious service format or delivery promise. Its output is a candidate question, which the team should rewrite in experimental form: - Name the action under consideration. - Define the alternatives, including the relevant status quo. - Specify the population whose behavior matters. - State the outcome that would change the product decision. - Set the evidence threshold before seeing the result. Subconscious research begins at this handoff. It runs a controlled discrete-choice experiment over the specified alternatives and population, returning a measured causal effect with a confidence interval. The practical advantage is decision evidence tied to the proposed action, not another set of open-ended impressions. ## What can't a causal test do? Naming what a causal test cannot do is what lets a buyer check the claim before funding the next step. A controlled experiment does not replace the qualitative habit: it does not generate an opportunity map, conduct an open-ended persona conversation, observe a usability session, or decide which customer problem deserves attention. Product and research leaders own those judgments. A method that draws its own boundary is a method a buyer can verify. Simulation and recruited research remain distinct. Audience reach describes the scale of a simulated experiment, not a participant count. When the stakes call for recruited real-human validation, the same causal question can be tested without relabeling the work as qualitative discovery, a usability study, a clinical trial, or proof of market performance. The limit sits next to the finding, so a buyer can check the claim against the tested population before funding a launch. The result is bounded evidence for a defined comparison. It can reduce uncertainty about which alternative causes a change in stated choice within the tested population; it cannot guarantee adoption after launch or substitute for production measurement. ## How do you put a build decision into experimental form? Before the next planning meeting, take the strongest discovery finding and write down the action it seems to support. If the team cannot name the alternatives, population, and outcome, keep exploring. If it can, the question is ready for testing. See how Subconscious structures the engagement, from decision framing through validation. The goal is not to replace continuous discovery, but to prevent a promising signal from becoming an expensive build commitment before the causal choice has been tested. --- # Why Insights Teams Lose Influence When They Avoid AI URL: https://subconscious.ai/blog/use-cases/why-insights-teams-lose-influence-when-they-avoid-ai Published: 2026-08-13 Insights teams lose influence when they avoid AI because stakeholders then adopt ungoverned tools on their own and treat unvalidated synthetic output as settled evidence, leaving the team without ownership of the evidence boundary. A stakeholder wants an answer before the evidence is ready. A manager asks whether the team can skip the disciplined first pass. A draft report appears before the researcher has finished reading the data. The insights leader decides one request at a time: build a governed evidence path for AI-assisted exploration, or let stakeholders turn unvalidated output into strategy. ## Influence is lost at the evidence boundary The demand for research is not disappearing. The U.S. Bureau of Labor Statistics expects roughly 7% growth between 2024 and 2034 for two roles, marketing specialists and market research analysts, plus about 87,200 average yearly openings across that span. The projection appears in the BLS Occupational Outlook Handbook. This labor-market projection is historical planning context, not a current Subconscious or vendor performance claim. The practical risk is organizational. If an insights team ignores AI-assisted exploration, stakeholders may adopt ungoverned tools and treat their output as settled evidence. If the team adopts those tools without review and validation, the business may base a launch, price, or public claim on an unverified or biased synthetic read. In both cases, the team loses influence because nobody owns the boundary between a directional answer and a defensible one. ## What are the four layers of a governed answer? The operating question is where synthetic exploration ends and validation begins. A useful evidence system separates four layers: - Exploration: generate hypotheses, objections, and alternative explanations. - Directional testing: use a synthetic population to compare controlled options. - Human review: confirm the audience is defined correctly, prompts stay neutral, sources are grounded, and business context is accounted for. - Validation: draw on real-participant or observed behavioral evidence when a decision is high-stakes or public. Research expertise no longer rests only on access to study design, data cleaning, or reporting. Its value is the judgment to define the causal question, test plausible actions, and decide what level of evidence the decision requires. A fluent synthetic answer remains an output until that judgment is applied. ## How does the handoff from direction to proof work? Subconscious provides a defined handoff between those stages. Teams can run controlled experiments on simulated populations, then validate the same study with real human participants without changing the underlying causal question. The output can therefore carry an honest evidence label: directional synthetic read or real-participant validation. The research approach explains the method, while case studies show the kinds of decisions it can support. ## The boundary that protects credibility This framework does not mean synthetic populations replace fielded research, and no single accuracy figure applies to every study. Aggregate pattern matching is easier to support than individual-level fidelity. Segment-level conclusions and high-stakes external claims still need validation with real participants before they become business evidence. [figure: Four stages left to right: exploration finds hypotheses, directional testing compares options on synthetic populations, human review checks audience and prompts, validation uses real participants.] Validation does not convert a causal action test into a usability session, clinical trial, or automatic forecast of market performance. It answers a narrower question: whether the tested pattern holds with the recruited participants under the study design. [figure: Two paths converge on one box. Top: team avoids AI, stakeholders adopt ungoverned tools. Bottom: team adopts AI without review, an unverified read becomes strategy. Both end: team loses influence.] ## An operating loop that keeps research in the decision - Select a live launch, pricing, positioning, or public-claim decision. - State the causal question in one sentence. - Define the audience and the consequence of acting on a wrong answer. - Use synthetic exploration for hypotheses and controlled directional testing. - Review the study design, audience, prompts, and result before anyone acts. - Name the evidence level and the condition that triggers real-participant validation. Applied across recurring decisions, this loop becomes an evidence system rather than a collection of tools. The insights team stays influential by owning the question, the test, and the standard of proof. How we work describes that operating model, and about Subconscious explains the company behind it. --- # Test a Backlog Item Before You Commit Sprint Capacity URL: https://subconscious.ai/blog/use-cases/test-backlog-item-before-commit-sprint-capacity Published: 2026-08-13 A product team debating a backlog item has two ways to decide: argue from opinion, or run a controlled comparison against a defined audience and read the behavioral result. Teams that skip the comparison find out whether they were right only after the sprint ships. ## The research gap in sprint planning Sprints run on a fixed cycle. User research usually doesn't. Interviews take weeks to schedule, and quantitative studies often arrive after planning is already over, so teams end up prioritizing on gut feel or whoever argued the ticket best. Nielsen Norman Group finds that agile teams need research findings on a schedule the standard research process cannot sustain; the cadence mismatch, not a lack of intent, is what keeps customer evidence out of planning. "Because of all these challenges, UX research often ends up neglected or abandoned altogether in Agile environments." Rachel Krause, Nielsen Norman Group (source) ## Turn the debate into a testable question ### Define what would change your mind Look at the top 5 to 8 items your team is debating. For each one, write the one question whose answer would actually change the priority: would this reduce a real problem, would it change how the buyer behaves, or is it a tie-breaker between two directions the team could take. Keep the question concrete. An abstract question produces an abstract answer. ### How do you define the audience and comparison? Specify the population the answer needs to hold for: the segment of buyers the backlog item is meant to affect. Then specify the alternatives worth comparing: the proposed feature against no change, or against a competing item fighting for the same sprint slot. Subconscious runs this as a controlled experiment on a simulated market. The alternatives are held constant except for the one thing being tested, and the output is a measured difference in behavior between them, not a summary of what a single respondent said it would do. A first comparison might define 6 to 10 distinct buyer segments to represent the primary population, weighted toward the mix that actually uses the product rather than a convenient sample. The underlying audience graph covers a person-level population of roughly 800 million real people, so a team can define a fairly specific segment and still draw a study population that matches it. That graph is the addressable population a comparison is drawn from, not a pool of recruited participants. ### How should you read the result? Publishing the uncertainty alongside the effect is what lets a buyer check the claim. The output is a causal effect for the population you defined, with its uncertainty, not a promise about any individual buyer. Separate two things when you read it: whether the item changes stated interest versus whether it changes modeled behavior. The second is the one that should move sprint capacity. ### Bring it into planning as a brief Summarize each item's result in one or two sentences and bring it into planning alongside the ticket. Across a set of ten debated backlog items, a team might find that one shows a strong behavioral pull, with 7 of 10 tested segments describing an existing workaround they would drop, while a second shows no measured change in behavior despite popular internal support. That split is the actual value: it separates the items worth arguing about from the items the comparison has already settled. ## What doesn't this replace? Naming what a method doesn't cover is what lets a team pick the right tool for the next question. A controlled comparison against a simulated population answers a narrower question than teams sometimes expect. It does not replace direct usability observation, moderated interviews, or product analytics. It won't show whether a real person can operate the interface, why they hesitate, or what they did last week with the product already shipped. A comparison establishes a causal effect for the tested population under the tested conditions. It does not guarantee individual-level accuracy, and it does not automate sprint planning. When the decision genuinely depends on it, a team can move from the comparison to a study with real human participants without changing the underlying question being tested. That step matters when the stakes of a wrong call justify it, not as a routine second pass on every backlog item. ## Common questions Is this the same as asking a chatbot what it thinks of the feature? No. The output isn't a generated opinion. It's a measured difference in behavior between two defined alternatives, run against a specified population, with everything else held constant. See how Subconscious runs these comparisons for the underlying method. What if the result contradicts what our analytics show? That's worth a conversation, not a dismissal. Qualitative and quantitative signal often disagree because they measure different things: stated preference versus revealed behavior, past usage versus a proposed change. A contradiction usually means one of the two questions was framed wrong, not that one source is useless. Does this replace the product manager's judgment? No. It narrows the set of things worth arguing about. The team still decides what to build; the comparison tells you which arguments are backed by a measured behavioral difference and which are backed by whoever spoke last in the meeting. [figure: A branching diagram from one comparison result splitting into two paths: stated interest, leading to a dead-end labeled doesn't move capacity; and modeled behavior, leading to moves sprint capacity.] ## Start with one contested item Pick one backlog item your team is currently split on. Define the population it's meant to affect and the alternative you're weighing it against, then run the comparison before the next planning session. Review current research and methodology or see how the approach applies across decisions already tested before you scope the first one. [figure: Five-step path: write the one question that would change your mind, define audience and alternatives, run the comparison, read the result as a behavioral effect not a verdict, then brief sprint planning.] --- # How a Research Team Avoids Becoming an AI Ticket Desk URL: https://subconscious.ai/blog/implementation-and-procurement/how-research-team-avoids-becoming-ai-ticket-desk Published: 2026-08-13 A head of research protects the team by publishing a routing rule before the request queue grows, not by defending headcount after it does. Low-stakes, reversible questions go to fast exploratory testing. Pricing, launch, and positioning decisions go through a controlled, validated causal experiment before they ship. Skip the routing rule and the team becomes either a slow bottleneck stakeholders route around, or a fast desk producing confident-sounding answers nobody validated. ## Why is the routing question arriving now? AI tools have made a plausible-sounding answer cheap to produce: a draft concept reaction, a first-pass read on a message, a quick comparison of options. That does not remove the need for a validated one. Per the U.S. Bureau of Labor Statistics market research analyst outlook, employment in market research and marketing specialist roles is still expected to keep expanding over the 2024-2034 window. The pressure is specific: when any stakeholder can generate a draft answer in minutes, the research team's advantage stops being access to tools and becomes judgment about which answers are safe to act on for a given decision. ## What stays scarce once production gets cheap Research expertise used to include partial ownership of access: knowing how to field a study, clean the data, and package the finding. AI weakens that access advantage. It does not remove the need for the judgment that decides whether an answer deserves trust. It means naming the business decision before any AI-assisted tool touches the question, and naming the caveat after the tool produces output. A useful research leader can say what the decision is, what evidence would change it, what confidence it requires, and where a fluent-sounding answer could still mislead the business. ## Three tiers of evidence, routed by decision risk The core design choice is not which tool to use. It is which requests get fast exploratory testing and which require a controlled, validated causal experiment before a decision ships. Subconscious runs the middle tier as a controlled discrete choice experiment and reports a causal effect with a confidence interval. That is the evidence a pricing, launch, or positioning decision needs once it is expensive or public to defend: a measured effect, not another fluent draft answer. ## Publish an intake rule before the backlog forms A routing rule beats a routing habit because it survives the person who wrote it going on vacation. A useful rule asks four questions of every incoming request: - What decision is being made, and what changes if the answer points one way or the other? - Who is the audience, and what does being wrong about them cost? - Is the question reversible and low-stakes, or does it precede a pricing, launch, or positioning commitment? - What evidence tier does that risk level require? A reversible message or concept choice can start and end in exploratory testing. A pricing decision, a launch commitment, or a public positioning claim should move to a controlled causal experiment, and escalate further when the question is genuinely novel or the result has to hold up outside the model. ## Carry the same question from exploration to validation When a finding needs to hold up under real human evidence, the useful move is continuity, not a restart. Subconscious can test or validate studies with real human participants, carrying the same causal question, actions, audience definition, and outcome from the modeled experiment into human validation. Only the evidence tier changes. Where scale of the modeled population matters to the decision, keep it distinct from the tiers above it: a person-level audience graph covering 800 million real people describes how large a modeled population can be, a controlled causal experiment describes a comparison run against that population, and recruited human validation describes evidence gathered from real, fielded participants. Collapsing any two of these into one claim is the fastest way to lose the credibility a routing rule is built to protect. ## Say which tier answered the question The failure mode that turns a routing rule back into a ticket desk is letting every request get a bespoke, unlabeled answer. Label the tier every time: "directional exploratory read," "modeled causal effect with confidence interval," or "validated with real human participants." Naming the tier makes the finding more credible because it tells the stakeholder exactly what evidence they are holding and what would strengthen it. A rejection or a redirect should name the tested action, the alternative, the target audience, the outcome, and the evidence limit, rather than a vague objection. That gives the stakeholder a defined next step instead of a stalled request. ## Where does a routing rule still break? A routing rule depends on the team enforcing it. It breaks when a stakeholder skips the intake questions and reports an exploratory read as though it were validated evidence, or when the team routes every request to full validation out of caution and becomes the bottleneck it was built to avoid. It also breaks when the causal question changes between tiers. If the audience, the compared actions, or the outcome shifts between passes, the escalation does not confirm anything. It starts a new study instead. ## Limitations A method that hides its edges is marketing. This one publishes its edge here: none of these tiers replace a research team. A controlled causal experiment narrows and tests options; it does not decide organizational priorities, negotiate with stakeholders, or write the intake rule itself. It is one validation option among fielded research, behavioral data, and expert review, not a substitute for the operating model that decides when to use it. [figure: Decision path: a request splits on reversibility. Low-stakes goes to exploratory testing. Pricing, launch, or positioning goes to a controlled causal experiment, escalating to human validation when novel.] To size a routing rule against a live backlog, see the research method behind the causal-effect tier, how causal experiments are run, review case evidence, or discuss a decision that needs a defined evidence path. --- # Choosing Between MaxDiff, Conjoint, and a Controlled Experiment URL: https://subconscious.ai/blog/methods-and-validation/choosing-between-maxdiff-conjoint-controlled-experiment Published: 2026-08-13 A research team choosing a study design before fieldwork has to answer one question: does the business decision need a preference ranking, an importance score, a satisfaction metric, or a causal estimate of what drives behavior? Picking the wrong instrument produces data that cannot support the decision it was meant to inform. ## The decision this instrument choice actually serves Before writing a single question, define three things: who should answer, what they are being asked to evaluate, and what the team will do with the result. A pricing decision, a message test, and a feature-tradeoff decision are not interchangeable research problems and do not share one correct method. The failure mode is not a badly worded question. It is choosing a method whose output shape does not match the decision. A ranking tells a team what people prefer, not what would happen if the team changed the offer. ## How do MaxDiff, conjoint, satisfaction metrics, and a controlled experiment compare? These four instrument families answer different questions, and none substitutes for the others. MaxDiff and conjoint analysis are both tradeoff methods: MaxDiff ranks items by relative importance, while conjoint analysis estimates the tradeoffs buyers make between attributes like price and features through a randomized-design choice task (Sawtooth Software). Neither produces a causal estimate of real-world market behavior beyond the tested choice task. ## When is a preference ranking or satisfaction score the wrong tool? A ranking or importance score is directional: it says people rate a feature or claim highly relative to alternatives. Treating a directional read as a causal estimate is a common way research gets misapplied: the study answers a different question than the one the business is asking. Satisfaction metrics have the same limit. NPS or CSAT can flag that something is wrong, but they do not identify which lever moved the number, because they were never designed to isolate cause from correlation. ## When does the decision need a causal answer? If the question is which action to take among a defined set of options, and the cost of choosing wrong is a launch, a price change, or a message that does not land, the study needs to isolate cause, not just rank preferences. A controlled discrete choice experiment varies all attributes simultaneously through a randomized design and reports a causal effect, averaged across the other attributes, with a confidence interval. Subconscious runs this kind of study against a population modeled on the affected buyers when the decision is which action drives an outcome. ## A short framework for choosing - State the decision in one sentence: which action, priced how, framed how. - Ask whether the answer needs to be causal (an estimate of what would change behavior) or directional (a relative preference or a sentiment snapshot). - If the answer is directional, MaxDiff, conjoint attribute exploration, or a satisfaction metric can fit. - If the answer needs to hold up under scrutiny as a causal claim, use a controlled experiment designed to isolate that one variable. - Decide, before fieldwork, what evidence would change the recommendation. A study designed after the fact to justify a decision already made is not evidence. ## Limitations A controlled discrete choice experiment does not replace early qualitative exploration, and it does not set the target-group definition; that judgment call stays with the research team. The experiment does not certify representative statistics for a regulatory or public claim. Subconscious can test or validate studies with real human participants to check the study's stated choices against real behavior, since stated preference runs high relative to what people actually do (hypothetical bias), and that step is required before any claim from the study goes external. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, a targeting frame rather than a probability sample of respondents. That reach describes the population available for a controlled study design, not a recruited sample delivered without further validation. ## Next step Teams that have used MaxDiff or conjoint analysis to narrow a set of options, and now need a causal answer, can see how a controlled study is scoped in case studies, or book a working session to map a pricing, message, or feature decision to the right method. [figure: Decision path: state the decision, ask if the answer needs to be directional or causal, then route to MaxDiff/conjoint/satisfaction for directional, or a controlled experiment for causal.] --- # AI Interview Practice: What Simulation Can and Cannot Test URL: https://subconscious.ai/blog/use-cases/ai-interview-practice-what-simulation-cannot-test Published: 2026-08-13 Interview practice works when it exposes weak answers, vague rubrics, and missing follow-up questions before a real interview. Generic question lists rarely reproduce the specificity of a role, company, or evaluation process. Subconscious is a causal behavioral platform, not an interview-coaching product. [figure: Two columns: left, what simulation tests (answer specificity, follow-up questions, rubric consistency); right, what it cannot test (physical presence and nerves, proof of a fair process).] ## Candidate rehearsal Define the interviewer by role, seniority, interview type, and known evaluation criteria. A technical interview at a Series B startup differs from a behavioral cycle at a large company; the scenario should reflect that without impersonating a real person. A manager might ask how a candidate would respond when a team of eight engineers includes two people publicly disagreeing about an architectural direction. A follow-up might challenge whether the candidate described their own work or the team’s work. Run the same interview three times. Compare whether the answer becomes more specific, concise, and evidence-based. A question about leaving a role after eight months tests whether the candidate can explain a difficult fact directly. ## Hiring-team rehearsal Hiring teams can test whether a rubric distinguishes: - a strong candidate with relevant experience; - a polished candidate who lacks depth; - a technically strong candidate who does not meet a stated requirement; - an underqualified candidate with high potential. The exercise can also reveal questions that reward familiarity with one background rather than the capability the job requires, the kind of unequal effect the federal Uniform Guidelines on Employee Selection Procedures direct employers to check for in any selection process (29 CFR Part 1607). New interviewers can practice probing, redirecting, and applying the rubric consistently. Reviewing how Subconscious structures a causal read on a decision shows what evidence-backed validation looks like before applying the same standard to a hiring rubric. "A selection rate for any race, sex, or ethnic group which is less than four-fifths ( 4/5 ) (or eighty percent) of the rate for the group with the highest rate will generally be regarded by the Federal enforcement agencies as evidence of adverse impact" Uniform Guidelines on Employee Selection Procedures, 29 CFR Part 1607 (source) "A selection rate for any race, sex, or ethnic group which is less than four-fifths ( 4/5 ) (or eighty percent) of the rate for the group with the highest rate will generally be regarded by the Federal enforcement agencies as evidence of adverse impact" Uniform Guidelines on Employee Selection Procedures, 29 CFR Part 1607 (source) [figure: Four candidate profiles as test cases for a hiring rubric: strong with relevant experience, polished but lacking depth, technically strong but missing a stated requirement, and underqualified with high potential.] ## What simulation does not reproduce Physical presence, body language, nerves, and human connection affect real interviews. A simulated interviewer is more consistent than a person whose context changes across a week of candidates. Do not use a synthetic rehearsal as proof that a candidate will perform well or that a hiring process is fair. Validate rubrics against real outcomes, audit adverse effects, and keep accountable people in every hiring decision, the same discipline Subconscious applies when it moves a simulated result to real-human validation before a decision ships. The useful output is a better question or clearer rubric, not an automated employment judgment. --- # AI for Product Discovery: Research Before You Build URL: https://subconscious.ai/blog/use-cases/ai-product-discovery-research-before-build Published: 2026-08-13 Product discovery asks what to build before delivery asks how to build it. Good discovery starts with a decision, evidence about the customer, and a hypothesis that can fail. AI can speed up early exploration. It does not remove the need for interviews, observation, behavioral data, or human validation. [figure: A five-step path: define the target audience and job context, run simulated comparisons across segments, narrow to the questions that remain uncertain, validate those with real customers, then decide what to build.] ## What discovery must establish A useful discovery process identifies the customer's real problem, tests whether a proposed action addresses it, separates essential features from nice-to-have work, compares segments, and challenges assumptions before engineering begins. Traditional discovery may take weeks because recruitment, scheduling, sessions, and synthesis all require time (User Research Participant Recruitment Guide, User Intuition). Simulation can help a team prepare and screen options while that work is arranged. ## Use simulation to narrow the question Define the target audience, job context, expertise, goals, and constraints. Then compare product or message alternatives under the same conditions. The output is a hypothesis about aggregate behavior, not a faithful copy of one individual. A planning example might run five rounds of simulated discovery in the time required to schedule one round of interviews. Another might test five concept directions in an afternoon and select the two worth taking to real customers. Treat both as workflow examples, not throughput guarantees. See how the research behind this approach is designed. ## Five applications ### Problem validation Ask how often the problem appears, what people do now, and which constraints shape the workaround. A buyer can only check a number against its stated limit if that limit is written down. Do not treat a simulated willingness-to-pay answer as a pricing decision. ### Solution-hypothesis testing Compare proposed solutions against the same outcome. Ask how each alternative fits the current workflow, where it adds friction, and what evidence would change the result. ### Feature prioritization Present candidate features to several defined segments. Use the result to identify tradeoffs for a causal experiment or real customer study. This miss is printed right next to the result it qualifies. A ranked simulated list is not a roadmap decision by itself. ### User-story validation Test whether the story reflects the customer's problem and whether the proposed action matches the expected workflow. Record edge cases for human review. ### Onboarding and adoption Onboarding is underused. Compare alternative instructions, steps, or messages for a new customer. Measure the intended behavior rather than asking a persona to narrate an entire interface it cannot actually use. ## Know where simulation fails Naming the failure mode here is what lets a team check the method before it relies on the result. Models often match aggregate patterns better than individual behavior. Novel use cases, genuine surprise, sensory experience, and unusual early adopters are hard to simulate. Prompt sensitivity and demographic flattening can also hide meaningful differences. Use simulated exploration to identify the most important questions. Use real customers and usage data for the discovery that shapes the product direction. [figure: A list of five discovery applications for simulation: problem validation, solution-hypothesis testing, feature prioritization, user-story validation, and onboarding and adoption.] ## A practical setup Define two to four key audience types, including the primary segment and important secondary segments. Describe each with only the context relevant to the decision. Focus each session on one topic. Compare the same stimulus across audiences. Capture what the exercise suggests, what remains uncertain, and what needs real-user validation. Subconscious supports controlled, decision-specific experiments on product, pricing, messaging, and go-to-market actions. The useful result estimates which action changes which outcome for which segment under the study conditions. It does not tell a team what to build without judgment and real evidence. See how the process runs or book a demo to test a specific discovery hypothesis. --- # Analytics, Tracking, or a Controlled Experiment: Picking the Right Target-Group Research Instrument in 2026 URL: https://subconscious.ai/blog/use-cases/analytics-tracking-controlled-experiment-picking-right-target-group Published: 2026-08-13 There is no single best tool for target-group research. There are three different questions, and each needs a different instrument. Analytics tools tell you who your audience is. Tracking tools tell you what they do. Only a controlled experiment tells you why, or what they would do if you changed the price, the message, or the concept in front of them. Picking the wrong instrument is expensive in a specific way: teams spend research budget and fieldwork weeks answering a question the method was never built to answer, then discover the result cannot support the decision it was meant to inform. ## Three layers, three different questions Target-group research splits into three functional layers. Confusing them is the most common way research budget gets wasted. Identifying-layer tools work from search behavior, competitor traffic, and audience-overlap data to draw the demographic and psychographic boundaries of a market. Observing-layer tools track how real visitors interact with an existing site or product, which is useful for understanding current customers but silent on people who never showed up. Neither layer can tell you how a target group would react to a concept, a price, or a message it has never seen. That is the asking layer's job. ## The asking layer has a speed problem and a validity problem Recruiting a human panel for every question is slow: fielding a study can take three to four weeks and cost real money before a single answer comes back. That cost pushes teams toward two bad habits: skipping research on decisions that deserve it, or running one broad study and stretching its answer to cover questions it was never designed for. The newer failure mode runs the other direction. Simulated research tools can return a directional read in minutes, but a fluent, plausible-sounding response from a generated respondent is not the same thing as evidence that a real market would behave that way under a specific alternative. Qualtrics's overview of synthetic data in market research is a useful primer on where synthetic methods help and where they still need a human check. ## What Does a Controlled Experiment Add That a Directional Read Doesn't? Subconscious runs a discrete-choice experiment: it compares defined alternatives, such as competing messages, prices, or concepts, across a defined population, and returns a measured causal effect with a confidence interval, not a paragraph of generated opinion. A directional read tells a team a concept felt more appealing. A causal effect tells a team how much more likely a defined population was to choose it over the alternative, and how much uncertainty is attached to that estimate. Marketbridge's case for a hybrid research approach makes a similar argument for pairing fast simulated methods with the rigor a real decision requires. For teams that want to see how Subconscious structures and validates these experiments, the replication leaderboard documents the method and tracks how estimates hold up against real outcomes. ## Where Isn't a Controlled Experiment the Right Instrument? A controlled experiment does not replace an analytics platform or on-site behavioral tracking. It cannot tell a team who is visiting a website right now or where they came from; that is still the identifying and observing layers' job. Naming this limit is what lets a buyer check whether the method fits a regulatory or audit need before they rely on it: it also does not produce population-scale market sizing or the kind of statistical proof a regulator or auditor requires. If a claim must hold up to outside scrutiny with a precise figure, say, 34 percent of a population sharing a given view, the number has to come from a study built and fielded for that purpose, typically with recruited human participants. The leaderboard puts misses next to hits in public, and this is one of the miss conditions worth stating outright: it is weakest on behavior with no real-world precedent. A method built on historical patterns and existing data will lag a genuinely novel product category, an unprecedented event, or a group underrepresented in the data it draws on. ## Screening before you field: a workflow, not a single tool The instruments are complementary, not competing, and the sequence matters more than any single choice: - Use identifying and observing tools to define the target group and see how current visitors already behave. - Use a controlled experiment to screen hypotheses before committing fieldwork budget: which concept, message, or price is worth testing further. - Refine the research instrument itself. A simulated pass on draft questions surfaces confusing phrasing or missing options before they reach a real panel. - For high-stakes calls, such as a multi-million-euro media buy, a final price, or a regulatory submission, move to recruited human participants. Subconscious can validate a study with real participants without changing the underlying causal question. Because the earlier steps already narrowed the field, that human study is smaller, more targeted, and more defensible than one built from a cold start. Audience reach and recruited participation are two different things worth keeping separate. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people; that scale describes who a study can reach, not a promise that 800 million people are personally recruited and interviewed. ## How Do You Choose the Instrument for the Decision in Front of You? The practical rule: match the instrument to the question, not to whichever tool is fastest to open. If the question is who the audience is, use identifying tools. If it's what they're already doing, use observing tools. If it's why they'd choose one option over another, or what would change that choice, that's a controlled experiment. Save recruited human participants for the calls where the cost of being wrong, or the need for outside-defensible proof, is highest. To see how this fits into a live research process, /how-we-work walks through the setup, and a demo is the next step for teams ready to test a specific decision. [figure: Three rows, each naming a layer plus the question it answers and the one it can't: Identifying (who, not why), Observing (what, not why non-visitors stay away), Asking (why, not population-scale proof).] --- # Audience Mapping or Causal Testing: Choose the Right Instrument URL: https://subconscious.ai/blog/comparisons/audience-mapping-causal-testing-choose-right-instrument Published: 2026-08-13 The instrument should match the decision. Audience mapping describes groups and their observable digital behavior. Open-ended dialogue develops possible explanations and creative directions. A controlled causal experiment compares defined actions to estimate which one changes buyer choice. Confusing those jobs can put budget behind a segment description or plausible conversation that never tested the proposed action. [figure: Four-step path left to right: map the audience, generate candidate actions, define the outcome to change, then test the action causally, showing the handoff from mapping and dialogue to a controlled experiment.] ## Match the method to the question Audience mapping fits the starting question: who is present, how do groups differ, and where can a campaign reach them? A current product page in this category describes segments built from social conversations or marketer-chosen traits (audience-mapping product description). Open-ended dialogue fits an earlier creative question: what objections, interpretations, or message directions should the team consider? It expands the option set but cannot establish that a generated response represents a market or that a proposed message will change behavior. A controlled causal experiment fits the action question: which message, price, or positioning alternative changes choice for the defined audience? Subconscious compares specified alternatives in a decision-specific experiment, estimates differences between actions, and reports uncertainty only when the study design supports it. ## When should you switch to causal testing? Move to causal experimentation once the team can name: - the audience whose choice matters; - the messages, prices, or positions under consideration; - the choice or behavior the campaign is meant to change; and - the decision the team will make from the result. Without those elements, a causal study has no clean intervention or outcome. With them, the team can test whether one action changes that group's choice. ## A campaign decision in practice Suppose an audience map identifies a promising buyer group and the creative team develops three positioning directions. The map defines whom to study; the creative work supplies the alternatives. The unresolved decision is which position to fund. A useful experiment holds the audience and outcome definition constant while varying the positioning action, addressing the budget decision directly rather than inferring effectiveness from affinity, engagement, or eloquence. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That figure describes audience reach, not the size of a recruitable participant pool. When the stakes require human evidence, Subconscious can test or validate the same study with real human participants without changing the causal question. ## What can the experiment still not establish? A controlled study is only as useful as its audience definition, alternatives, outcome, and design; weak inputs produce a precise answer to the wrong question. Naming what a method does not cover is what lets a buyer size the remaining gap before committing budget. Subconscious does not replace the observation of social conversations or digital audience mapping needed for campaign and influencer targeting. It does not replace direct qualitative work during problem discovery or idea development. The boundary of a method belongs next to its claim so a buyer can weigh both before deciding. Real-human validation narrows uncertainty for the same causal question. It does not turn the study into an observed usability session, guarantee post-launch performance, or remove the need to monitor the market after the decision ships. ## Bring a defined choice, not a broad audience question Start with audience mapping to find and describe the group. Use direct qualitative exploration to generate candidate actions. Once the audience, alternatives, and outcome are defined, review the research method and the study workflow. Teams with a decision ready to compare can bring it to a demo. --- # AI for UX Researchers: Add Speed Without Losing Depth URL: https://subconscious.ai/blog/buyers/ai-ux-researchers-add-speed-without-losing-depth Published: 2026-08-13 AI adds speed to UX research by running behavioral simulations that narrow concepts, segments, and questions before human researchers validate the survivors through real sessions. UX research teams face two demands at once: move faster and preserve rigor. Behavioral simulation can accelerate discovery and pre-testing, while human researchers retain responsibility for study design, interpretation, and validation. [figure: Four-step path: frame the decision, simulate to narrow segments, run human sessions on survivors, then the researcher interprets the results.] ## Where AI can help a UX researcher AI tools cover several jobs. Some synthesize transcripts, tag themes, analyze open text, or help draft interview guides. Simulated-buyer research serves a narrower purpose: comparing product or message actions across defined audiences before a team commits to a build or full human study. Useful applications include: - early discovery that turns a broad problem into testable hypotheses; - concept pre-testing before prototype investment; - comparison across audience definitions; - interview-guide rehearsal to find ambiguous or leading questions; - rapid checks between larger human studies. ## A research process that keeps humans in the loop ### Frame the decision Name the product action, target audience, alternatives, and behavior to compare. A good brief names which decision the research must change. ### How do you run an early comparison? Use decision-specific experiments to compare concepts or messages. If the team is considering five real user sessions for each of four segments, an early simulation can help decide which segments and questions deserve that investment. ### Why is human research still necessary? Real participant work remains the validation layer. Focus the study on the hypotheses that survived the early screen. No simulated result replaces observation of actual use, individual context, or the relationship between researcher and participant. ### Interpret the evidence Automation can reduce transcription and first-pass coding work. The researcher still checks themes against raw material, resolves contradictions, and translates findings into product decisions. [figure: Two-column comparison. Left, simulation can show: aggregate pattern, which question to ask next. Right, only human research confirms: individual behavior, consent and data handling, researcher-participant relationship.] ## What limits should UX teams keep visible? Aggregate patterns are easier to reproduce than individual behavior. Persona prompting can collapse variance, flatten demographic differences, and react to small prompt changes (Investigating Persona Collapse and Homogenization in Large Language Models, arXiv). A simulation may identify a question worth asking without proving that real customers will act the same way. Privacy also requires a separate review. Do not send participant data to a third-party system until the team has confirmed consent, data handling, retention, and the applicable legal requirements. Synthetic audiences reduce some collection concerns, but they do not make every workflow compliant by default. The best use of AI is not less research. It is earlier comparison, better questions, and more human attention on the decisions that need it. --- # From Uncertainty to Insight: What Bayesian Reasoning Means for a Business Decision URL: https://subconscious.ai/blog/methods-and-validation/uncertainty-insight-what-bayesian-reasoning-means-business-decision Published: 2026-08-13 A marketing or analytics leader deciding how much stock to buy, how to price a product, or when to worry about churn usually has a forecast in hand. What that forecast rarely carries is an honest statement of how wrong it might be. A single point estimate, such as "sales will be 1,200 units next month," hides the range of outcomes that are also plausible given the data. Commit budget against that number alone, and the business gets blindsided when real demand lands outside the range nobody wrote down. Bayesian reasoning exists to fix that gap. It does not replace machine learning; it changes what a decision-maker is allowed to see before acting. ## Why a point estimate is not enough Traditional machine learning models are tuned to produce one answer. They are fit on historical data, optimized for accuracy on a validation set, and used to generate a prediction. That workflow works well when data is abundant and the cost of being wrong is small. It works poorly when either condition breaks down: when a business has too few observations to trust a black-box function, or when the decision at stake (a stocking commitment, a price change, a churn intervention) is expensive to reverse. The core Bayesian idea is simple: start with a prior belief about an uncertain quantity, update it with new data through a likelihood function, and arrive at a posterior, an updated belief that blends what was already known with what the data just showed. With little data, the posterior stays close to the prior. With more data, it shifts toward what the data says. Either way, the output is a distribution of plausible values, not a single number. That distinction matters most when data is scarce: new products, small markets, or infrequent events like churn spikes. A model that requires large samples is the wrong tool there; a Bayesian model that starts from a stated prior and updates as evidence arrives is built for it. ## Where the uncertainty comes from Bayesian models are not automatic. Building one requires specifying, in advance, an assumption about how the data was generated: the process a business believes produces the sales, spend, or churn numbers it observes. That upfront cost is real: a data scientist has to think about the shape of the problem before fitting anything. The payoff is a model that can be interrogated, updated, and reasoned about, instead of a black box that only answers the question it was trained for. Hierarchical modeling extends the same idea across related groups. Instead of estimating each marketing channel, region, or product line in isolation, a hierarchical model lets information flow between them: a channel with thin data can borrow strength from the pattern across all channels, while still preserving its own individual effect. This is one reason Bayesian methods show up often in marketing attribution and mix modeling: channels rarely have identical data volume, and a hierarchical structure keeps a low-volume channel from producing a wildly unstable estimate. ## The decision this changes Uncertainty quantification is not academic. A posterior that is still wide, spanning a broad range of plausible unit counts, tells a buyer to gather more evidence, hedge the order, or delay the price change. What justifies acting is where the posterior mass sits relative to the decision threshold, not how narrow the posterior is; a narrow posterior can be confidently wrong. A point estimate alone cannot make that distinction; a distribution can. This is the same reasoning that sits behind causal experimentation more broadly. Subconscious runs controlled causal experiments on a simulated population that test product, pricing, messaging, and go-to-market actions before capital gets committed, and returns the resulting effects with confidence intervals attached: not a single predicted lift, but a range produced by a procedure with a stated, pre-chosen confidence level. A capability's boundary belongs next to the claim itself. A team gets confidence intervals around preference and willingness-to-pay parameters from a controlled experiment, not the demand or churn posterior a hierarchical Bayesian model would provide, without first building and maintaining that modeling infrastructure in-house. When the decision depends on validation beyond a simulated estimate, Subconscious can test or validate studies with real human participants, carrying the same causal question from a simulated experiment into a real-human study rather than switching to an unrelated method. ## What this does not replace Bayesian data science, done properly, is a modeling discipline: specifying priors, checking them against domain knowledge, fitting hierarchical structures, and validating posteriors. That discipline still requires expertise, whether built in-house or brought in as outside modeling help. A causal experimentation platform is a different tool for a related problem: testing which action produces the better outcome, with a confidence interval attached to the result. Naming what a tool does not do lets a buyer check the fit before committing budget. It does not replace an in-house Bayesian modeling team, and it does not treat every study as a fully specified conjoint or MaxDiff design; those are structured techniques within a broader toolkit, not synonyms for it. A capability claim needs its scope stated alongside it. The audience a causal experiment can reach and the participants recruited for a real-human validation study are two distinct capabilities: reach is not the same claim as recruitment, and neither should be read as a claim about sample size for any specific study design. [figure: Two-column comparison. Left: Bayesian modeling answers how uncertain a forecast is. Right: causal experimentation answers which action produces the better outcome. Neither replaces the other.] ## The practical takeaway Recent applied work continues to formalize this approach for marketing decisions specifically. A 2025 paper on Bayesian marketing mix modeling works through building hierarchical structure into channel-level spend and response estimates so that low-data channels do not distort the overall attribution picture (Bayesian Marketing Mix Modeling for Roy Ravid Lemonade, Inc., arXiv). A related 2023 framework addresses cross-channel effects and channel influence bias directly, which is a confounding problem that uncertainty quantification alone does not resolve (A new framework for Marketing Mix Modeling: Addressing Channel Influence Bias and Cross-Channel Effects, arXiv). The underlying lesson holds whether a team builds the Bayesian model itself or turns to a platform that returns causal effects with confidence intervals: a decision made against a range of plausible outcomes is better-informed than one made against a single number that hides how uncertain it really was. Before committing budget to the next stocking, pricing, or churn decision, ask what the estimate's uncertainty actually looks like, and whether that uncertainty is narrow enough to act on. Explore how Subconscious tests actions under uncertainty. [figure: Path diagram: a single point estimate feeds a Bayesian update producing a posterior distribution, which splits into a wide posterior leading to gathering evidence or hedging, and a narrowed posterior leading to acting.] --- # Persona Simulation Tools vs. Causal Choice Experiments: A Buyer's Comparison URL: https://subconscious.ai/blog/comparisons/persona-simulation-tools-vs-causal-choice-experiments-buyers-comparison Published: 2026-08-13 The persona simulation and synthetic-research market splits into four categories: conversational persona platforms, data-grounded persona generators, template builders, and specialized simulators. None of the four answer the question that actually drives a launch, pricing, or positioning decision: which specific action changes what a customer chooses. ## Four categories, one gap Conversational persona platforms let a team query a synthetic persona like a chat partner, asking it to react to a message, a feature, or a pitch. Data-grounded generators build a persona from analytics, CRM records, or interview transcripts, trading conversational flexibility for a documented data source behind each trait. Template builders produce a static one-page persona document for a deck or a kickoff meeting, with no simulation layer at all. Specialized simulators cover narrower jobs, such as scripted engineering tests against a synthetic user or documentation testing. All four produce a description of a customer, not a controlled comparison between real decisions. ## What is the real decision underneath the category? The real choice is between a tool that generates persona conversations or documents, and a platform that runs a controlled experiment to determine which specific action changes customer choice. A team that buys a persona-chat or template tool for a question that needs a causal, replicable answer ends up shipping a positioning, pricing, or launch decision with no evidence it changes real behavior. A persona will agree with almost anything asked of it in conversation; agreement is not evidence of a choice. ## How does a controlled choice experiment answer it? Subconscious runs randomized, controlled discrete choice experiments against the specific action a team is deciding between: a price point, a headline, a feature bundle, a launch sequence. The discrete choice experiment itself is decades-old applied economics, not a new invention: it is the reference method built on McFadden's Nobel-recognized framework for modeling how people choose among a fixed set of alternatives. That lineage, not a benchmark percentage, is the rigor bar a comparison should use. Subconscious's own application of the method is documented on the replication leaderboard. ## Where the two approaches actually differ Use the row that matches the question. A team that needs a shareable persona document for a kickoff still needs a template builder. A team that needs to know whether Price A or Price B changes signup rate needs the experiment. ## What does the causal approach not replace? Subconscious does not replace open-ended persona conversations for early ideation, and it does not replace direct human interviews for exploratory discovery. Those are different jobs: understanding a customer's language and context, rather than comparing fixed alternatives. Audience-graph reach used for targeting and recruitment is also not the same thing as a recruitable research panel. ## From simulated experiment to real-human validation When a decision is large enough to warrant it, Subconscious can test or validate the same study with real human participants, without changing the causal question being asked. That matters most for high-stakes launch, pricing, or positioning calls. ## Choosing the right tool for the decision in front of you Start from the decision, not the tool category. A team that needs a document or a conversation partner should buy a persona or template product. A team that needs an answer that holds up under scrutiny needs a controlled choice experiment. See how Subconscious runs a study or book a walkthrough to scope which one applies. [figure: Two-column comparison: persona and template tools yield a conversation or document; a controlled choice experiment yields a causal effect with a confidence interval, plus optional real-human validation.] --- # AI Market Research Automation Tools in 2026: Choose the Bottleneck URL: https://subconscious.ai/blog/use-cases/ai-market-research-automation-tools-2026-choose-bottleneck Published: 2026-08-13 Market research automation in 2026 covers three different jobs: collecting human responses, generating simulated responses, and analyzing research output. The right tool depends on which step is limiting the team. [figure: Decision path from "what's the constraint?" branching into collection, simulation, and analysis, each with its bottleneck, converging on: automate only the binding layer.] ## What is the collection and fielding layer? Platforms such as Cint, Lucid, and Prolific recruit respondents, field questionnaires, and return datasets for analysis (Prolific: participant recruitment). Planning examples for hard-to-reach samples range from 50 to 150 EUR per complete and 24 to 96 hours for fielding, with complex programs taking weeks. Use this layer for verified human respondents or primary evidence. Recruitment quality, sample design, and response quality require scrutiny. ## What is behavioral simulation? Simulated-buyer tools compare product, price, message, or GTM actions without recruiting a new panel per exploratory question. Results can arrive in minutes. Published category examples describe stated-preference agreement moving from an “interesting demo” in 2023 to 80 to 95 percent against selected human benchmarks in 2026. Those ranges do not prove accuracy for a new decision. Aggregate agreement is easier than individual fidelity, and prompt design, calibration, and validation matter. Subconscious belongs in this decision-specific layer. It uses causal experimentation and discrete-choice-style modeling rather than generic roleplay. The supported claim is directional comparison across configured actions, with uncertainty language only when the study supports it. ## What is the analysis and reporting layer? Platforms such as Dovetail, Notably, Looppanel, and Voxpopme apply AI to coding, theme extraction, sentiment analysis, and report drafting. Category planning examples claim 60 to 80 percent less time in analysis and reporting. Treat that range as an example, not a guaranteed saving. Analysis automation cannot repair a weak sample or poorly framed question. Human researchers must check themes against the underlying evidence. [figure: Decision path from "10 hypotheses a day, or 1 per quarter?" One branch leads to automating exploration; the other to preserving human validation. Both converge on automating one bottleneck at a time.] ## Match the tool to the constraint One planning example burns through the full year's research budget by June because respondents cost too much per study. Compare fielding and simulation options when recruitment is the binding cost. If a six-week cycle is caused by slow fielding, a synthetic exploratory screen may narrow what needs human validation (how a controlled study is run). If transcripts take three weeks to analyze, fix the analysis layer instead. Some planning models propose replacing 50 to 80 percent of stated-preference exploration with synthetic work. Another example runs the three layers in sequence for two quarters and targets two to three times the research surface, including 12 exploratory panels in one week. These are workflow examples, not Subconscious guarantees. A more useful procurement question: does the team need to test 10 hypotheses in a day, or validate 1 hypothesis per quarter? Start with that decision, preserve human validation where it matters, and automate one bottleneck at a time. Review the research basis for the simulation layer, or discuss a specific decision. --- # How to compare simulated and human experimental results URL: https://subconscious.ai/blog/methods-and-validation/how-compare-simulated-human-experimental-results Published: 2026-08-13 Compare simulated and human experiments only when they measure the same alternatives, population, and outcome. Then test whether both experiments rank the alternatives in a similar order. A high rank correlation supports agreement for that study. It does not prove universal accuracy. This is the validation discipline behind Subconscious for ordering claims. A causal experiment earns weight by reproducing a human baseline's ordering under matched conditions, not by producing a plausible response; rank correlation does not validate effect magnitudes, attribute coefficients, WTP, or confidence-interval width. [figure: A four-step scale of rank correlation values from below .52 to .96, each paired with a scatter pattern from scattered points to a near-perfect diagonal line.] ## Define the comparison before reading the result The unit of comparison is a matched experiment. Subconscious can use the same intervention, choice set, target population, and measured outcome for simulated and recruited-human runs. This distinction matters because a model can match an overall average while missing the ordering of alternatives or the response of an important segment. Validation should preserve the decision a buyer will make. ## How was the human baseline established? The historical method grouped responses from a published experiment with 25 levels into three subsets. Pairwise comparisons between the subsets produced Spearman rank correlations from 0.52 (p=.006) to 0.86 (p<.001). Publishing the baseline's own caveat keeps the threshold honest. The analysis used at least 0.52 as its within-human agreement baseline, though this subset-based estimate is likely attenuated relative to a full-sample comparison and should not be treated as symmetric with a full-sample simulation result. [figure: Three pairwise comparisons between subsets of a human experiment.] [figure: A second pairwise comparison between human response subsets.] [figure: A third pairwise comparison between human response subsets.] The source used a rank correlation of at least 0.52 as this threshold. Rank correlation measures order agreement, not variance explained in the underlying decisions, so the threshold does not imply a percentage of variation explained. Keep that interpretation tied to the historical experiment. It is not a general product guarantee. ## How do you read a rank-correlation chart? Spearman rank correlation measures whether two result sets order alternatives similarly. The coefficient does not require the values to be identical. [figure: Rank correlation .32 with p .234. The points are scattered and the chart is outlined in red.] This miss sits on the leaderboard next to the hits. A coefficient below .52 did not meet the historical baseline, though the p-value shown tests only whether the correlation differs from zero, not whether it reaches .52. [figure: Rank correlation .54 with p .040. The points form a wide diagonal band and the chart is outlined in green.] Naming the spread here is what lets a buyer check the claim. The .54 point estimate clears that study's threshold, but its confidence interval is wide enough to overlap much lower values, so the wide spread still matters. [figure: Rank correlation .76 with p .001. Points follow the diagonal with visible spread.] The .76 result shows stronger agreement in ordering. [figure: Rank correlation .96 with p less than .001. The points form a clear diagonal line.] The .96 result shows very strong agreement for the compared experiment. ## What can the comparison support? A matched result can support a narrow claim: the simulated study recovered the ordering found in the human study under the tested conditions. It cannot show that every audience, intervention, or outcome will behave the same way. Subconscious uses this distinction to separate a tested decision from synthetic roleplay. Agreement and disagreement both become useful evidence when the intervention, population, outcome, and comparison rule remain fixed. The company publishes a replication leaderboard so a buyer can inspect the human baseline, result, and failure condition rather than accept an unqualified accuracy claim. Current replication claims belong with the causal fidelity paper, where the metric and evidence can be defined. They should not be derived from the historical 0.52 threshold on this page. Use the counterfactual causal inference guide to see how model assumptions affect an estimated effect. ## Limitations A correlation number without its limits is marketing copy. Rank correlation does not measure calibration, individual-level fidelity, subgroup validity, or the cause of disagreement. A complete validation program should examine those questions separately and report failed replications as well as successful ones. The charts and thresholds above describe a historical validation method. They require evidence review before publication as current Subconscious proof. The decision rule is narrow: report what replicated, what did not, and which conclusion the evidence can support. --- # AI Research Panels: What They're For, and Where the Answer Needs to Come From Real People URL: https://subconscious.ai/blog/use-cases/ai-research-panels-what-theyre-where-answer-needs-come-real-people Published: 2026-08-13 A multi-persona AI panel puts several synthetic personas in the same session and asks them all to react to the same concept, message, or positioning at once, instead of interviewing one persona at a time. That's useful for a product or marketing lead who needs an early, directional read before committing budget. This page answers a narrower question: when is a panel discussion enough to act on, and when does the decision behind it need a controlled, causal experiment? ## What is a research panel? In a traditional market-research panel, real participants are recruited to respond to research questions, often repeatedly over time. Formats vary by who's recruited: consumers, B2B buyers, or subject-matter experts. All of them run on real scheduling, real incentives, and a real wait between fielding and analysis. An AI research panel replaces those participants with configured personas: each one defined by demographics, psychographics, role, industry, and behavioral profile. A session can run from two to fifty personas, and because they respond simultaneously and in character, the output is a structured comparison of how different segments talk about the same issue. ## What a panel is good for - Early hypothesis generation. Before a concept, message, or positioning idea is worth testing formally, a panel session surfaces which reactions, objections, and phrasings are worth investigating. - Directional depth over statistical breadth. A panel is built to expose the texture of different perspectives and the reasoning behind a reaction, not to produce a representative estimate of how a market will respond. - Segment comparison. Running the same prompt across differently configured personas shows where reactions diverge, useful groundwork for pricing, packaging, or feature-prioritization questions. - Structured discussion without groupthink. Traditional focus groups carry failure modes: groupthink, dominant personalities, and social-desirability bias, where a participant answers to look good rather than to say what they think (SAGE Journals, Qualitative Health Research). This caveat sits next to the strength it qualifies, the same way a public leaderboard shows misses next to hits. Because each persona responds independently, a panel avoids the groupthink and dominant-personality problems, though social desirability applies differently to a simulated respondent than a real one. ## Where does a panel fall short? Naming where a panel stops working lets a buyer check the claim before spending against it. A panel session is simulated, not real. It isn't a substitute for a randomized, causal experiment when the decision at stake is high-stakes: a launch, a price, or a positioning spend. Treating a directional panel discussion as proof of what a market will do is the failure mode a controlled test exists to prevent. ## What the decision actually needs Publishing this limit is what keeps a directional read from being read as proof. A panel can tell a team which messages feel stronger to a set of configured personas. It cannot tell them which message will change behavior in the market, because a simulated persona's reaction is not a causal estimate. For the highest-stakes version of a question raised in a panel, the next step is a controlled experiment against a real audience, with confidence intervals attached rather than a synthesized opinion. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That's a distinct capability from recruiting a panel of participants: the audience graph describes who a study can be run against, not a pool of people recruited and waiting. ## Moving from directional signal to a decision you can defend [figure: Five-step path: define the decision at stake, run a directional panel discussion, design a causal experiment from its sharpest question, validate with real participants, then commit budget with a confidence interval.] - Define the decision at stake. Not "what do people think of this concept" but the actual call that follows: ship it, price it, or fund the spend. - Run a directional discussion. Use a panel to surface the range of reactions and sharpen the research question before committing to a formal test. - Design a causal experiment. Take the highest-stakes version of that question into a controlled test that isolates cause from correlation, rather than reading intent off simulated reactions. - Validate with real participants. When the decision depends on it, the same causal question can move from a simulated experiment to a study with real human participants: the question stays fixed, only the source of the response changes. - Commit budget with confidence. A decision backed by a causal estimate with a confidence interval is a different kind of evidence than a directional panel read, and it's the standard a high-stakes call deserves. [figure: Two columns: Panel discussion (configured personas, directional read, no estimate) vs Causal experiment (real 800M-person audience graph, causal estimate, confidence interval). Arrow: "sharpens the question."] ## When does this not apply? Not every question needs this ladder. If a panel session is being used to sharpen internal thinking, draft messaging variants, or rule out weak directions before anything is spent, the directional read is the right tool for that job on its own. The causal step matters when the answer changes what gets funded. Ready to take a directional finding to a defensible test? See how Subconscious runs causal experiments, browse prior studies, or book time to scope the specific decision. --- # NielsenIQ vs Causal Testing: Choosing Before You Ship a CPG Change URL: https://subconscious.ai/blog/comparisons/nielseniq-vs-causal-testing-choosing-before-ship-cpg-change Published: 2026-08-13 A VP of Consumer Insights at a CPG or retail company already has NielsenIQ panel and point-of-sale data. The open question is what to do about a price change, a new claim, or a packaging update that has never run in-market: greenlight it on historical panel trends, or test the specific action first. ## What does NielsenIQ measure? NielsenIQ aggregates retail sell-through, point-of-sale, and omnichannel purchase data. It tells a category team what already moved: which SKUs sold, at what price, through which channel, over what period. That is a different question from what would happen if a team changes something it has never tried. Panel and point-of-sale data describe the market as it existed. They do not run a controlled comparison of an untried price point, claim, or pack design against an alternative. ## What is the cost of greenlighting on panel data alone? Historical trends can look stable right up until a price move, reformulation, or claim underperforms in a way the panel never observed. The write-off shows up at the scale of a go-to-market cycle, not a study fee: a national rollout, a shelf reset, or a season of trade spend built around a launch that missed. ## Testing the specific action before it ships Subconscious runs controlled discrete choice experiments on synthetic populations to estimate the causal effect of one proposed action, such as a specific price point, claim, or pack change, before it launches, using discrete choice and Mixed Logit-style modeling rather than aggregating what already happened. When the decision calls for it, a team can move from a simulated experiment to real-human validation on the same causal question. This does not replace NielsenIQ's ongoing category measurement. It answers a narrower question: which of the specific alternatives under consideration is more likely to move the outcome, before any of them ship. ## Where each method fits ## Proof and its limit The causal-effect estimates rest on discrete choice experiments and Mixed Logit and ICLV modeling, validated against real human studies. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, detailed in the causal fidelity paper. It is a validation result, not a claim about any single CPG pricing or claims decision. Subconscious does not aggregate historical retail sell-through or omnichannel purchase data the way NielsenIQ's panel does. It answers the causal effect of one proposed action, not a continuous read on category performance. ## Which method should a team run first? A team with an untried price point, claim, or pack change to evaluate before committing capital should test that specific action first, then use NielsenIQ's panel and point-of-sale data to track how the market responds once it ships. Teams already running category decisions in CPG can review past decisions tested this way or see how a study is structured before scoping a specific decision. Read more about the underlying causal research methods. [figure: Two-column comparison table with four rows: past sell-through, channel tracking, causal effect of an untried change, and sim-to-human validation. NielsenIQ covers the first two rows; Subconscious covers the last two.] --- # Best Incrementality Testing Tools in 2026, Compared by the Decision They Serve URL: https://subconscious.ai/blog/comparisons/best-incrementality-testing-tools-compared-by-decision Published: 2026-08-14 The best incrementality testing tool for 2026 depends on which question you're asking. Every tool in the category (Meta Conversion Lift, Google's Meridian GeoX, Measured, Haus, Sellforte, Recast/GeoLift) measures spend that is already live in market. A randomized discrete-choice experiment measures a price, message, or feature before spend commits, which none of the lift-test tools can do. A VP of growth comparing vendor renewal quotes against Google's free geo tool is really choosing between those two questions, not between vendors. - Platform-native lift tests (Meta Conversion Lift, Meridian GeoX) are free and strictly retrospective. - Vendor platforms (Sellforte, Measured, Haus, Recast) differ mainly in test-type coverage and channel reconciliation, not in whether they can test something before launch. - DIY geo-holdouts are cheapest and most underpowered: their sample size is markets, not people. - No tool in the category can evaluate a price, message, or product that has not shipped yet; that requires a randomized experiment run before spend commits, not a lift test run after. - A framing now common among measurement vendors treats incrementality results as priors that calibrate a marketing mix model (MMM), not as a standalone source of truth. ## What does incrementality testing actually measure in 2026? It measures what would have happened to sales or conversions if a specific, already-live ad spend had not occurred. Measured's own comparison draws the line cleanly: MMM shows what spend correlates with sales across history; incrementality testing shows what would have happened absent the spend you already committed. Both questions look backward. Neither tells a buyer what will happen to a campaign, price, or product that hasn't launched, because the method needs a treatment and control group already in market, and you can't randomize people into a product that doesn't exist. ## The 2026 field: platform-native tools, vendor platforms, and DIY geo-holdouts Three categories cover the market this year. Platform-native tools live inside one walled garden or one MMM: Meta Conversion Lift measures Meta spend, and Google's Meridian GeoX (announced at Google Marketing Live 2026) runs holdback, go-dark, and heavy-up geo designs that feed the open-source Meridian MMM. Vendor platforms (Sellforte, Measured, Haus, Recast) sell cross-channel geo and holdout testing as a managed service, differentiated by test-type coverage and integrations. DIY frameworks, built on open-source packages like GeoLift, give a team full control over a synthetic difference-in-differences design at the cost of building and powering it themselves. ## Comparing the tools by the decision they serve The honest comparison is which question each tool answers, before or after the money moves. ## Why are geo lift tests so hard to power correctly? Because the sample size in a geo test is the number of markets you have, not the number of people in them, which makes statistical power scarce by design. A geo experiment analyzed with synthetic difference-in-differences draws its power from how many designated market areas you can split into treatment and holdout, not from the much larger number of individual customers inside those markets (power analysis walkthrough). That same walkthrough found that with a handful of DMAs, brands often need a minimum detectable effect in the 5 to 10 percent range sustained over four to eight weeks just to see a signal above noise. A heuristic is only useful if its limits ship with it. That range moves with market count, baseline variance, and category, so treat it as a planning heuristic from one analysis, not a guarantee for your test. [figure: Two parallel paths. Top: spend goes live, then holdout markets are assigned, then lift is measured after four to eight weeks, then the result feeds the next quarter's plan. Bottom: choice scenarios are designed, respondents are randomized, preference and effect are measured, a decision is made, and spend commits afterward.] ## The calibration loop: how incrementality, MMM, and MTA fit together now Measurement vendors increasingly frame the three methods as a loop rather than competitors. Incrementality tests set the priors an MMM starts from, the MMM decides which channels are worth testing next, and MTA optimizes spend inside channels the MMM has already validated (Liftlab's breakdown of the calibration loop). The loop improves on treating the three as competing verdicts, but every input to it comes from spend that is already live. It calibrates what happened; it cannot say what to launch next. ## What none of these tools can do: deciding before you spend Every tool above shares one precondition: the spend has to exist in market first. None can be pointed at a price you haven't set, a message you haven't written, or a feature you haven't built, because there's nothing live to hold out against. For a buyer whose actual decision this quarter is which of three prices to launch, or which of two messages to run, the entire incrementality category is the wrong tool, not because it measures poorly, but because it measures too late to inform the decision at hand. ## Where a randomized discrete-choice experiment fits instead This is the gap a randomized, pre-launch experiment closes. Subconscious runs randomized experiments on a simulation of a market, validated against real human behavior, before the action is live. The experiments are analyzed with discrete choice models (McFadden discrete choice, Mixed Logit, ICLV); these are estimators, and the causal read comes from the randomization in the experiment design, the same logic that makes an A/B test causal. A method's assumptions get published alongside its results. A basic logit assumes independence of irrelevant alternatives; Mixed Logit and ICLV relax that assumption when substitution patterns matter. Naming the failure mode is what lets a buyer check it. For pricing, one caveat is non-negotiable: stated willingness to pay runs high relative to real purchases unless the design ties responses to real stakes. A known direction to correct for, like a geo test's power constraint. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. The leaderboard publishes the miss next to the hit so a buyer can check the number themselves. It is a validation result, not a guarantee for a new market, per the causal fidelity paper, and the protocol accounts for the risk that published studies sit in a model's training data. Per-market performance is on the leaderboard; estimator details are in methods and validation. ## Which tool should you actually pick? It depends on which question you're actually asking. If the question is "did the spend I already committed work," pick from the table above by budget and channel: Meridian GeoX if you're already on Meridian MMM and want a free option, Sellforte if you need broad test-type coverage, Haus if you want a lighter managed contract, GeoLift if you'd rather own the model in-house. If the question is "which price, message, or feature should I launch," none of those tools can answer it, because all of them require the thing you're deciding on to already be live. That decision needs a randomized experiment run before spend commits, not a lift test run after. Some teams need both: a discrete-choice experiment to decide what to launch, then a geo or platform lift test to confirm it worked once it's live. Recent applications of this sequencing are documented in the case studies archive. Write down which of the two questions your next budget cycle is asking, then check the leaderboard for replication performance in a market like yours. If you want to talk through where a pre-launch experiment fits next to the incrementality stack you already run, the team is at /meet. --- # Incrementality Testing: What It Proves, What It Cannot, and How to Run It Causally URL: https://subconscious.ai/blog/methods-and-validation/incrementality-testing-what-it-proves-how-run-causally Published: 2026-08-14 A marketing leader deciding next quarter's budget needs to know one thing about incrementality testing before commissioning one: it answers the wrong question at the wrong time for that decision. Incrementality testing proves that a specific campaign, already launched with real budget, caused a measurable lift over a holdout or control group. It cannot tell a marketing leader, before that budget goes out the door, which creative, price point, or channel would have produced a larger lift, because the test has no data on the options that never ran. That gap is where budgets get spent twice: once on the campaign, once on the test that only confirms what already happened. - Incrementality testing proves causal lift for the campaign you already ran, using a holdout, test/control split, or geo-experiment. - A number without its limits is marketing copy. It has no counterfactual for creative, price, or channel options that never received spend, so it can't compare what you did to what you didn't do. - Platform-run lift studies (Meta, Google) are the most common version in use, and the seller grading its own media is a conflict of interest the industry now names openly. - Independent geo-experiments, built on methods like Meta's open-sourced GeoLift, are more rigorous but require real budget, weeks of runtime, and enough markets to reach significance. - The decision that actually moves next quarter's budget, which option to fund next, has to be answered with a randomized experiment before the campaign launches, not after. ## What Does Incrementality Testing Actually Prove? It proves that one specific campaign, already run, caused an outcome beyond what would have happened anyway, not that a different campaign would have caused more of it. A geo-experiment withholds treatment from a set of control markets and compares the difference against markets that ran the campaign; a platform lift study does the same with user-level holdouts inside a single ad account. Across 225 geo-based tests run between August 2024 and December 2025, median incremental ROAS came in at 2.31x, with 88.4 percent of tests reaching statistical significance, according to a 2026 geo-testing guide's dataset, one vendor's client base rather than an industry census. The same dataset found that incremental ROAS differed from platform-reported ROAS by 30 to 70 percent, which is the clearest evidence that the platform's own number and the causal number are not the same thing. Each measurement method also has a documented blind spot: multi-touch attribution is blind to offline conversion and prone to over-crediting, marketing mix modeling overfits without an experimental check, and incrementality tests can't economically cover every channel at once, per Measured's comparison of the three. ## Why a Platform Grading Its Own Media Isn't Independent Proof Because the company selling the media is also the company scoring whether the media worked. Meta and Google run their conversion lift studies as test/control audience splits entirely inside their own walled gardens, with no external party checking the math. That structural conflict is a large part of why 52.8 percent of US advertisers report plans to add incrementality testing this year, per an AI Digital industry review. Independent geo-experiments avoid the self-grading problem by using synthetic control methods to build a counterfactual from markets that never saw the campaign, but they cost real money and time: one 2026 testing guide puts platform lift tests at roughly $30,000 to $50,000 in minimum spend over a four-week window before noise overwhelms signal, and geo tests at twenty or more markets with four to six weeks of exposure. That's the price of an honest answer to a question you've already committed to. ## Which Incrementality Method Should You Trust? It depends on what you're using it to decide. The table below compares the three approaches a senior buyer is likely choosing between right now. ## What Can't Incrementality Testing Tell You? Naming this failure mode lets a buyer check the test before trusting it. It can't tell you what would have happened with a creative, price, or channel you never launched, because every geo-experiment or lift study is built around the specific intervention it measures. GeoLift, Meta's open-source geo-testing package, constructs its counterfactual with synthetic control: it builds a weighted combination of untreated "donor" markets to estimate what the treated markets would have done without the campaign, per its published methodology. That counterfactual is for the campaign that ran, not for the campaign you didn't run. If the real question is "would a different price or a different hero creative have moved more volume," a geo-test has no answer, because it never observed those alternatives in the market at all. "arguably the most important innovation in the policy evaluation literature in the last 15 years" Athey and Imbens (2015), cited in Meta's GeoLift methodology documentation (source) [figure: A five-step path showing that incrementality testing occurs after budget is already committed and campaign launched, and concludes without answering which of the original options would have performed better.] ## Where Does the Budget Decision Actually Get Made? It gets made with a randomized experiment run before any money is spent, on a simulated population, analyzed with discrete choice models that estimate which option people would actually choose. Discrete choice estimators, McFadden's foundational discrete choice model, Mixed Logit, and ICLV, are not themselves causal methods; the causal identification comes from randomizing which product, price, or message a respondent sees within the experiment design. That's why the framing is randomized experiments analyzed with discrete choice models, not "causal methods like DCE." A plain multinomial logit also carries the independence of irrelevant alternatives assumption, meaning it treats a substitution between any two options as unaffected by a third; Mixed Logit and ICLV relax that assumption when preference share and substitution questions matter. The misses sit on the public leaderboard next to the hits. In validation testing, our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. Any confidence interval from that kind of experiment covers the effect within the simulated population, not the real market unconditionally. The current results and how they're scored are public on the leaderboard. ## Does Incrementality Testing Replace MMM and MTA? No, and 2026 practitioner consensus has settled on treating attribution, marketing mix modeling, and incrementality testing as complementary layers rather than competing ones. In practice, few teams run all three together: only 39 percent of organizations use attribution, incrementality testing, and MMM in combination, despite the methods being designed to cover each other's blind spots, per IAB/BWG Global's State of Data 2026 findings cited by House of Martech. That unified stack is still a post-spend stack: MMM runs quarterly on historical spend, MTA runs daily on historical touches, and incrementality testing runs on a campaign that already launched. None of the three answers which option to fund before the money moves. That's a separate layer that has to run first. More on how these methods fit together is in the methods and validation hub. ## How Should You Sequence These Tests? Run the pre-spend causal experiment first, to pick the creative, price, or channel most likely to win, then commit budget to that option, then use a platform lift study or an independent geo-experiment to prove the committed campaign actually worked, then feed that result into the MMM baseline for ongoing allocation. Reversing that order, spending first and asking which option was best afterward, is how a 30 to 70 percent gap between platform-reported ROAS and true incremental ROAS becomes the first time anyone finds out the campaign wasn't the strongest option available. Before your next campaign locks a creative, price, or channel, run the pre-spend comparison as a randomized experiment rather than a guess, then hold the winner to a geo-test to prove it out. If you want a second opinion on where that experiment should sit in your measurement stack, talk to us. --- # When a Marketing Mix Model Recommends a Budget Shift, Test the Claim Before You Move the Money URL: https://subconscious.ai/blog/methods-and-validation/when-marketing-mix-model-recommends-budget-shift-test-claim-before-move Published: 2026-08-14 A production Bayesian marketing mix model (MMM) can trace spend through a real funnel: upper-funnel spend shapes lower-funnel demand, demand runs into budget caps, caps shape observed spend, and observed spend produces leads. That chain is a modeling architecture, not a proof. Before a marketing analytics or measurement lead reallocates a multi-million-dollar budget on the model's recommendation, the specific causal claim behind it still needs to be tested against controlled experimental evidence. [figure: A left-to-right chain of five boxes tracing how one stage of the funnel is assumed to drive the next, with each arrow labelled as an assumption to test rather than a measured effect] ## The case study behind this pattern A marketing analytics team building a production MMM for an insurer's marketing budget ran into three problems a strong architecture alone did not solve. ### 1. Why don't campaigns change effectiveness smoothly? Real campaigns shift discontinuously: new creative, a new targeting approach, a different concept, so treating channel effectiveness as constant or smoothly trending misrepresents what happened. The team's fix segmented the timeline at known campaign-change dates and gave each segment its own multiplier, constrained by a Dirichlet prior so the multipliers stay identifiable relative shares rather than an unconstrained set that degrades sampler efficiency. Published accuracy numbers mean little without stating where they fall short. This decomposition did not improve out-of-sample predictive accuracy. It was built anyway, because the question the team needed answered, "did switching to performance creative change ROI, and by how much," is a causal question about a specific decision, not a forecasting benchmark. ### 2. Why isn't influencer spend one input? Standard MMM treats a dollar of influencer spend as fungible, but follower counts, audience quality, and conversion rates vary widely between creators. The team split influencer effectiveness into two factors: observable reach, modeled with diminishing returns so a creator with 2 million followers does not get credited with twice the impact of one with 1 million; and a latent quality factor, estimated jointly with the rest of the model, that captures the residual effectiveness a raw follower count misses. The result is a spend multiplier that shifts with whichever mix of creators was actually deployed in a given period. ### 3. New channels break standard cross-validation Channels enter mid-series as new platforms launch. Standard time-series cross-validation assumes a fixed feature set across every fold, so a channel with zero spend in an early fold either gets estimated on no signal or gets excluded, changing the model specification fold to fold. The team's time-slice cross-validation excludes any channel with zero training-set spend from that fold only, so each fold's specification matches the information available at that point in the series. A validation number that skips its own limits is a sales pitch. Fold scores are not on a common model and cannot be pooled into a single accuracy figure without qualification, while still using every channel, fully, in the final model trained on all data. ## What does the case study report as its outcome? According to the source case study, the production model's recommendations were adopted into the team's budget planning, and the engagement reports a double-digit reduction in cost per lead alongside continued MMM investment. A result posted without the boundary of where it holds is marketing. Those figures describe that team's reported result on their own data, not a benchmark that transfers to another company's funnel, and they are not a Subconscious claim. Treat them as a planning example, not a guarantee. ## Why the chain still needs testing, not just architecture Every link in that funnel is a structural assumption the model was built to encode. A well-specified MMM makes those assumptions transparent and internally consistent. Naming what a model architecture cannot prove is what lets a buyer test the claim before spending against it. It does not, by itself, prove that a given assumption reflects the true causal mechanism rather than a correlation the model was free to fit. That gap matters most at the moment a specific recommendation turns into a spending decision: raise the paid search cap, shift budget toward performance creative, concentrate influencer spend on fewer, higher-quality creators. Each of those is a claim about what would happen if the team acted on it. Google's Meridian documentation frames marketing measurement in exactly this causal-inference vocabulary: an MMM's coefficients answer "what would have happened under a different spend pattern," and that answer carries the assumptions built into the model. ## Where Subconscious fits, and where it doesn't Subconscious does not build or replace a Bayesian MMM, and does not do the censored-data handling, the campaign-segmentation modeling, or the cross-validation work described above; that is MMM construction, distinct from what Subconscious tests. What Subconscious does: once an MMM surfaces a candidate causal driver, a team can design a discrete choice experiment that presents the underlying attribute to the affected buyer segment and estimates how it drives stated choice, as evidence to weigh before committing spend against the driver. Subconscious can then validate that experiment with real human participants, without changing the causal question being tested. [figure: A left-to-right chain: observable reach with diminishing returns and a latent quality factor estimated jointly with the model combine into a spend multiplier, which sets the effectiveness credited to a creator's spend.] ## The buyer decision An MMM's output is a decisive-looking number attached to an uncertain modeling choice. Acting on it directly assumes the model got the causal structure right. Testing the specific claim first, whether this campaign change, this influencer mix, or this budget reallocation actually changes stated choice, is the step that separates a defensible reallocation from a bet on the model. Before moving budget on an MMM recommendation, that's the question worth answering: has the underlying causal claim been tested, or only modeled? Explore how Subconscious tests causal claims, see related case studies, or book time to walk through a specific MMM output. --- # One Oil Forecast, Five Independent Models: A Case Study in Trusting a Number URL: https://subconscious.ai/blog/implementation-and-procurement/one-oil-forecast-five-independent-models-case-study-trusting-number Published: 2026-08-14 A procurement team whose costs track crude oil faces a binary choice: lock in supply now at an elevated price, or wait for the market to normalize. Locking in early wastes money if prices fall. Waiting too long means paying the premium for longer, or missing the window entirely. Either mistake is expensive, and a single model's point forecast hides exactly the disagreement a buyer needs to see before committing. ## The setup: a price above its normal range WTI crude (the benchmark price for U.S. crude oil futures) was trading at $90.54 per barrel, roughly a third above the $68.26 threshold defined as "normal." That threshold marks the top quartile of where WTI sat across all of 2025: a year in which the price moved within a $55-to-$80 band and averaged $64.74. The question a procurement team needs answered isn't "will oil come down," it's "what is the probability it comes down within three months, six months, a year, and how much should that probability change a purchasing plan." The U.S. Energy Information Administration publishes its own probabilistic WTI outlook on a comparable cadence, which is the kind of independent benchmark a team should check any internal forecast against rather than trusting a single pipeline in isolation (EIA Short-Term Energy Outlook). "We now forecast the Brent crude oil spot price to average around $85 per barrel (b) in the third quarter of 2026 (3Q26)." U.S. Energy Information Administration, Short-Term Energy Outlook (source) ## Five forecasters, one dataset, no coordination Rather than fit one model and report one number, the analysts calibrated five independent Bayesian forecasting models on 19 years of daily WTI price history (2007–2026, 4,744 observations), plus four supplementary series as optional context: an oil volatility index, the S&P 500, three Asian equity indices, and a group of shipping and energy-transport stocks. Each forecaster chose its own statistical method from a library of ten time-to-event techniques (survival models, regime-switching, jump-diffusion, and others) without seeing the others' work, then a separate reviewing pass compared the five outputs, scored them on convergence diagnostics and internal consistency, and selected the best-calibrated one as the headline. That structure matters more than the specific numbers. If five differently-configured analysts independently reach the same answer, it's more likely to reflect the data than an arbitrary modeling choice. If they disagree, the disagreement itself is information: it shows a buyer where the uncertainty lives instead of hiding it behind one confident number. ### What did the five forecasters find? All five independently picked the same backbone method: mean-reversion, the idea that a price stretched away from its long-run equilibrium tends to drift back toward it. They split only on whether to add explicit jumps (sudden, discrete price moves like an OPEC announcement or a demand shock) on top of that backbone. Two forecasters used pure mean-reversion; three added jumps to account for fatter tails in the historical data. Near-term estimates cluster tightly: at three months the five forecasters range from 18.7% to 23.3%, a narrow spread relative to each individual model's own credible interval. At twelve months the spread widens and splits along method lines: the two pure mean-reversion models land near 66%, the three jump-diffusion models near 49 to 50%. That gap is a real, disclosed disagreement about whether today's elevated price is a transient spike or evidence of fatter-tailed risk. ## Checking the forecast against what actually happened A forecast is only as trustworthy as its track record on data it wasn't fit to. The headline model was validated with time-slice cross-validation: refit on an earlier window, then checked against what prices actually did afterward. Its 94% credible bands covered 74% of the held-out six-month slice, 99% of the twelve-month slice, and 100% of the twenty-four-month slice. The twelve- and twenty-four-month results are well-calibrated. The six-month under-coverage is an honest signal that the model understates near-term volatility, which means the near-term probabilities in this analysis are best read as lower bounds, not final answers. ## What it means for a procurement decision Near-term normalization is unlikely, and all five forecasters agree on that. Put a number on it: the odds sit near 77% that WTI hasn't dropped back under $68.26 by early September, so a team needing supply within about three months is mostly buying insurance by waiting. At six to twelve months the picture is genuinely balanced: close to a coin flip by six months (44%), tilting toward normalization by twelve months (66% under the mean-reversion view, closer to 49% under the jump-diffusion view). Staging purchases at trigger prices, buying more as the price crosses successive thresholds, captures early normalization while hedging the chance prices stay elevated through the year. ## What are the limits of this analysis? Five forecasters sharing one prompt, one method library, and one underlying model family is weaker evidence of robustness than five genuinely independent teams working in isolation. The agreement here rules out a single unlucky configuration, not every source of correlated error. The $68.26 threshold is anchored to a single calm year and sits close to the model's own fitted long-run equilibrium (~$70), which is why the long-horizon odds look as high as they do; a different definition of "normal" would move that number. And the models see only price and volatility history: an OPEC decision, a supply disruption, and a drop in demand all look identical to the model, so it can't tell which is behind the current elevation, only that price jumps of this size have historically reverted. None of this establishes a forecasting capability for commodity prices, and it isn't one. What it demonstrates is a discipline worth borrowing for any consequential business decision built on a single model's output: run more than one credible method, disclose where they agree and where they don't, and check the result against data the model never saw. ## Where does this discipline apply to customer behavior? Subconscious.ai applies the same discipline of multiple independent methods, disclosed disagreement, and out-of-sample checking to a different class of decision: not a commodity-price forecast, but whether a specific pricing or positioning change moves buyer behavior, tested as a controlled causal experiment with a confidence interval attached to the answer rather than a single number. Teams that need a causal answer, not a market forecast, can move that same experiment design from a simulated study to real-human validation without changing the underlying question, then bring the results into their own case studies. [figure: Comparison diagram: five oil-price forecasting models converge closely at 3 and 6 months, then split at 12 months into two groups: two pure mean-reversion models near 66%, three jump-diffusion models near 50%.] Full detail on the forecast methodology, including the historical calibration window and each forecaster's method selection, is available from the EIA's published WTI price comparisons. --- # AI Positioning Research: Testing a Value Proposition Before It Ships URL: https://subconscious.ai/blog/use-cases/ai-positioning-research-testing-value-proposition-before-ships Published: 2026-08-14 A CMO choosing between two or three competing positioning statements does not need another opinion. The decision is which statement changes how a buyer understands the category, places the product against alternatives, and decides to learn more, a question a controlled comparison can answer before the campaign budget is committed. [figure: Five-step path: write variants by framework, define specific segments, run one controlled comparison, map four gaps (understanding, competitive, value, segment fit), stress-test the winner.] ## Why the decision matters Positioning sets the frame for everything downstream: messaging, content, sales conversations, and creative. Most teams choose it by internal consensus, then find out whether it works only after months of campaign performance, sales-call recordings, and win/loss analysis have accumulated, by which point the campaigns built around the wrong frame are already live. The cost compounds. A framework that is slightly off produces messaging that is slightly wrong, which produces segmentation that targets the wrong buyer, which produces a campaign that performs 30% worse with no clear signal why. Traditional research methods (customer interviews, surveys, competitive analysis) are valuable but slow, and a single rating on a five-point scale tells a team almost nothing about why a positioning statement succeeds or fails with a given segment. ## What causes a positioning statement to succeed or fail? Positioning is not a tagline exercise. Following April Dunford's framework, a positioning statement stands or falls on five components: - Competitive alternatives. What the customer actually compares the product to, not what the team assumes. - Unique attributes. What is genuinely different about the approach, not a feature list. - Value. What that difference is worth, stated in the customer's language. - Target customer. Which segment feels the difference most urgently. - Market category. The category the product occupies, or the one it is trying to create. A statement that scores well on a rating scale can still fail on any of these five components, because a scale score does not reveal which alternative the buyer had in mind, what objection surfaced first, or which segment understood the claim at all. That gap between intended and perceived positioning only becomes visible when a buyer is asked to explain their own reasoning rather than rate a statement. ## Evidence: comparing statements instead of rating them Subconscious runs controlled experiments on simulated buyer segments rather than asking a single persona to score a statement. The comparison holds the segment constant and varies only the positioning statement. That design surfaces the same signals a strong positioning research process is built to find: - Understanding gap. Does the segment describe the product's category and function the way the team intended? - Competitive-framework gap. Which alternatives does the segment name unprompted? A segment that consistently names a product the team does not consider a competitor is a positioning signal, not noise. - Value-perception gap. Which benefit does the segment actually respond to, versus the one the team is leading with? - Segment fit. Which segment shows the strongest response, which speaks to ideal-customer-profile prioritization rather than messaging alone. Subconscious can run these controlled studies against a person-level audience graph covering 800 million real people, which matters for testing whether positioning holds across segments rather than one hand-picked persona. Subconscious can also test or validate a resulting study with real human participants, moving from the simulated comparison to a real-human check without changing the underlying question. ## A recommended decision process - Write two or three positioning variants that differ in framework, not word choice. For example, a category-first variant ("the market research platform for product teams"), a problem-first variant ("shrink the research cycle from months to hours"), and an alternative-first variant ("replace a costly research engagement with a faster comparison"). - Define three to five buyer segments with real specificity. Naming the role, company stage, and evaluation context, say, a product VP at a Series B SaaS firm shopping for a research tool, produces a more useful comparison than a generic "product leader." - Run each variant against every segment in one controlled comparison, capturing, in the segment's own words: what the product does, who it is for, the first objection or question, how it compares to what the segment currently uses, and whether the segment would investigate further. - Map the gaps across segments and variants, using the four gap types above to decide which statement to advance and which segment to prioritize. - Stress-test the strongest variant by asking what a competitor would say to counter the claim, what a reader would need to see next on a landing page to keep reading, and whether any scenario makes the positioning actively repel a buyer it should attract. Naming the estimate as illustrative is what keeps it useful for a reader who plans around it. Teams that follow a version of this pattern elsewhere describe it as roughly an afternoon of setup and a few hours of comparison, well short of a six-figure agency engagement run over a twelve-week timeline, though time and cost scale with the number of variants and segments tested and are illustrative, not a Subconscious delivery commitment. ## Where does Subconscious fit in positioning research? Subconscious's advantage over a rating-scale survey is that it tests a comparison, not a single statement, and reports what changed and for which segment rather than a single aggregate score. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. The misses sit on the public record next to the hits, so this score comes with its boundary stated. It is a validation result, not a guarantee for a new market, and is the relevant proof point for trusting a simulated comparison as a first pass before a real-human check (the causal fidelity paper). ## Limitations and failure conditions A number without its limits is marketing, so this section states plainly what the comparison cannot do. A controlled comparison of positioning statements does not replace customer discovery, sales-call evidence, competitive win/loss analysis, or observed market performance after launch. It is a pre-launch filter that narrows which statement is worth taking to market, not a guarantee of market response. A statement that wins the comparison can still underperform if the launch executes poorly, the channel mix is wrong, or the competitive landscape shifts between the test and the launch date. Teams that skip the real-human validation step for a high-stakes launch are trading a cheap check for a small amount of remaining uncertainty, not eliminating it. [figure: Five components a statement depends on: alternatives customers compare against, what is different, what that is worth, which segment feels it most, and the category the product occupies.] ## Where to go next Review the methodology and validation evidence behind the replication figure, look at a documented decision that used a controlled comparison before launch, or set up a comparison for a specific positioning decision. --- # AI Brand Positioning: Test Your Brand Story First URL: https://subconscious.ai/blog/use-cases/ai-brand-positioning-test-brand-story-first Published: 2026-08-14 A marketing leader picking which brand story to fund is making a bet that is hard to unwind. Once media and production dollars commit, the company lives with that story for months. Choose wrong and the cost is not one bad campaign. It is a stretch of budget spent teaching customers a version of the brand they find confusing or easy to forget. ## What does a brand positioning statement need to survive? A workable brand position needs a defined target, a category the buyer already recognizes, and a reason to believe that holds up once customers compare it with alternatives (Harvard Business School Online). Getting there means answering questions a team cannot settle by debating in a room: - How do target customers describe the brand today, unprompted? - Which competitors occupy the same mental shelf? - Which candidate direction shifts perception or intent, and for whom? - Which language in the story feels native to the buyer instead of internal jargon? Interviews, brand-association studies, and structured tests with recruited participants answer these questions well but slowly, and that cost is a reason teams often skip straight to a single favored direction instead of comparing several. ## Why is brand positioning testing comparative instead of a single verdict? Brand teams typically draft three to five candidate directions and need to know which one wins, and against whom. The same story can read as authoritative to one buyer segment and as inaccessible to another, so a single average reaction hides the decision that matters. This is where a controlled comparison run as a simulated experiment is more useful than reading each direction in isolation. Subconscious tests competing positioning directions or narrative drafts as alternatives against a defined buyer population and measures which one produces the strongest, most consistent response before a team commits production budget to it. ## Testing the pieces before the story is final ### Narrative drafts Present each narrative under identical conditions and ask what the company appears to do, who it appears to serve, and which claims a buyer would want proof of. A flat or negative reaction is informative: it can mean the story puts the brand in the wrong category or promises something the segment does not value. ### Candidate directions Treat the drafted directions as alternatives in one design, tested against a consistent target behavior. This is most useful when a team is split on which direction "feels right," because the disagreement gets resolved by measured reaction instead of internal opinion. ### Category language Ask buyers to describe the problem, the category, and the alternatives in their own words. Comparing that language against the brand's internal phrasing is often how a team finds out its copy sounds native to itself and foreign to the market it is trying to reach. ### Taglines and headlines Compare candidate lines on memorability, clarity, category fit, and whether they move the target behavior at all. Traditional rounds of this kind of testing carry meaningful cost and lead time; a faster comparison pass narrows the set before that spend happens, without claiming to replace it. ### Segment by segment Run the same comparison across each buyer segment separately before averaging anything. A position that lands with one group and misses with another is a segmentation finding, not noise to smooth over. ## A sequence for pressure-testing a direction before it ships [figure: Five-step path: define target segments, draft competing directions, compare responses across segments, fund the direction with the clearest lift, validate with real customers before the campaign goes wide.] The workflow a team can run before committing to a single direction: - Define the target segments. Pick the two to four buyer groups the strategy actually depends on. - Draft competing directions. Write three to five distinct positions rather than one preferred story plus straw men. - Compare across segments. Run every direction through the same design and the same measured behavior, segment by segment. - Fund the strongest direction. Pick the direction with the clearest, most consistent lift, not the one with the most vocal internal support. - Validate with real customers. Confirm the funded direction holds up before the campaign goes wide. ## What can't a simulated brand positioning test settle? A simulated comparison is weaker on genuinely novel ideas that break the category's existing patterns; there is less established behavior to draw on. It also cannot stand in for the sensory or emotional weight of an actual brand encounter: a film, a retail environment, packaging, or a sound identity all need a real audience in the room. Use the simulated pass to cut weak directions and surface hidden assumptions early. Reserve real people for the experiential dimensions and for the final call before a story goes into wide production. [figure: Five separate checks before a brand story ships: narrative drafts, candidate directions, category language, taglines, and a segment-by-segment pass, as distinct items, not one combined judgment.] ## Carrying the direction to real customers The point of the comparison stage is not to skip real research. It is to walk into that research already knowing which direction deserves the budget. Subconscious can test or validate the resulting study with real human participants, using the same causal question the simulated comparison was built around. Teams that want to see the comparison method applied to their own candidate directions can talk with the team. --- # Continuous Feedback Monitoring vs. Controlled Behavioral Experiments URL: https://subconscious.ai/blog/comparisons/continuous-feedback-monitoring-vs-controlled-behavioral-experiments Published: 2026-08-14 Continuous feedback monitoring tracks whether customers notice or react to a change, while a controlled behavioral experiment isolates which specific variable caused a behavior change and by how much. A pricing change, a positioning shift, or a feature bet needs more than a reaction. It needs a test that isolates which specific change moved customer behavior, and why. A day-to-day product tweak usually doesn't need that much rigor. Confusing the two wastes either a research cycle or a decision. Discrete choice experimentation is the research method for that isolation problem: it holds some attributes constant, varies others, and estimates which attribute change actually drove the choice (ISPOR Conjoint Analysis Good Research Practices Task Force Report, ScienceDirect / Value in Health). A reaction stream and an open-ended research conversation can feed into that design. Neither is that design by itself. ## Two workflows product teams often mix up Some tools give a continuous stream of reactions: a lightweight signal that flags whether something might be working, checked as often as a team ships. The question is narrow: did people notice this change? Other workflows are deliberate. A team defines the question, designs a comparison between specific alternatives, runs it, and extracts an answer before a launch, a price change, or a positioning shift. The question is broader: which alternative moves behavior, and by how much? Only a controlled comparison answers it. Neither cadence is universally better. The mistake is applying the wrong one: treating an ambient reaction trend as proof that a pricing change will work, or running a full controlled study on a routine tweak a lighter check would have answered just as well. ## Cadence is not the same as causal certainty A method's limits belong on the record next to its use cases, so a buyer can check the fit before relying on it. A continuous feed of reactions tells a team whether something changed. It does not isolate which change caused the shift, or rule out other things that moved at the same time. That's a structural limit of always-checking-in workflows, not a flaw in execution. A controlled experiment holds everything constant except the one variable under test: a price point, a message, a feature bundle, and estimates that variable's effect on behavior. A team that needs to know "will this price move conversion" needs that answer, not a reaction trend. ## Comparing the three common approaches ## When is continuous monitoring enough? If a team ships frequently and wants a running check on whether customers are reacting well or badly to incremental changes, a lightweight continuous check fits. Naming what a tool can't do is what lets a buyer pick the right one for the decision at hand. It isn't built to isolate causes; it's built to catch something worth a closer look. ## When the decision needs a controlled experiment Getting that kind of decision wrong is expensive: a team reads a reaction trend as proof, ships the change, and later finds the trend had nothing to do with the actual driver. It needs a test that isolates the one variable in question and estimates its effect, not a directional read on general sentiment. Subconscious is built to help here. It runs controlled experiments on a simulated market, comparing specific alternatives, such as a price, a message, or a bundle, and estimating which one is more likely to change a defined behavior, with a person-level audience built for that comparison. It is the deliberate side of this distinction, proven with a randomized experiment rather than an open-ended conversation. Subconscious can also test or validate a study with real human participants, moving from a simulated run to real-human validation without changing the underlying causal question, which matters when a pricing or positioning call is consequential enough to warrant that second check. Subconscious runs controlled studies against a person-level audience graph covering 800 million real people. A capability number without its boundary is marketing copy, so the boundary goes on the record too. That graph is not a recruitable panel of standing participants; recruiting real people for the validation step above is a separate, explicit part of the workflow, not an automatic guarantee attached to every study. [figure: Three methods compared: monitoring flags if people noticed a change; qualitative research surfaces what customers say; a controlled experiment isolates which variable caused a behavior change and by how much.] ## How do you choose between them? Most strategic decisions, such as pricing, positioning, or a major feature bet, benefit from a controlled comparison. Most day-to-day development cycles benefit from a lighter, continuous check. For a decision that fits the second category, see how Subconscious approaches case studies of causal action testing, or book a walkthrough of a controlled experiment on your own market. --- # What Should Transfer to a New Context? A PyMC Walkthrough for Evaluating Causal Simulation Predictions URL: https://subconscious.ai/blog/methods-and-validation/what-should-transfer-new-context-pymc-walkthrough-evaluating-causal Published: 2026-08-14 A causal simulation predicts a segment, product, or time period it never observed directly. Before a team acts on that prediction, someone has to answer one question: which learned relationships were assumed to carry into the new context, and which were left to be re-estimated or bounded by fresh uncertainty? If that assumption is silent, the error stays invisible until the real outcome misses the simulated one. PyMC's Bayesian workflow makes this question explicit, and its worked examples are a checklist for anyone evaluating a causal simulation output, whether they wrote the model or are reviewing someone else's. ## The mechanism: what carries forward and what gets relearned A Bayesian model infers a posterior distribution over its parameters from observed data. Predicting in a new context means building a second model and deciding, variable by variable, whether each parameter's posterior draws should be reused or resampled from a prior. PyMC's sample_posterior_predictive function makes this decision by name matching: any variable in the new model whose name matches a variable already estimated in the original model reuses that variable's posterior draws by default. Any variable that has no match, or that the analyst explicitly asks to resample, gets sampled fresh from its prior, or from its conditional distribution given any already-reused parent variables, instead (pymc.sample_posterior_predictive documentation). The prior distribution assigned to a reused variable doesn't matter, since only the name match controls whether that variable's learned posterior is inherited or replaced. ## Four ways to test what should transfer ### 1. Reusing a parameter across model forms A linear model infers a slope parameter from observed predictor-outcome pairs. A second model, built for prediction only, can reuse that slope's posterior draws even though the second model's likelihood has a different functional form (a Student's T noise distribution instead of Normal). The slope is assumed to transfer; the noise structure is allowed to change. ### 2. Simulating groups a hierarchical model never saw A hierarchical model estimates group-level parameters (a shared mean and a shared spread) from a fixed set of observed groups, plus a per-group offset for each observed group. To predict an unobserved group, the predictive model reuses the shared group-level parameters but draws a fresh per-group offset from the prior for each new group, since no data exists yet to estimate it directly. In the source's worked example, this pattern predicts two new groups (called the "9th and 10th schools," extending an eight-group dataset) using the group-level mean and spread learned from the original eight, while giving each new group its own offset prior. ### 3. Forecasting under a partial-transfer assumption A time-series forecast can assume that some learned parameters hold into the future and others don't. Reusing every parameter, including the drift, produces one forecast band. Deliberately resampling the drift parameter from its prior, while still reusing the volatility parameter, produces a wider forecast band, because the model no longer relies on a possibly-stale trend estimate. A middle option keeps the learned drift but adds new noise around it, narrowing the band without pretending the drift is certain. ### 4. Recovering an unobserved variable, not just an outcome The same reuse-or-resample logic applies to variables the original model never directly reported, such as which of several underlying groups most likely produced a given data point. Reusing the posterior draws for the group-level parameters lets the predictive model back out that assignment with its own uncertainty, higher for points that sit near the boundary between groups, lower for points that clearly belong to one group. ## Why does this distinction matter for a causal simulation vendor? The same generalization question underlies any causal behavioral platform that predicts a scenario, segment, or time period it did not directly test. A platform that lets a team test an action, compare scenarios, and estimate a causal effect with uncertainty for the specific study it ran is answering a narrower and more defensible question than one that implies its estimates transfer automatically to any new population or period. Evaluating a vendor's out-of-sample claim means asking the same question this walkthrough answers for each example: which estimated relationships is the vendor assuming still hold in the new context, and where does the vendor's own uncertainty widen to reflect what it does not know? A vendor that cannot answer that question for a specific prediction is asking for trust rather than showing scope conditions. When the decision depends on evidence beyond a simulated estimate, a team can move from a simulated study to testing with real human participants without changing the underlying causal question, a different check than generalizing a model's own draws, but often the harder validation before committing budget. ## Where this method runs out A method description that skips its own scope reads like marketing copy. Stating the scope here is what lets a reader check the claim before using it. This is a general-purpose open-source Bayesian modeling mechanism. It is not a benchmark, a validation study, or a result reported by any specific vendor's platform, and no accuracy, replication rate, or customer figure should be read out of it. It does not by itself establish that any particular causal simulation's predictions generalize correctly to a new context; that still requires evidence specific to that platform and that study. A failure mode named in the open is one a reader can test for before trusting the result. Reusing posterior draws also assumes the analyst correctly identified which relationships are stable. Nothing in the mechanism itself checks that assumption. Getting it wrong, reusing a parameter that does not actually hold in the new context, produces a confident, well-calibrated-looking prediction that is simply wrong. ## What should you ask before trusting an out-of-sample estimate? Before acting on a simulated prediction for a segment, product, or period the underlying study did not directly cover, ask what was assumed to transfer, what was left to widen the uncertainty, and what evidence supports treating the transferred piece as stable. A team weighing how a causal platform structures that evidence or checking published study results is applying this walkthrough's discipline at the scale of a full causal behavioral platform. [figure: Flowchart: a variable in the new model branches on whether its name matches the original model. Matched reuses posterior draws into a narrower band; unmatched resamples from the prior into a wider band.] --- # A Price-Guessing Benchmark Is Not a Pricing Decision URL: https://subconscious.ai/blog/comparisons/price-guessing-benchmark-not-pricing-decision Published: 2026-08-14 A price-guessing benchmark measures how closely a model recalls a known product price, while a pricing decision depends on the causal effect that price has on real buyer demand. A large language model that scores well on a price-estimation benchmark raises a question: does that say anything about how real buyers respond to a specific price or market entry? It does not. A model that recalls plausible grocery prices has demonstrated background knowledge, not a tested causal effect on demand. [figure: A five-step path: a benchmark score reflects recall of a static price, not buyer response, so the pricing or market-entry decision routes to a controlled causal test of buyer choice.] ## What Does a Price-Estimation Benchmark Actually Measure? One published benchmark tests large language models on a Price Is Right–style Showcase game: models see ten example products with known prices, then estimate the total cost of three unseen products without going over. Performance is scored three ways: Mean Absolute Percentage Error (MAPE) for how close the bid lands to the true price, an overbid rate for how often the model breaks the no-overbid rule, and an Elo rating that combines both across simulated head-to-head matchups. The dataset behind it holds 820 real grocery items pulled from the show, priced at west-coast manufacturer suggested retail levels. Each model runs 50 to 100 Showcases, with results replayed in simulated tournaments rather than live competition. ## Why the score is background knowledge, not a demand signal The benchmark's own stated caveats matter as much as its scores. Prices in the dataset are static, drawn from a single source, and limited to west-coast retail; the benchmark does not model promotions, inflation, or regional price differences. Elo ratings can also vary meaningfully at 50 to 100 Showcases per model, since the random assignment of items to tournaments introduces sampling variance. That is not a flaw in the benchmark's design for its stated purpose. It is a boundary on what the score can support. A model that produces a low MAPE has shown it can recall or infer a plausible price for a can of coconut water. It has not shown how a real buyer's willingness to purchase shifts when that price moves, how a competitor's price changes their choice, or what happens to demand when the product enters a market it has never been sold in. ## Two questions that look similar and are not The benchmark's leaderboard shows models trading raw accuracy for rule compliance in different ratios. A model with a low MAPE can still carry a high overbid rate, and vice versa, because minimizing average error and following a hard constraint are different optimization targets. That tradeoff illustrates a broader point: a single accuracy number rarely captures the property a decision depends on. The same gap separates price recall from a tested pricing action. Model coverage varies by provider and version, so any comparison of "which model estimates prices best" is also a comparison of specific model releases at a point in time; check current model documentation for what a given release supports before citing its benchmark score (Anthropic model overview). ## Why Do Market Entry, Elasticity, and Compliance Still Need a Causal Test? The benchmark's own framing connects price estimation to market entry, price elasticity, economic indicators, and regulatory compliance. Each of those applications depends on knowing how a price or entry decision changes buyer behavior, which a static price-recall score cannot show. Subconscious runs a controlled discrete choice experiment against a defined population before a pricing or market-entry decision ships, and returns the causal effect of the specific price or entry action, with a confidence interval, rather than a single point estimate of what a price "should" be. A team can move from that simulated experiment to real-human validation without changing the causal question tested: the same experimental design, run with recruited human participants instead of a synthetic population. This distinction cuts both ways. A causal experiment on a defined population answers how a specific pricing or entry action changes choice, not what the objectively correct price is, and it does not replace demand forecasting, cost accounting, or regulatory review. A price-estimation benchmark is not a market-entry study or a proof of demand response; it is a test of whether a model's background knowledge produces plausible numbers under game rules. Shipping a price on the strength of a benchmark score alone means the market's actual response was never tested. Before that decision ships, see current model coverage on the leaderboard, or book a demo to scope a pricing or market-entry test. --- # 10 Audience-Evidence Methods Agencies Are Using Before a Pitch in 2026 URL: https://subconscious.ai/blog/buyers/10-audience-evidence-methods-agencies-using-before-pitch-2026 Published: 2026-08-14 Clients now expect a pitch deck to carry audience evidence, not just a creative point of view. A pitch team whose best answer is a promise of three weeks of qualitative research before anyone knows anything is losing ground to teams that walk in with an AI-generated synthetic-panel read already in slide four. That shift has pulled a wide set of AI research tools into the agency stack, raising one buyer question before any of them get budget: what does this specific method prove, and what would it be wrong to conclude from it? Industry research backs the shift in expectation, even where individual tool claims vary: a 2026 survey of agencies found AI research and multi-step, self-directed AI workflows moving from experiment to standard practice (Digital Applied's 250-agency adoption survey). [figure: A decision-path diagram routing five agency scenarios (pitch prep, tight budget, regulated clients, hard-to-reach B2B buyers, pre-launch validation) to the audience-evidence method suited to each.] ## The 10 audience-evidence methods agencies reach for Agencies rarely pick one tool for every brief. The methods below cover the ten situations that come up most often across a retainer. None outranks the others in the abstract; each answers a narrower question than "which tool is best," so fit depends on the decision in front of the account team. ### 1. What is a controlled experiment with a human-baseline check? For a client-facing recommendation that has to survive scrutiny, the strongest evidence path runs a controlled test on defined alternatives and then checks the result against real human participants without changing the underlying question. Subconscious runs this way: it tests actions as causal experiments and can validate a study with real people afterward, reporting the effect rather than a raw preference score. ### 2. Self-serve synthetic focus groups for a tight budget Smaller shops and boutique agencies without a research function often need the cheapest viable way to get a directional read on a creative or message before it ships to a client. ### 3. Qualitative AI respondents for product-led clients Agencies working B2B SaaS or fintech accounts use conversational AI respondents to pressure-test product positioning and feature framing in a format that mirrors how a product manager would read the output. ### 4. Synthetic crowds for large consumer brands Enterprise consumer and media accounts sometimes need audience simulation built and maintained at the brand's scale, with a dedicated onboarding process rather than a self-serve signup. ### 5. Behavioral simulation for campaign rollout dynamics Strategy-led engagements that need to model how a message or feature spreads through a population, rather than test a single reaction, reach for multi-agent simulation platforms built for adoption and virality questions. ### 6. Panels built for hard-to-reach B2B decision-makers CFOs, IT buyers, and other scarce respondent pools are difficult to recruit for traditional qualitative research. Platforms built specifically around B2B decision-maker audiences fill that gap for enterprise-facing agency work. ### 7. What are audit-trail platforms for regulated industries? Agencies serving automotive, finance, energy, or pharma clients need a documented, defensible research trail alongside the result itself, which points them toward platforms built around audit and compliance requirements rather than speed. ### 8. Verbatim synthesis on top of human qualitative research Agencies that still run traditional qualitative studies use AI synthesis tools to summarize and cluster open-ended verbatim responses at scale, keeping the human fieldwork and adding a faster read on the output. ### 9. What is simulated pre-launch testing for product features? Product-strategy agencies use simulated-user testing to screen a feature or roadmap item before it reaches real users, catching an obviously wrong direction before a live beta would. ### 10. Local-market digital twins with real-time data feeds Agencies with a regional client base sometimes need a research partner with local data integration and a local support relationship, rather than a global platform run from a different market. ## Matching the method to the decision The table below is the check to run before a result reaches a client deck: what question each evidence path can actually answer, and what happens when a team asks it to answer a different one. ## What the agency workflow actually looks like Across a retainer, the same four moments recur: pitch prep, where a directional read goes into the deck before the client asks for one; onboarding, where a client-specific evidence approach gets set up once and reused across briefs; creative and message testing, where a shortlist of variants gets narrowed before client sign-off; and a pre-launch or high-stakes call, where the team decides whether a human-baseline check is worth the extra step before the recommendation ships. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That describes audience reach for study design, not a recruitable panel of 800 million participants standing by to answer questions. ## Historical planning figures, labeled Earlier agency planning examples described a 10-person audience test, screening five to ten creative variants, a three-week qualitative panel, and a roughly $40,000 baseline-research line item, kept here only as reference points with different scopes. They are not current Subconscious prices, delivery times, or guarantees, and should not be read as benchmarks for any tool named above. ## Where this breaks if a team skips the check None of these methods replaces the others, and none is a self-serve chat tool an account lead can treat as a live oracle in a pitch room. A controlled experiment with human-baseline validation takes longer than an open-ended synthetic-panel chat because it answers a narrower, better-supported question. Using it as a pitch-room parlor trick, or skipping the human-baseline step on a decision that will get expensive if wrong, both undercut the reason to run a controlled test at all. The check before anything goes on a client slide is simple: name the decision, name what would have to be true for the test to answer it, then pick the method. Teams weighing whether a specific decision needs a human-baseline step can review how Subconscious structures that workflow, see the method applied in past studies, or compare it against the broader research approach before choosing a path. --- # 11 Ways to Test Consumer Reactions Before You Spend Budget URL: https://subconscious.ai/blog/use-cases/11-ways-test-consumer-reactions-before-spend-budget Published: 2026-08-14 A brand, insights, or innovation lead rarely has one thing to pre-test. Before media, production, or product budget moves, several stimuli usually need a reaction check: the concept, the message that sells it, the price attached to it, the channel it runs in. Committing budget on an untested version means finding out how the audience reacts only after launch, when the spend is gone. Running a controlled test on the specific stimulus first, instead of an open-ended chat with a synthetic panel, gives a comparative answer to the same question every time: does this version perform better than the alternative for this audience. Demand for that kind of check is not shrinking: between 2024 and 2034, the U.S. Bureau of Labor Statistics expects the market research analyst and marketing specialist workforce to keep expanding (BLS market research analyst outlook). Below are eleven stimuli worth testing that way before they carry budget. ## Set the rule before you run the test Every one of the eleven checks below works the same way: - Write the one-sentence decision the test needs to answer. - Define the audience and how much is riding on the answer. - Run a controlled experiment on the stimulus, comparing it against a realistic alternative rather than judging it alone. - Have a person review the audience definition, the stimulus wording, and whether the read actually answers the decision. - If the decision is expensive or will support a public claim, move the same question to real-human validation before committing. A directional read without its limits is marketing. Skipping step five is where a directional read gets asked to do a public claim's job. The ICC/ESOMAR International Code on Market, Opinion and Social Research and Data Analytics sets the same expectation for how audience and stimulus definitions should be documented and reviewed before a study result is used for a decision (ICC/ESOMAR Code). [figure: Five-step horizontal path: write the decision, define audience and risk, test against a realistic alternative, human review, then real-human validation if expensive or public.] ## 11 ways to test a consumer reaction before you commit budget ### 1. A product or campaign concept Compare the concept against a realistic alternative, not against silence. A concept judged in isolation tells you it sounds fine, not that it beats the option you would otherwise run. ### 2. A core message or claim Test the specific sentence the campaign will lead with. Confusing or polarizing language shows up faster against a comparison message than in an open reaction. ### 3. A pricing story or price point Test how the price is framed and justified, not just the number. A price that reads as reasonable next to one comparison and expensive next to another tells you the story needs work before the number does. ### 4. A campaign route or creative direction When two creative directions could carry the same message, run them against each other on the same audience definition before production budget picks a side. ### 5. A feature idea Test the feature description against the current alternative the audience already uses. A reaction to the idea alone does not tell you whether it changes a real choice. ### 6. Where should you test a journey moment? Test reactions at the specific point in the journey where a decision or drop-off happens, not the journey as a whole. The stimulus should match the moment, such as a specific screen, message, or offer. ### 7. A headline Headlines carry a disproportionate share of first impressions. Compare candidate headlines against each other on the audience segment that will see them. ### 8. A channel or context pairing The same message can land differently depending on where and when it appears. Testing the message in its intended channel catches a mismatch that testing it alone would miss. ### 9. A landing page or packaging treatment Test the version of the page or pack that a person will encounter, including the surrounding layout and claims, not an isolated line of copy. ### 10. A strategic assumption behind the plan Some budget decisions rest on an assumption about what the audience wants or believes, not a specific creative asset. State the assumption as a testable claim and compare it against an alternative assumption before the plan locks around it. ### 11. A claim intended for public use Naming a failure mode here is what lets a buyer check it before committing budget. A claim that will appear in advertising, packaging, or a press statement carries more risk than an internal one. Treat it as the case where a directional read is not the finish line: move it to real-human validation before it goes out. ## How do you move from a simulated read to real-human validation? Subconscious runs controlled experiments on a simulated audience to compare specific actions before capital is committed, then can test or validate the same study with real human participants when the decision is expensive or public. The audience definition and the causal question stay the same across both steps; only the evidence standard changes. A number without the limit attached to it is a marketing claim. Subconscious can run these controlled studies against a person-level audience graph covering 800 million real people. That describes the audience the experiment can be defined against, not a group of 800 million people available to answer a study in person. [figure: Four-column grouping of the eleven test types, ending in one column for public claim, flagged as needing real-human validation.] ## What can't these tests decide on their own? What a method cannot do belongs on the record next to what it can. A controlled test on any of these eleven stimuli answers what changes choice among the versions tested. It does not replace human judgment, and it does not observe a real person using the interface or product. A simulated read still needs real-human validation before it supports an expensive commitment or a public claim. Set the audience definition, the comparison, and the escalation rule before the first result comes back, and walk through the research process that fits the size of the decision. To scope a specific concept, message, or price test, talk to the team. For examples of how these tests get used, see the case studies. --- # Which Consumer Insight Workflows Should Move to AI First? URL: https://subconscious.ai/blog/implementation-and-procurement/which-consumer-insight-workflows-should-move-ai-first Published: 2026-08-14 A head of consumer insights does not need to decide whether AI touches the research function. It already has. The real decision is narrower: which workflows move to AI-assisted or synthetic-panel exploration this quarter, and which stay locked to real-respondent validation. Get the sequencing wrong and the cost is not one bad study. It is a research function that stakeholders stop trusting for a full budget cycle. ## The decision this quarter Five workflow types sit at the center of this call: concept screening, message iteration, segment hypothesis generation, journey questions, and report drafting. None of them is automatically safe just because a tool can produce fluent output for it. The variable that matters is not tool capability. It is decision cost and stakeholder exposure. A concept screen that narrows ten ideas to three before a creative sprint is a low-stakes, high-volume call: wrong here costs a redirected week of exploratory cycles. A pricing decision, a public claim, or an executive strategy call is a high-stakes, low-volume call: wrong here erodes stakeholder trust in the research function, not just in one study. ## Why is the wrong sequence expensive? Skipping validation on a high-stakes call and shipping a plausible-but-wrong synthetic read does more damage than the study itself. It teaches stakeholders that "AI-assisted" and "unverified" mean the same thing, and that lesson generalizes to every research output that follows, including the ones that were properly validated. The opposite mistake carries a cost too. Routing a low-risk, exploratory question through a full fielded-validation process wastes a week that a lighter governance step would have caught in an afternoon. Both failure modes trace back to the same root cause: treating every workflow as if it carried the same stakes. ## A sequencing framework, not a tool checklist This table is a starting sequence, not a permanent map. A concept screen for a regulated product category can carry executive-level stakes. The governing question stays fixed regardless of category: what does it cost the business if this specific answer is wrong, and who sees it? ## Where does causal discrete-choice testing fit? Subconscious runs controlled discrete-choice experiments against synthetic populations and returns effects with confidence intervals rather than a single fluent-sounding answer. That distinction lets a research lead label a result honestly as directional, not as fact. That property maps onto the exploration and directional-testing layers of this framework: concept screening, message iteration, and segment hypothesis generation. These are the workflows where a comparative directional read against defined alternatives is the useful output, not a final claim. The research methodology and how a Subconscious study runs end to end both describe how the confidence-interval format supports this kind of governed, sequenced use. ## What limit has to stay explicit? This is a sequencing and governance framework, not a claim that synthetic panels replace fielded validation for expensive or public decisions. Pricing, public claims, and executive-level calls still require real-respondent or behavioral validation before anything ships externally. Subconscious can test or validate studies with real human participants, which means a team can move from a directional synthetic read to real-human validation without changing the underlying causal question. The audience definition and the tested alternatives stay fixed while the evidence source changes. Treat that boundary as a hard governance gate, not a suggestion. A finding that started as directional AI-assisted exploration does not become validated evidence through repetition or stakeholder pressure to ship faster. It becomes validated evidence when a defined population responds to the same tested alternatives under real conditions. ## Building the habit The workable version of this is not a one-time policy memo. It is a recurring practice: before each new research request, name the decision it feeds, place it against the sequencing framework above, and route it accordingly. Employment for market research analysts is projected to keep growing from 2024 to 2034, according to the U.S. Bureau of Labor Statistics, a sign the function itself is not disappearing. The workflows within it are what get resequenced. Teams that build this habit end up with something more durable than a list of approved tools: a working system that can explain, for any output it produces, whether it is a directional hypothesis or validated evidence, and what has to happen before that difference stops mattering. [figure: Five-step path: a question hits a gate asking cost-if-wrong and audience. Low-stakes routes to AI-assisted exploration; high-stakes routes to real-respondent validation before shipping.] --- # AI Persona Tools vs. a Controlled Behavioral Experiment: Which One Answers Your Question? URL: https://subconscious.ai/blog/use-cases/ai-persona-tools-vs-controlled-behavioral-experiment-which-one-answers Published: 2026-08-18 Marketing and product teams comparing AI persona tools are usually asking the wrong first question: not which persona tool is best, but whether a persona, however fluent, can tell you what a real buyer will do when you change a price, a message, or a launch decision. A persona is a description. A controlled experiment produces a comparison: this action versus that one, on the outcome that matters. ## Why the persona-tool comparison misses the buyer's actual decision Persona software has moved through three phases. Early tools produced a static document: a name, a quote, a few attributes, meant for a slide. A second wave grounded that document in analytics, CRM records, or interview transcripts, trading imagination for real customer data. A third wave made personas conversational, so a team could ask a simulated buyer questions instead of reading about them. Each phase makes the persona more detailed. None of them changes what a persona fundamentally is: a description of a type of person, not a test of what that person does under a specific change. Asking a persona "would you buy this at $49?" produces a plausible-sounding answer. Research on persona-conditioned language models finds that transparency into how a synthetic persona was built, and how its answers should be interpreted, lags far behind how convincingly the persona talks (Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency, arXiv). A separate reliability study of persona-conditioned models used as synthetic survey respondents found their answers do not track real respondent behavior consistently enough to substitute for it across question types (Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents, ACM Web Conference 2026 Companion Proceedings). "Our analysis shows substantial differences in user representation, with most studies focusing on limited sociodemographic attributes and only 35% discussing the representativeness of their LLM personae." Batzner and colleagues, "Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency," arXiv (source) ## Four things buyers say they need from a persona tool Teams evaluating persona software are usually trying to solve one of four separate problems: - A static reference document for a deck or brief. - A queryable profile they can ask follow-up questions. - A panel of profiles they can run the same question across, to see how answers vary. - A programmatic library engineers can wire into other tools or simulations. These are four different jobs. Even the best version of each produces a description or a conversation, not a causal answer to "which version of this decision changes behavior." ## Comparing what each method actually produces ## What does a controlled experiment add that a persona conversation can't? A controlled experiment holds most things constant and changes one thing at a time, then measures which version moved the outcome. Subconscious runs that kind of experiment against a person-level audience graph covering 800 million real people, and can test or validate the resulting study with real human participants without changing the underlying causal question. A persona tool can tell you what a "typical" buyer might say about two headlines. A behavioral experiment tells you which headline more people actually chose, and by how much, when everything else about the offer stayed the same. ## When is a persona tool still the right call? A static document is still the fastest way to align a team before a decision exists to test. A conversational tool is useful for early exploratory thinking, before a concept is specific enough to run a controlled comparison. And a programmatic persona library is the right layer for engineering teams building their own simulation infrastructure rather than testing a single go/no-go decision. Subconscious is built for a narrower, later moment: once a team has a specific action to decide on. A price. A message. A launch narrative. A feature to ship or kill. That's the point where a description of a buyer stops being enough, and a measured comparison between real alternatives becomes worth the setup. ## The practical next step If the open question is "who is our buyer," a persona document or conversation is the right tool. If it's "which of these two or three actions will change what our buyer actually does," that's a causal experiment, worth testing before the decision ships. [figure: Three persona formats grouped as "description," each unable to say if a message or price changes a buyer's choice. Controlled experiment stands apart as "measured comparison," showing which version moved the outcome.] --- # Beyond AI Persona Interviews: Choosing a Method for Consequential Decisions URL: https://subconscious.ai/blog/comparisons/beyond-ai-persona-interviews-choosing-method-consequential-decisions Published: 2026-08-18 A research or insights team that adopted an AI persona-interview tool for fast, pre-research hypothesis generation eventually asks a different question: can the same chat transcripts support a budget-level decision, such as pricing, positioning, or a launch call? The answer is no. Single-persona conversational interviews are built for early qualitative direction, not a measured, segment-level estimate with quantified uncertainty. ## What AI persona interview tools do well, and where they stop Tools in the Synthetic Users category let a researcher define a target user, generate a persona, and question it like a real participant. That focus is the product's strength: - Quick to launch: pick who the persona represents, build it, and start the questioning. - The interview format feels natural for product and UX teams already used to qualitative discovery. - Clear positioning around pre-research and hypothesis generation, before a team recruits real participants. The same focus creates the limit: a single persona in a single conversation has no built-in comparison group, no sample logic, and no way to quantify confidence. Once the question moves from what a customer might say to which action changes which outcome, for which segment, with what certainty, a conversational interview cannot answer it alone. ## Where does population-scale simulation fit, and where does it fall short? At the other end of the market, Aaru runs multi-agent population simulation aimed at large-scale prediction work. Implementations typically take weeks to months, and contracts start in six figures, fitting organizations modeling behavior across an entire population, not those validating a single pricing or messaging decision quickly. Between interviewing one persona and simulating an entire population, most buyer-level product, pricing, and marketing decisions need a controlled test that isolates one action's causal effect on one outcome, for a defined segment. ## The comparison ## Where does a causal test change the answer? Subconscious is the causal AI company. Randomized experiments on a simulation of your market, validated against real human behavior, tell you why people choose and which action drives the outcome. Once the question is which action changes which outcome, with quantified uncertainty, a randomized experiment is the right tool: it holds a comparison condition constant and reports a measurable effect rather than one persona's opinion. When a decision needs real-world confirmation, Subconscious can test or validate studies with real human participants without changing the causal question. Read more on the research page. ## Where is a causal test the wrong tool? Subconscious does not offer open-ended, single-persona conversational interviews for fast qualitative hypothesis generation. That remains a legitimate, different job. Causal experiment design is wrong for early-stage, low-stakes exploratory questions where speed and conversational depth matter more than measured certainty. Use a persona interview to explore; use a randomized experiment to decide. ## Choosing the right method Stick with an AI persona-interview tool for pre-research before recruiting humans, when no segment comparison is needed. Move to a controlled, causal test once the output has to survive a stakeholder asking why real customer behavior might diverge from what one simulated persona said. See how the method works on how we work, or check current results on the leaderboard. The next step for a consequential pricing, positioning, or launch decision is to book a demo and scope it against a controlled experiment design, rather than extending a persona interview tool past the job it was built for. [figure: Four methods ranked by rigor: persona interviews (no comparison group), population simulation (population-level), real-participant recruiting (real behavior), causal experiments (controlled, confidence intervals).] --- # AI Ad Creative Testing Platforms in 2026: Three Approaches URL: https://subconscious.ai/blog/use-cases/ai-ad-creative-testing-platforms-2026-three-approaches Published: 2026-08-18 A performance marketing lead running paid social can end up with more creative variants queued in a single week than any pre-launch study could review before the budget goes out the door. The instrument that screens those variants isn't the same instrument that explains why an audience responds the way it does, and picking the wrong one for a given decision either wastes spend or wastes weeks. ## Why one instrument stopped being enough Paid-social teams routinely push fifty to two hundred creative variants through a single week of production. Testing all of them with live spend is expensive at that volume, and a conventional pre-launch study can't turn results around fast enough to keep pace. That gap has pulled in a dozen or so vendors by 2026, each addressing a different slice of the problem. [figure: Four-item list: automated scoring for fast triage, panel testing for directional reads, causal experiments for isolating what moved a choice, and simulation for population-scale forecasts.] ## Four instruments, four different questions ### What is automated creative scoring? A scoring tool trains on a large set of historical ad performance and returns a number for each new asset, built from visual, copy, and structural features. Its value is throughput: a team can route every new variant through an API and drop the bottom 30 percent before any spend commits. What it can't do is explain itself. Two assets that land five points apart on the same scale may differ for reasons the model never surfaces, so a losing score tells a team what to cut, not what to change. ### What is panel-style reaction testing? A panel-style study shows a defined audience a still, a video frame, or ad copy and records open reactions before aggregating them. Framed around a specific question, this kind of session can surface whether a hook registers in the opening seconds, whether a headline reads as confident or apologetic, and which segment of the audience is unmoved. Framed around a vague question, such as whether people simply like the ad, the session mostly returns opinion, and opinion doesn't tell a creative team which direction to take next. ### Large-scale campaign simulation A simulation models how a campaign's effect could spread across a stratified population over the course of a launch, forecasting outputs such as a share-of-attention curve or a conversion funnel. This is a different job than screening a single asset: it exists for decisions where population-level dynamics, not one creative's clarity, decide the outcome. Setting one of these up commonly takes weeks, which puts it out of reach for a routine weekly variant and in reach for a campaign large enough to justify the wait. ### Controlled causal experiments A controlled discrete choice experiment shows a defined audience creative or message variants that differ in one attribute at a time, then measures which change actually moved a stated choice. Documented practice for this design, laid out by the ISPOR Conjoint Analysis Good Research Practices Task Force, calls for isolating one attribute's effect at a time and reporting it as a measured contrast rather than a single blended score (Statistical Methods for the Analysis of Discrete Choice Experiments, ScienceDirect / ISPOR). This is where Subconscious fits in the stack: not a faster scoring pass and not a generic reaction panel, but the layer built to answer which specific change in a message, price, or go-to-market alternative moved a defined audience, backed by an effect size and a confidence interval rather than a hunch. The causal fidelity paper covers how the experiment is designed and read. ## Matching the instrument to the decision ## What does a causal experiment not cover? A controlled causal experiment isn't a substitute for an in-platform, spend-based live test, and it isn't a scoring endpoint a creative tool can call between every save. It also doesn't forecast how a full campaign's effect diffuses across a population; that stays the job of large-scale simulation when the budget and timeline support it. What the experiment adds is narrower and more defensible: a measured answer to which specific change moved the choice for a defined audience. ## Building a stack instead of picking one tool Teams producing more than a handful of variants a week rarely settle on a single instrument. A common pattern: scoring clears volume by killing the weakest assets early, a causal experiment settles a contested creative or message direction before it reaches production, and simulation gets reserved for the rare campaign where population dynamics justify its cost and timeline. Before comparing platforms on a feature list, name the decision that's actually on the table, the evidence needed to defend it, and the cost of getting it wrong. Current experiment results are on the leaderboard, and a specific decision can be worked through on a call with the team. --- # How to Choose a Customer Simulation Method URL: https://subconscious.ai/blog/implementation-and-procurement/how-choose-customer-simulation-method Published: 2026-08-18 Choose a customer simulation method by the evidence the decision requires. Ungrounded roleplay can generate hypotheses. A vendor platform can organize simulated reactions. A custom multi-agent build can inspect an unusual workflow. When the question is which price, message, or product action changes buyer response, use a randomized causal comparison. Committing to the wrong method can turn an unvalidated output into decision-grade evidence or consume engineering budget on a build the team did not need. [figure: Four buyer questions branch to four methods: described reaction to roleplay, cross-profile themes to a vendor platform, live workflow to a custom build, which action changes response to a causal experiment.] ## Match the method to the evidence The four common approaches are not substitutes. Each supports a different question. Subconscious is built for the fourth question. It runs randomized comparisons of the actions under consideration against a person-level audience graph covering 800 million real people. This is audience reach for controlled studies, not a count of recruited participants or a claim that every simulated result predicts market performance. When the stakes require it, Subconscious can test or validate the study with real human participants while preserving the causal question. Simulation and recruited-participant validation remain different sources of evidence. ## Define the buyer and action before the study The study should isolate a choice the team can make. Four inputs keep that choice concrete: - Demographics. Where the buyer lives and works, what they earn, who they live with, and what stage of life they're in. Include only what plausibly affects the decision being tested. - Psychographics. Values, motivations, and identity drivers, each stated in one sentence. If a trait cannot be stated in a single sentence, it is too vague to test against. - Historical voice. Anonymized excerpts of real reviews, support tickets, sales-call notes, or survey comments. These examples constrain the language used in the study without turning past comments into proof of future behavior. - The job to be done. The actual problem the buyer is solving, stated specifically. Not "buy a laptop," but "look credible on a sales call without admitting a recent job change." Then define the action. A price, headline, feature, onboarding sequence, or name can be randomized. "What do buyers think?" cannot. ## Set the boundary before reading the result Simulation narrows a decision. It does not certify that the market will behave exactly as modeled. Four boundaries matter: - Prompt sensitivity. A leading prompt can produce an agreeing answer. Academic work comparing large language model outputs with real survey responses found systematic divergence between them (Cambridge University Press, Political Analysis). A single simulated reaction is not decision-grade evidence. - No true unpredictability. This gap goes on the record so a buyer can weigh it against the study's confidence before acting on the result. Real buyers behave inconsistently in ways a model approximates without fully replicating. The larger the decision, the more a team should validate a simulated result against a real-human sample before acting on it. - Data lag. A simulated profile reflects historical behavior, not this morning's news cycle or a sudden cultural shift. Trend-sensitive decisions need a current, real-time signal alongside any simulation. - Regulatory and longitudinal limits. A regulator, or a study that tracks the same cohort over time, needs real-human data. Simulation is useful for narrowing the field of ideas before that study runs, not for replacing it. Subconscious does not replace real-human usability observation or moderated research. It also does not publish a live automated pricing or catalog optimizer, confidence-interval output, or decision-memo workflow as a standard, unconditional product feature. ## Plan scale without turning examples into requirements Historical manual setups used group sizes of 8, 15, 50, or 100 simulated buyer profiles per run. Other planning examples used 3 to 5 profiles for a directional read or 15 to 50 for a smaller group. These figures are examples from prior setups, not Subconscious minimums, current product specifications, or guarantees. Scale should follow the decision. A headline choice may need contrast across a few well-defined buyers. A pricing decision may require more segments, stronger controls, and real-human validation. Adding simulated buyers does not correct a vague action, a leading frame, or a missing validation plan. ## Before funding a custom build A bounded first pass can reveal whether the team needs roleplay, a causal study, or custom engineering: - Pick one decision already on the table: a campaign headline, a pricing change, a feature launch. - Define three buyer profiles against the four inputs above: demographics, a one-sentence psychographic, and the job to be done. - Ask the same question of each profile and identify what would change the answer. - Read the contrast across profiles, not any single response. - Decide whether the contrast justifies a randomized comparison or real-human validation. One historical planning example allotted 30 minutes to this exercise. Treat that figure as an agenda placeholder, not a current delivery estimate or an efficiency claim. If the choice warrants a controlled experiment, discuss the decision with Subconscious. --- # Seven Survey Biases That Distort Market Research Numbers URL: https://subconscious.ai/blog/methods-and-validation/seven-survey-biases-distort-market-research-numbers Published: 2026-08-18 A consumer insights lead is about to greenlight a launch, a price, or a message on a survey number: does it describe what the market will actually do, or how the market wanted to answer a question? Survey bias is any systematic error that pushes a response away from the truth. Most fielded surveys carry at least two or three of the seven biases below, and most teams never test for them. Shipping against a biased number wastes the spend behind it and points the roadmap at demand that was never there. ## The seven biases, and what each one costs That divergence is what a launch, pricing, or messaging decision built on the survey alone will inherit. ## Where a controlled experiment closes the gap, and where it does not A causal behavioral platform runs controlled experiments on a defined population and checks the result against real human outcomes. That structure speaks directly to two of the seven biases above: - Sampling bias, because the population under test is defined by segment rather than assembled from whoever self-selects into a panel. - Non-response bias, because every defined segment participates in the experiment; there is no silent majority outside the sample. It does not structurally remove the other five. Social desirability, acquiescence, framing, recency, and recall bias are instrument-design problems: they call for balanced scales instead of agree/disagree statements, multiple question frames tested against each other, and timing controls that separate a transient event from the general pattern. Social-desirability bias in particular is a property of how a respondent answers under observation, not of who is asked, so precise sampling alone does not touch it. ## What "checked against a human baseline" actually proves The misses belong on the record next to the hits. A causal result carries the same instrument-design risk as any method until it is checked against real behavior. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. That figure is an aggregate replication rate across the historical corpus; it supports a prior, not a per-study validation of a new causal result, and it is not a guarantee for a new market. It is not a claim that self-report bias disappears inside the simulation, or a substitute for designing the questions well. Where the decision depends on it, a team can move from a simulated experiment to real-human validation without changing the causal question being tested. Simulation is the pre-decision check; human research remains the validation and discovery layer. ## Before the number goes in the deck Treat any survey-derived number consistently: name which of the seven biases could be operating, decide whether the instrument was designed to control for it, and decide whether the decision is big enough to warrant a controlled check against a human baseline before budget moves. Case studies and the research methodology behind this approach are worth reading before that decision. [figure: Two columns. Left, closed by a defined population: sampling bias and non-response bias. Right, untouched, still requiring instrument design: social desirability, acquiescence, framing effect, recency bias, recall bias.] --- # How to Evaluate Customer Simulation Platforms in 2026 URL: https://subconscious.ai/blog/use-cases/how-evaluate-customer-simulation-platforms-2026 Published: 2026-08-18 The most important question when evaluating a customer simulation platform in 2026 is not how realistic its personas sound. Ask whether the company publishes a study-level replication rate against real human behavioral studies, defines exactly how that rate is calculated, and states where the method can fail. Without that evidence, treat simulated output as a hypothesis to investigate, not a basis for a launch, pricing, or messaging decision. A polished response can look convincing while untested against what people choose. [figure: Five-step path: define the behavior at stake, reject plausible dialogue, require a disclosed replication protocol, match the experiment to the decision type, keep audience scale separate from validation.] ## What should you start with when choosing a platform? A buyer should define the action before comparing platforms. “Learn what customers think” is a topic. “Determine whether message B increases preference over message A for this buyer group” is a decision. That distinction matters because stated preference and revealed preference are not interchangeable. A 2025 study found that large language models can be inconsistent between what they say they prefer and what they select in choice tasks (Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?). "We repeat the analysis across different categories of preferences and on four mainstream LLMs and find that a minor change in prompt format can often pivot the preferred choice regardless of the preference categories and LLMs in the test." Gu, Wang, and Han, "Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?" (arXiv:2506.00751) (source) Open-ended dialogue can help a team explore language, objections, and hypotheses. It does not establish which of two or more defined alternatives will change a behavioral outcome. A consequential decision needs a controlled comparison and evidence that the method reproduces results observed in real human studies. ## What should you require instead of a confidence claim? Replication accuracy should mean how often simulated studies reproduce both the direction and the outcome of the original human study. A useful report states what counted as a reproduction, which studies were included, which results did not reproduce, and which populations and decisions were tested. This is different from a general accuracy score, a realistic transcript, or a claim that a panel resembles its target audience. Ask the platform provider to show the evidence, not merely summarize it. The answer should make unsuccessful reproductions and known method limits visible. If the denominator is unclear, the headline rate is not decision-grade. ## Apply the filter to the decision in front of you The same evidence standard should produce a different experiment for each business question. - Launch decisions: compare a defined launch treatment with a defined alternative for a named buyer population and outcome. Do not accept broad enthusiasm as a substitute for the comparison. - Pricing decisions: test specific price presentations or offers against each other. Do not assume that a simulation automatically optimizes pricing or produces substitution and cannibalization matrices. - Messaging decisions: compare concrete messages against the behavior that matters. Theme summaries can explain reactions, but the decision rests on the controlled result. A scale figure on its own is a marketing number. Historical planning examples for population-opinion systems have used tens of thousands of simulated agents and a multi-week enterprise setup. Treat those figures as examples from those setups, not as current delivery commitments, universal category limits, or current Subconscious or vendor claims. Scale does not replace validation. ## Why should audience scale and human validation be read separately? Audience reach, simulated experiment size, and recruited human participant count describe different parts of a study. They should never be combined into one scale claim. Subconscious uses a person-level audience graph that reaches 800 million real people to define relevant populations. Publishing the boundary of that number is what lets a buyer check it. That reach is not a claim that 800 million people participate in an experiment. A simulated study uses a modeled population. A real-human validation study separately recruits real participants. Keeping those quantities separate lets a buyer ask a clean question at each stage: Is the target population defined correctly? Is the simulated experiment designed around the intended action? Does a comparable study with real participants reproduce the result? ## Put Subconscious through the same test Subconscious is a causal behavioral platform that runs randomized experiments on a simulated market and validates results against real human behavioral studies. Its research program is built around that connection. The published leaderboard reports replication accuracy, defined as how often simulated studies reproduce the direction and outcome of the original human study. The fit is strongest when a team must choose among defined actions for a defined population and wants to preserve the same causal question from simulation through real-human testing. The misses belong on the same public leaderboard as the hits. The simulation does not replace exploratory interviews, observed usability research, a clinical trial, or a real-world launch test. It is a controlled estimate whose limits travel with the result, not automatic proof of market performance. Naming what the product does not do belongs in the published record. Subconscious is also not a persona-chat product or a general replacement for focus groups. Automated pricing optimization, substitution and cannibalization matrices, and decision memos should not be assumed to be standard outputs. ## Make the evidence standard part of procurement Before committing budget, require every platform under consideration to answer the same questions: - What exact human studies form the replication set? - How is replication accuracy defined? - Which direction and outcome must the simulation reproduce? - Which results failed to reproduce? - Which populations, behaviors, and decision types remain untested? - Can the team move from simulation to a comparable real-human study without changing the causal question? Use how we work to inspect the study path. If your decision is already framed as defined alternatives, a target population, and a measurable behavior, request a demo to evaluate the method against that decision. --- # Prolific or a Controlled Choice Experiment: What Each Choice Proves URL: https://subconscious.ai/blog/comparisons/prolific-controlled-choice-experiment-what-each-choice-proves Published: 2026-08-18 Prolific and Subconscious solve different parts of a research decision. Prolific supplies recruited human participants. Subconscious runs controlled choice experiments and can carry the same causal question from simulation into real-human validation. Choose based on what the study must prove. [figure: Two-column diagram: recruited participation proves real people gave data; controlled experiment proves a tested change altered choice, with a confidence interval. An arrow links both into shared real-human validation.] ## First decide whether the constraint is access or inference Use a recruitment platform when the immediate need is access to real people for a survey, task, or experiment. Prolific describes its service as recruiting research participants for human data collection (Prolific: participant recruitment). Use a controlled choice experiment when the harder question is causal: which change in price, message, feature, or concept altered preference, and by how much? Subconscious supports McFadden discrete choice experiments and Mixed Logit analysis with confidence intervals and replication. That design separates the tested action from background variation instead of treating a plausible reaction as decision evidence. These choices can be combined. A team can define the causal question once, test it in simulation, and validate it with real people before committing a consequential decision. ## Is the evidence from both methods equivalent? A recruited sample establishes that real people supplied the observations. Recruitment alone does not determine whether the study isolates a causal effect. That depends on the experimental design, assignment, comparison conditions, and analysis. A simulated experiment can compare many controlled alternatives against a specified audience. Simulation alone does not establish that the effect will replicate in real people. Confidence intervals describe uncertainty inside the experiment. A real-human validation step tests whether the result survives outside it. The practical buyer question is two-part: - Does the study require direct human participation from the start? - Does the decision require a causal estimate that will be checked against real-human evidence? ## Compare the research roles ## Stakes determine the sequence For a low-consequence exploratory question, direct recruitment may be enough if the team needs reactions from real people and does not need to estimate the effect of a controlled action. For a price change, campaign, or product concept that is expensive to reverse, the safer sequence is to define the action and outcome, randomize the alternatives, estimate the effect with confidence intervals, and validate the same causal question with real people. This addresses the core failure condition in simulated research: a convincing result that has no holdout or replication evidence before budget is committed. Subconscious can run the controlled study against a person-level audience graph covering 800 million real people, then move the same causal question into real-human validation. The audience graph is not a recruitable participant pool, and Subconscious does not claim to recruit, screen, or vet people as a dedicated recruitment operator does. ## What cases need a different research method? This workflow is not a substitute for an observed usability session, task-performance study, clinical trial, or any process whose validity depends on direct human behavior under its own protocol. It also does not prove market performance automatically. A replicated preference effect is evidence about the tested choice under the study conditions. Launch conditions, distribution, competition, and execution can still change the commercial result. The procurement rule: buy participant access when direct human participation is the requirement. Use a controlled choice experiment when the business decision depends on identifying the effect of a specific action. Pair the two roles when the consequence of a wrong answer justifies validation before the decision ships. Review the research basis, see how a controlled study is run, examine case evidence, or discuss a specific decision. --- # Industry Awareness vs. a Decision-Specific Test: What Each One Actually Answers URL: https://subconscious.ai/blog/comparisons/industry-awareness-vs-decision-specific-test-what-each-one-actually-answers Published: 2026-08-18 A research or insights lead who reads trade coverage and attends industry events stays current on how the category is moving. That is different from testing whether a specific positioning line, price point, or launch message will work with the audience it targets. Confusing the two is where teams get burned: trend awareness shows what the field is doing in general, not what your buyers will do when they see your specific offer. ## The Question Behind the Question Before a positioning, pricing, or launch decision ships, the real question is not "have we kept up with the category" but "have we tested this specific choice against the audience it affects." General industry reading answers the first question, not the second: it was never built to isolate one variable for one audience. The cost of getting this wrong shows up after the decision is live: a positioning statement ships on general sentiment rather than evidence tied to the actual buyers who see it, and the gap surfaces only once it is too late to cheaply reverse. ## What Does Trend-Watching Actually Give You? Trade publications, conferences, and supplier directories serve a real function: they surface what peers are trying, which methods are gaining adoption, and which vendors are worth a closer look. For a team building internal literacy or scoping a vendor shortlist, that coverage is the right tool. A general awareness claim without its scope is a marketing line. Naming what it cannot answer is what lets a team check it against their own decision. What it does not do is answer a question specific to your product, your audience, and your choice. General coverage describes the category, not the decision in front of you. ## Where the Two Diverge: Iteration Cost The two approaches diverge most clearly on what a follow-up question costs. Neither column is strictly better; the table makes the tradeoff explicit, not a winner. ## How Does Subconscious Fit the Gap? Subconscious is a causal behavioral platform built for the second column: running a controlled experiment on the specific action a team is considering, rather than reading about what the category is doing in general. It complements industry awareness reading rather than replacing it. That distinction matters because language-model-assisted approaches to consumer research are still an active area of methods work, not a settled substitute for controlled experimentation. Recent research into whether large language models can assist choice modelling finds that prompting strategy and model choice materially affect how well these tools approximate real preference structure, and that performance still varies by category and setup (arXiv, 2026). That is the reason a causal behavioral platform treats a discrete choice experiment as the unit of evidence, rather than treating any single model output as a finished answer. "proprietary LLMs can generate valid and behaviourally sound utility specifications, particularly when guided by structured prompts (CoT). Open-weight models such as Llama and Gemma struggled to produce meaningful specifications." Sfeir, Nova, Hess, and van Cranenburgh, Journal of Choice Modelling (source) "proprietary LLMs can generate valid and behaviourally sound utility specifications, particularly when guided by structured prompts (CoT). Open-weight models such as Llama and Gemma struggled to produce meaningful specifications." Sfeir, Nova, Hess, and van Cranenburgh, Journal of Choice Modelling (source) When a decision genuinely depends on validation beyond a simulated experiment, a team can move the same causal question to recruited human participants without changing what is being tested. That step matters when the stakes of the specific choice, not general category awareness, are what the team cannot afford to get wrong. ## A Practical Loop: Read, Test, Confirm Teams that use both well run something like this: read the trade coverage to know what is worth testing, run a controlled experiment on the specific choice in front of them, and reserve recruited-participant validation for the calls where the cost of being wrong is highest. Reading generates candidate questions. Testing narrows them to the one decision that matters this quarter. ## When Is General Awareness Still the Right Call? Not every question needs a dedicated test. Scoping vendors, building shared vocabulary, or staying current ahead of budget season are cases where general reading is the right tool and a formal experiment would be overkill. The judgment call is knowing which side of that line a decision sits on. ## What This Doesn't Replace The scope of a method belongs next to its results. A buyer checking the method needs to see exactly what it does not cover. A controlled experiment answers a specific causal question about a specific audience. It does not replace staying current on industry practice, nor does it replace the judgment of when a decision is big enough to warrant its own test rather than general reading. See how this plays out in practice in case studies built from real decisions, or read more on how the method works. ## Next Step If a decision is sitting on general industry sentiment rather than evidence tied to the specific audience it affects, that is the signal to stop reading and start testing. Book a walkthrough to see what a controlled test on your specific decision looks like before it ships. [figure: Two columns: staying current answers what the industry is doing, fits vendor scoping, costs near zero per follow-up. Testing your decision answers audience behavior, fits launch calls, costs a real round-trip.] --- # Structured Study, Synthetic Conversation, or Causal Experiment: Which One Answers Your Decision URL: https://subconscious.ai/blog/comparisons/structured-study-synthetic-conversation-causal-experiment-which-one-answers Published: 2026-08-18 A pricing, packaging, positioning, or launch decision usually gets routed to whichever AI-assisted research tool is open, not the tool that answers the question. That routing mistake is expensive: a team runs a structured study or a synthetic conversation, gets a confident answer, ships on it, and discovers after launch that what people said didn't predict what they did. ## Three different questions, three different tools Before picking a platform, name the question being asked. Three shapes come up repeatedly in consumer and B2B research: - A structured, study-shaped question. "How does this segment rate five concepts on a fixed set of attributes?" This fits a survey-and-respondent model with a report at the end. - An open-ended, conversation-shaped question. "What does this segment think about this idea, and why?" This fits a free-form dialogue where a researcher can follow up on a surprising answer. - A causal, action-shaped question. "Which specific price, message, or feature change moves the outcome we care about?" This fits a controlled experiment with random assignment and a comparison condition, not a single wave of opinions. The first two produce stated preference: what a respondent or persona says they'd do. The third produces a causal estimate: which alternative, tested against the others under controlled conditions, moves a defined behavioral outcome. Conflating the three is how a team ends up with a well-organized study or a fluent transcript that still can't say which option to ship. ## Where does stated preference break down? Naming where a method breaks lets a buyer check it against their own decision. Stated-preference methods, structured or conversational, share the same limit: self-reported interest isn't revealed choice. Recent research testing whether large language models can reproduce human purchase intent found they approximate stated survey responses well: evidence that a structured study and a synthetic-persona conversation measure the same thing, what respondents say, not what they do (LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings, arXiv). A plausible synthetic conversation or a well-designed opinion study can both replicate the say-do gap instead of closing it, in a faster interface. "SSR achieves 90% of human test-retest reliability while maintaining realistic response distributions (KS similarity > 0.85)" Maier and colleagues, arXiv preprint 2510.08338 (source) Discrete choice experiments exist for this reason. Rather than asking someone to rate or describe a preference, they present controlled trade-offs and infer preference weights from the choices made, a method with a long track record in health and consumer economics (Discrete Choice Model and Analysis, Columbia University Mailman School of Public Health; The use of discrete choice experiments in applied economic analysis, Oxera). The distinction that matters: does the platform ask people what they prefer, or observe which option they choose under controlled, comparable conditions? ## Comparison at a glance ## How does Subconscious test the causal question? Subconscious is a causal behavioral platform: it runs controlled experiments on a simulated population and estimates which action moves a decision-specific outcome, rather than eliciting opinions or generating persona conversation. That's the third shape above, not a faster version of the first two. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). That fidelity number sits on the public leaderboard next to every miss, so a buyer can check it before trusting it. It is a validation result, not a guarantee for a new market. Subconscious can also test or validate studies with real human participants, so a team can move from a simulated experiment to a real-human check without redesigning the study. ## What are Subconscious's limitations? A published number without its limits is just marketing. Subconscious doesn't replace open-ended qualitative discovery, multi-persona panel conversation, or structured survey research outright: those remain legitimate methods for different questions, particularly early-stage discovery. The fidelity figures only mean something attached to their definition and a specific study comparison; they're not a blanket accuracy guarantee. Confidence intervals and segment-level heterogeneity are outputs of a specific study design, not standard features of every study. ## Matching the method to the decision Three questions decide which tool fits: - Is the question about what people say, or which option they'd choose? If the answer only needs to describe attitudes or generate ideas, a structured study or open-ended synthetic conversation is the right shape. - Does the decision carry real downside if stated preference doesn't hold up? A pricing, packaging, or launch call with real spend behind it is where the say-do gap gets expensive, and where a controlled causal comparison earns its cost. - Do you need to compare several alternatives, or just understand one? Comparing which of several prices or messages moves behavior needs random assignment and a comparison condition; understanding how one segment feels about one idea doesn't. Where the decision is real and reversible mistakes are costly, run the causal comparison before the spend, not after the launch. [figure: Branching path from three question shapes to their matching methods: structured study, synthetic conversation, and controlled causal experiment, the only one producing a causal estimate.] Explore how the method works, see it applied in the research library or the leaderboard, or book time to walk through a live experiment. --- # The Future of Market Research: Where Simulation Stops and Human Evidence Starts URL: https://subconscious.ai/blog/use-cases/future-market-research-where-simulation-stops-human-evidence-starts Published: 2026-08-18 The future of market research is not a choice between simulation and human studies, but a clearer division of labor. Teams can use simulated experiments to test defined product, pricing, messaging, and go-to-market actions before committing capital. Decisions where validation is critical should still reach recruited human participants before they reach the market. That boundary belongs in the research budget and headcount plan. Without it, a team can mistake a simulated result for human evidence or keep every question in the same fielded workflow, even when the questions carry very different costs of error. ## The adoption question has already been answered The 2025 GRIT Business Outlook reports that 72% of insights buyers use generative AI in at least one stage of a research project, up from 23% in 2023. It also reports that concerns about synthetic respondents and survey fatigue rose 40% year over year. Greenbook documents both sides of that shift. More research workflows now include generated or simulated material, while confidence in the resulting data remains an active concern. The planning question is therefore not whether these methods belong in the stack, but which decisions they can support on their own and which need another evidentiary stage. For decades, a familiar model treated recruitment, questioning, analysis, and recommendation as one study. A mixed research stack separates those jobs. Exploration can frame the hypothesis. A controlled experiment can compare defined actions. Human research can calibrate or validate the result when the decision requires it. [figure: A four-step path: name the action and consequence, run a simulated experiment comparing the actions, check if the cost of error is high, then add human validation before market commitment if so.] ## Allocate rigor according to the cost of error Novelty is not the deciding factor. Neither is the volume of questions. This division prevents two expensive mistakes. The first is spending fieldwork effort on every preliminary question. The second is committing budget, roadmap capacity, or brand equity on evidence that never reached people. ## Convert the business choice into an action test The most useful research question names an action and a consequence. Instead of asking what an audience thinks about a launch, define the launch choices and the behavior that would distinguish them. Instead of asking whether a price feels acceptable, compare the actual price actions under consideration. Subconscious structures this work as controlled causal experiments on a simulated population. The fit is strongest when a buyer must choose among defined product, pricing, messaging, or go-to-market actions before committing capital. The result applies to the population, alternatives, and outcome represented in that experiment, and should not be generalized beyond them without further evidence. This approach differs from open-ended qualitative exploration. Qualitative work can explain language, surface concerns, and generate hypotheses. A causal action test asks which defined intervention changes the modeled behavioral outcome. ## Preserve the human evidence boundary A simulated population is not a recruitable human group. Audience reach in simulation and recruited participants in a validation study are separate stages. When a decision depends on human confirmation, a team can move from the simulated experiment to real-human validation without changing the causal question or redesigning the study. This does not make the simulated result human evidence, and it does not make the human stage an observed usability session, a clinical trial, or automatic proof of market performance. Fielded interviews and surveys can calibrate assumptions, investigate language, and reveal conditions that a defined experiment did not represent. They should remain in the plan wherever the decision needs direct testimony, population-specific recruitment, or evidence that a simulation cannot supply. ## Write the boundary into next year's plan The budget decision becomes more defensible when every major research question is assigned an evidence path before a study begins: - Name the action the business may take. - Record the cost of choosing poorly, including capital, roadmap, brand, or compliance exposure. - Define the population, alternatives, and behavioral outcome that the experiment will represent. - Decide in advance whether the result can inform the decision directly or must pass through recruited human validation. - Keep exploratory findings, simulated experimental evidence, and human evidence labeled separately in the final recommendation. This produces a portfolio of evidence rather than a contest between methods. ## Choose the decisions that deserve a causal test Start with the decisions that carry the largest cost of error and the clearest competing actions. Review how the workflow runs from question to evidence, then discuss a specific decision and its validation boundary. --- # How to Evaluate an AI Simulation Tool Before You Trust It With a Launch Decision URL: https://subconscious.ai/blog/use-cases/how-evaluate-ai-simulation-tool-before-trust-launch-decision Published: 2026-08-18 Evaluating an AI simulation tool means answering one question: can this output carry a positioning, pricing, or launch decision, or is it plausible-sounding text never checked against real behavior. Persona chatter that reads as specific and confident is easy to mistake for evidence. Shipping a launch on it, then finding the market didn't behave as the personas predicted, costs a wasted campaign and a decision built on a false read of demand. The dividing line isn't persona fidelity, conversation depth, or how many personas a tool can hold in memory. It's whether the tool runs a controlled, causal experiment whose output can be checked against real human behavior, or produces unverified conversational text with no such check. [figure: A four-step path of increasing rigor: general LLM prompting, survey automation tools, dedicated simulation platforms, and causal experiment platforms with human validation, only the last checkable against real behavior.] ## What do these AI simulation tools actually do? Most AI simulation tools do three things: model target audiences as AI personas calibrated to a role or segment, run research sessions where those personas respond to concepts and messages, and synthesize where personas agree or diverge. The workflow supplements or replaces interviews and focus groups. That's legitimate for early-stage work: sharpening a concept, stress-testing messaging, or generating a first read before a team commits budget to real fieldwork. It answers "what might people say." It does not answer "what will people do, and why." ## The four categories, and where each one breaks Buyers evaluating this space run into four broad approaches, each with a different failure mode once the stakes rise past a gut-check. Subconscious sits in the fourth category: it runs controlled studies against a person-level audience graph covering 800 million real people, and can validate a study with real human participants, moving from simulation to human testing without changing the causal question (research). The audience graph is a targeting and modeling asset, not a recruitable panel of 800 million people standing by to answer surveys. Keep those two ideas distinct. ## Questions worth asking before you buy - What does the tool actually test? Ask whether it runs a controlled comparison of one action against a baseline, or produces open-ended text a human has to interpret. A structured causal result with a confidence range differs from a wall of persona commentary. - Can the output be checked against reality? Ask whether the vendor can run the same question with real human participants and show whether the simulated result held up. A vendor with no answer is asking you to trust simulation on faith. - Who owns your input data? If personas are calibrated on your customer interviews or CRM notes, know where that data is processed and under what terms before uploading it. - Is the reach claim about targeting or recruitment? A large audience graph is not the same claim as a large recruitable respondent panel. Ask which one a vendor is describing. ## When is simulation alone the wrong tool? Simulation-only tools fit when speed matters more than certainty, the work is early-stage (concept, positioning, messaging), or the target customer is hard to reach directly. They are the wrong primary method when the decision needs real behavioral evidence, the stakes require customer validation, or a stakeholder needs proof that real customers were tested, not just modeled. For those higher-stakes calls, the question isn't how good the personas sound, it's whether the same causal question can be run again with real people and produce a result you can stand behind. See how that validation step works in practice on how we work, or look at completed studies in case studies. To scope a specific decision, book a walkthrough. "Digital personas improve alignment with human response distributions, especially in domains tied to stable attributes and values, but remain limited for individual prediction and fail to recover multivariate respondent structure." Jia and colleagues, arXiv preprint 2605.10659 (source) "Digital personas improve alignment with human response distributions, especially in domains tied to stable attributes and values, but remain limited for individual prediction and fail to recover multivariate respondent structure." Jia and colleagues, arXiv preprint 2605.10659 (source) Related research on this question: a controlled comparison of when digital personas can approximate real survey results (arXiv, 2026), and an evaluation of LLMs as human surrogates in controlled experiments (arXiv, 2026). --- # Simulated Marketing Panels: What They Test Well, and Where Real Buyers Still Decide URL: https://subconscious.ai/blog/methods-and-validation/simulated-marketing-panels-what-they-test-well-where-real-buyers-still Published: 2026-08-18 Marketing teams increasingly run early positioning, pricing, and messaging questions through a simulated panel before committing production or media budget. The method has a well-documented shape: build a panel of AI personas calibrated to a target audience, present a stimulus, and read back a distribution of reactions. Independent research points toward this approach being useful for some marketing decisions and unreliable for others, though the closest available studies evaluate political-survey replication and persona approximation rather than marketing stimulus testing directly, and knowing the difference is the whole game. ## What is the buyer actually deciding? A brand or marketing leader choosing between several concepts, headlines, or pricing structures faces a specific risk. A variant that reads well internally can still fail with the target segment once real budget is behind it, and shipping the wrong one wastes production and media spend, sometimes forcing a mid-campaign reversal. The opposite failure matters just as much: treating a directional read as a firm purchase-behavior forecast and building a revenue plan on a number the method was never built to produce. ## Where are simulated panels reliably useful? These panels are strongest on relative comparison and rank-order questions when the bias is consistent across the compared alternatives, weaker on absolute, individual-level prediction, and unreliable for rank order when bias is stimulus-specific. That maps onto a specific set of marketing decisions: Each row shares a shape: comparing options against each other, not predicting an absolute number. ## Where the method is honest about its limits The same research base is explicit about where these panels underperform: - Sensory testing. If a respondent needs to taste, smell, touch, or wear something, a simulated respondent cannot substitute for a real one. - Genuinely novel categories. A category with no public precedent gives a language model little to ground personas in, and quality degrades accordingly. - Precise purchase-behavior prediction. A simulated panel can show that a concept resonates. It should not be the basis for a claim about what percentage of an audience will pay a given price next month. - Regulatory or legal substantiation. Claims that require substantiation evidence need real-human research; a simulated panel is not admissible in most jurisdictions. - Trend-tracking past a model's training cutoff. Asking about very recent news returns a model's best guess, not a real audience's reaction. Research on when digital personas can approximate human survey findings and a related evaluation of the perils of large language models as survey-data replacements both describe this divide, and both flag demographic flattening as a known failure mode worth watching for. ## A workflow that keeps the test question intact A five-step workflow holds up in practice: - Define the panel. Specify the audience as precisely as a traditional recruited panel: demographics, psychographics, market, segment composition, persona count. - Frame the stimulus concretely. Provide the actual headline, body copy, or pricing structure being tested rather than a description of it. Response quality tracks stimulus quality. - Run the panel and capture the distribution. Read results as a distribution with segment cross-tabs read as directional only, not a single top-line score. - Read the result against the right benchmark. Treat the output as strongest for comparisons between variants and rank-order decisions, weaker for absolute-number predictions. - Decide the next action. Ship the winning variant, escalate the shortlist to a real-human validation round on the same question, or refine the stimulus and re-run. The fifth step is where the method's ceiling matters most. Subconscious can test or validate studies with real human participants, so a team that needs a higher bar of evidence can move from a simulated experiment to real-human validation on the same test question, without redesigning the test from scratch. That path matters for a launch decision or a claim that needs stronger proof than a directional read. ## How is the audience different from the panel? A simulated panel's participant count describes a simulated experiment, not a recruited human panel of the same size: dispersion across synthetic respondents reflects decoding temperature and prompt variation, not sampling from a target population, so it does not support standard errors or confidence intervals in the frequentist sense. Separately, a person-level audience graph used to define who a campaign should reach is not the same thing as recruitable survey respondents. Reach and recruitment answer different questions, and conflating them overstates what either number means. ## Limitations and failure conditions - The method is a comparative tool first. Treat any single-number probability output as directional, not as a forecast to plan revenue against. - It does not replace sensory testing, novel-category research, or regulatory substantiation. - It cannot see past its training data, so recency-sensitive questions need a different method. - Moving to real-human validation should preserve the same test question the simulated round tested. A validation round that quietly turns into an unrelated usability session or clinical-style trial answers a different question, not a stronger version of the same one. ## Next step Historical figures on turnaround time and cost for this kind of test are worth noting only as planning examples from the source workflow they came from, not as current Subconscious pricing or delivery commitments. For a specific comparison, positioning, or pricing decision, the more useful next step is running one comparative test end to end and, where the decision warrants it, escalating the winning variant to real-human validation before committing budget. See how Subconscious teams typically run this workflow or review prior comparative studies. [figure: Five steps: define the panel's audience, frame the actual stimulus, run it and read a distribution, judge it against the right benchmark, then branch to ship, escalate to real-human validation, or refine and re-run.] --- # How Agencies Can Test a Client-Audience Decision URL: https://subconscious.ai/blog/buyers/how-agencies-test-client-audience-decision Published: 2026-08-18 An agency should use a controlled behavioral experiment when a recommendation depends on a defined choice for a defined client audience and fresh customer evidence is unavailable. Name the alternatives before testing. Measure how the choice changes. Report uncertainty. A number published without its limits is only marketing. Do not use the result for open-ended discovery or as a guarantee of market performance. This distinction protects the client relationship. An assumption presented as evidence can lose a pitch, misdirect a paid engagement, or conflict with the client's own customer data. A simulated result presented as direct customer validation creates the same credibility risk. ## Match the Evidence to the Client Question ## How Do You Turn a Brief Into a Testable Choice? A defensible experiment begins with the decision, not a request for general audience feedback. The agency and client should agree on four elements: - Audience: the specific buyer or market segment the recommendation concerns. - Alternatives: the messages, positions, concepts, or actions under consideration. - Outcome: the choice, preference, trust, or intended behavior that would change the recommendation. - Decision rule: the evidence that would support, reject, or narrow the proposed direction. This design makes the causal question visible and exposes weak briefs. If the team cannot name the alternatives or the outcome, the work is still in discovery. Subconscious research uses causal experimentation and discrete-choice-style modeling to compare defined actions for a defined audience. The study estimates the difference between alternatives. When the design supports it, the result can include quantified uncertainty such as a confidence interval. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. The misses sit on the record next to the hits so a buyer can check the number. That figure describes audience reach. It does not describe a recruitable group of 800 million human participants. [figure: Five-step path: define audience, name alternatives, define outcome, set decision rule, run experiment. A branch off the last step points to escalating to human validation when risk is material.] ## What Belongs on the Slide? The experiment earns a place in the pitch or strategy deck when the client must choose among concrete options. Useful questions: - Which position is more likely to move the intended behavior for this segment? - Which message changes trust relative to the agreed alternative? - Which concept should advance to direct customer validation? - Does the proposed direction behave differently across defined audience segments? The result should state what changed, what was compared, which audience was modeled, how uncertain the estimate is, and that the finding is simulated. Naming what the estimate cannot do is what lets a buyer check it. A causal estimate supports a recommendation. It does not guarantee adoption, conversion, revenue, or any other market outcome. ## When Should You Escalate the Evidence? Current client research should anchor the recommendation when it matches the audience, decision, and market context. The controlled experiment can then test a gap or a new alternative without overriding stronger direct evidence. Subconscious can test or validate studies with real human participants. A team can carry the same causal question from a simulated experiment into real-human validation by preserving the audience definition, alternatives, and outcome where the study design permits, making the validation question explicit. Use recruited human research before action when the recommendation carries material financial, regulatory, reputational, or customer risk, and keep direct observation for questions of lived context and unmet needs that a defined comparison cannot capture. ## Reject a Result That Cannot Survive These Checks Do not present the result as client evidence if: - the client has not accepted the audience definition; - the alternatives do not represent the real decision; - the outcome is a vague proxy for the behavior that matters; - the estimate lacks enough uncertainty information for the decision; - current customer evidence contradicts the finding and the conflict has not been investigated; or - the deck describes a simulated result as observed customer behavior. These are not disclosure details. They determine whether the experiment answers the client's question at all. ## Start With the Decision, Not the Method Write one sentence that names the client audience, the actions being compared, and the outcome that would change the recommendation. Then bring that sentence to a working session. The quality of the decision statement sets the ceiling on the evidence that follows. --- # AI-Coded Survey Platforms vs. Conversational Persona Panels: Which One Answers Your Decision? URL: https://subconscious.ai/blog/comparisons/ai-coded-survey-platforms-vs-conversational-persona-panels-which-one Published: 2026-08-18 Choosing between an AI-coded survey platform and a conversational persona panel is choosing a timeline: a structured research cycle measured in days to weeks, or a self-serve session measured in minutes. That choice does not resolve whether either tool tells you what a real customer would actually do. ## Two shapes of AI-assisted research ### What is a structured survey coding platform? One shape is a platform built around traditional survey methodology adapted for digital delivery: structured questions, aggregated responses, and results delivered through dashboards and reports (GroupSolver, "Platform Features"). It is typically operated by a dedicated research or insights function with budget and process built around it, and connects into the analytics stack teams already use for reporting (GroupSolver, "How GroupSolver Works"). The tradeoff for that depth is time: a research cycle runs from question design through fielding to analysis before anyone gets an answer. ### What is a conversational persona panel? The other shape lets a team build synthetic personas, each with a described role, demographics, and behavioral traits, and hold a conversation with them individually or as a panel. The workflow is designed for daily, self-serve use by marketing, product, sales, or research staff rather than a dedicated research team, and results appear inside a single session instead of a multi-week engagement. Some vendors in this category describe their persona construction as drawing on roughly 100 times more public-web research than a typical model prompt, though that figure describes a vendor's own stated process rather than an independently verified benchmark. Both shapes are legitimate at what they are built to do: structured, defensible data collection in the first case, fast qualitative exploration in the second. Neither establishes that a real customer's choice would change under a real alternative. ## What each approach actually measures A survey response and a persona's conversational answer are both descriptive: they report what was said, not what would happen if the choice were real. Treating either as proof that a pricing move, a message, or a launch decision will work is where the cost of guessing wrong shows up: budget, positioning, or a go-to-market plan gets committed on an answer with no confidence interval and nothing to replicate. [figure: A branching path from a question mark. Exploration leads to survey coding platform and persona panel. Budget commitment leads to controlled experiment, then human validation.] ## How does a controlled experiment change the answer? Subconscious runs a controlled, randomized experiment that estimates which specific action (a price point, a feature, a message) is more likely to change a real choice, and reports that estimate with confidence intervals rather than a single descriptive sentence. Where the decision depends on it, a team can move from that simulated experiment to real-human validation on the same causal question instead of switching methods or re-scoping the research. Subconscious can test or validate studies with real human participants, so the question tested and the question validated stay the same from simulation through human confirmation. That validation step, described further in how Subconscious approaches experiment design, is the practical difference from a survey report or a persona conversation: an estimate that can be checked against a human baseline, not generated once and trusted. ## What a controlled experiment does not replace A controlled experiment is not a substitute for early qualitative discovery. Drafting message variants, stress-testing a rough concept, or exploring an unfamiliar customer segment before a bigger commitment is legitimate work for a fast, self-serve conversational tool. It is also not a substitute for a structured research program that a dedicated insights team needs for a large, multi-stakeholder study with its own cadence and reporting requirements. Subconscious is the step for a narrower moment: a specific decision, with real capital behind it, that needs a defensible causal answer before it ships. ## Matching the tool to the decision The right starting question is not which tool is faster or cheaper, but whether the decision is exploratory or a real commitment with a measurable cost of being wrong. Early-stage exploration and broad sentiment tracking are reasonable jobs for either a structured survey platform or a persona panel, depending on team structure and timeline. A pricing change, a launch claim, or a positioning bet that competes for scarce budget calls for a test of what actually changes behavior. A demo walks through how to design that test around a specific decision. --- # What a Rigorous Causal-Inference Pipeline Checks Before You Trust Its Output URL: https://subconscious.ai/blog/methods-and-validation/what-rigorous-causal-inference-pipeline-checks-before-trust-its-output Published: 2026-08-18 A causal-modeling vendor's output is only as trustworthy as the engineering discipline behind the pipeline that produced it. Before a technical or data-science leader bases pricing, messaging, or go-to-market decisions on a vendor's causal effects, the question worth asking is not "does the model run," but "which stage of verification can this vendor actually show me." ## The decision: trust the pipeline, or demand more proof first Basing a budget or go-to-market decision on causal output from an opaque inference pipeline risks discovering, only after the money is spent, that gradients were never checked, uncertainty was never quantified, or the model's internal structure was never inspected. The fix: ask each vendor to demonstrate, not just describe, the stage of verification their pipeline actually passes. ## A general pattern for turning a simulator into an inspectable model One PyMC example gallery walkthrough shows a concrete version of this discipline: taking an existing JAX-based generative function and wrapping it as an operation inside a PyMC probabilistic model, so it can be run through Markov chain Monte Carlo (MCMC) sampling rather than treated as a black box (wrapping a JAX function for PyMC use). The pattern generalizes past this specific example, regardless of vendor or software stack. ### Step 1: Define the function and its compiled version The generative process, the function that turns inputs into an output, whether that output is a market share, an ODE trajectory, or a classification score, has to be written down explicitly and compiled once, so every later evaluation runs the same defined computation. ### Step 2: Obtain gradients, and check them numerically If the sampler needs gradient-based inference, the pipeline needs a way to compute how the output changes with respect to each input. JAX's automatic differentiation can supply this directly for functions already written in JAX, without hand-deriving gradient expressions. Before that gradient is trusted, it gets checked: a numerical gradient check compares the analytic gradient against a finite-difference approximation and confirms they agree. ### Step 3: What does keeping the model graph inspectable show? Once the wrapped function sits inside a model, that model can be rendered as a graph, showing exactly which prior feeds which deterministic node, which likelihood is observed, and how they connect. This is not decoration. It's the difference between a pipeline a reviewer can audit and one they have to take on faith. ### Step 4: Register the function so the full graph compiles together The wrapped operation has to be registered with the backend so the whole model, not just the individual function, can compile and run as one program. Without this step, the wrapped piece and the surrounding model run in two disconnected modes, and results between them can silently drift apart. ### Step 5: Confirm the output matches under both execution paths The source example runs the same model's log-probability and its gradient in both a standard mode and a JAX-compiled mode, and confirms the two outputs match to the precision reported. That match is the actual proof the wrapping worked, not the fact that the code executed without an error. These stages confirm the computation is correct; they say nothing about whether the sampler converged, which requires separate diagnostics such as R-hat, effective sample size, and divergence checks across multiple chains. ## Do some samplers require a manual gradient? The source example includes a case where a more complex simulator, such as a small neural network, is wrapped without implementing a manual gradient method at all. JAX-native samplers can differentiate through the entire compiled model graph automatically, so a manual gradient implementation becomes unnecessary for that specific class of sampler. That's a valid implementation choice, but it does not remove the verification requirement: the numerical gradient check in Step 2 still matters whenever a manual gradient is written, and the automatic differentiation path still has to be confirmed against the model's actual log-probability, the way Step 5 does. ## Apply the checklist to a behavioral-experimentation pilot Subconscious is a causal behavioral platform. It uses causal experimentation and discrete-choice-style modeling to test product, pricing, messaging, and go-to-market actions before teams commit capital. Where a study design supports it, results can include causal effects with credible intervals or other uncertainty quantification. That is the general family this checklist describes, a causal-modeling pipeline that needs its own verification stages, not a claim that Subconscious's internal tooling matches any specific library in the example above. Read more about the method. ## What can this checklist not establish? This is a discipline checklist, not a product claim. It describes what rigorous causal-inference engineering looks like in general; it does not confirm which specific technical stack any vendor, including Subconscious, runs internally, and it is not a substitute for a buyer's own technical due diligence. Credible-interval and uncertainty output should be described as available where the underlying study design supports it, not treated as a universal guarantee attached to every result a vendor produces. [figure: Two paths side by side: manual gradient checked numerically, or automatic differentiation only. Both converge on one box: output confirmed against the model's log-probability.] ## Turn vendor claims into an evidence request Ask which of the five stages above the vendor can actually demonstrate, not describe in a deck. A vendor that can show a gradient check, an inspectable model graph, and matching output across execution paths has done more verification work than one that only shows a final number. That work verifies the pipeline computes what it was written to compute; it does not by itself establish that the experimental design supports a causal interpretation, since causality comes from the randomized design, not the estimator. For teams evaluating how a causal-modeling vendor's process holds up before a pilot, how Subconscious runs a study and a live demo are concrete next steps, alongside the vendor's own technical documentation. [figure: Five-step path of pipeline verification: define and compile the function, check gradients numerically, keep the model graph inspectable, register for full compile, confirm outputs match across execution modes.] --- # Persona Chat or a Structured Choice Experiment: Which Evidence Should Back a Market Decision? URL: https://subconscious.ai/blog/comparisons/persona-chat-structured-choice-experiment-which-evidence-should-back-market Published: 2026-08-18 Use a one-on-one persona chat to explore language and rehearse an argument. Do not use one simulated character's reaction to greenlight a pricing change, message, or launch. That commitment needs a structured choice experiment across a panel, with a causal effect size and confidence interval. [figure: Two columns: persona chat gives one modeled character's qualitative reaction, suited to drafting; choice experiment across a panel gives a causal effect with confidence interval, suited to pricing or launch commitments.] ## Match the evidence to the decision The useful distinction is not which interface feels more conversational. It is whether the decision needs qualitative exploration or quantified evidence. A persona chat can help a team rehearse a sales pitch, challenge a message draft, or review a landing page from one simulated point of view. A published product demonstration presents persona-based website feedback as candid qualitative critique (product demonstration). That is appropriate while the team is still shaping the question. A pricing, positioning, or launch commitment asks a different question: how does changing an attribute affect choice across a defined audience? A single conversation cannot estimate that population effect or its uncertainty. ## How does a structured choice experiment work? Subconscious.ai runs discrete choice experiments using McFadden DCE, Mixed Logit, and ICLV models across panels of synthetic respondents at defined audience reach. The design varies the attributes under consideration, such as price, message, or feature bundle. The result is a causal effect size with a confidence interval. That evidence is appropriate when a team must choose among concrete alternatives before committing budget or market exposure. The causal methodology is documented in the causal fidelity paper, with applied evidence in case studies. ## The practical split ## When should you use persona chat versus a choice experiment? Start with persona conversation when the work is exploratory. Turn the strongest alternatives into explicit attributes and levels once the team can state the choice it needs to make. Then run the structured panel test as the deciding evidence. When the commitment warrants it, the same causal question can move from a simulated experiment to validation with recruited people. The populations remain distinct. Keeping the experimental question fixed makes the comparison interpretable. ## What can the result tell you and what can't it? A persona conversation remains qualitative input, not measured buyer preference. A synthetic panel estimates effects within its defined audience and experimental design; it does not automatically prove market performance or replace recruited-human evidence. The right workflow preserves those boundaries. Use conversation to form the alternatives, structured choice evidence to compare them, and human validation when the risk requires it. Review the full research workflow or scope a decision experiment. --- # Compiling Code Is Not Validating a Model URL: https://subconscious.ai/blog/methods-and-validation/compiling-code-not-validating-model Published: 2026-08-18 A data science team asks an LLM agent to generate a PyMC model from a plain-language description, and the code compiles on the first or second try. Does that mean the model is right? No. Compiling is a syntax check. It confirms the code runs without a Python traceback. It says nothing about whether the priors, likelihood, or causal structure match the decision the model is supposed to inform. ## What a compile-and-revise loop actually checks A recent hackathon prototype built by PyMC contributors demonstrates the pattern. The agent generates a candidate PyMC model from a natural-language prompt, sends it to a remote sandbox that attempts to compile it, and, if compilation fails, feeds the resulting Python traceback back to the agent so it can revise the model. In one documented example, a capture-recapture model first failed with: Compiling and modeling correctness are different checks: The agent used that traceback to produce a version that compiled and ran. The revised model sets up the same three inputs as before: 25 bears tagged on the first pass, 20 captured on the second, and 4 of those recaptures already tagged. It then bounds the unknown total population at 500 and gives it a DiscreteUniform prior no smaller than either capture count: The likelihood swaps in the correctly ordered HyperGeometric call, pm.HyperGeometric("k", N, n1, n2, observed=k_observed), and the model samples 3,000 draws after 1,000 tuning steps before summarizing the posterior over N. That fix is real. The prototype's own builders flag the population upper bound and the choice of likelihood as modeling decisions, not compiler outputs, and they're explicit that other formulations of the same problem remain open: a different, equally valid prior on population size would compile just as cleanly. ## Why did prompt wording change the model's structure? A prompt-sensitivity test on a separate Think Bayes problem, the "lions and tigers and bears" classification task, shows this most clearly. A Dirichlet distribution gives this task a concise structure. After Python type annotations became a system-level requirement, the generated result shifted toward more convoluted categorical models with additional places to attach annotations. Removing that requirement restored the simpler, better-suited model. Both versions compiled. Only one matched the structure of the actual problem. ## What is the difference between code that runs and code that's correct? Research on LLM-generated code shows execution success and correctness are separate properties: code can hallucinate a plausible-looking function or import that never existed, and it can pass an execution check while still encoding a wrong assumption or invalid dependency; see the analysis in Library Hallucinations in LLMs: Risk Analysis Grounded in Package Fabrication. A companion study on execution-based verification for generated code, CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification, reaches the same conclusion from a different angle: verifying that code executes catches a narrower class of errors than verifying that the code does what it's supposed to. For a statistical or causal model, "supposed to do" means: the likelihood matches how the data were generated, the priors encode defensible assumptions, and the model's structure supports the actual decision at hand, not just the actual numbers in the training example. ## Where independent review has to sit A compile-and-revise loop catches outdated syntax, hallucinated function signatures, and shape mismatches. It is not a substitute for methodological review of whether the model's assumptions are the right ones for the question being asked. Subconscious's approach to causal experimentation treats this as a hard line: execution success is necessary but not sufficient for causal validity. A model that compiles still needs to be checked against the decision it's meant to support. When that check matters, Subconscious can test or validate studies with real human participants, so a team can check a model's predictions against real behavior and catch miscalibration, though matching human data cannot certify the model's causal structure is correct. How we approach validation covers this in more depth, and current benchmark results are on the leaderboard. ## The takeaway Treat "it compiled" as the first gate, not the last one. Before a generated statistical or causal model informs a real decision, confirm someone with modeling expertise has reviewed its assumptions, independent of whether the code ran. [figure: Five stages left to right: generate model, compile-and-revise on tracebacks, code runs, check structure against decision, validate against real behavior. Stage three can't see if priors or structure are wrong.] --- # Five checkpoints before you trust a latent-trait score URL: https://subconscious.ai/blog/implementation-and-procurement/five-checkpoints-before-trust-latent-trait-score Published: 2026-08-18 A personality or preference score from a psychometric model is never observed directly. It is inferred from a person's answers, then compressed into a single number that a hiring, pricing, or segmentation decision leans on. The question is not whether the model runs. It is whether the number is trustworthy enough to act on. ## What a trait score actually claims Models that infer an unobservable trait from observed responses fall under item response theory, a family of statistical models relating a latent characteristic to the answers a person gives. A graded response model extends this to ordered categories, such as a five-point agreement scale, so "strongly agree" and "somewhat agree" count as different amounts of evidence, not the same category. The trait itself is never measured. Only the responses are. Everything past that point is inference, and inference carries assumptions that can be checked or left unchecked. ## The checkpoints a defensible workflow doesn't skip A public engineering discussion of building this kind of model for a workplace personality assessment moved through a fixed sequence of checks before trusting the resulting score. Each checkpoint catches a different way the model could be silently wrong: This is the shape of a Bayesian workflow: not one model-fitting step, but a sequence where each stage has to produce a checkable result before the next stage is trusted. "Beyond inference, the workflow also includes iterative model building, model checking, validation and troubleshooting of computational problems, model understanding, and model comparison." Gelman and colleagues, "Bayesian Workflow" (arXiv:2011.01808) (source) ## What breaks when a checkpoint gets skipped? A team that stops after "fit a candidate model" ships a score with no evidence that it means anything outside the questionnaire, and no uncertainty band around it. The score still looks plausible and still ranks people from low to high; nothing about the output signals that a step was skipped. The failure shows up downstream instead: a hiring rule built on the score screens out a candidate who would have performed well, a price segment built on a preference score misclassifies a customer, and there is no confidence interval to point to when someone asks why. The cost of skipping a checkpoint is paid by the decision the score was supposed to inform, not by the model itself. ## Where does the same discipline apply to a causal question? Subconscious runs controlled discrete-choice experiments on the same discipline described above: a model of an unobservable preference or trait is fit, checked against alternative specifications, and reported with a confidence interval rather than a single number. When a decision depends on evidence beyond a simulated audience, a study can move to real-human validation with recruited participants without changing the underlying causal question being tested. A simulated audience and a recruited panel are different sources of evidence, and neither substitutes for reporting which one produced a given result. See how a study moves through this process. ## What this kind of discussion doesn't prove The source behind this article is one engineering team's account of building one personality-scoring model. It documents a workflow, not a benchmarked outcome. It contains no comparative accuracy or speed figures that hold up as a general claim, and none are reproduced here. Treat it as a description of a defensible process, not as evidence that any particular implementation is more accurate than an alternative. ## What is a practical gate before you trust a score? Before a psychometric or preference model reaches a hiring, pricing, or segmentation decision, ask which of the five checkpoints above it passed through. A score that only cleared "fit a candidate model" is not yet a number a decision should depend on. Recent case studies show what the fuller version of this workflow looks like when the decision on the other end is a real one. [figure: A five-stage chain: simulate the data-generating process, fit a candidate model, compare parameterizations, validate against real outcomes, report the score with uncertainty. Each stage gates the next.] --- # 10 Ways to Pair Causal Testing With a Research Stack Already in Place URL: https://subconscious.ai/blog/implementation-and-procurement/10-ways-pair-causal-testing-research-stack-already-place Published: 2026-08-18 Most research and analytics stacks already answer what happened. Few of them answer why: which action moved which outcome, for which segment. That gap is the right place to insert a causal test, not a replacement for the analytics, conjoint, survey, or testing tools already doing their job. Getting the sequence wrong is expensive. Running a causal test after a launch decision is already locked wastes budget on data nobody can act on. Running it as a swap-in for a method it should complement leaves stakeholders holding two answers to the same question instead of one causal explanation. [figure: Four-step path: analytics finds what happened in a funnel, conjoint or DCE surfaces stated preference, a causal test explains why by isolating the driving change, an A/B test validates it under live traffic.] Ten places that gap shows up, and how to sequence a causal test into each. ### 1. Behavior analytics and drop-off tracking Product and web analytics tell you where users stall in a funnel: a pricing page, a checkout step, an onboarding screen. They don't tell you why. A causal test then isolates which specific change in that step (price framing, copy, form length) moves completion, holding everything else fixed. ### 2. CRM and segment definitions CRM tools define who your segments are and what they've done. A causal test explains why two segments respond differently to the same offer, so the difference can be built into messaging and outreach instead of guessed at from historical response rates. ### 3. Qualitative research Interviews and focus groups are strong at surfacing the emotional or contextual reasons behind a result, but they're expensive to run broadly. Use a causal test to find which messaging tone or feature moves behavior, then send that finding into a smaller, targeted round of qualitative follow-up. ### 4. Traditional surveys Surveys capture stated opinions and preferences at scale. A causal test can validate whether a preference stated in a survey actually predicts a choice, then a follow-up survey wave can confirm the relationship holds across a broader sample. ### 5. How does conjoint analysis differ from a causal test? Conjoint and DCE are the closest existing method to causal testing: stated-preference data from a conjoint study and revealed-choice data from a controlled causal test answer related but distinct questions about the same decision, which is why researchers pair them rather than substitute one for the other (NCBI PMC, "Investigating the complementary value of discrete choice experiments for the evaluation of barriers and facilitators in implementation research"). Run the causal test after a conjoint study to test which attribute is actually driving the choice, not just which attribute respondents say matters. "Unlike the traditional questionnaire, the results of a DCE provide implementation researchers and clinicians with a relative attribute importance ranking that can be used to prioritize potential barriers and facilitators to change, and hence to better fine-tune the implementation strategies to the specific problems and challenges of a particular implementation process." van Helvoort-Postulart and colleagues, Implementation Science (source) ### 6. Reporting and data visualization Dashboards are built to show what changed over time, not why. Report a causal test's result alongside the existing dashboard view, trend on one axis and causal driver on the other, so stakeholders see the pattern and the explanation in the same review. ### 7. How does A/B testing differ from a causal test? A/B tests confirm which version performs better in production; they don't explain why one wins. Run a causal test before an A/B test to narrow which variants are worth building, then use the A/B test to validate the causal finding under live traffic and real-world constraints. ### 8. Machine learning feature pipelines Models trained on correlated features can pick up signals that don't hold up outside the training window. Feed those findings into feature selection to prioritize inputs shown to actually drive the outcome, not inputs that merely move together with it. ### 9. What do longitudinal tracking studies measure? Long-running tracking studies are built to detect drift over months or quarters, not to explain a single inflection point. When a metric shifts mid-study, a bounded causal test can explain that specific shift without disrupting the tracking cadence already in place. ### 10. Campaign and messaging planning Before committing budget to a campaign, a causal test on messaging, incentive framing, or calls-to-action gives a read on which version is likely to work before it's built and shipped, so the campaign team chooses between validated options instead of untested drafts. ## What this pairing is not This is a sequencing framework for where a causal test adds distinct value in a workflow that already exists, not a certified integration, connector, or data pipeline into any specific analytics, CRM, BI, or marketing-automation product. [figure: One causal-test finding splits into four downstream uses: narrowing A/B variants, ranking model features, explaining a tracking-study inflection point, and picking campaign messaging before spend.] ## Where to start Pick the one stage in the list above where the current stack already produces a what without a why, usually the analytics or descriptive-research stage right before a decision gets made, and insert a single causal test there before extending it further. Read how a causal test is structured and validated, or see worked examples across research programs. --- # What a World Cup Forecasting Model Teaches About Trusting a Model at All URL: https://subconscious.ai/blog/methods-and-validation/what-world-cup-forecasting-model-teaches-about-trusting-model-all Published: 2026-08-18 A model can hit its headline accuracy target and still be wrong on the exact numbers a decision depends on. That gap is the reason a calibration check belongs before every go/no-go call built on a probabilistic or simulation-based forecast, not after. ## What decision is this really about? A team weighing a market-entry date, a launch window, or a resource allocation off a model's probabilities is not asking "is this model roughly right." It is asking whether the model is right on the specific cells the decision turns on. Those are two different questions; a model can pass the first while failing the second without it ever showing up in a top-line accuracy number. A public forecasting project built to predict the 2026 World Cup group stage makes the gap concrete. It is not a Subconscious result, a customer study, or a causal-effect estimate: it is one modeler's independent forecasting exercise, published for its methodology. But the discipline it demonstrates is what separates a model worth acting on from one that only looks worth acting on. ## Build the model, then interrogate it The forecast represented each national team's attacking and defensive strength as an unobserved rating. Goals were count outcomes, and new matches changed the ratings instead of leaving them fixed. The result was a distribution over possible scores, group standings, and advancement outcomes rather than one supposedly certain answer. Producing probabilities is only the beginning of validation. In a posterior predictive check, draws from the fitted model generate plausible new datasets. Those replications are then tested against observed patterns, especially the individual quantities that control the decision. The Stan project's documentation describes this method as a way to discover when a model fails to reproduce important features of the observed data (Stan Documentation, "Posterior and Prior Predictive Checks"). ## Where the check caught the model The forecast's aggregate numbers looked fine: the share of teams scoring 0, 1, 2, or more goals in a match, and the 0-0 scoreline rate, all fell inside the model's predicted range. Real matches ended level more often than the model's assumptions allowed. The model treated each team's goal count as statistically independent of the other's, a common simplification traceable to score-modeling work going back to Maher (1982), but a posterior predictive check flags which statistics the model fails to reproduce, not why; one plausible explanation is that two sides sitting at 1-1 late in a match play differently than the independence assumption implies. Whatever the mechanism, the result showed up as a small, specific miscalibration: too few predicted 1-1s and 2-2s. The gap looks small in isolation, roughly one percentage point on the draw rate. But it sits exactly on the cells a group-stage table is built from: points, goal difference, and goals scored, all determined by exact scorelines. So a deficit that small sits on the exact cells a group-stage table is built from, and whether it shifts which teams the model says will advance is a question the model can answer by simulating advancement probabilities under both specifications, not something to assume. Work on visualizing Bayesian model checks makes the same point: predictive checks find where a fit is locally wrong, not just whether it is globally acceptable (Gabry, Simpson, Vehtari, Betancourt, and Gelman, "Visualization in Bayesian Workflow," Journal of the Royal Statistical Society: Series A, 2019). ## How do you fix the flagged cells without fixing the whole model? Because the check pointed at specific cells rather than a vague "the model is off," the fix could be specific too: a small mixture term that adds extra probability to level scorelines at 1-1 and 2-2, tuned so it moved only the cells the check flagged and left the calibrated cells, including the 0-0 rate, untouched. Re-running the same predictive check afterward showed every flagged statistic back inside the model's interval. That re-check confirms the correction moved the flagged cells as intended, but a fix tuned to those exact statistics will pass the same check by construction, so it is not independent evidence. A model is "fixed" only when held-out evidence, not the same diagnostic that caught the problem, confirms it. Before the forecast was trusted with the 2026 field at all, the corrected version and its predecessors were also backtested against the actual 2022 tournament: scoring group matches the model had never seen and comparing the result against baseline forecasts. [figure: Five steps: aggregate fit passes, but checking the specific cells the decision depends on finds draw-related rates outside the interval; fix only those cells, then re-run the check to confirm.] ## What does this mean for trusting a model before a decision? The lesson generalizes past football. A model built to inform a consequential call, such as where to launch, when to enter a market, or which segment to fund, should be checked against the specific quantities that call depends on, not against a single top-line accuracy figure. The same discipline applies when a study moves from a simulated experiment to real-human validation without changing the causal question. Subconscious can validate studies with real human participants, comparing modeled responses against evidence from people while preserving the question under study. The replication leaderboard shows how that checking process holds up across published studies, and how Subconscious works explains where it fits into a team's existing decision process. For background on the research discipline behind that validation step, see the research program. Before a team commits budget on the strength of any model's output, probabilistic, simulation-based, or otherwise, the standing question is the one this forecast had to answer for its own scorelines: not "is the aggregate accuracy good," but "is the model calibrated on the exact numbers this decision is riding on." --- # AI Customer Satisfaction Research Beyond NPS URL: https://subconscious.ai/blog/use-cases/ai-customer-satisfaction-research-beyond-nps Published: 2026-08-18 NPS can move two points while CSAT holds at 4.1, and a team can still have no explanation for either result. A score records an attitude. It does not identify the experience that produced it or the action that would change it. A score without its limits is marketing copy. AI-assisted satisfaction research can probe possible drivers by segment. The output is a diagnostic hypothesis, not a substitute for real customer evidence. [figure: Five-stage chain: an aggregate score splits into segments (enterprise 4.6 vs SMB 3.4); each segment runs a decision probe on what worked, failed, and what's next; the probe yields a hypothesis; an experiment confirms it.] ## Why the score is shallow NPS asks one question. A CSAT survey asks a handful. An open field may produce something useful from 12% of respondents while everyone else leaves it blank or writes a short answer. Typical post-interaction completion is 5-15%. Very happy and very angry customers can be overrepresented while quietly indifferent customers disappear (Pew Research Center). Timing also changes the response: emotion is fresh right after an interaction and detail is lost weeks later. An overall CSAT of 4.2 can hide enterprise customers at 4.6 and SMB customers at 3.4. Within SMB, customers who joined in the last 90 days may score 2.9. The useful story may sit three levels deep, where a conventional sample becomes too small. ## What can replace the rating prompt? Instead of asking for a rating from 1-5, define the customer segment and walk through the relevant experience. Ask what worked, what failed, and what the customer would do next. Build separate definitions for enterprise accounts, SMB users, customers in their first 30 days, and experienced users. Run the same protocol across all of them so differences are visible. A traditional satisfaction study can take 6-8 weeks. An AI-assisted diagnostic can be run in an afternoon, making it possible to form hypotheses before the next quarter. That speed comes with a limit worth stating plainly: it does not establish that a product change caused the score to move. ## Questions scores cannot answer ### What are the satisfaction drivers? Identify which parts of the experience matter for each segment: speed, reliability, onboarding, support, or pricing relative to value. ### Detractor paths Ask what would need to change, and in what order, for a dissatisfied customer to reconsider. Translate the answer into a product or service action that can be tested. ### Emotional and functional experience A product can work while the customer feels ignored or constrained. A customer can like a brand while struggling with the product. Do not collapse those conditions into one number. ### How does competitive comparison work? Use public evidence to model how an alternative's customers may define satisfaction. Treat the result as positioning research, not a statement from real competitor customers. ## Use the diagnosis across teams Product teams can test whether a proposed roadmap item addresses an important satisfaction driver before spending a quarter on it. Customer-experience teams can map the full path from discovery through onboarding, daily use, and renewal rather than relying only on post-interaction surveys. Retention teams can study at-risk segments before dissatisfaction appears as cancellation. Churn is a lagging indicator, and the relevant intervention may occur months earlier. Competitive teams can compare possible loyalty drivers and investigate where an alternative appears vulnerable. [figure: A score box splits into four categories: drivers by segment, what changes a detractor's mind, emotional vs. functional gaps, and how an alternative's customers might score the same experience.] ## A three-step setup First, define 5-10 customer profiles by company size, plan, tenure, use case, or geography. Include goals, context, alternatives, and category experience. Second, design a consistent protocol. Begin with the overall experience, then probe onboarding, support, pricing relative to value, trust, recommendation, and switching. Third, compare segments. Variance can show which audience needs investment, which pain point should be fixed first, and which satisfaction driver deserves protection. The aggregate score is a signal. The explanation is a hypothesis. A controlled experiment and real customer behavior determine whether the proposed action works. --- # Which Causal Method Fits Your Data When a Randomized Trial Isn't Possible URL: https://subconscious.ai/blog/methods-and-validation/which-causal-method-fits-data-when-randomized-trial-isnt-possible Published: 2026-08-18 A marketing or analytics leader wants to know whether a pricing change, a campaign, or a launch actually caused a shift in customer behavior. Randomized controlled trials give the cleanest answer, but a live GTM decision often can't wait for one, and sometimes randomization isn't available at all. When a team reaches for existing observational data instead, the question becomes which method fits the data shape in front of them, and what happens if the identifying assumption behind that method silently fails. ## Reading a causal effect out of data you already have Causal claims are strongest when they come from randomization: differences in outcomes then reflect the treatment, not some hidden confounder that varies between test and control groups. In practice, randomization is frequently out of reach. Two examples make the constraint concrete: - TV advertising: nobody can flip a coin to decide which households watch a given linear TV spot, but a business still needs to know what that spot did to customer behavior. - Environmental proximity: households cannot be randomly assigned distance from an industrial facility, yet the causal health consequence of that proximity is a real policy question. Quasi-experimental methods exist for exactly this gap. When their identifying assumptions hold, they support a causal claim from observational data without an experimental randomization step. CausalPy, an open-source Python package, packages four of these designs into one Bayesian API. ## Four designs, four data shapes Each method below applies to a different data structure. Picking the wrong one doesn't produce an error: it produces a plausible-looking estimate resting on an assumption your data can't support. Synthetic control applies when multiple units exist and only one receives the treatment. It blends the untreated units into a single weighted composite standing in for the missing counterfactual, then measures the treated unit against that composite once the intervention begins. The gap is the estimated effect. Typical use: evaluating a change rolled out in one region, country, or business unit while others were left alone. Interrupted time series applies when only one unit was ever treated, all you have on it is a running series of measurements over time, and there's no other group to compare against. It fits the pre-intervention trend, extrapolates it forward as the counterfactual, and compares that projection to what actually happened after the intervention. Typical use: assessing a policy change, product launch, or platform update against one tracked metric over time. Difference in differences fits situations with one before-and-after reading on each side: a group that got the treatment and a group that didn't. It takes the before-to-after change within each group, then credits the treatment with whatever separates the two. Typical use: evaluating a program when pre/post data exists for both treated and untreated populations. Regression discontinuity applies when treatment is assigned by a threshold on a continuous running variable, often something other than time, such as an age or score cutoff. A sharp break in the outcome right at the threshold counts as evidence of a causal effect, because confounders that vary smoothly across the running variable can't plausibly produce a sharp jump. Typical use: an eligibility rule or policy line drawn at a cutoff, whether that's an age limit, a passing test score, an income ceiling, or a geographic border. A published example of this last design: Carpenter and Dobkin's study of the US legal drinking age used regression discontinuity to estimate the effect of turning 21 on all-cause mortality, finding a discontinuity in the mortality rate right at the threshold (NBER working paper; published version, American Economic Journal: Applied Economics). The logic: no treatment was randomly assigned, but if mortality breaks sharply exactly at age 21 and nowhere else, a smoothly varying confounder can't explain it. "This increase in alcohol consumption results in a discrete 9 percent increase in the mortality rate at age 21." Carpenter and Dobkin, NBER Working Paper 13374 (source) ## Do these causal methods check their own assumptions? None of these four methods verify their own assumption automatically. Synthetic control assumes the untreated units can be weighted into a valid counterfactual, checked with in-space and in-time placebo tests. Interrupted time series assumes the pre-intervention trend would have continued unchanged, checked with placebo-date tests. Difference in differences assumes parallel trends between groups absent the treatment, checked with pre-trend and event-study tests. Regression discontinuity assumes nothing else changes discontinuously at the same threshold, and that units can't sort precisely around the cutoff, checked with a McCrary density test. Each diagnostic can catch a broken assumption, but none of them run on their own: skip the check, and the method still returns a confident-looking estimate that attributes an unrelated shift to the action you took, wasting the budget decided on that estimate. ## Running the controlled experiment instead Where quasi-experimental methods extract a causal estimate from data that already exists, Subconscious runs the controlled experiment upfront: pricing, messaging, or launch actions get tested against a person-level audience graph before the decision ships. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people: that's audience reach for a controlled study, not a recruitable panel of 800 million participants. When the causal question calls for it, the same study design can move from simulation to real-human validation. That matters because a quasi-experimental analysis estimates a causal effect on realized market behavior, while a designed choice experiment estimates a causal effect on stated preference within the choice task itself, and stated preference tends to run high relative to realized behavior (hypothetical bias). The two answer different questions with different risk profiles. ## What does Subconscious not replace? Subconscious does not implement synthetic control, interrupted time series, difference-in-differences, or regression discontinuity as product features, and moving to real-human validation doesn't turn a controlled causal test into a clinical trial, a usability session, or an automatic guarantee of market performance. Quasi-experimental methods remain the right tool when the data already exists and no new study is possible. The choice in front of a buyer is whether that's the situation they're actually in, or whether the decision is big enough to justify running a controlled test before committing the budget. [figure: A four-row list, one row per causal method, each paired with the specific unverified assumption its causal claim depends on.] ## Where to go next If you're deciding between reading a causal effect out of existing data and running a controlled test before you commit budget, Subconscious's research covers how the controlled-study approach works, and how we work covers what a study looks like end to end. [figure: Branching diagram from "what data do you have," splitting into four paths by data shape, each ending in its matching causal method.] --- # Synthetic User Research Platforms: Which Method Fits Your Decision? URL: https://subconscious.ai/blog/use-cases/synthetic-user-research-platforms-which-method-fits-decision Published: 2026-08-18 Synthetic user research is now a practical category with real tools and real buyers. The harder question isn't which vendor to pick. It's which type of synthetic method answers the decision a research, product, or marketing leader faces, before budget or engineering time is committed. ## What synthetic user research covers The category spans two jobs. Some tools simulate open-ended conversation: an AI persona or panel a team can talk to and read for directional impressions. Others run a controlled experiment: a defined set of alternatives tested against a defined population, measured as a behavior rather than a conversation. Both are useful. They are not interchangeable, and picking the wrong one is the real risk. ## The buyer's real choice A team evaluating this category usually chooses between two shapes of tool: Directional tools answer "what do people generally think about this?" A controlled experiment answers "which of these specific alternatives do people actually choose, and by how much?" ## Where do directional synthetic tools fit? Open-ended persona and panel tools are the right choice earlier in a process, before the team has a short list of alternatives to compare. They also fit teams whose primary need is a lightweight, ongoing sense of a customer type across product, marketing, and sales, without a designed experiment for every question. Focus-group-style tools and one-off report generators serve a narrower version of the same job: an occasional, self-contained read, useful when the research cadence is infrequent and the stakes are low. The tradeoff is the same: they simulate conversation and impression, not a measured choice. Persuasive language, a strong stated preference, or a plausible-sounding persona response is not evidence the underlying behavior would shift. A review of AI-generated persona research calls this the category's central risk: a confident synthetic read can still misrepresent the population it claims to speak for, with nothing inside the tool to flag the error (ACM Interactions). [figure: Two-column diagram: left lists three directional methods producing impressions from simulated conversation; right shows a controlled discrete-choice experiment producing measured effects with confidence intervals.] ## Where does a controlled experiment fit? Subconscious runs a controlled discrete-choice experiment: defined alternatives, tested against a defined population, with causal effects and confidence intervals as the output rather than a transcript. That's the fit when the buyer already has a short list of product, price, or message alternatives and needs to know which one moves the outcome, not just which one sounds more appealing. A structured choice format, rather than open-ended reaction, lets a comparison trace a stated preference back to a specific, testable alternative: the property a funded decision needs, per a think-aloud comparison of choice-based research methods (PMC). "The majority (18,75%) of participants indicated a preference for DCE, as they felt this enabled comparison of alternative full profiles." Whitty and colleagues, PLoS ONE (source) The comparison runs against a person-level audience graph covering 800 million real people. That's a measurement scale, not a recruitment claim: not a panel a team schedules interviews with. See the replication leaderboard for how the experiment is structured and how results are validated. ## What doesn't a controlled experiment replace? A controlled causal experiment doesn't replace direct usability observation, moderated qualitative research, or in-market results. It answers which tested alternative changes a defined behavior; it doesn't watch someone struggle through an interface, surface untested objections, or confirm how a launch performs once it ships. It isn't automatic proof of market performance on its own. Where the decision depends on it, a team can move from the simulated experiment to real-human validation of the same comparison without changing the causal question being asked. For smaller, lower-stakes calls, the simulated comparison is often the whole answer a team needs. ## Making the call Four questions determine which method fits: - Do you have a short list of alternatives, or are you still forming one? A short list points toward a controlled experiment. An open question still being explored points toward a persona or panel. - What's the cost of guessing wrong? A pricing, launch, or positioning decision justifies a measured comparison. A low-stakes, easily reversible call doesn't need one. - What's your research cadence? Frequent, exploratory conversations fit a persona tool. A decision that needs to be defensible, not just plausible, fits a designed experiment. - Does the decision need to survive scrutiny after the fact? If a stakeholder will ask "how do we know," a directional impression won't hold up as well as a causal effect with a confidence interval. Most research programs use both: directional tools to explore and frame the alternatives, then a controlled experiment to measure which one changes the outcome once the list is short enough to test. Book time to see which fits a specific decision. --- # Audience Research Tools Compared: Query, Conversation, or Causal Experiment URL: https://subconscious.ai/blog/comparisons/audience-research-tools-compared-query-conversation-causal-experiment Published: 2026-08-18 A research, product, or market leader should choose an audience-research tool by the decision it must support. Use a query tool for population-level measurement. Use a conversation tool to explore reasons and objections. Use a controlled causal experiment when the consequential question is which action will change behavior. The wrong instrument can produce a confident answer that does not justify the budget or roadmap choice. ## Start with the evidence job Interface is a poor way to compare research tools. A chart, a chat window, and an experiment can all look persuasive while answering different questions. Aaru presents its product as predictive analytics for decisions across a population (Aaru). Synthetic Users presents its product as a generated user-research platform (Synthetic Users). Those first-party descriptions establish two useful categories: structured population queries and exploratory conversation. Neither description, by itself, establishes that a specific intervention caused a behavioral outcome. The third category begins with an action. It compares defined alternatives on the same target population and estimates which alternative moves a specified outcome. ## Three instruments answer three questions [figure: Three instruments side by side with their question: query for population measurement, conversation for reasons and objections, experiment for which action changes behavior.] These instruments can complement one another. Conversation can reveal objections worth testing. A query can identify a segment or market pattern. A controlled experiment can then compare the actions a team might take. The methods become dangerous only when one output is used to make a claim that belongs to another. ## When description is not enough Suppose a team must choose between two launch messages. A query tool can measure current preference across segments. A conversation tool can surface possible reasons for hesitation. Both can improve the alternatives. The investment decision still asks a different question: which message is more likely to change the target behavior for the defined population? Answering it requires holding the relevant conditions constant, changing the message, and comparing the outcome. Without that contrast, a team may mistake correlation, stated preference, or persuasive language for evidence about an intervention. Subconscious is designed to fit here: a causal behavioral platform that helps teams test product, pricing, messaging, and go-to-market actions before committing capital. Its method centers on controlled experiments over simulated populations rather than open-ended roleplay. The team defines the action, alternatives, population, outcome, and constraints. The experiment estimates the directional difference between alternatives and reports uncertainty where the study design supports it. ## Put proof and scope in the same sentence A fidelity number without its limits reads as marketing. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. It is a validation result, not a guarantee that every new study will predict market performance. Audience reach is a separate property. Subconscious can use a person-level audience graph covering 800 million real people when population scale matters. Stating what that figure covers lets a buyer check the claim against the study itself. That figure describes the graph available for audience definition. It does not mean a study recruits 800 million participants. Subconscious can test or validate studies with real human participants. A team can move from a simulated experiment to real-human validation without changing the causal question. Spelling out the boundary of real-human validation keeps the comparison honest. That step does not turn the work into an observed usability session, a clinical trial, or automatic proof of commercial performance. ## Apply a procurement test before choosing Ask each provider the same practical questions: - What exact decision will this output change? - Does the method measure a population, explore possible reasoning, or compare actions? - What is held constant, and what is deliberately changed? - Which population, alternatives, and outcome bound the result? - How is uncertainty reported, and what would make the result unreliable? - Can the causal question proceed to real-human validation when the stakes require it? A query tool is the better choice when the deliverable is a recurring measure or a segment comparison. A conversation tool is the better choice while the team is still discovering language, objections, or hypotheses. A controlled experiment is the better choice when the alternatives are defined and the cost of choosing poorly is material. Subconscious does not replace open-ended qualitative conversation or ongoing dashboard-style sentiment monitoring. It is built for a bounded action against alternatives. Teams can review the underlying research, examine decision examples in case studies, and see how the workflow is scoped. When the decision, alternatives, population, and outcome are ready, the next step is to scope a controlled test. --- # AI-Moderated Interviews or Causal Experiments: Which Fits Your Decision URL: https://subconscious.ai/blog/comparisons/ai-moderated-interviews-causal-experiments-which-fits-decision Published: 2026-08-18 An insights or marketing lead facing a product, pricing, or messaging decision usually has to pick between two very different tools: an AI-moderated interview platform that talks to real people, or a causal experiment platform that tests actions on a simulated market. Picking the wrong one wastes a recruitment cycle, or trades away the depth a real conversation gives you. The right choice depends on the question you're actually asking. ## What decision is actually on the table Before comparing tools, name the decision. Two questions get confused constantly: - "What do people think, and why?" This needs open-ended human narrative: contradictions, tone, unscripted objections, the reasoning behind a preference. - "Which specific action moves the outcome?" This needs a controlled comparison: show one group action A, another group action B, and measure the difference in a decision-relevant outcome like preference, adoption, or willingness to switch. Choosing an interview tool when the real question is which lever to pull leaves a team with rich quotes and no quantified answer about what to do next. Choosing a causal-experiment tool when the question needs emotional nuance or open-ended discovery loses exactly the depth that format is built for. The cost isn't just the study itself; it's shipping the next decision on the wrong kind of evidence. ## Two different answers to "what will people do" AI-moderated interview platforms recruit real participants and use an AI moderator to run one-on-one conversations, then synthesize themes and quotes across the transcripts. Teams reach for this category when they want that same depth but can't put a live moderator on every session (listenlabs.ai). Causal experiment platforms run randomized experiments on a simulation of the market instead of asking one group of real people what they think. They compare how a simulated population responds to alternative actions, such as different prices, messages, or product concepts, and estimate which action is more likely to change the outcome that matters, with uncertainty reported where the study design supports it. Neither approach is a strict upgrade on the other. They answer different questions. ## Comparing the two approaches Use the table to decide which job needs doing, not to pick a category winner. ## How to route the decision - Is the question "why" or "which"? If the team needs to understand reasoning and language, start with real conversations. If the team needs to know which specific action performs better, start with a controlled comparison. - How many alternatives are being compared? A handful of messages, prices, or concepts across several segments favors a method built for running many controlled comparisons quickly. - What's the cost of being wrong? High-stakes, capital-committing decisions usually deserve both: a fast causal read to narrow the field, then real-human depth or validation on the finalists. This is where a causal behavioral platform like Subconscious fits: it runs randomized experiments on a simulation of the market, validated against real human behavior, to estimate which action is likely to change a decision-specific outcome before a team commits budget, roadmap capacity, or brand equity to it. Learn how that experiment design works. That is a different job than synthesizing interview transcripts. [figure: A two-column comparison of AI-moderated interviews and causal experiments on a simulated market, scored across respondent type, output, strongest use, recruitment need, and weakest use.] ## Where does each method run out of road? A method's limits belong on the record next to its strengths, so a buyer can weigh both before choosing. AI-moderated interviews are bound by recruitment. Every study needs real participants sourced, screened, and incentivized, and the depth that makes the format valuable also limits how many alternatives a team can practically test. Naming where a method stops working is what lets a buyer check it against their own decision. Causal experiment platforms have their own limit: a simulated population is not a substitute for the open-ended narrative, contradictions, and unscripted objections a live conversation surfaces. A simulated study answers which action moves the outcome, not what a person actually means when they say something. ## How do you move from a fast read to a validated one? For decisions where the answer needs confirmation from real people, the practical path is to move from a simulated experiment to real-human validation without changing the causal question being tested. Subconscious can test or validate studies with real human participants. See how Subconscious runs that validation step. That sequence, not a single tool, is usually the right answer for a high-stakes decision. ## Adjacent questions Can you use both in the same project? Yes. A common pattern runs a fast simulated comparison to narrow a wide set of alternatives, then follows with real-human interviews or validation on the finalists, using each method for the job it's built for. Does a causal experiment replace qualitative research? No. It answers which action is likely to move a specific outcome. It doesn't replace the open-ended discovery and emotional nuance a real conversation surfaces. What proof should a causal-experiment vendor be able to show? Ask what the simulated population is validated against, and how. Subconscious runs controlled studies against a person-level audience graph covering 800 million real people, kept distinct from any recruited participant panel, and can validate specific studies against real human responses on request. Prior case evidence shows the pattern in practice. --- # AI Tools for Product Managers: Research at Decision Speed URL: https://subconscious.ai/blog/buyers/ai-tools-product-managers-research-decision-speed Published: 2026-08-18 Product managers work between customer needs, business goals, and technical constraints. The cost of a weak assumption rises once it becomes a specification, sprint, and launch. Customer evidence often arrives too late. A researcher may have a dozen priorities. A formal study may take four to six weeks while the team runs two-week sprints. AI can help prepare and screen decisions before the higher-cost research begins. [figure: A horizontal timeline of five product-calendar points in sequence, each with a small AI-screening checkpoint before an arrow leads to the next point.] ## Use the right claim AI does not create an on-demand version of a real customer. Simulated audiences can support controlled, early experiments when the team defines the decision, audience, alternatives, and outcome, using the method described on the research page. A planning example might prepare a study next week rather than promise a complete customer-research session in the next 20 minutes. A number without its limits is marketing. Speed does not establish validity. ## Five product-management applications ### How do you discover and prioritize product features? When a backlog contains fifteen possible features and a sprint has room for three, compare the candidates against an explicit customer behavior and business constraint. Define two to four audience segments. Ask which action changes intended use, which is merely preferred, and which creates a barrier. Treat the result as input to prioritization, not proof of demand. ### What should a user-story review check? Check whether a story reflects the customer's problem, current workflow, and decision. Use the review to find hidden assumptions and edge cases before engineering begins. ### Specification and problem framing Compare the experience implied by a specification with the problem it claims to solve. Naming a model's failure mode is what lets a buyer check it before relying on it. A language model can inspect the written artifact. It cannot experience the actual product, so prototypes and customer observation remain necessary. ### Why does onboarding research matter? Onboarding is one of the highest-impact product problems. Compare alternative instructions, sequences, or messages for a new customer. Measure completion or adoption behavior when possible. ### Stakeholder preparation Use simulated CFO, engineering, or design perspectives to rehearse objections. The misses sit next to the hits so a reviewer can weigh both. These perspectives do not replace finance, technical, or design review. They help the PM prepare the evidence those reviewers will need. Competitive-feature analysis follows the same rule. Models cannot reveal a competitor's private roadmap or predict a specific customer's response. They can help structure testable alternatives. ## Add experiments to the workflow Sprint planning: use a 30-minute planning session on stories where customer intent is unclear. Define what needs evidence. Backlog refinement: compare the priority order with the stated customer behavior and business constraint. Specification writing: add an assumption review as the last step before engineering review. Launch planning: test launch-message and adoption alternatives before production. After launch: use simulation to generate hypotheses about usage metrics, then check analytics and speak with customers. The 30-minute duration and last-step placement are planning examples, not universal product limits. [figure: Branching path: a question hits a decision point on frequency and stakes. High-frequency, early-cycle routes to AI screening. High-stakes, direction-setting routes to real customer research.] ## Keep people and behavior in the loop Publishing this limit is what lets the rest of the claim hold up under scrutiny. Real usage data is irreplaceable. People often act differently from what they say. Breakthrough discovery and the rarest early adopter are also difficult to reproduce because simulated audiences represent patterns of the many more readily than the few. Use AI for high-frequency, early-cycle questions. Use real research for the highest-stakes product direction, following the escalation path described in how we work. A causal behavioral experiment is strongest when it connects a product action to an observed outcome and states the remaining uncertainty, per the causal fidelity paper. --- # How to Pressure-Test a PRD Before the Engineering Kickoff URL: https://subconscious.ai/blog/methods-and-validation/how-pressure-test-prd-before-engineering-kickoff Published: 2026-08-18 A PRD flaw that slips past review is the costliest bug a product team can ship. Design ends up reworking the flow twice before engineering finishes, and the buyer meets the finished feature with a shrug. That is 6 weeks of build time gone, plus the opportunity cost of whatever the team did not ship instead. The decision a product leader needs to make before an engineering kickoff is narrow: proceed with the spec as written, cut scope, or kill it. A full round of user interviews is too slow to run before every kickoff. The PRD needs a pressure test between the draft and the meeting, one that answers whether the proposed feature actually changes what the target buyer does, compared with not shipping it. ## Why the decision matters Every PRD that reaches an engineering kickoff carries an assumption about who will use it and why. When that assumption is wrong, the team does not find out until adoption data comes back, by which point the cost is already spent: engineering time on a feature the target segment does not adopt, and the roadmap slot that feature occupied instead of something buyers would have used. ## Where does an ad hoc review break down? A hallway review or a standing kickoff meeting can catch obvious problems, but it has a structural weakness: everyone in the room helped write or approve the spec, so they share the author's blind spots. ### Wrong-segment bets A PRD is often written for the buyer the team imagines it has, not the one it actually has. In one review team's account, the group representing 60 percent of the active base said "this is not for me," while a group representing 15 percent said "this is exactly what I need." The spec was quietly solving for a minority the whole time, and nobody in the room noticed because nobody was asking the question that way. ### Hidden first-mile friction A PRD assumes the buyer is already inside the workflow. A newcomer has no idea where to begin. That distance between what the author pictured and what an unfamiliar buyer sees is where a large share of feature-adoption failures start; one estimate puts it around 80 percent. This is the kind of directional, qualitative signal that conditioned personas tend to reproduce well even though they diverge from individual human responses on quantified questions (arXiv, "When Can Digital Personas Reliably Approximate Human Survey Findings?"). ### Scope nobody will use A feature can ship with 7 capabilities when buyers only ever touch 2. The pattern is visible before launch if someone walks a fresh reader through the spec and asks what they would use, ignore, and actively turn off, but a review room rarely runs that exercise. ### Competitive blind spots A PRD that stays silent on the alternative a buyer is already using gets outflanked at the sales conversation. It needs a differentiation section, or it needs to be cut. ## How does Subconscious test a PRD before kickoff? Subconscious treats a PRD pressure test as a controlled comparison rather than a discussion. The setup starts with defining the buyer segments the spec actually targets, not the ones the team assumes. The proposed feature is then compared against the status quo, or against a named competing alternative, as two distinct actions inside a decision-specific experiment. The result reports which segment's behavior actually shifts, with uncertainty attached where the study design supports it. That is a different question than whether a feature sounds good in a room; it is whether the action changes what a defined segment does. A team can run that comparison on the same buyer segment and the same competing alternative a review room would have argued about anyway, using research built around the decision instead of opinion collected in a meeting. Subconscious can also test or validate studies with real human participants: a team can move from a simulated comparison to a real-human check without changing the underlying question, so the pressure test and the confirmation round answer the same thing. ## Comparing the options ## Limitations A causal result without its limits attached is marketing copy. Publishing where the method stops is what lets a product leader judge whether it fits this decision. Subconscious does not replace direct usability testing, willingness-to-pay interviews, or small real-user alpha rounds for habit-breaking workflow changes. For those changes, a small alpha with 10 to 20 real users still catches what a simulated comparison cannot: whether the change breaks a habit already built into someone's day. For pricing questions tied to a PRD, a 5-user round of structured interviews still answers what a comparison of actions cannot: the exact price a buyer will pay. Naming a failure mode here is what lets a buyer check the method before committing engineering time to its result. Subconscious also does not access a buyer's existing account state or live product data. It cannot confirm whether a migration path works inside a real account, and it does not guarantee the exact market outcome of any shipped feature. Treat the comparison as the first pass on whether the spec is worth the engineering investment, not a substitute for those checks. [figure: A draft PRD forks into an ad hoc review room versus a causal action test comparing the feature to the status quo for the real segment, yielding a result that feeds a proceed, cut, or kill decision.] ## Next step Pick the next PRD on the roadmap. Define the segment it targets, the status quo it replaces or the competitor it is up against, and the result that would tell the team to proceed, cut scope, or kill it. Run that comparison before the kickoff, not after the sprint starts. Book a demo to scope the first comparison. --- # Live moderated video research or causal simulation: choose by the evidence required URL: https://subconscious.ai/blog/comparisons/live-moderated-video-research-causal-simulation-choose-evidence-required Published: 2026-08-18 Choose live moderated video research when watching a real respondent is part of the evidence. Choose a causal simulation when the decision is which action is most likely to change a defined behavior. A simulated result cannot supply body language, tone, or an observed reaction, while a moderated conversation does not by itself isolate the effect of one action against another. [figure: Decision path from "name the action, alternative, outcome" branching to "live moderated research" for observation risk, "causal simulation" for action risk, and "sequence both" when both risks remain.] ## Start with the evidence the decision needs A live moderated research platform offers structured, moderated conversations with real participants (platform overview). This method is the stronger fit when a researcher must see how someone handles a prototype, hear hesitation, follow an unexpected answer, or interpret a reaction in context. The limits of a simulated panel belong on the page next to what it can do. A simulated panel produces directional reasoning from a defined audience model. It can help a team form hypotheses or compare lines of inquiry, but it does not show what a recruited respondent did in a live session. ## The wrong method leaves a specific gap Naming where a method falls short is what lets a buyer check it against the decision at hand. Using moderated sessions for a causal choice can produce rich explanations without showing which action caused the outcome. This gap goes on the record too, since a method's misses matter as much as its hits. Using simulation when direct observation is required creates the opposite gap. The team receives modeled reasoning but never sees a person hesitate, navigate, misunderstand, or respond on camera. For usability, ethnography, and other observation-dependent work, that missing evidence is decisive. The buyer should name the action, outcome, and required evidence before committing fieldwork budget or calendar time. The cost of choosing poorly is not just the research expense. It is reaching the decision point with an answer to a different question. ## Causal experiments answer the action question Subconscious is the causal-experiment layer in this research system. It tests which action moves a defined outcome in a simulation of the market. The practical advantage is decision alignment. A team can compare the actions it might take against the behavior it wants to change, then use the result to decide what deserves further investigation. How the method works explains the experiment design, while /research covers the validation approach. Subconscious can also test or validate studies with real human participants, carrying the same causal question into recruited human research. It does not turn the study into a moderated video session. ## Build the research sequence around the unresolved risk Begin with the decision itself: the action under consideration, the alternative, and the behavior that should change. Then choose the next method based on the risk that remains. - If the unresolved risk is causal, compare the actions. - If the unresolved risk is how a real person behaves or reacts, recruit and observe real respondents. - If both matter, keep the methods separate and sequence them deliberately. Use the causal comparison to test the action question, then use live moderated research for the observation question. For a decision centered on which action changes behavior, bring the action, alternative, and target outcome to a working session. For a decision centered on direct observation, choose live moderated research. --- # Pareto/NBD: Finding Silent Churn Before It Shows Up in Revenue URL: https://subconscious.ai/blog/methods-and-validation/pareto-nbd-finding-silent-churn-before-shows-up-revenue Published: 2026-08-18 A customer who buys on demand, not on a contract, never clicks "cancel." They just stop. By the time a revenue report shows the drop, the budget window to win them back has usually closed. The Pareto/NBD model exists to catch that customer earlier. It estimates, from purchase history alone, which buyers are still active and which have likely already gone quiet. ## What the model separates Pareto/NBD couples two mechanisms per customer: - A purchasing process: while active, purchases follow a Poisson process, so short gaps between orders are more likely than long ones. - A dropout process: the time until a customer goes inactive follows an exponential distribution, modeling the silent churn that non-contractual businesses never observe directly. A customer can have high historical purchase frequency and a low probability of still being active at the same time, exactly the profile revenue reports miss until it's too late. Schmittlein, Morrison, and Colombo introduced the model in a 1987 paper that asked how a business could identify its individual buyers and forecast what each one would purchase next (Management Science, 1987); later work extended it with hierarchical Bayesian estimation (Marketing Science) and a simplified alternative formulation (Marketing Science). It remains the reference model for non-contractual, continuous-purchase settings: grocery, retail, subscription-adjacent commerce with no cancellation event. ## What does Pareto/NBD need, and what does it give back? The model runs on four fields already sitting in a purchase-history table: a customer_id, frequency (repeat purchases), recency (time of the most recent purchase), and T (time since first purchase). No survey, no additional tracking, no new instrumentation. In return it produces four estimates for each customer: - Expected purchases over a future window. - Probability the customer is currently active ("alive probability"). - Probability of making an exact number of purchases in a future window. - Expected purchases for a brand-new customer with no history yet. A customer with high historical frequency but a low alive probability is a specific, targetable signal: someone who used to buy often and appears to have quietly stopped. That is the population retention budget should reach first, before spend goes to customers who were never actually at risk. ## What does Pareto/NBD not tell you? A model's output means something only when its boundaries are stated alongside it. Pareto/NBD predicts purchase occurrence and churn risk. It does not estimate monetary value; that requires pairing it with a separate model such as Gamma-Gamma. And it does not model contractual or discrete-purchase businesses, where a cancellation event already exists and this non-contractual framing doesn't apply. Naming a limit here tells a team where the model's job ends and a different kind of evidence has to start. It also does not tell a team what to do about an at-risk customer. The model outputs a probability, not a causal answer. Historical purchase data can show that a customer's alive probability has dropped; it cannot show whether a discount, a personalized email, or a loyalty credit would actually change that customer's behavior, because none of those interventions happened in the historical data the model was fit on. ## Where the two methods connect Pareto/NBD narrows a full customer base down to the accounts worth spending retention budget on. Subconscious can run randomized choice experiments with real human participants to answer the next question: how a given offer, message, or price change shifts choice probabilities in a population resembling the at-risk segment. It does not replace the purchase-history model; it picks up where the model's probability estimate stops. [figure: Flow diagram: purchase history splits into a Poisson purchase process and an exponential dropout process, combining into an alive-probability estimate that routes low-probability customers into a retention test.] ## Where does Pareto/NBD fit in a retention workflow? Use Pareto/NBD as the triage step: it works directly on purchase logs a business already has. It answers "who is at risk," not "what should we do about it" or "how much are they worth." Pair it with a monetary model for value, and with a causal test for the intervention decision itself. [figure: Four boxes left to right: Pareto/NBD ranks at-risk customers; Gamma-Gamma prices that list; a causal test checks which offer changes behavior; then a retention offer goes out.] ## Related reading Explore causal behavioral research methods or see how validated studies move from simulation to real participants. --- # The Model Was Right. The Decision Came First Anyway. URL: https://subconscious.ai/blog/methods-and-validation/model-was-right-decision-came-first-anyway Published: 2026-08-18 A data scientist builds a strong predictive model. Executives respond with a barrage of what-ifs: what happens if the budget shifts, if volume drops 10%, if three variables move at once. The data scientist spends a week writing new queries and adjusting parameters. By the time the answers are ready, the business has already decided, on a gut feeling, because the decision could not wait. The model was right. It arrived after the decision had been made. ## Why is the bottleneck structural, not personal? The gap does not come from a shortage of skill. It comes from three misalignments that compound. Translation. Stakeholders talk in business outcomes: volume, share, revenue. Analysts talk in code, parameters, and statistical significance. Turning "what if we tweak the budget?" into a validated query is manual work, and every what-if becomes its own small project. Timeline. Analysis runs in sprints. Business pressure runs in real time. When a decision is due Tuesday and the analysis lands Friday, the model arrives too late to shape the outcome it was built to inform. Burnout. The first two compound into a third. Stakeholders start treating the analytics team as a black box where urgent requests disappear. Analysts spend their time re-running yesterday's questions instead of improving the model, and trust erodes on both sides. McKinsey's research on data culture describes the same pattern at the organizational level: teams report being data-driven while decisions still lean on intuition once the analysis lags the decision window (Why data culture matters). [figure: Two paths from question to answer. Left: agent translates the question into a query on the spot, answer arrives after asking. Right: decision scoped before the question, answer exists already.] ## What are two ways to close the gap? One response, common among teams building conversational layers over existing models, is to let a stakeholder ask a business question in plain language and have an agent translate it into a query against the model on the spot. For example: a stakeholder asks what happens to Q3 revenue if the digital ad budget drops 15% with half reallocated to TV, and the agent runs the scenario and returns a summary with confidence intervals. The translation step still happens, just automated and moved closer to the moment the question is asked. A second response changes when the translation happens. Instead of routing every scenario through an analyst after the fact, the experiment is structured around a specific business decision before the question is asked: what action is being weighed, what outcome it should move, for which segment. Subconscious's causal experiments work this way: the decision comes first, and the experiment is scoped to answer it directly, with no ad-hoc translation step. Both approaches attack the same translation gap. The difference is where the design work happens: at query time versus at experiment-design time. ## What does this approach not solve? Subconscious is a causal experimentation platform, not a conversational agent that queries an existing predictive model on demand. It is not a substitute for the interactive dashboards or notebook tooling teams already run against their own models. Decision-specific experiment design reduces how often a team needs an ad-hoc query cycle in the first place; it does not replace the value of iterating within an experiment that is already running. [figure: One-off explorations lead to a better query interface; a recurring decision with a deadline leads to designing the experiment around the decision, then optionally human validation.] ## When it is worth restructuring the analysis If most requests are one-off explorations of a model that already exists, a better query interface addresses the actual problem. If the same handful of business decisions keep arriving with a deadline the ad-hoc cycle cannot beat, the higher-leverage move is to design the experiment around the decision itself, so the answer exists before the question is asked. Teams moving from a simulated read to validation with real human participants can do that without changing the underlying causal question, useful when the decision is big enough to warrant both. See how this fits into a working process, or talk through a specific decision. --- # How to Test a Product Name Before You Commit to It URL: https://subconscious.ai/blog/buyers/how-test-product-name-before-commit Published: 2026-08-18 A founder naming a product before launch, fundraising, or a rebrand has three options: pick from gut, ask a handful of friends who already like the founder, or hire a naming agency. None answers the question that matters: what a buyer who has never met the founder assumes about the product the first time they read the name. ## Why the name decision is expensive to get wrong A product name appears on every URL, every contract, every pitch deck, and every customer's first impression for the life of the company. If the name misleads on category, is hard to recall, or undercuts trust in a sensitive category like finance or health, that friction repeats on every future interaction. By the time collateral, a domain, and brand equity are sunk into the name, reversing it is expensive. ## Why the usual methods don't answer the buyer question Three fallback methods dominate naming decisions, and each has a structural blind spot: - Gut check. The founder has spent months attached to the project, which contaminates their read on how a stranger encounters the name cold. - Friends and family. A small group that already likes the founder and wants the launch to succeed cannot hold the frame of a buyer who has never heard of the company. - A naming agency engagement. The paid alternative, priced as a full strategic engagement. Useful for a team with the budget and runway for a multi-week process, out of reach for a one-to-three-person founding team deciding this week. None of these tests what a real buyer population assumes about the product, or whether that assumption holds up under a controlled comparison against the alternatives. ## What a structured comparison answers, and what it doesn't The naming decision splits into two kinds of question, and only one of them is a research problem with a defensible answer. Subconscious runs controlled experiments on a simulation of the market to test which of several candidate names changes comprehension, recall, category fit, or trust for a defined buyer population, rather than collecting unstructured opinions from a small, biased group. ## The buyer questions worth asking, regardless of method Whatever route a founder uses, five questions distinguish a name that holds up from one that doesn't: - Reading the name cold, what product category does a buyer assume it belongs to? - Is it easy to say out loud, without hesitation? - Would a buyer remember it a day later? - Does it stand apart from the closest competitor's name, or does it get confused with one? - What does the name imply about the product before anyone reads a word of copy? A name that wins on all five is rare; most founders optimize for the three that matter most for their category. Trust and category fit usually dominate for money, health, or sensitive data. ## How do you build a candidate set for name testing? A comparison only produces a useful answer if the candidate set spans genuinely different naming strategies, not variations on one preferred idea: Populate each structure with a handful of candidates, then remove anything that collides with a major brand or is hard to spell or say out loud. ## What does this naming method not replace? A naming comparison does not substitute for checks that sit outside comprehension and recall: - Trademark clearance. No comparison, human or synthetic, can tell a founder whether a trademark application will be rejected. That requires a trademark search or an IP attorney. - Domain availability. Whether a domain sits parked, was grabbed by a squatter, or belongs to a running business elsewhere in the market is a registrar-level check, not a research question. - Cultural and acoustic fit for non-English markets. A name that reads cleanly in English can trigger unintended associations elsewhere; that calls for a native-speaker review, not an English-language comparison. A pre-launch name test does not guarantee market reception after launch, and it does not replace founder judgment on the qualitative texture a name carries beyond the dimensions tested. ## Where does human validation fit in the process? For a founder who wants both a fast comparison and a human check, the practical path is to run the controlled comparison first, then validate the shortlist with real people before committing. Subconscious can test or validate studies with real human participants, moving from a simulated comparison to real-human validation without changing the underlying question: which name changes buyer comprehension and trust, and by how much. This is the same discipline behind controlled comparisons in market research: isolating one variable, comparing it against defined alternatives, and reading the result against a baseline (Sawtooth Software on choice-based conjoint methodology). It matters here for the same reason it matters in pricing or messaging research: what people say they'd choose in conversation often diverges from how they'd actually respond to a real comparison (the say/do gap in market research). ## A practical sequence for the decision - Build a candidate set spanning different naming strategies rather than variations on one idea. - Compare the shortlist against a defined buyer population using the five questions above. - Take the top few names into a second, deeper pass that asks specifically about trust and category assumptions. - Run trademark and domain checks on the finalists in parallel. Do not wait until after the name is chosen. - If launching outside English-speaking markets, get a native-speaker read before committing. No Subconscious case study covers a naming decision specifically. The relevant proof is the experiment design and validation process behind the platform, not a borrowed result from an unrelated use case. ## Next step Read how Subconscious structures and validates causal behavioral experiments on /research, see applied comparisons on /case-studies, and review the workflow for standing up a study on /how-we-work. For a founder ready to test a specific naming decision, book time. [figure: Two columns: comprehension questions (category, recall, trust) answered by a buyer comparison; legal questions (trademark, domain, language fit) answered by a trademark search, registrar, and native speaker.] --- # Why Thin Segments Need Partial Pooling Before You Trust a Causal Estimate URL: https://subconscious.ai/blog/methods-and-validation/why-thin-segments-need-partial-pooling-before-trust-causal-estimate Published: 2026-08-18 A segment-level causal estimate can look strong for a reason that has nothing to do with the segment. When a niche buyer persona, a single region, or one price tier has few observations, its estimate carries more noise than the pooled estimate for the whole population. Treating that noisy number at face value is how a team rolls out a change to a segment that never actually outperformed: the apparent lift was sampling variance, not a real effect. ## What's wrong with estimating each segment alone? An unpooled model fits every segment as if it were unrelated to every other segment. Each estimate uses only the data in its own bucket. For a segment with a large sample, that is fine: the estimate converges on something close to the true effect. For a segment with a small sample, the estimate can swing widely based on a handful of observations, and nothing in the unpooled model corrects for that. The opposite extreme, complete pooling, ignores segment differences entirely and reports one estimate for everyone. That erases real variation between segments, which is usually the reason a team wanted a segment-level view in the first place. ## What is partial pooling? Hierarchical models, sometimes called multilevel or partial-pooling models, treat each segment's estimate as related to, but not identical to, the estimates of other segments. A segment with a large sample keeps most of its own signal. A segment with a thin sample gets pulled, or "shrunk," toward the broader pooled baseline, because there isn't enough segment-specific evidence to justify treating its raw estimate as reliable on its own. PyMC's own documentation for hierarchical generalized linear models works through this shrinkage-toward-group-mean mechanism directly, using varying intercepts and slopes across groups (PyMC: GLM hierarchical linear regression). A broader explainer on partial pooling in Bayesian multilevel models lays out the same shrinkage behavior with worked examples (R-bloggers: How to fit hierarchical Bayesian models in R with brms, partial pooling explained). [figure: Five-stage decision path: a raw segment estimate goes through a sample-size check into partial pooling shrinkage, producing a shrunk estimate pulled toward the pooled baseline, ending at a rollout decision.] ## What this means for reading a causal experiment's segment output A discrete-choice causal experiment can report an effect at the level of a persona, a region, or a price tier. Before acting on any one of those numbers, ask how many respondents actually sit in that segment, and whether the reported breakdown already comes from a hierarchical or mixed-logit specification, in which case it has already been partially pooled and should not be shrunk again. A segment estimate reported as a raw, unpooled figure from a small slice of the sample deserves less weight than the pooled result, not equal weight. Whether an estimate should be treated as reliable enough to act on is a judgment call that depends on sample size, segment variance, and how much is riding on the decision. Teams that need more certainty on a specific segment before committing budget can move from a simulated study to real-human validation on that segment without changing the underlying causal question. [figure: Five-step path: estimate arrives, sample size checked, weighed against pooled baseline, branch asks if it survives that pull, then act, validate with real humans, or discard as noise.] ## The takeaway for a buyer decision Before shrinking budget toward a niche persona, a single region, or a narrow price tier based on its isolated number, check whether its marginal effect, choice probability, or WTP estimate, not its raw utility coefficient, would survive being pulled toward the pooled estimate for the full population. If it would not survive that pull, treat it as not yet enough evidence to act on, not as proof the effect was noise. Comparable segment breakdowns from published causal studies are available on the leaderboard. --- # How Marketing Managers Can Test Campaign Actions Before Spending the Budget URL: https://subconscious.ai/blog/buyers/how-marketing-managers-test-campaign-actions-before-spending-budget Published: 2026-08-18 A marketing manager rarely gets to choose between "test it properly" and "ship it now." The decision that lands on their desk is narrower: which tagline, hero image, offer, or campaign sequence should go out the door, this week, with the budget already committed. Getting that choice wrong costs wasted spend, a delayed launch, and a rebuild cycle that eats the next sprint too. [figure: A four-step flow: a campaign decision enters a controlled comparison against the target audience; the winning action ships with budget; high-stakes spend gets an added in-market validation step before running at scale.] ## Why the usual research options don't fit this decision Marketing managers absorb every decision that is too small for a formal research project but too consequential to leave to whoever argues loudest in the room: a hero image for Monday's paid social launch, a tagline going onto the landing page, whether to lead with price or features in a demand-gen email, which of three agency creative directions earns the budget. Traditional research tools were not built for that cadence: These cost and timing figures come from published market-research pricing guidance (Drive Research's 2026 market research cost guide), not current Subconscious pricing or delivery commitments. They illustrate why campaign decisions this small and this frequent usually get decided by opinion instead of evidence. ## What does a controlled comparison actually answer? The buyer question is not "what do people think of this idea?" It is "which of these actions is more likely to produce the outcome we want, for the audience we're targeting?" That is a causal question, and it calls for a randomized comparison rather than a stated-preference poll. Subconscious runs that comparison as a controlled experiment: it defines the audience, exposes it to the alternatives under consideration, and estimates which action moves the target behavior: click-through, message recall, stated intent, or another outcome the decision depends on. The result is a directional answer with its uncertainty, not a single number presented as guaranteed truth. A few planning examples of the kind of decision this fits: - A launch-day headline decision. A team compared three headline variants for a paid-social ad against a defined target audience of 25 buyers and had a clear winner, plus the objections the losing headlines raised, before the campaign shipped. - A stalled homepage debate. Two competing hero-copy options had been argued over for a week. A comparison against the target audience split preference 70/30, and the reasons the minority gave for preferring the losing option surfaced a consideration the team had missed. - An agency pitch review. Three creative territories were compared against the audience segments a campaign needed to reach. One worked across all three segments; one failed with the most important one, evidence that changed the agency conversation. These are illustrative, not a guarantee of a specific outcome, turnaround, or price for a given engagement. ## Where does this method stop? A controlled pre-launch comparison does not replace in-market testing for a high-stakes spend, the kind of decision where a €500K TV buy is on the table. The responsible sequence: compare the candidate actions first, then confirm the winner with a real-world test. Subconscious can move from a simulated comparison to validation with real human participants without changing the underlying causal question. It also does not replace foundational discovery work: comparing candidate messages assumes the team already knows what to test, not the interviews or field research that generate those candidates in the first place. And it will not hand a marketing manager a lift number, or say a tagline lifts conversion by a fixed percentage. What it estimates is which candidate action performs better against the audience and outcome the team defined, and by how much, with the uncertainty that estimate carries. ## When audience granularity matters Some of these decisions are local. A campaign built for the UK does not automatically work in Germany or France, and guessing which version to run in each market is its own expensive bet. Because Subconscious runs controlled studies against a person-level audience graph covering 800 million real people, a comparison can be scoped to a specific market's audience instead of one generic group. ## What does this replace in the marketing manager's week? The realistic alternative to a controlled comparison is not "no research." It is a Slack thread, a design review, or a coin flip disguised as consensus. The value of testing every asset is not a perfect prediction; it's a paper trail. When a campaign underperforms, "we compared the messaging, the creative, and the positioning, and this one still missed" is a materially different conversation with leadership than "we made the best guess we could." The practical next step: pick one action already in the queue, a headline, a hero image, a positioning line, and compare it against the alternatives before it ships, rather than after. See how other teams have used a controlled comparison to make a specific campaign or pricing decision, and why the buyer's decision, not a simulated customer conversation, is the right starting point for this kind of test. --- # AI Buyer Persona Tools in 2026: A Comparison Guide URL: https://subconscious.ai/blog/comparisons/ai-buyer-persona-tools-2026-comparison-guide Published: 2026-08-18 "AI buyer persona tool" covers three unrelated products: a document generator, a customer-data clustering platform, and an interactive persona a team can question. Buying the wrong one is expensive in a quiet way. A team pays for a one-page export nobody reopens, or a clustering platform with too little customer history to find real segments, while the question that actually mattered, which message, price, or feature moves this buyer, never gets tested. ## Same label, three different products A document generator. Tools such as HubSpot's Make My Persona take a short form (industry, role, company size, stated pain points) and return a formatted persona page in under a minute (HubSpot, Make My Persona). The result reads well and works fine as a placeholder in a kickoff deck. It is not built to be reopened once the team needs to test a specific headline or price. A clustering platform built on your own data. A second group ingests CRM records, survey responses, and site analytics, then groups customers into named segments using clustering and language-model summarization. This only works once a company has enough history to cluster: a business with a few hundred paying customers and several thousand trial signups has enough signal; a pre-revenue team, or a company entering a market it has never sold into, does not. An interactive persona profile. A third group builds a standing profile a person can question directly instead of filing a research request. Synthetic Users is one example, aimed mainly at product and UX interview work rather than segmentation or head-to-head action testing. ## Where each one stops being useful The document generator's weakness is durability, not quality. It gets saved once, cited in one meeting, and then nobody returns to it when a new campaign or feature needs a fresh read on the same buyer. The clustering platform's weakness is data. It describes the customers a company already has, not the ones it is trying to reach. Ask it about a segment with no purchase history behind it, and there is nothing to cluster. The interactive-persona group's weakness is scope. A profile a team can talk to is useful for exploring reactions and gathering directional color. That is a different job from running a controlled comparison between two prices, two messages, or two feature cuts and reporting which one actually changed the outcome, and by how much. [figure: Four items show where each tool stops: document generator once revisited, clustering with no purchase history, interactive persona needing a measured comparison, causal experiment testing which action moves the buyer.] ## How the three compare ## What question can none of the three tools answer? Which specific choice, a message, a price point, a feature cut, changes what a defined buyer segment actually does is a different job: a controlled experiment that compares the actions under consideration and reports the causal effect of each one. Subconscious runs that kind of study. It tests defined actions against a defined buyer segment inside a person-level audience graph covering 800 million real people, kept distinct from any recruitable interview panel, and reports the effect of each action with its uncertainty. When a decision's stakes justify it, the same causal question can move to real-human validation without changing what is being measured or asked. ## What does Subconscious not do? Subconscious does not export a persona document, generate a PDF for a deck, or run CRM-based clustering to produce customer segments. It is not a substitute for direct interviews with real customers, or for a leader's own judgment on a call only a human can make. It answers a narrower question than the three categories above: which of the specific actions on the table moves the buyer, not who the buyer generally is. ## Matching the method to the decision Start from what happens after the output ships, not from which category sounds most advanced. A deck that gets shown once needs a document generator. A mature customer base worth segmenting needs a clustering platform. A concept that benefits from open-ended reaction needs an interactive persona. A decision that still has two or more live options on the table, with real budget riding on which one wins, needs a controlled experiment that names the action and measures it. Teams that want to see how that comparison is built and reported can review the leaderboard of published studies before scoping a first test, or go straight to setting one up. --- # Pre-Testing a Thought Leadership Thesis Before It Goes Out Under the Firm's Name URL: https://subconscious.ai/blog/use-cases/pre-testing-thought-leadership-thesis-before-goes-out-under-firms-name Published: 2026-08-18 Every agency and consulting firm hits the same wall before a thought leadership post goes out: is the thesis strong enough to publish, or does it need to die before a partner's name goes on it. What matters is not polished prose, but whether the argument is worth the firm's distribution and the partner's credibility. The draft usually gets written late, under deadline, judged only by people who already agree with it. It goes live on LinkedIn, reaches four hundred people, convinces no one of anything new, and the cycle repeats next month. Fixing the writing doesn't fix that; the thesis has to be tested before the keyboard is touched, not after. Research from LinkedIn and Edelman on B2B thought leadership documents how much buying influence a strong, specific argument carries with the readers a firm is actually trying to reach. ## Why the post usually fails before the writing starts Three reasons repeat, and none get fixed by a better edit. The thesis is too vague to argue with. "AI will change marketing" is a weather report, not a claim. Broad claims read safe and get skimmed; specific claims get shared and argued with, the only distribution mechanism that compounds over time. The thesis is calibrated to the wrong reader. Partners write for other partners, so the draft collects internal nods before it flops with the actual audience: clients and prospects who read in a different register and stop at the second paragraph. The thesis is not new. Most firm content restates a position the industry settled three to five years ago, dressed up as a fresh "take." A reader who has seen the argument before scrolls past it faster than an internal reviewer catches the repetition. ## How do you compare reactions before the post ships? Pressure-testing a thesis means running a controlled comparison of reactions across the reader types who decide whether a post travels or dies. These are not buyer profiles; they are the people who will read, share, or argue with the post once it is live. Comparing reactions across these reader types, rather than relying on one internal reviewer's opinion, is the point of a controlled experiment: it produces a causal read on which version of the claim moves agreement, sharing intent, or credibility. ## How do you test the thesis at each stage? The comparison does not have to slow down publishing. It fits into five checkpoints that already exist in most drafting processes. Before drafting: the single-sentence thesis check. State the thesis in one sentence: "X is getting too much of the firm's investment, Y too little, and that's about to flip." Compare reactions across reader types: who signs on, who pushes back, who wants more before deciding, and why. A thesis that fails here goes back to the thesis, not the draft, before hours go into writing around a claim that was never going to travel. First draft: the framing check. Compare reactions to the opening paragraphs and ask what the post is about and who it's for. This catches a post meant to be about strategy but reading as tactics, before a week of revisions goes into the wrong frame. Second draft: the example check. Compare reactions to the examples: do they read as current, credible, and relevant to the thesis, or as stale, narrow, or lifted from the firm's own client list. Examples are usually where a post loses credibility first. Pre-publish: the shareability check. Ask whether each reader would share the post, and why. The exact reader the firm is trying to reach and the early-career reader matter most here, since they drive the distribution that makes the post work. This step often rewrites the opening line, the line a reader sees before deciding whether to click through. Post-publish: the rebuttal check. Once the post is live, compare reactions in the voice of a public critic: the strongest counter-argument, and what a skeptic would write in response. This lets a firm plan the follow-up post instead of losing the thread when the pushback starts. ## What this comparison tends to surface Patterns that repeat across firms running this comparison: - The thesis is usually over-hedged. Firm training pushes writers to caveat every point until the position is sanded down to something safe and forgettable. - The opening line is wasted on setup instead of the claim. On LinkedIn, the first line decides whether the second line gets read at all. - The practitioner's voice is missing. Partners write from the partner's chair, and the comparison catches when a draft reads as advised-from-a-distance rather than done-the-work. - The post proves the claim but skips the implication. Three case studies and no answer to "what does this mean for my company" reads as correct but useless. - A sales call to action, where one exists, is usually the weakest sentence and the first to get cut. Thought leadership needs a question or a prompt for a counter-view, not a pitch. ## The second use: deciding whether a practice area is worth building Every new practice area starts as a thesis: the firm believes something is changing and wants to build a service around it. That belief is usually validated through partner debate, client conversations, and conference-circuit reactions, three channels that all tilt toward telling the firm what it wants to hear. Partners have already talked each other into it. Clients say what they think the consultant wants said. Conference audiences reward confidence over accuracy. A controlled comparison of reader reactions is a fourth, independent channel: it estimates which framing of the belief moves agreement or skepticism before the firm hires against it or names it in a capabilities deck. When the practice-area thesis is worth building the firm's public reputation on, the same causal question (which claim actually changes a reader's mind) carries over from a simulated comparison to real-human validation without changing what's being measured. That step matters here because a practice area, unlike a single post, commits hiring, positioning, and client-facing claims to the answer. A controlled comparison of reactions does not replace an editor's judgment or a subject-matter expert's critique of whether the argument is sound. It estimates which claim resonates with which reader, not whether the claim is factually correct. ## Where to start Most firms already have a backlog of thesis ideas that never get written because no one is confident they're worth the time. Running a single-sentence check across the backlog ranks those ideas by which are most contestable, most relevant to the target reader, and most likely to travel. Testing the thesis, not just the prose, is the difference between a post that gets argued with and one that gets scrolled past. See how this fits into a broader decision-testing workflow, or start a comparison on the next thesis in the backlog. More on the firm behind this approach is here. [figure: Four sequential gates: thesis check before drafting, framing check on the first draft, example check on the second draft, shareability check pre-publish. Each compares reader reactions before the next stage.] --- # Social Listening vs. Causal Testing: When Is Signal Enough to Act On? URL: https://subconscious.ai/blog/comparisons/social-listening-vs-causal-testing-when-signal-enough-act Published: 2026-08-18 A team running social listening has a real signal: what people are saying about a category, a competitor, or an emerging trend. The open question is whether that signal is enough to greenlight a specific action, a new message, a repositioned product, a price change, or whether the action needs a controlled test first. Confusing the two research categories is the actual risk: observed sentiment tells you what already happened in the market, not what will happen if you ship a message nobody has reacted to yet. ## What social listening actually measures Social listening and market-intelligence platforms scan social media, news, forums, and review sites to surface sentiment, share of voice, and emerging trends. Quid is one example: it analyzes public conversation to show what is being said about a brand, competitor, or topic, and how that conversation is shifting. This is observational research. It reports what real people have already said, without asking anyone a new question. It fits brand tracking, crisis monitoring, and spotting a trend before a competitor names it. ## Where does the signal run out? Listening data cannot tell you how an audience will react to something that does not exist yet: a monitoring platform has no way to test a headline you have not published, a price you have not charged, or a product concept you have not launched. Treating a favorable sentiment trend as proof that an untested message will land is a correlation read as causation: the trend can be real, and the inference about your specific untested idea can still be wrong. A launch built on the assumption that current sentiment predicts the reaction to a new, specific action can fail with no way to trace which variable caused the miss. The reverse mistake carries its own cost: running a full controlled experiment for a question ongoing monitoring already answers wastes cycles that belong on the next test. ## What does a controlled causal experiment add? Subconscious is a causal behavioral platform. It runs controlled, randomized experiments against a specific action, such as a message, a price, or a concept, and reports the causal effect of that action with a confidence interval. The question changes from "what are people saying" to "which specific version of this message, price, or concept changes the outcome, and by how much." Those experiments run against a person-level audience graph covering 800 million real people, and a study built this way can move into real-human validation without changing the causal question: the team tests the same action, against the same design, with recruited participants instead of the simulated panel. ## Comparing the two approaches ## Use both, in sequence The two categories are not competing for the same budget line. Listening data is the input that tells a team which questions are worth testing. A controlled experiment is the step that turns one of those questions into a decision before the team commits budget. For a team already running social listening, the practical move is not to replace it, but to add a testing step for any action specific enough that no one has reacted to it yet. See how Subconscious runs a study, or review published results and case evidence. ## Limitations Subconscious does not do social listening, sentiment monitoring, or real-time trend detection across social and news data. That is a distinct, observational category Subconscious does not claim to replace. The audience graph is also not a recruitable panel of respondents; real-human validation draws from a separate, dedicated participant pool. [figure: Two columns: social listening reports observed sentiment and trends from existing conversation; a controlled experiment tests an untested action and reports its causal effect with a confidence interval.] --- # Subconscious vs. Brandwatch: Which Job Does Your Team Actually Need? URL: https://subconscious.ai/blog/comparisons/subconscious-vs-brandwatch-which-job-team-actually-need Published: 2026-08-18 A VP of Consumer Insights weighing a pricing change, a repositioning, or a launch message this quarter needs to answer one question before any budget moves: will this specific action change behavior? Brandwatch and Subconscious both get filed under "consumer intelligence," but they answer different questions, and confusing them means shipping a decision built on the wrong kind of evidence. [figure: Two-column table. Monitoring: existing conversation, sentiment/volume, ongoing tracking. Testing: controlled experiments, causal effects with confidence intervals, pre-launch decisions.] ## What is Brandwatch built to answer? Brandwatch is a consumer intelligence platform that aggregates social and web conversation, then surfaces it through dashboards and reports for research and insights teams. It answers "what are people already saying about my brand or category?" That is a monitoring job: mentions, sentiment, and volume for conversation that already exists. Brandwatch fits organizations with a dedicated insights function, an established research cadence, and a need for that conversation data flowing into an existing analytics stack. ## What is Subconscious built to answer? Subconscious answers a different question: if a specific action changes, such as a price, a message, or a launch claim, what will people do? It runs controlled experiments on a simulated population and returns causal effects with confidence intervals, backed by a person-level audience graph covering 800 million real people. When a team wants to move from a simulated result to real-human confirmation, the same causal question can be tested or validated with real participants without changing what is being measured. That is a testing job: it pressure-tests a decision before it ships, not after. ## Where the two jobs diverge Publishing where a tool stops is what turns a claim into something a buyer can check. A brand tracking dashboard cannot tell a team whether a new price point will change purchase behavior, and a causal experiment cannot tell a team what people are saying about a brand right now. ## The cost of using the wrong one for this decision Naming this failure mode on the record is what lets a buyer catch it before the budget moves. Treating conversation volume or sentiment trend as evidence that a proposed action will work skips the step where that action is actually tested. Sentiment describes the past; it does not predict a decision that has not happened yet. ## What does Subconscious not do? A capability list that states what's missing is one a buyer can verify against the product. Subconscious is not a social listening or brand monitoring tool. It does not track ongoing conversation volume, sentiment trends, or real-time mentions across social platforms and the open web. A team that needs continuous monitoring of what customers are already saying still needs a listening platform for that job. ## Deciding where the budget goes this quarter If the open question is "what is the market already saying," a monitoring platform is the right tool. If it's "will this specific price, message, or launch decision work," that calls for a controlled test before it ships. Teams can see how that test is structured on how we work, review completed examples in case studies, or look at the underlying method on research. For a specific upcoming decision, a demo is the fastest way to see the test run against it. --- # AI Content Strategy Research: Test the Decision Before Writing URL: https://subconscious.ai/blog/use-cases/ai-content-strategy-research-test-decision-before-writing Published: 2026-08-18 Content teams publish, measure, and learn weeks or months later, after thousands are spent. A number without its limits is marketing. A team may publish 10-20 pieces per month while only 2-3 generate meaningful engagement, traffic, or pipeline. AI-assisted audience research can test the topic, angle, format, and headline before production. [figure: A left-to-right path of five boxes: Topic, Angle, Format, Headline, Publish. Each of the first four boxes is tested against an audience before the piece moves to the next stage.] ## The questions keyword data cannot answer Naming where keyword data stops is what lets a content team check the gap for themselves. Keyword research shows what people search for, not what would make a specific buyer stop and read. A topic such as improving a sales process can support 20 different angles. A content team must decide which problem, evidence, and format fit the audience. Format is also a decision. An executive may want a short data brief. A director may need a comparative guide. An individual contributor may need a tutorial. A technical buyer may need documentation and benchmarks. ## Test the content choices ### Topic Ask the audience definition about current job friction, missing industry coverage, and the last material that changed a work decision. Turn the answers into topics checked against search and customer evidence. ### Angle Compare distinct arguments for the same topic. One planning example contrasts a general market-research critique with a cost argument: "$50,000 on research that takes 12 weeks." Preserve the amount and duration as an example, not a current Subconscious claim. The useful output: why one angle is clearer or more relevant, then a real test. ### Which content formats should you compare? Compare a guide, brief, analysis, tutorial, or opinion for the same audience and subject. ### Headline Draft five headlines, then put each in front of an audience panel sized 4-6. Check whether each headline sets accurate expectations, sounds distinct, and matches the intended channel. ## A mechanical workflow ### What happens during weekly content planning? Spend 30 minutes testing proposed topics with the top 3 audience segments before the editorial meeting. Ask whether each topic addresses a current decision and what would make it useful. ### Before production Spend 15 minutes per piece comparing 2-3 angles. Put the selected angle and the audience reasoning in the brief. ### Before publication Spend 10 minutes comparing the working headline with 3-4 alternatives. Do not choose a headline that wins attention by misrepresenting the article. ### After publication Compare the simulation with real engagement, qualified traffic, and pipeline. Record where the audience model was wrong, and use that to improve the next test. [figure: A four-step loop diagram: Weekly planning feeds Before production, which feeds Before publication, which feeds After publication, which loops back to the next Weekly planning session.] ## What does simulation not replace? Stating what a method cannot do is what makes the result next to it worth trusting. It does not replace performance data, search fundamentals, distribution, or skilled writing. It cannot invent original reporting, a real case study, or proprietary data. AI research can help package an existing proof point. The promise is to shorten part of the learning cycle from weeks to minutes. The value comes from a tighter loop: define the audience, compare content actions, publish the selected work, and validate against behavior. Subconscious is relevant when topic, message, or format can be framed as a decision-specific experiment; see the causal fidelity paper for how that kind of causal read is measured. Keep SEO evidence, human editorial judgment, and observed performance in the workflow. Read more about the research behind the method, and see how the process runs end to end. --- # Feature Naming Is a Testable Decision, Not a Team Vote URL: https://subconscious.ai/blog/use-cases/feature-naming-testable-decision-not-team-vote Published: 2026-08-18 A product or product-marketing leader choosing which candidate feature name to ship should run a controlled comparison against a defined buyer population before the name locks into the UI, docs, and sales decks. The name that wins should be the one that changes correct identification of the feature, not the one the team likes best in a planning meeting. ## Why the decision matters A feature name lives in the UI, the changelog, the sales deck, the help center, and the customer's own conversations about the product. Once it ships, it is expensive to change: docs reference it, onboarding explains it, sales has pitched it, and customers have built a mental model around it. A name that new users misread on first contact produces months of support tickets, onboarding friction, and confused sales conversations. Teams skip proper naming research for a structural reason: recruiting real customers for comprehension interviews takes time and money, and feedback usually arrives after engineering has already committed the name to the codebase. So the decision defaults to whichever name a handful of people in a meeting happen to like. ## What causes teams to pick a bad feature name? Internal teams suffer from the curse of knowledge. A name that feels obviously clear to the people who wrote the feature spec is not the same name a customer encounters cold, with no tooltip or onboarding, just a word in a navigation bar. Structured concept testing exists to separate what a team already understands from what a first-time viewer infers from the name alone (UX Army, Concept Testing In Market Research & UX). Naming quality is a four-axis decision, and most internal debates collapse all four into "which name sounds best": A name can win on memorability and lose badly on comprehension, a tradeoff a single popularity vote hides. ## Evidence A sound naming process separates candidate generation from evaluation: brainstorm broadly first, then test the surviving candidates against a defined audience rather than judging them as they're written (SmashBrand, How To Create An Effective Product Naming Framework). A structured comprehension test on candidate names for a given feature typically shows: a metaphorical name driven mostly by internal appeal scores well below a plainer, descriptive alternative on correct-guess rate and confidence, because new users read the metaphor as unrelated to the feature's actual function. This is a historical planning example of the pattern naming research catches, not a current Subconscious benchmark figure. Comprehension variance, not average preference, best predicts which name will need a tooltip or onboarding tour. A name that different testers guess differently ("could be reporting, or maybe an alert system") should be eliminated even when its average preference score looks fine. A name that is clear and inoffensive in one language can fail category-fit, or, less obviously, translate cleanly but lose its category signal in another market. ## Options and comparison ## What is the recommended process for naming a feature? Subconscious runs a controlled experiment comparing defined candidate names across a defined buyer or user population and measures which name changes correct identification of the feature. The output is a causal effect with a confidence interval, not an aggregated self-reported score. That distinction matters: the question is not "which name do people say they prefer" but "which name, when substituted for another, changes whether people correctly identify what the feature does." A workable process: - Generate a working set of candidate names spanning descriptive, metaphorical, proper-noun, and action-led patterns. Otherwise the comparison ends up testing several variations on the same idea. - Define the population the comparison should represent: new prospects, recent signups, or existing customers, depending on who encounters the name first. - Run the controlled comparison and pick a name with both a high correct-identification rate and tight agreement across the population, not just a strong average score. - Carry the top candidates into direct conversation with real customers before finalizing. See /research and /how-we-work for how Subconscious structures and validates causal experiments. When the name needs to work in more than one market, run the same comparison against a population defined for each target market. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, supporting those market populations without committing to one name and finding out later. Naming this limit is what lets a buyer check the claim before relying on it. That audience graph describes reach for defining a population; it is not a pool of participants recruited for open-ended interviews. ## What are the limitations of this method? A number without its limits is marketing copy, so here is where this method stops. A controlled naming comparison does not replace watching real customers encounter the name in a live product. It does not constitute real-human validation, and it does not resolve emotional resonance or brand-fit judgment calls that benefit from a direct conversation with a customer. Subconscious can test or validate studies with real human participants, letting a team move from a simulated comprehension experiment to real-human validation without changing the causal question. ## Adjacent questions Why not just pick the name the team likes? Because the team already understands the feature. That's not evidence of how a new customer reads the name cold. Does this replace talking to customers? The miss sits on the record right next to the hit. No. Use a controlled comparison to narrow candidates to a short list based on comprehension and category-fit, then validate the finalists with real customers before locking the name in. What if the name needs to work in more than one market? Run the same comparison against a population defined for each target market. A name that is fine in one language can carry the wrong category signal, or an embarrassing meaning, in another. [figure: Five-step path: generate candidate names, test comprehension and category-fit against a buyer population, check comprehension variance, select the name that improves identification, then ship it.] Talk to us at /demo, or read how other teams have used this approach in /case-studies. --- # Persona Chat, Product Simulation, or Causal Experiment: Choosing How to Validate a Roadmap Call URL: https://subconscious.ai/blog/comparisons/persona-chat-product-simulation-causal-experiment-choosing-how-validate Published: 2026-08-18 A product leader deciding whether to build a feature, change pricing, or reorder the roadmap can reach for three different kinds of evidence: an open-ended conversation with an AI persona, a simulated model of likely outcomes, or a controlled experiment that measures the causal effect of the specific change. Picking the wrong one costs a sprint spent building against a signal that doesn't hold once it reaches real customers. ## The question each method actually answers Open-ended persona conversation answers "how does our customer reason about this, and what am I not anticipating?" It is exploratory by design: a team talks to a simulated customer type and follows the conversation wherever it goes, often surfacing objections nobody wrote into the brief. Product-outcome simulation answers "if we build X, what happens?" A tool such as Seldon (formerly Blok) is built around that question directly, framing its own product as a way to simulate how users will behave before a team launches. A team defines the decision space: the experiments, the options, the variables. The platform then models which direction has the most upside, aimed at structured roadmap planning rather than open-ended discovery. A controlled causal experiment answers a narrower, more specific question: does this specific change cause this specific outcome, and how confident is the team in the size of the effect? That is the question Subconscious runs controlled discrete choice experiments against, and it produces a measured effect with a confidence interval rather than a modeled scenario or a qualitative read. ## Where a simulator and a causal experiment diverge Both a product-outcome simulator and a causal experiment produce a quantitative answer, which makes them easy to conflate. The difference is in what backs the number. A simulation models likely outcomes across a decision space the team has already structured. A controlled experiment instead randomizes the specific choice under test against a defined audience and measures the difference in outcome that choice actually causes. Neither approach replaces exploratory persona conversation. Discovering an objection nobody anticipated is a different job from measuring the size of an effect once the team already knows what it's testing. ## Comparing the three approaches ## What does Subconscious add to this stack? Subconscious's fit is the experiment column above, not a replacement for the other two. A study runs against a person-level audience graph covering 800 million real people, kept distinct from any recruited panel. When a team needs to move past a simulated result, a study can go from a simulated experiment to real-human validation: the same experimental design, tested against recruited participants instead of the simulation. This is a narrower claim than either alternative makes for itself: a measured causal effect, with a stated interval, for the specific decision under test. ## What are the limitations of a causal experiment? A causal experiment only answers the question it was designed to test. If the roadmap decision itself is still unclear, and the team doesn't yet know which options belong in the decision space, an experiment run too early is answering a question nobody asked yet. Naming what a step does and doesn't do is what lets a buyer check the claim against the method. Real-human validation, when used, replicates the causal design already run in simulation. It does not turn the study into an observed usability session or a guarantee of market performance; it is one more data point on the same causal question, tested against a different population. ## Next step Teams evaluating how to validate a specific decision can see the mechanics of a controlled study in how Subconscious runs an experiment or review completed studies to see what a measured effect and its confidence interval look like in practice. [figure: Three boxes, one per method: persona conversation asks why customers reason this way, simulation asks what happens if we build X, experiment asks whether this change causes this outcome.] --- # How to Validate a Product Idea Before You Commit Engineering Budget URL: https://subconscious.ai/blog/implementation-and-procurement/how-validate-product-idea-before-commit-engineering-budget Published: 2026-08-18 Most product ideas fail slowly and expensively: a team builds for months, ships, and finds that customers don't want it, don't understand it, or won't switch from what they already use. Nearly half of startups that fail point to no market need as the cause (User Intuition). The decision in front of a product or growth leader isn't whether to validate, but which method to trust before greenlighting build work: an open-ended AI chat session, a structured comparison of concepts, or interviews with real customers. ## Why is "will people buy this" really four separate questions? "Will people buy this?" is really four separate questions, and jumping straight to the last one is the most common way validation goes wrong (MIT Professional Education). "Nearly 30,000 new products are introduced each year, and 95% of them fail according to Clayton Christensen, a professor at Harvard Business School." Clayton Christensen, Harvard Business School, cited by MIT Professional Education (source) Each layer needs a different kind of test. Treating them as one question, usually the demand question asked too early, produces a confident answer to the wrong thing. ## Define the segment before testing anything Define the buyer segment whose reaction matters. A useful segment design mixes distinct viewpoints rather than one uniform "target customer": - 3-4 target-buyer variants. Different company sizes, roles, and sophistication levels inside the ideal customer profile, since not everyone in a target market reasons the same way. - 1 adjacent-market buyer. Someone outside the primary target who could plausibly use the product; adjacent buyers often surface positioning angles a narrow segment misses. - 1 active skeptic. Someone aware of the problem who decided not to solve it, or tried and gave up. This is where the real objections show up, not the polite ones. - 1 competitor's customer. Someone already solving the problem with an alternative, who can point to what that alternative does well and where it falls short. These are example segment sizes from a common validation framework, not a fixed requirement. The right composition depends on how the market is already segmented. ## Why use a controlled experiment instead of an open-ended AI chat to compare concepts? An unstructured chat with an AI persona produces a plausible-sounding transcript, but it doesn't isolate what actually moved a buyer's answer. A controlled experiment does: hold the segment constant, vary one thing at a time (the concept, the positioning statement, the price frame), and measure which variant moves stated intent for that segment. This is closer to how Subconscious approaches the same four-layer question: a randomized, controlled experiment run against a simulated buyer population, comparing concept or positioning variants and estimating which one moves stated intent for a defined segment, rather than a single freeform conversation. Applied to each layer: - Problem layer: compare how strongly different segments recognize and rate the pain, rather than asking one open-ended conversation to surface it. - Solution layer: run two or more solution framings against the same segment and compare reactions directly, instead of iterating a single conversation and trusting a read of the tone shift. - Positioning layer: test several positioning variants against the same segment and check whether each one is correctly attributed to the right buyer and the right problem. - Demand layer: compare price and framing variants for directional differences in stated intent, without treating any single number as a forecast. ## What is a simulated comparison good for, and where does it stop working? A simulated comparison is strongest at killing bad ideas early. A comparison that turns up universal indifference or positioning that gets misread is more valuable, sooner, than one more polished pitch deck. It is weaker at predicting exact conversion rates or willingness to pay, and it should not stand in for real customer input in a novel category with no grounding data to compare against. No willingness-to-pay figure, confidence interval, or buy/no-buy recommendation from a simulated comparison should be read as a guarantee. The sequence matters more than the tool: use the simulated comparison to get to a strong hypothesis, then check that hypothesis against real buyers before committing meaningful spend. A team can move from a simulated experiment to real-human validation without changing the underlying causal question: same segment definition, same comparison, real participants in place of the simulation. ## Before greenlighting build work Run all four layers as a controlled comparison against a defined segment, then validate the resulting hypothesis with real buyers before the engineering budget is committed. How Subconscious works covers what that experiment design looks like end to end, and research has more on the causal approach behind it. [figure: Four-step path: Problem (pain severe enough to act on), Solution (beats the status quo), Positioning (right buyer understands and wants it), Demand (would they pay). Each has a labeled cost if skipped.] --- # Can an LLM Stand In for a Human Survey Respondent? What One Benchmark Found URL: https://subconscious.ai/blog/methods-and-validation/llm-stand-human-survey-respondent-what-one-benchmark-found Published: 2026-08-18 A large language model prompted to answer as a person with a given age, income, and education can predict that person's political party about as well as a supervised model trained for the task, though individual-level party identification is only modestly predictable from demographics to begin with. Ask the same model to predict how many hours that person watches television, and its accuracy is inconsistent. That gap is what a research team should check before trusting an LLM persona on a real go-to-market decision. ## What was actually tested An independent benchmark study gave several large language models a real respondent's traits (age, sex, race, education, income, occupation, religion) and prompted each to answer in that person's place (arXiv, "Large Language Models as Virtual Survey Respondents"). The source data came from the General Social Survey (GSS), a public dataset of adult US residents' attitudes and behaviors, and the models were scored on two questions: - Party identification, self-reported on a seven-point spectrum running from Strong Democrat at one end to Strong Republican at the other. - Hours of daily TV viewing, sorted into five buckets ranging from one hour or fewer up to six or more. Each model's error was scored as the mean absolute distance between its answer and the real respondent's, averaged over a random sample of 100 held-out respondents, and compared against the baselines below. That comparison is intentionally asymmetric: the random forest is supervised on in-domain data, while the language models rely only on pretraining, so it contextualizes the LLM output rather than claiming the two methods are equivalent. GSS microdata and its published cross-tabs are public, so the language models' pretraining may itself include the demographic-attitude relationships being tested, a possible source of overlap the study does not rule out. ## Where did the model do well? On political party identification, most large models tested, including GPT-4o, Claude 3.7 Sonnet, and Gemini 2.0 Flash, landed within the range between the naive baseline and the random forest, several approaching its accuracy, though the study reports no standard errors, so these differences cannot be called equivalence. Claude 3 Opus came close to the random forest despite no task-specific training. A control run removed all demographic information from the prompt and asked the same question generically. Performance collapsed below the naive baseline across every model tested. That comparison is suggestive but not conclusive: under mean absolute error a naive median predictor is hard to beat, so falling below it in the no-demographics arm is close to expected regardless of whether the model uses demographic signal, and the ablation changed the prompt's content and framing along with the demographic fields. ## Where did the model break down? Daily TV-viewing hours is a weaker-signal task: it correlates less tightly with standard demographic variables, and the random forest itself had less training data available. Model performance here was substantially more variable: runs typically fell between the naive baseline and the random forest, but plenty landed outside that band on either side, and no model consistently beat the rest in the reported results, a contrast with the more stable rankings for party identification. That instability is the practical risk: a model can perform like a trained classifier on one question and a coin flip on the next, and the study did not measure model confidence or calibration, so there is no signal available to distinguish which case is which. "Bisbee et al. ( Bisbee et al., 2024 ) found that even when distributions appear statistically similar, they can lead to substantively different inferential conclusions." Bisbee and colleagues (2024), cited in Zhao et al., "Large Language Models as Virtual Survey Respondents," arXiv (source) ## Why does this matter before a GTM decision ships? This failure mode is exactly what a human-baseline check is designed to catch before a team acts on it. This benchmark scores individual-level point prediction of a marginal attitude, not recovery of the conditional response distribution or the treatment effects a choice experiment needs; an insights or research lead checking whether an LLM persona's answers are usable for a specific question is at most checking how demographically predictable that question is, and an ungrounded model roleplay does not surface that on its own. Subconscious can test or validate studies with real human participants, so a team can move from a simulated experiment to a real-human baseline check without changing the underlying causal question. A result that looks strong in simulation can then be confirmed, or caught, before it becomes a pricing, messaging, or positioning decision. ## What this study does not establish None of the specific accuracy figures, model rankings, or benchmark results in this study are Subconscious findings, and Subconscious did not run or endorse this experiment. A separate industry review of synthetic survey samples has also documented significant limitations in AI-generated responses relative to real respondent panels (Verian Group, "Synthetic Sample in Social Research"). The lesson to take from both is procedural, not a number to reuse: test the demographic predictability of the question before trusting a model's answer to it, and check a weak-signal result against a real human baseline before it drives a decision. ## Next step Compare methods for testing a causal marketing question, including where a real-human baseline changes the answer, at Subconscious's research and leaderboard pages, or see how the validation workflow runs end to end. [figure: Comparison: party ID, where LLM personas cluster near the trained random forest; TV-viewing hours, where results scatter with no consistent winner. Stripping demographics drops every model below the naive baseline.] --- # Set the Evidence Tier Before a Simulated Finding Reaches a Launch Decision URL: https://subconscious.ai/blog/use-cases/set-evidence-tier-before-simulated-finding-reaches-launch-decision Published: 2026-08-18 A research leader should set the evidence threshold before a simulated finding can support a public claim, price change, or budget-committing launch. The threshold should rise with the cost of being wrong, or a plausible market read can reach a decision before the underlying behavior has been tested. ## What Is a Simulated Finding Allowed to Change? The first question is not which method produced the finding, but what action it is allowed to support. Internal exploration, shortlist selection, a public claim, and committed launch spend carry different consequences and should not share one proof standard. The U.S. Bureau of Labor Statistics expects the market research analyst and marketing specialist occupation to keep expanding over the 2024-2034 window. (U.S. Bureau of Labor Statistics, Occupational Outlook Handbook) That projection is evidence about the research function, not proof of any Subconscious result. ## Four evidence tiers for one business decision An evidence ladder makes the permitted decision explicit at each tier: Fluency is not evidence. A clear explanation can help define what to test, but it cannot establish that one action caused a different behavioral response. [figure: Four-step path: exploration frames hypotheses internally; directional test narrows a shortlist, not an external claim; decision review sets required proof; validation supports public or budget-committing launches.] ## Keep the causal question stable as proof increases At the directional tier, Subconscious can run a controlled, randomized experiment on the specific action under consideration against a real or simulated audience. The result is a causal effect with a confidence interval for that action, not a general market summary. The person-level audience graph covers 800 million real people. That reach is not a recruitable pool of respondents, and size alone does not make a simulated experiment human research. Real people participate only when a study explicitly uses real-human validation. For an expensive or public decision, the team can carry the same causal question into real-human validation. Keeping the action, comparison, and outcome stable makes the two tiers easier to interpret, but does not turn either result into automatic proof of performance after launch. ## Who Sets the Evidence Threshold for a Launch? Subconscious does not choose the evidence threshold for a launch, replace executive judgment and accountability, or substitute for the decision owner's knowledge of the commercial stakes. Nor can it make one experiment answer a different question from the one the team tested. The buyer owns three checks: - Name the action under consideration. - State the cost of choosing poorly. - Set the minimum evidence tier before any result is reviewed. - Use lower tiers only to narrow the decision. - Require real-human validation when the decision is public or commits material budget. ## Put the rule into the launch process Choose one active launch, pricing, or messaging decision and write its evidence threshold into the approval criteria. Then review how the study moves from a directional test to fielded evidence and examine documented study designs. When the causal question and decision boundary are clear, scope the study around that action. --- # Two Synthetic-Audience Models, and the Question Neither Answers URL: https://subconscious.ai/blog/comparisons/two-synthetic-audience-models-question-neither-answers Published: 2026-08-18 A consumer insights or growth marketing lead evaluating synthetic-audience tools usually runs into two families of product, built on different foundations, before ever reaching the harder question: what happens after the screen comes back positive. ## How do vendors build a synthetic audience? The first is survey-grounded: a synthetic segment is built directly from a large, recurring program of survey interviews, so each answer traces back to a real respondent's self-reported data at a known point in time. The second is source-modelled: a reusable AI persona is assembled from scoped external signals (public reports, review and search data, permitted partner research) organized into knowledge bases and reused across interviews rather than rebuilt from a fresh survey wave each time. Both models answer the same underlying question well: which ideas are worth pursuing further. Neither is built to answer what comes next: how much a specific change in price, message, or feature moves a specific decision, with a number a team can defend to a stakeholder who wasn't in the room. ## What can't a directional screen carry on its own? A synthetic-audience read is a screen. It narrows a wide set of options to a short one quickly and cheaply. The risk shows up when a team treats that narrowing as proof: committing media spend, a price change, or a public positioning claim on the strength of a directional read, then having the real launch contradict it. The cost isn't just the wasted spend. It's the credibility hit with stakeholders who were told a fast synthetic read and a validated decision were the same thing. The fix isn't picking a better screening tool. It's routing the surviving option through a step neither screening model performs: a controlled experiment that measures the effect of a specific change, on a specific population, with a confidence interval attached to the answer. ## Where does a causal experiment pick up? Subconscious.ai runs controlled discrete-choice experiments across a precisely defined population and returns a measured causal effect with a confidence interval, before a decision ships. It sits downstream of exploratory synthetic-audience and persona tools rather than competing with them on the same axis: it doesn't generate ongoing directional reads, and it isn't a source of segment benchmarking. It validates one decision at a time: does this price, message, or feature change move the outcome, and by how much. Subconscious documents how it structures and validates these experiments at research, including the fidelity work behind the causal method. ## Three things worth keeping separate Buyers evaluating this space tend to blur three distinct capabilities into one claim: - Audience reach: how broad a population a tool can represent or simulate. - Simulated experiments: a controlled discrete-choice test run on that population, which is what produces a measured effect. - Recruited real-human validation: testing the same causal question with real respondents once the stakes justify it. A wide audience reach doesn't make a simulated read a validated one, and a simulated experiment isn't automatically upgraded to a clinical trial or an observed usability session just because real humans eventually weigh in. When a decision's stakes call for it, a team can move from a simulated experiment to recruited human validation on the same causal question without changing what's being measured. See how that validation path has supported real decisions in case studies. ## The practical next step If a synthetic-audience screen has already narrowed a decision to one or two live options, the next move isn't running another directional read. It's defining the specific alternatives being compared and the population that matters, then measuring the effect with a controlled experiment before committing budget or making the claim public. [figure: Two boxes, survey-grounded segments and source-modelled personas, converge into a shared directional-screen box, feeding a controlled-experiment box that outputs a causal effect with a confidence interval.] --- # How to Scope a Simulated-Market Study Before You Procure One URL: https://subconscious.ai/blog/implementation-and-procurement/how-scope-simulated-market-study-before-procure-one Published: 2026-08-18 A vague research brief produces a vague result. Before an insights, product, or pricing leader commits budget to a simulated-market study, the team needs one decision-specific question: the action being considered, the audience it affects, the alternatives on the table, and the outcome that changes what happens next. [figure: Four labeled boxes for action, audience, alternatives, and outcome, each with an arrow feeding into a single box labeled decision-specific research question.] ## Why does a thematic question waste the study budget? "What do customers think of our product?" is not a research question. It has no answer structure, so any result it produces is directional color, not something the team can act on. A standard market research process moves from defining the problem through data collection to a reportable conclusion, and each later stage depends on the problem being defined clearly at the start.[^1] A vague brief can lead a team to treat an underspecified study as settled evidence for a launch, price, or positioning call it was never designed to support, spending budget and the decision window without moving the decision. Compare a weak brief to a strong one: If a question cannot be written that specifically in one sentence, the study is not ready to scope. ## The four inputs a decision-specific brief needs ### 1. The action under consideration Name the concrete thing being compared: a price point, a message variant, a product concept, a positioning statement. "Improve our messaging" is not an action. "Lead with the outcome claim instead of the category claim" is. ### 2. The audience the decision affects Name the buyer segments whose response actually changes the decision, not every segment the company could theoretically reach. A pricing decision for enterprise buyers does not need a self-serve consumer's reaction. ### 3. The alternatives being compared A study needs at least two things to compare. Testing one message in isolation tells the team whether people reacted, not whether it works better than the current one or a competing option. ### 4. The outcome that changes what the team does next Decide in advance what result would change the plan. If the team would ship the same roadmap regardless of what the study shows, the study is not tied to a decision and should not be funded. ## Sizing the study once the question is scoped These figures are planning examples from a comparable panel-style research setup, not current Subconscious defaults or guarantees: - 4 to 6 segments is a workable range for comparing distinct buyer perspectives in one round; combine two segments if they would give essentially the same answer. - 5 to 8 questions per session keeps the results manageable; more than that and synthesis becomes hard to act on. - 15 to 30 minutes was a typical session length in that setup, a fraction of the time a real focus group or interview series takes. - 3 to 5 iterations in a single day was achievable when a first round revealed confusing messaging and a revised version needed a same-day retest. Use these as starting anchors when sizing a proposal, not as fixed rules for every engagement. ## What this scoping work rules out This scoping work also rules out engagements the method cannot support: - Deep relationship building. A single study estimates a directional response; it does not replace ongoing customer relationships. - Statistical-scale quantitative validation. A simulated study is a fast first pass, not a substitute for a large-sample field study when the decision requires that scale of proof. - Emotionally nuanced topics. Some questions need the texture only real human participants provide. - Regulatory mandates for real participant data. When a rule requires human data, a simulated study cannot satisfy that requirement. None of these are failures of the method. They mark which questions belong in a simulated study and which do not. ## How does the scoped question run as a causal comparison? Once scoped, the engagement can take the form of a controlled experiment rather than an open-ended conversation. Subconscious.ai runs this kind of comparison on simulated markets: it tests specific alternatives, such as price points, messages, or product concepts, against a defined population, and estimates which action is more likely to move the stated behavioral outcome, reporting uncertainty where the study design supports it. The public validation result behind that comparison: our best configuration reaches 87% of the measured human ceiling on one study, 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, drawn from roughly 300 replicated human studies across 9 domains. See the causal fidelity paper. That result covers the simulated pass, not a guarantee of market performance, so the scoping boundaries still apply once the study runs. ## When the scoped question still needs a human study The same four boundaries apply after a simulated study comes back. A simulated result is directional evidence: it does not turn a causal action test into an observed usability session, a clinical trial, or a guarantee of results. Where the decision is high-stakes, emotionally nuanced, or under such a mandate, the practical step is real-human validation: Subconscious can test or validate studies with real human participants. That step confirms the simulated answer; it does not replace the scoping work above. Write the one-sentence decision question before requesting a proposal. If a vendor cannot state back the action, audience, alternatives, and outcome the study is meant to test, the engagement is not scoped yet. Review current research and methodology, or book a walkthrough. [^1]: The Market Research Process, Lumen Learning, Introduction to Business. --- # AI Panels for Luxury Brands: Testing Positioning Before It Ships URL: https://subconscious.ai/blog/industries/ai-panels-luxury-brands-testing-positioning-before-ships Published: 2026-08-18 A luxury brand leader deciding on a new positioning, price point, category extension, or retail concept cannot test it the way a mass-market brand does. Recruiting real luxury customers for research runs into hard constraints: the qualified base is tiny, professional respondents and affluence fraud contaminate consumer panels, and scheduling senior or founder-level participants is difficult even under a third-party agency's NDA-bound, blind recruitment. Shipping the wrong narrative or price framing costs more than a failed campaign. It costs trust with a customer base that does not give second chances. De-risking that decision means running the positioning, price, or concept through a controlled experiment on a simulated audience, then confirming the direction with real people before it ships. Subconscious runs randomized experiments on a simulated market and validates the result against real human behavior, producing a causal read on what changes perception, not a survey score. [figure: A four-step path: define a tightly specified audience segment, run a randomized experiment on the simulated audience, validate the result against a small sample of real customers, then ship the confirmed positioning.] ## Why Luxury Research Breaks the Standard Playbook Three constraints separate luxury research from everyday consumer research. The qualified base is small. Pulling together thousands of survey participants means little for a mass-market brand. For a luxury house it means something immediately: true affluent buyers are recruited blind, through third-party fieldwork agencies under NDA rather than open advertising, and the pool of people who both qualify and will show up is small. The customer is busy and skeptical. Affluent, senior, and founder-level customers respond to surveys at low rates, a pattern documented in research on rich non-responders, and the ones who do respond rarely represent the base. The topic resists direct elicitation. Asking a customer to explain their own taste changes how they think about it. Luxury research has historically leaned on observation and long-form interviews rather than direct questions for this reason. A simulated audience sidesteps the first two constraints: nothing leaks, and there is no recruiting cost or scheduling to fail. It does not remove the third, but a well-specified audience and a well-designed experiment can probe reactions indirectly, closer to the tradition than a direct-ask survey. ## What Does a Well-Specified Luxury Audience Require? As a planning example, a typical luxury-audience study is sized from 50 to 200 simulated respondents representing a tightly defined affluent segment. A generic "wealthy households" audience produces generic answers. A luxury decision needs a granular definition: income band, geography, generational wealth versus first-generation buyer, category loyalty, and brand-skepticism level all change how a segment reads a positioning. Teams evaluating luxury audiences commonly work with definitions this specific: Subconscious runs controlled studies against a person-level audience graph, making a definition this specific reachable rather than theoretical. A capability claim only means something once its edges are stated plainly. That audience graph is a targeting and study-design resource, not a recruitable panel of participants. As a hypothetical illustration of what that specificity buys: a manufacturer of luxury sanitary fittings might assemble a study of 100 affluent households screened for premium-residence ownership, bathroom renovations underway, and a lean toward European design houses, to pre-test brand-story drafts and launch positioning before any creative is committed. ## Six Decisions Where This Method Applies ### 1. Brand story validation A new collection story, a heritage anniversary, or a refreshed narrative gets committed to a season-long campaign, so a narrative that reads as manufactured rather than earned is costly. Testing the draft narrative against the target audience before commitment surfaces which version lands as authentic. ### 2. Price-point strategy Luxury pricing signals identity: too low erodes it, too high alienates part of the base. Testing multiple price points framed as an in-boutique moment ("you see this product at price X") shows whether a price reads as coherent with the brand story rather than as a bare number. ### 3. New category entry A brand entering an adjacent category, a leather house into eyewear or a fashion house into hospitality, risks diluting the core brand if the extension reads as incredible. Testing the extension against current and prospective customers separately shows whether it needs more positioning work before launch. ### 4. Retail experience concepts The boutique is the highest-value touchpoint and the most expensive to pilot physically. Testing a proposed retail experience in detail (greeting, appointment flow, follow-up) before any physical pilot shows which elements read as an upgrade and which read as a departure. ### 5. Communications and editorial voice Luxury communications are minimal and considered, and customers detect commercial intent quickly. Testing draft communications against the target audience checks whether the voice respects the customer's intelligence and whether the implicit positioning holds. ### 6. Demographic expansion Expanding to a younger buyer, a new geography, or a new wealth segment risks confusing the existing customer. Testing the proposed positioning against both the new target and the existing base together shows whether the expansion reads as coherent brand growth, or as the brand abandoning its own customer for someone else's. ## A Worked Example, as a Planning Reference The following is a planning example from luxury brand-strategy work, useful for understanding the shape of a multi-stage engagement rather than as a Subconscious case study. A luxury bathroom-fittings house wants to launch its most expensive flagship product yet and needs the launch to read as a brand statement, not a price increase. The work would run across 8 weeks: In that scenario, such a study would surface which concept framing resonates, where the audience shows skepticism toward marketing claims, and which price-framing approach reads as coherent with the brand story. None of this would replace the brand team's own instinct; it would sharpen decisions the team has not resolved and could catch a price-framing problem before it reaches market. ## What Does This Method Not Replace? The misses belong on the record next to the hits. A simulated audience is not a substitute for the boutique relationships and client-advisor history luxury houses build over years, or for the creative director's taste. It can tell a team whether a story lands with a defined audience. It cannot tell the team what story to tell. When a decision is consequential enough to need certainty beyond the simulated read, Subconscious's human-baseline research line can re-run the same causal question with real human participants, moving from simulation to real-human validation, without changing the question being asked or starting a new research effort from scratch. Naming this failure mode is what lets a buyer check the method before trusting it. Treat this as one input to weigh against the brand's own conviction, not a source of ready-made direction. A team that ships whatever a simulated audience says it wants is not doing luxury brand work. ## Getting Started Pick one upcoming decision, a positioning refinement, a price change, or a brand-story draft, and run it as a single experiment against a precisely defined target audience of roughly 50 simulated respondents to start. Read the research methodology behind how these studies are validated. Book a walkthrough to scope the first test against your own audience definition. --- # Stakeholders Don't Need More Data. They Need a Decision. URL: https://subconscious.ai/blog/use-cases/stakeholders-dont-need-more-data-they-need-decision Published: 2026-08-18 A stakeholder wants the answer tomorrow. A report draft appears before the analyst has finished reading the data. A manager asks whether the team can automate the first pass. The answer is not another dashboard: use a causal experiment to produce a directional read, review the assumptions, then validate consequential findings with real people before the launch, price change, or campaign ships. [figure: Four-step path: a simulated experiment gives a directional comparison, an assumption review checks audience and source quality, real-human validation checks the same question with real people, then the decision.] ## More output does not resolve the decision For a market-research team, the risk is not that every task disappears. It is narrower: being asked for more dashboards while decisions stay stuck. As drafting and summarization get easier to automate, the analyst has to move closer to the decision, not further from it. Between 2024 and 2034, the Occupational Outlook Handbook expects the market research analyst and marketing specialist role to keep growing (U.S. Bureau of Labor Statistics). The role is not disappearing. It is shifting toward the judgment and validation work a fluent-sounding output cannot substitute for. A number without its limits reads as marketing. A stakeholder acts on a plausible finding as if it were proven, then greenlights a product, pricing, or campaign decision based on an unvalidated read of customer behavior. ## What actually changes in the analyst's job? Expertise used to live partly in access: knowing how to field a study, clean responses, and package a finding. Automated drafting and synthesis narrow that advantage. A first-pass survey, transcript summary, or reaction to a concept is easier to produce. The bar for proving expertise has moved: with a fluent answer available to anyone, value sits with whoever can judge which answer deserves trust, and trace which stage of the workflow it came from. ## How do you separate a directional read from a validated claim? Early-stage work such as hypothesis generation, directional comparison of concepts, and first reactions to a pricing story or campaign route can run through a simulated experiment. Claims that are expensive, public, or will drive a stakeholder decision need a separate step: checking whether the finding holds with real people before it reaches a deck. Subconscious runs controlled randomized experiments in a simulation of the market. The same causal question can then move to real-human validation without changing the experimental design. A large audience number means nothing without its limits stated plainly. Studies can be defined against a person-level audience graph covering 800 million real people. That coverage describes the audience a study can represent; it is not a claim that 800 million people participate in a validation study. Results are reported with confidence intervals and documented limitations rather than as a single point estimate. Naming what still needs review is what lets a buyer check the work. Audience definition, source quality, possible bias, and the level of evidence the decision requires still need human review. ## Interpretation is the analyst's product The mistake is confusing data access with understanding. It usually comes from pressure: a tool gives a fluent answer, and the deck needs a conclusion. An automated first pass can produce a useful hypothesis. It cannot decide on its own whether that hypothesis is valid for the decision in front of the team. Make the boundary part of the deliverable. State what the simulated work tested, what it did not test, and what still needs validation before the business acts. ## What the workflow deliberately leaves to the team The boundary belongs on the public record, next to what the workflow does provide. Subconscious does not provide an automated recommendation engine, financial or ROI modeling, or a scored ranking of a team's existing dashboards. It does not decide which claim is consequential enough to validate. The analyst still owns the business question, the audience definition, the evidence review, and the recommendation. The team can keep the causal question consistent across a simulated experiment and real-human validation, while keeping a directional signal distinct from evidence suitable for a consequential decision. ## Put one live decision through the workflow Do not rewrite the whole research process at once. Start with one visible workflow: - Pick a real project tied to a live decision. - Write the business decision in one sentence. - Define the audience and how much confidence the decision requires. - Use a simulated experiment for the early, directional stage. - Have a person check the output by hand, flagging what holds up, what is shaky, and what should not drive action. - Present the answer with a clear caveat and, where the decision is expensive or public, a recommended real-human validation step. Repeat that once a week for a month: directional evidence for exploration, explicit human review, and validation where the business is about to act on the answer. For the buyer-decision side of that process, how Subconscious runs a causal experiment and the worked examples in the case studies show how the sequence applies to a product, pricing, or campaign decision. Reviewing active research is a useful next step once a team has a specific decision to test. --- # Does a Synthetic Panel Rank Subcompact Car Features the Way Thai Buyers Did? The Wu Replication URL: https://subconscious.ai/blog/case-studies/synthetic-panel-rank-subcompact-car-features-way-thai-buyers-did-wu Published: 2026-08-18 An automotive product or insights leader deciding which vehicle features to prioritize before committing R&D and marketing spend to a new subcompact model needs a way to check a simulated experiment against a real fielded study before trusting its feature-importance ranking. Prioritizing the wrong feature, such as underweighting appearance or fuel efficiency, wastes engineering and marketing budget and risks a launch built around what buyers were assumed to want rather than what they actually value. [figure: Two ranked lists of six car features (appearance, fuel efficiency, price, safety, power, gadgets), one from a 2014 published study and one from a matched synthetic-panel run, linked by a correlation of .76.] ## What the published study found Wu, Liao, and Chatwuthikrai (2014) ran a conjoint analysis with 201 Thai consumers, asking them to choose among subcompact car profiles that varied by appearance, fuel efficiency, price, safety, power, and gadgets. Their published result is a feature-importance ranking across those six attributes. ## What the replication compared Publishing that no dataset exists here is what lets a buyer verify the claim before relying on it. A matched, unpublished internal run against a synthetic panel used the same six-attribute design and produced its own feature-importance ranking, with no public dataset or report available for independent verification. Comparing the two orderings gives a Spearman rank correlation of rs=.7429 (p=.08, not significant at conventional thresholds) between the synthetic panel's ranking and the published human ranking. A correlation number reported alone is marketing. That single correlation is the entire evidence base here: one replication, one market (Thailand), one product category (subcompact cars). It is a directional but statistically inconclusive agreement, not identical rankings, and it describes stated preference among the six attributes the study measured, not a prediction about novel or future vehicle features. ## Why this matters before a launch decision A commercial vehicle launch commits real engineering and marketing budget to a feature set months before the car reaches a lot. A feature-importance study that gets the ranking wrong, for example treating appearance as a low priority when buyers weight it heavily, means that budget is misallocated before anyone can catch it. ## Where a validation path fits A team does not have to commit to a full fielded conjoint study before learning whether its synthetic design tracks real buyer preference. Subconscious can move a study from a simulated experiment to real-human testing without changing the underlying causal question, serving as a first-pass filter before fielding a new vehicle-feature study. Related replication comparisons are collected in case studies, and the underlying method is documented on the research page. [figure: Branching path: synthetic design compared to a published study gives a strong but imperfect match, then splits on same market and features, routing to trust-as-filter or field-a-new-study.] ## What this replication does not establish Naming what this replication doesn't cover is what lets a buyer check it before relying on it. This is one historical replication, not a live customer engagement or a benchmark across markets or vehicle categories. It does not establish accuracy for other countries, product categories, or feature sets, and rs=.74 should be read as a directional, non-significant correlation rather than a guarantee of matching outcomes. A team applying this approach to a different market or vehicle segment needs its own matched comparison, ideally reviewed alongside a full field conjoint, before treating a synthetic ranking as a substitute for fielded research. --- # AI Competitor Analysis Through Buyer Decisions URL: https://subconscious.ai/blog/use-cases/ai-competitor-analysis-through-buyer-decisions Published: 2026-08-18 AI competitor analysis through buyer decisions uses public signals and approved evidence to model the reasoning behind a buyer's choice of a competitor and the conditions that would make them switch. Pricing pages, launch notes, job listings, and feature grids show what a competitor is doing. They do not explain why a buyer chose that competitor, what creates loyalty, or what would trigger a switch. AI-assisted competitor research can model those decision questions using public signals and approved evidence. The output is a set of hypotheses for sales, positioning, and product work. Stating the boundary here is what keeps the output checkable against real evidence. It is not private knowledge about a competitor or its customers. [figure: Five-stage flow: public signals feed a decision profile built from adoption history and win-loss evidence, which generates a switching question routed to sales, positioning, or product, then validation.] ## Focus on decision dynamics - Why a buyer chose the alternative. - Which parts of the experience create real loyalty. - Which frustrations are tolerable and which prompt evaluation. - What event, feature gap, or price change could overcome switching costs. - Which alternatives the buyer may not know. A statement such as "easy to implement" is less useful than a decision account: the team needed a system before Q4 planning and accepted a promised 2-week deployment. ## How do you build competitor-customer profiles? Define the role and company context. A mid-market director and an enterprise vice president can use the same product for different reasons. Add an adoption history: the previous approach, the trigger for evaluation, and the reason the current product won. Distinguish a power user with embedded workflows from a casual user with low switching costs. Use public reviews, support forums, social discussion, and win-loss evidence to calibrate strengths and weaknesses. Win-loss interviews surface the deciding criterion and the runner-up alternative more reliably than a CRM close code does (Win-Loss Analysis for Competitive Intelligence, Corporate Visions). Do not fabricate proprietary experience. Use 3-5 profile types: a loyal power user, a frustrated but constrained user, a new customer, and a customer evaluating alternatives. ## Ask questions tied to a choice ### Original decision Ask which alternatives were considered, which criterion decided the outcome, and whether the same buyer would make the same choice today. ### Satisfaction and friction Identify the one capability the buyer would miss, the largest current frustration, and any expectation the product has not met. ### Switching Ask what an alternative would need to offer, how much operational pain a move would create, and which event would make evaluation worthwhile. ### Which segments are vulnerable to switching? Look for use cases where the current product is tolerated because no better option is visible. A planning scenario can ask whether a 30% price increase would trigger a search for alternatives. ## Combine market facts with simulation Traditional competitive analysis covers features, pricing, positioning, and market share. Simulated buyer research explores motivation, loyalty, and triggers for change. Public facts constrain the simulation. The simulation points to questions that win-loss interviews, sales calls, product analytics, or market experiments can test. See how the research behind this approach is designed and read. ## Where can the results be used? Sales teams can turn likely decision criteria into battle-card questions without presenting simulated statements as customer proof. Positioning teams can test whether a claimed advantage changes consideration. If implementation speed appears important, compare the messages and supporting proof rather than assuming the claim will win. Product teams can investigate repeated frustration with a competitor's limitation. Naming this failure mode is what stops a hypothesis from being treated as data. A simulated pattern is a reason to research the opportunity, not evidence that demand exists. [figure: Four categories: Original decision (deciding criterion); Satisfaction and friction (largest frustration); Switching (trigger event); Vulnerable segments (tolerated for lack of a better option).] ## Keep the limits explicit Naming the limit here is what lets a buyer verify the method before relying on it. The representation is simulated, not a real competitor customer. It cannot know a private roadmap, internal metric, or upcoming feature. Calibration quality sets the ceiling on the exercise. Subconscious is relevant when the next step is a controlled comparison of product, pricing, messaging, or go-to-market actions for a defined buyer segment. Use the competitor model to form the experiment. Use observed behavior to validate it. See how the process runs or book a demo to test a specific differentiator claim. --- # A Donor Value Model Told This NGO Who Would Give. It Didn't Say What to Change URL: https://subconscious.ai/blog/methods-and-validation/donor-value-model-told-ngo-who-would-give-didnt-say-what-change Published: 2026-08-19 A data science team inside a global children's rights NGO spent months building a Bayesian model to forecast donor value. It worked. It still could not tell the fundraising team which action would move that value. That gap is the case study here, whether the budget in question is donor retention or customer acquisition. ## The decision this model was built to inform The NGO's fundraising and marketing teams needed to compare donor value, at the individual and segment level, to allocate budget toward expected long-term impact rather than short-term donation revenue. They wanted the resulting lifetime-value number to work as a core KPI: a common yardstick for judging acquisition channels and reading A/B test results. That is a forecasting problem. Predicting a number and knowing what changes it are different questions, and the second one is where a budget decision actually lives. ## What are the two donor types? The team's data covered two overlapping donor behaviors: committed recurring giving ("pledge donors") and flexible, ad hoc contributions ("cash donors"). A single supporter could show both patterns over time. Most records started in 2018, with a subset of pledge histories going back to the 1990s. The team set 2024-01-01 as the train/test split, holding out roughly two years for out-of-sample testing. They built one model per behavior: - Cash pillar. A beta-geometric/negative binomial (BG/NBD) model forecast donation frequency, following the framework Fader, Hardie, and Lee proposed as an alternative to the Pareto/NBD model in Probability Models for Customer-Base Analysis. A Gamma-Gamma model estimated each donor's expected average cash donation amount. Cash CLV is the product of the two. - Pledge pillar. A Shifted Beta-Geometric survival model, building on Fader and Hardie's approach to projecting customer retention in a discrete-time contractual setting, estimated donor survival curves. Multiplying survival probability by the most recent monthly pledge amount and summing over time produced pledge CLV. Both pillars applied a 2% annual discount rate to account for currency devaluation, kept configurable rather than fixed. The team capped the forecast horizon at 10 years, judging that their observation window could not support reliable inference beyond that range given their donor base's loyalty patterns. Total donor-level value is cash CLV plus pledge CLV. The team checked that donation frequency and donation amount stayed roughly independent, a required assumption for the Gamma-Gamma model, using a correlation threshold below 0.3 as their working rule of thumb. ## Why MAP replaced MCMC at this scale PyMC, the open-source probabilistic programming library underneath this work, supports several fitting methods. The team started with MCMC (Markov chain Monte Carlo) sampling, which returns full posterior distributions rather than single point estimates, useful because it lets a team assess uncertainty directly rather than assuming it away. Publishing where a method breaks down is what makes the rest of the result trustworthy. At their scale, more than a million data points, that advantage came with a cost. Sampling was compute-intensive, tuning draws, chains, and target acceptance rate took sustained iteration, and the team hit convergence and autocorrelation problems even after switching samplers. They eventually downsampled the data to keep MCMC runtime feasible. They moved to MAP (Maximum a Posteriori) estimation instead: point estimates from an optimization over the posterior density rather than a full sampling process. For a dataset this size, MAP ran with less tuning overhead and avoided the convergence headaches, at the cost of losing built-in uncertainty quantification. MAP is still Bayesian: priors regularize the fit, but the team had to reason about uncertainty separately from the estimate itself. The broader lesson for any team evaluating Bayesian CLV work: the full-posterior advantage of MCMC is most compelling in small-data regimes. At large scale, it can become the obstacle rather than the benefit. ## What the model still leaves open The misses go on the public record next to the hits. None of this tells the fundraising team which lever to pull. A well-fit CLV model ranks donors and segments by predicted value. It does not test whether a different ask amount, a different retention message, or a different acquisition channel would change that value. That's the distinction Subconscious is built around: testing the specific product, pricing, messaging, or go-to-market action before a budget commits, using causal experimentation and discrete-choice-style modeling. It's a complement to value-prediction work like the CLV model above, not a replacement for it. Subconscious does not build donor or customer lifetime-value forecasts, survival curves, or the production modeling pipeline described here; that statistical work stays with a team's own data science function. A team can validate a simulated result against real human participants without changing the underlying causal question, useful when a channel or message decision is expensive enough to justify a second check. ## What are this model's limitations? The NGO's own team flagged two open gaps in their model: it is revenue-driven and does not yet account for cost, and it does not model donors moving between cash and pledge behavior over time. Naming a method's boundary is what lets a buyer check it before relying on it. Separately, a causal action test answers a narrower question than this CLV model does. It will not forecast ten-year donor value or produce a survival curve. It tells you whether a specific change to price, message, or offer moves the outcome you tested, with uncertainty stated qualitatively rather than promised as a fixed confidence interval. [figure: Two-column comparison. MCMC: full posteriors, fine at small scale, hit convergence problems past a million data points. MAP: point estimates, ran at that scale, lost built-in uncertainty quantification.] ## Where does this fit in a budget cycle? Use a lifetime-value model to decide where to look: which segments, which channels, which donors justify more attention. Use a causal test to decide what to do about it: which specific action actually changes behavior in that segment before the budget ships. Treat the two as sequential steps in the same decision, not competing methods. [figure: Two-column comparison. Left: CLV forecast feeding a ranked list of donors by predicted value. Right: an open question over which lever moves value, resolved by testing the action before budget commits.] --- # When an AI Product Manager Mindset Helps You Prioritize the Roadmap URL: https://subconscious.ai/blog/buyers/when-ai-product-manager-mindset-helps-prioritize-roadmap Published: 2026-08-19 A founder or product manager choosing what to build next is choosing where engineering capacity goes for the next quarter. Get the sequence wrong and the cost is not the build itself. It is the months spent shipping a feature that never changes what the target customer does, discovered only after release. [figure: A split diagram: roadmap options feed into two parallel paths, trade-off reasoning on one side and a controlled experiment measuring customer behavior on the other, both converging on a single roadmap decision.] ## Where roadmap debate runs out of evidence Few roles in a growing company carry as much piled onto them as product management: strategy, discovery, prioritization, stakeholder alignment, specification, launch coordination, and customer feedback synthesis. Several situations strip a team of the structured judgment that usually holds those trade-offs together. - Founder-led product teams. A technical founder is good at building. Deciding what to build is a separate skill. The common failure is not building bad things. It is building the right things out of sequence: a team can burn three months on a feature that pleases existing power users while doing nothing to grow acquisition, or ship ten features at 70% quality rather than three at 95%. - Teams between PM hires. A PM leaves, the replacement starts in six weeks, and in the gap the engineering team still needs priorities while three stakeholders push conflicting requests. Absent a framework, the loudest voice in the room decides. - Growing teams without senior PM leadership. Junior PMs can execute but have not yet navigated a platform pivot, a pricing model change, or an entry into a new market, and no one senior is there to reason it through with them. In each case, the team still has to decide: which of several roadmap items to sequence first, what an MVP should and should not include, whether a PRD's success metric actually means anything, or whether to build a platform versus stay a point solution. Structured trade-off reasoning surfaces blind spots a team might otherwise miss. Publishing where reasoning alone falls short is what lets a buyer check the claim before trusting it. But reasoning through a trade-off is not the same as measuring how the target customer responds to each option: a well-argued roadmap sequence can still be wrong about what moves the outcome the team cares about. ## How does Subconscious turn roadmap debate into a measured comparison? Where trade-off reasoning stops at "here is my best judgment on the options," Subconscious runs a controlled experiment. It defines the competing alternatives, such as feature A versus feature B, platform versus point solution, or enterprise-first versus self-serve-first, as a randomized comparison against the same population and outcome, producing a measured behavioral result before the roadmap is locked. This does not replace the judgment call between a platform pivot and staying focused; it gives that call a comparison to check itself against. The discrete choice experiment behind this kind of comparison is a standard method in management research more broadly (Organizational Research Methods, SAGE). Subconscious can also test or validate studies with real human participants, so a team can move from a simulated comparison of roadmap options to real-human validation without changing the underlying causal question. /research documents how these experiments are structured, and /case-studies shows results across other product and pricing decisions. ## What does a measured comparison not settle? A number without its limits is marketing copy. A causal comparison between roadmap options answers one question: which defined option changes customer behavior, and by how much. It does not replace: - Naming what a method does not cover is what lets a buyer check it against the method itself. Customer conversations. Primary research still requires talking to real customers; a controlled comparison can sharpen which questions to ask next, not stand in for the conversation. - PRD and prototype review. Checking a specification for missing edge cases, unclear success metrics, or weak reasoning behind a feature is a different exercise than measuring customer response to a defined option. - Stakeholder and organizational navigation. Building consensus across conflicting stakeholder requests is a negotiation problem, not a measurement problem. - Judgment from a lived failure. A team that has shipped and watched a product fail carries a kind of calibration that no measured comparison replaces. [figure: Measured comparison at center, labeled "answers: which option changes behavior." Four boxes ring it, each marked "not answered": customer conversations, PRD review, stakeholder navigation, failure judgment.] ## How do you frame a roadmap decision as a test? Before locking a roadmap sequence, a team can name the two or three options actually in contention: which features, which market segment, which platform direction, and treat that as the experiment's design question rather than a debate to win. /how-we-work walks through how that experiment gets scoped, and /demo is the next step for a team ready to run one against a live roadmap decision. --- # Network Simulation, Persona Chat, or a Causal Test: Picking the Right Tool Before a Launch Decision URL: https://subconscious.ai/blog/comparisons/network-simulation-persona-chat-causal-test-picking-right-tool-before Published: 2026-08-19 A network-propagation platform and a persona-chat tool can both describe an audience. Neither, by itself, tells a marketing or product research lead which specific message, price, or launch audience actually moved a behavioral outcome. That distinction matters most right before a team commits budget to one version of a launch over another. ## Two ways vendors currently model an audience One category builds interconnected networks of AI agents and observes how a message or idea moves through that network over time, a lineage that traces back to agent-based modeling in computational social science (societies.io, checked 2026-07-27). The output is a picture of propagation: who reacts, who influences whom, how an opinion spreads across a simulated population. A second category builds individual AI conversation partners calibrated to a specific customer type or stakeholder profile, and lets a researcher talk to each one directly, following up and redirecting the way a qualitative interviewer would (societies.io, checked 2026-07-27, describing the contrast with network-style simulation). The output there is a set of individual reactions, closer to a structured interview transcript than to a market-level answer. Neither is built to compare two or more specific alternatives head to head and say which one caused a behavioral outcome to move, with the uncertainty of that estimate reported alongside it. ## What the propagation-vs-conversation choice can't settle A team evaluating these platforms is usually trying to answer a narrower question: which exact message, price point, or launch audience should ship. Neither output is a caused outcome. Treating spread or a set of individual reactions as proof that a specific decision is correct means finding out after launch, from real spend, whether the plausible answer was also the right one. ## How do you test the alternatives directly? Subconscious runs controlled experiments on a simulated market: two or more concrete alternatives compared head to head, with the platform estimating which one moved the outcome and reporting uncertainty where supported. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. A fidelity number by itself is marketing until its limits are attached. It is a validation result, not a guarantee for a new market. ## Where does a causal test stop? Naming what a method cannot do belongs next to what it can, so a buyer can check the fit before committing budget. A controlled experiment of this kind is not a network-propagation model; it does not simulate how an idea moves through interconnected profiles over time. It requires a defined decision and a specific set of alternatives to compare, not open-ended conversation with a single profile. When a team needs a recruited panel of real respondents rather than a simulated comparison, this approach does not replace that panel; it is a different step in the same research process. ## How do you move from a simulated comparison to real people? When a launch decision is large enough to warrant it, the same causal question can be extended from a simulated comparison to real-human participant validation without changing what is being tested. The alternatives, the outcome, and the comparison stay the same; only the source of the responses changes. That step is not always necessary. It matters most when the cost of being wrong is high enough that a team wants the causal estimate confirmed by real respondents before committing spend. For teams weighing a network-propagation platform against a persona-chat tool, the more useful first question is often neither "how would this spread" nor "what does this profile think," but "which of these specific options actually wins." Methodology and study design and worked comparisons cover how that estimate is produced end to end. How the process runs and a walkthrough are the next steps for a team ready to test a specific decision. [figure: Three approaches compared: propagation shows diffusion, persona chat shows per-profile reactions, a causal experiment compares named alternatives with an effect and uncertainty, then extends to real-human validation.] --- # AI Panels for Beta Testing: Find Questions Before Real Users Arrive URL: https://subconscious.ai/blog/use-cases/ai-panels-beta-testing-find-questions-before-real-users-arrive Published: 2026-08-19 Beta programs often discover major friction only weeks before launch. Simulated-audience work can help teams identify questions and likely failure points earlier. The real beta remains the source of behavioral evidence: independent reviewers of AI-generated research participants draw the same line, noting that participant simulation can approximate a stated response but does not substitute for watching how someone actually uses a product (Nielsen Norman Group, "Synthetic Users: AI 'Participants'"). "Synthetic users are fake users generated by AI. While there may be a few use cases for them, user research needs real users." Kate Moran, Nielsen Norman Group, "Synthetic Users: AI 'Participants'" (source) "Synthetic users are fake users generated by AI. While there may be a few use cases for them, user research needs real users." Kate Moran, Nielsen Norman Group, "Synthetic Users: AI 'Participants'" (source) [figure: Five steps: define flow and a mixed audience; compare segment reactions; map strengths vs hesitation; write targeted questions; check which predicted issues the real beta confirms or misses.] ## Frame the pre-beta study ### What should you define when scoping the beta? List each feature and flow, what the person sees, what action is expected, and the intended outcome. Describe the current experience, including known rough edges. ### 2. Define the launch audience Do not model only enthusiastic beta volunteers. One example panel uses 10 to 15 audience representations: - 2 to 3 enthusiasts; - 4 to 5 pragmatists; - 2 to 3 skeptics; - 2 to 3 low-tech users. ### What should you compare at each step? At each step, ask what the person believes will happen next, whether the value is clear, and what would cause abandonment. The result is a set of hypotheses for the beta plan. ### 4. Build a reaction map Separate shared strengths from segment-specific hesitation and repeated confusion. ### 5. Improve the beta instrument Turn broad prompts into targeted questions. If confusion appears at step 3, ask real beta participants what they expected there and observe what they did. ## What can an early screen reveal? A comparison can expose unclear messages, missing context, or assumptions about how feature A connects to feature B. It can also flag multi-step flows for usability testing. [figure: Three-item list showing what it means when a predicted issue is confirmed by the beta, when a predicted issue does not appear, and when the beta surfaces an issue the panel never flagged.] ## Calibrate against the real beta Compare modeled concerns with observed feedback: - predicted issues that appear become evidence about that specific setup; - predicted issues that do not appear may reflect selection bias or model error; - unpredicted issues identify missing context or an audience-definition gap. If a beta launches in the next quarter, an example planning exercise is to identify three to five issues for human testing in a single session. Use that list to improve the beta, then let actual behavior decide what needs to change. --- # What Is a Silicon Sample? URL: https://subconscious.ai/blog/use-cases/what-silicon-sample Published: 2026-08-19 A silicon sample means feeding a large language model a target population's demographic and psychographic makeup, then treating its outputs as a stand-in for how that population would answer research questions. Where a traditional sample means recruiting and surveying 500 real people, a silicon sample generates and queries 500 language-model outputs instead. The economics flip: minutes instead of weeks, a running query instead of a per-study field budget. ## Where does this method come from? In 2023, Argyle, Busby, Fulda, Gubler, Rytting, and Wingate published the paper that grounds the field, titled Out of One, Many, subtitled Using Language Models to Simulate Human Samples (Political Analysis, Cambridge University Press) (Cambridge University Press). Their setup: take a frontier LLM, give it the demographic backstory of a real ANES respondent, where ANES is a benchmark survey of US political attitudes, and have the model answer as that respondent would, then aggregate the results across many conditioned samples. Their finding: the resulting opinion distributions tracked the real ANES distributions closely on consistent attitude clusters, such as party affiliation, ideology, and policy preference, and less closely elsewhere. Political science, sociology, marketing, and economics picked up the citation, and that follow-on work turned silicon sampling into a named, studied method. ## How a research-grade sample gets built Five steps recur across implementations: - Define the target population: the demographic and psychographic parameters that matter, such as geography, age, income, occupation, attitudes, and prior brand exposure. - Determine sample composition: build proportions along those parameters so the sample mirrors the real population's makeup, rather than generating a flat batch of respondents. - Calibrate against prior real data: condition on panel data, prior survey waves, or CRM segments where available. This step separates a research-grade sample from a thin model wrapper. - Generate the sample: produce the conditioned, addressable units. - Query the sample: submit the research instrument, aggregate, analyze. ## What it answers well, and what it doesn't This gap is stated directly here so a buyer can weigh it before committing budget to a launch, pricing, or messaging decision. The right-hand column is the gap that matters for a launch, pricing, or messaging decision. A directional read tells a team what a population is likely to prefer. It does not establish that the preference is real, how large the effect is, or how confident the team should be before committing spend. ## What does a typical deployment look like? One historical example used a 200-persona silicon sample to screen a dozen concepts and narrow the field to 2 to 3 candidates, compared the same campaign across 4 to 8 country silicon samples in a single sitting before committing media spend, and, before launch, confirmed the surviving 1 to 3 options with a small study of real respondents. This note is included because a borrowed figure without its origin misleads a reader, and naming that origin here lets the reader place it correctly. Treat these as inherited planning figures illustrating how the method has been sequenced elsewhere, not a current Subconscious product specification, benchmark, or guarantee. ## Closing the gap: from directional signal to a decision you can defend Subconscious is a causal behavioral platform: controlled discrete-choice experiments run against a simulated population, returning causal effects with confidence intervals rather than a single directional read. That answers a different question than "what did the sample say": it answers "what caused the preference, and how much would changing the input change the outcome." Where a decision genuinely depends on it, a team can move from a simulated experiment to real-human validation without changing the causal question being tested. Subconscious can validate studies with real human participants or, where audience scale matters, run controlled studies against a person-level audience graph covering 800 million real people. That audience graph is a basis for structuring a study at scale, not a recruitable panel of people waiting to be surveyed. ## Does the terminology for this method vary? Commercial platforms describe this underlying method with several different labels. Some name the group, others name the individual unit inside it. The framing changes depending on whether the source is academic literature, a platform's marketing page, or a sales deck. The mechanism underneath stays constant: condition a model on a target population, generate outputs, aggregate. ## What this means for a pending research decision Before treating a silicon-sample readout as a green light for a launch, price, or message decision: - Confirm it was calibrated against real prior data from the target audience, not a flat, uncalibrated batch. - This absence is stated plainly here because a buyer needs to know when a number is missing its confidence interval before trusting a directional read. Treat the output as directional signal, not a population estimate. It carries no confidence interval unless the platform generating it produces one. - Where the decision is expensive to get wrong, route it through a causal experiment, and add real-human validation before committing budget. [figure: Two columns: strong for (directional reads, hard-to-reach audiences, multi-market comparison, iteration) vs not built for (validated estimates, sensory response, novel categories, unstated accuracy claims).] Related reading: research, how a study moves through this process, the leaderboard, about Subconscious. --- # Simulated Markets vs Real Participants: How Much Evidence Does the Decision Need? URL: https://subconscious.ai/blog/comparisons/simulated-markets-vs-real-participants-how-much-evidence-decision-need Published: 2026-08-19 A simulated-market experiment can carry an exploratory decision when its calibration is relevant and the cost of a wrong call is limited. It should not carry every decision alone. Pricing commitments, major launches, regulated-market choices, and emotionally sensitive questions can justify testing the same causal question with real participants before the organization commits. [figure: A rising scale of five decision situations, concept screening to regulated or emotional choices. Low-consequence end runs on simulated-market evidence; high-consequence end needs real-participant confirmation.] ## Trust should rise with consequence A low-cost concept screen can use a simulated experiment to reject weak options or identify a direction worth studying. A choice that commits budget, changes price, or affects a regulated population needs a stronger basis. The 5-concept row is an inherited historical planning example, not a current Subconscious limit or recommendation, and the table itself is a decision framework rather than a claim that one method is sufficient for every study in a row. ## Why does calibration matter for a simulated result? Calibration determines how much weight a simulated result deserves. A result checked against known human outcomes carries different evidence from an untested population description. Relevance matters too: evidence from one corpus does not automatically transfer to a new market, question type, or decision. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. The validation corpus covers roughly 300 replicated studies across 9 domains. The causal fidelity paper describes that evidence. That result is a validation finding for a specific published-study corpus. It is not a guarantee for every market, segment, question, or commercial outcome. ## What does the wider research record add? Research on language-model-assisted choice modelling reports sensitivity to prompting strategies and difficulty representing heterogeneity across people. The choice-modelling paper examines those limits. "Our findings reveal that proprietary LLMs can generate valid and behaviourally sound utility specifications, particularly when guided by structured prompts (CoT)." Sfeir and colleagues, Journal of Choice Modelling (arXiv:2507.21790) (source) A separate Nature paper presents a foundation model for predicting and capturing human cognition. Neither paper establishes the performance of a particular commercial study. Together, they make a narrower point useful to buyers: population-level pattern reproduction and individual-level fidelity are different standards of evidence. ## Where simulation should yield to human validation Real-participant confirmation matters when: - the market has thin or irrelevant calibration data; - legal, regulatory, clinical, or emotional context could change behavior; - segment heterogeneity is central to the decision; - the study design does not support the confidence intervals, segment comparisons, or rankings the buyer wants to use; - the downside could threaten the product, brand, or business. An inherited hundred-million-dollar launch scenario is best treated as a historical planning example: the scale illustrates why evidence requirements should increase with exposure, not a benchmark for when human confirmation begins. Subconscious can test or validate studies with real human participants. A team can therefore move from a simulated-market experiment to human validation without changing the underlying causal question. That continuity helps isolate whether the evidence, rather than a newly framed question, changed the conclusion. This does not turn a causal action test into an observed usability session, clinical trial, or automatic proof of market performance, and it does not make confidence intervals, segment breakdowns, or scenario rankings universal outputs; those depend on the specific study design. ## How should you design the study around the commitment? Start by naming the action the evidence will authorize and the consequence if the direction is wrong. Then choose a causal experiment that preserves that question across simulation and real-human validation. Review case studies for examples of decision-focused research, or discuss the decision before fixing the study design. --- # When Last-Touch Attribution Breaks: A Funnel-Aware MMM Read URL: https://subconscious.ai/blog/case-studies/when-last-touch-attribution-breaks-funnel-aware-mmm-read Published: 2026-08-19 A marketing leader reallocating budget under GDPR-constrained tracking faces a specific question: which channels actually cause leads, when user-level attribution can no longer see the path between them? Get that wrong and the usual failure mode is quiet. Bottom-funnel, last-touch channels keep looking dominant because they are the last thing tracking can see. Upper-funnel demand generation keeps looking weak because its effect shows up days later, in a different channel, after tracking has already lost the thread. Budget drifts toward the channels that are easiest to measure rather than the channels that cause the outcome. ## Why did GDPR break the old measurement? Multi-touch attribution with last-touch steering worked when user-level tracking could follow a customer across a long journey. GDPR and browser tracking restrictions shortened what that tracking can see, so mid- and upper-funnel touches became partially invisible and channel credit skewed toward the bottom of the funnel. Marketing Mix Modeling exists to fill that gap: it estimates channel contribution from aggregate spend and outcome data instead of individual-level tracking, which is why it has re-emerged as a standard response to privacy-constrained measurement (Analytic Edge). Standard MMM also treats each channel as independent, missing the case where upper-funnel spend does not convert directly but drives demand that a different, lower-funnel channel later captures. ## Modeling the funnel as one causal chain, not separate channels A published Bayesian MMM engagement with a German insurance marketer illustrates the fix. Instead of one model per channel, the team built a funnel-aware model with two linked likelihoods: one for how upper-funnel spend (video, demand generation) drives lower-funnel search demand, and one for how that lower-funnel spend converts to leads. The model also treated budget caps as a measurement problem, not a ceiling. When a paid-search campaign hit its daily budget cap, a naive read looks at the resulting plateau and mistakes it for saturation. The Bayesian model instead treated the capped periods as censored observations and estimated the latent demand above the cap: the recommendation shifts from "this channel is maxed out" to "raise the cap, because unmet demand is estimated at a specific level." A recent Bayesian framework for privacy-safe attribution anchored in MMM describes the same principle: aggregate, privacy-safe models can still recover granular, channel-level signal when the model's structure matches the funnel it measures (arXiv:2606.16878). Over a six-month rollout, the marketer's team reported cost-per-lead improving by more than 27% as budget shifted toward a full-funnel allocation rather than a channel-by-channel one. A number without its limits is marketing. That figure is a published account of a third party's own result: a historical example of what a well-specified funnel model can surface, not a Subconscious measurement or a guarantee for any other advertiser's funnel. ## What the model made visible The reported shift was gradual, according to the published account, though it does not detail a month-by-month trajectory. The team's account: agreement between predicted and observed outcomes mattered more to stakeholders than any single fit statistic. ## What can't this reallocation tell you before you spend? Naming this limit is what lets a buyer check the model against their own funnel before they rely on it. A funnel-aware MMM is still an after-the-fact read. It estimates what already happened once the budget was spent, using historical spend and outcome data. It cannot tell a team, before the money moves, whether a specific creative, offer, or channel message would have caused more leads than the one that ran. It also depends on enough historical variation in spend to identify the funnel's structure: a channel that has never moved much, or a demand pattern that has never been tested, is hard for any observational model to separate from noise. ## Testing the action before the model has to explain it Subconscious runs controlled causal experiments on a simulated population before a team commits the spend, so the question shifts from "what happened after we spent the money" to "which channel, message, or offer would move the outcome, before we fund it." That is a pre-decision test, not a replacement for post-spend measurement: the two answer different questions in the same budget cycle. Current examples of that kind of pre-decision testing are on the research page and in current case studies. Publishing this scope limit is what lets a buyer check the claim against the tool. Subconscious does not build or run marketing mix models, does not estimate censored demand from historical ad-spend data, and does not replace the post-spend measurement of actual media performance that a model like the one above provides. A team that wants both, a causal read on which action to fund and a later observational check on what the spend actually did, runs them as separate, complementary steps, not one system. Subconscious can extend this further: a team can move from a simulated test of a funnel action to a real-human validation study without changing the causal question being asked. ## Next step Teams weighing a similar reallocation decision can see how Subconscious structures a pre-spend causal test, and where it stops short of post-spend measurement, by walking through how the work is run or booking a session against a specific funnel decision. [figure: Left-to-right chain: upper-funnel video and demand-gen spend drives lower-funnel search demand; a capped paid-search budget is flagged as censored, not saturated; demand converts to leads; the model informs reallocation.] --- # Does a Causal Experiment Match a Fielded Conjoint Study? The Claret Fish-Preference Replication URL: https://subconscious.ai/blog/case-studies/causal-experiment-match-fielded-conjoint-study-claret-fish-preference Published: 2026-08-19 A causal experiment on Spanish consumers' fish preferences reached 87% of the measured human ceiling (0.832 of 0.959; mean 0.73 across the 43 studies passing design filters) against Claret, Guerrero, and Aguirre's 2012 conjoint study, per the causal fidelity paper. For a CPG or food and retail insights leader, that number is one data point of directional agreement, not proof the method generalizes to a new product or population without its own check. ## What did Claret, Guerrero, and Aguirre study? The original researchers ran a conjoint experiment on which fish characteristics move a Spanish shopper's choice: country of origin, how the fish was obtained, storage condition, and purchase price. Their published result, reported by EurekAlert, found that Spanish consumers preferred national fish over imported fish, with obtaining method, storage, and price as the other ranked factors. ## What did the replication compare? A matched causal experiment ran the same four attributes through Subconscious's experiment design and compared the resulting preference ranking against the original human conjoint ordering. The misses sit on the leaderboard next to the hits, so this section states both. A fidelity level this high says the two orderings agree on which attributes mattered most and least. It does not say the two studies agreed on the size of any single preference. ## Why this matters before committing a research budget Naming this failure mode is what lets a buyer check it before committing budget. An insights leader who mistakes a single directional replication for a general accuracy claim risks two errors: trusting a simulated ranking on a product or market it hasn't been checked against, or dismissing a method that in fact reproduces established human preference orders when checked. One spends budget on a study that didn't need commissioning; the other writes off a method before it was tested on the category that mattered. ## Reading the correlation without overreaching A number published without its limits is marketing. Treat this 87% fidelity figure as evidence that this one causal experiment tracked the direction of a single 2012 study, drawn from a mean of 0.73 across the 43 studies passing design filters. It doesn't establish accuracy across other product categories, and it isn't a substitute for commissioning a new human study on a live product decision with commercial stakes attached. ## Where to take this next The replication leaderboard documents this kind of check; see the case studies page for this study's writeup. A team weighing whether to run its own matched check before a launch decision can see how that process works on how we work. [figure: Two ranked lists of the same four fish-purchase attributes, one from the fielded conjoint study and one from a matched causal experiment, converging in the same order with rank correlation rs = .9442.] --- # How to Compare AI Focus Group Software URL: https://subconscious.ai/blog/comparisons/how-compare-ai-focus-group-software Published: 2026-08-19 A traditional focus group may require eight to twelve participants, recruitment, moderation, transcription, and analysis. One planning example puts the work at two to four weeks and $8,000 to $25,000 for a single session with one segment. An AI focus group changes the mechanics, but product claims about twenty-minute fielding, $0 to $30 platform spend, unlimited segments, or automatic cross-tabs need proof from the specific tool and study. Compare methods by the decision they support, not the loudest speed claim. [figure: Branching flowchart: define audience and decision, run a simulated panel, then check if the stakes are physical, regulatory, or unprecedented. If yes, branch to a human group. If no, ship on simulated evidence.] ## Three properties to inspect ### A defined panel The audience definition should be explicit. "Ask a model to imagine eight customers" is not enough. Record the segment, context, decision, and assumptions used to construct the simulated participants. ### Structured moderation The workflow should support consistent questions, follow-ups, and probes. A collection of one-off answers is closer to a survey than a moderated group. ### A reusable artifact The study should preserve the prompts, audience definition, stimulus, transcript, and outcome. Reuse matters only when the team can see what stayed constant and what changed. ## Compare method types before vendors Synthetic panels generate responses from simulated audiences. They help with early concept, message, and objection screening, but need calibration and human validation. Real-human research with AI moderation recruits people and uses software to guide or analyze the session. It preserves human evidence while changing fieldwork and analysis. Video-first qualitative tools emphasize recorded interviews, observation, and synthesis. They fit questions where voice, expression, or the interview itself matters. Population-scale simulation targets larger modeled populations rather than a panel-of-twelve format. Ask what data grounds the population, how it is validated, and whether the output matches the decision. Asynchronous interviews let real participants respond on their own schedule while software asks follow-ups. They trade live group dynamics for easier scheduling. The vendor list can change. The method distinction is more durable. ## How many AI focus group platforms exist in 2026? The category moved from experimental to buyable in 2025. By 2026, one market snapshot counted ten platforms shipping AI focus group software. Neutral examples in that snapshot include Remesh, Discuss.io, Synthetic Users (Synthetic Users describes itself as an AI user-research platform), Aaru, Outset.ai, Voxpopme, Lakmoos, Evidenza, and Persuva, formerly Pollie. They span real-human research with AI moderation, video-first qualitative research, synthetic panels, population-scale simulation, asynchronous interviews, and concept or message testing. The names provide a starting set, not a current ranking. Verify each product's present capabilities, validation, terms, and price before procurement. ## A buyer's comparison table Score each candidate against the same criteria: Do not treat a stated 80 to 95 percent accuracy range, ~90 percent correlation, a 100+ participant capacity, or 6-7 figure contract size as comparable measures without definitions. Those numbers can refer to different tasks, populations, and validation designs. ## Where do simulated focus groups help? Simulated groups shorten the path from a question to a testable hypothesis. A team can compare three segments under the same prompt, revise a stimulus, or run a planning session on a Sunday at 2 a.m. One operating example compares three weeks of traditional fieldwork with twenty minutes of simulation. Another compares $15,000 with $30 and calls the difference two orders of magnitude. Treat both as planning examples from a particular setup, not as current price or delivery guarantees. The strongest use is hypothesis triage: deciding which questions deserve real-human follow-up. ## Where are real focus groups still necessary? Use people when the decision depends on food, smell, touch, fit, ergonomics, physical behavior, or body language. Use audited human evidence for regulatory or legal substantiation. Use human research when a category has no useful precedent or when the investment is too consequential for directional evidence alone. The practical 2026 pattern is sequencing: simulated work for early triage, then human research for final validation when the decision warrants it. ## Model the operating economics honestly A planning scenario might assume five to twenty simulated groups per month, followed by two or three human-research questions per quarter. The right cadence depends on decision volume, validation risk, and the cost of mistakes. Do not promise 100x throughput or a 70 percent budget reduction from the method alone. Measure the actual cycle time, research spend, discarded concepts, and decision quality in your own workflow. Subconscious supports decision-specific causal experiments on product, pricing, messaging, and go-to-market actions. See how Subconscious structures a causal experiment or review case studies built from that method. The relevant comparison is whether a tool helps estimate which action changes which outcome for which segment, with uncertainty where the study supports it. --- # How to Spot Bad AI-Generated Consumer Insights Before They Drive a Decision URL: https://subconscious.ai/blog/implementation-and-procurement/how-spot-bad-ai-generated-consumer-insights-before-they-drive-decision Published: 2026-08-19 A fluent AI-generated finding does not fail by looking wrong. It fails by looking finished: a clean narrative, a confident tone, no visible gap where the evidence should be. The question a research operations or consumer-insights lead has to answer before that finding reaches a marketing or product decision is not "does this sound plausible," it is "what would make this wrong, and did anyone check." [figure: A four-item checklist a research lead runs before an AI-generated finding counts as evidence: audience specificity, causal design versus narrative, escalation path, and confidence labeling.] ## Why a checklist beats a gut check AI-assisted analysis is now part of the daily research workflow: drafting hypotheses, summarizing transcripts, running a first directional read against a described audience. None of that removes the need for evidence discipline; it raises the cost of skipping it, because a well-written answer is easy to mistake for a validated one. The ICC/ESOMAR International Code on Market, Opinion and Social Research and Data Analytics, the industry-wide standard for research conduct, exists precisely because fluent output and sound evidence are not the same thing. National bodies are still adopting the 2025 edition into local codes, including MRSI's recent adoption. A polished but ungrounded finding gets treated as proof, and a team commits budget or roadmap time to a claim that never had real support behind it. Fixing that does not require rejecting AI-generated or simulated research. It requires a gate before any such finding is allowed to count as evidence. ## Four checks before a finding earns trust A useful gate has four parts. Naming the failure mode here is what lets a team catch it in its own decisions. Most bad decisions trace back to skipping the second or third check: treating a narrative as a controlled comparison, or having no route to firmer evidence when the stakes rise. ## What does a controlled comparison add that a narrative can't? A single AI-generated summary answers one question: what does this tool say about this audience. It does not, by itself, tell you whether one message, price, or concept performs differently from another, or how confident you should be in that difference. A controlled experiment answers that second question. That is the gap a causal behavioral platform is built to close. Subconscious runs controlled causal experiments, including discrete-choice-style tests, against a person-level audience graph covering 800 million real people, kept distinct from a recruitable panel. When a decision is expensive or public enough to warrant it, a study can move from that simulated experiment to real-human participant testing on the same causal question, without redesigning the study from scratch. ## What does this platform not replace? Stating a limit here is what lets a buyer check the tool before they rely on it. Subconscious does not automatically certify an output as valid, does not generate ranked next-best-action recommendations, and does not run financial or ROI modeling on top of a study's results. A team still has to decide what level of evidence it actually requires, and a platform result should be read as one input into that judgment, not a substitute for it. ## Put the checklist to work The first move for a research operations lead is not adopting a new tool. It is applying the four checks above to one live decision this week, and writing down which check the current process actually skips. From there, see how the causal-to-real-human validation path works, or review replicated study designs to see the audience-definition-and-comparison pattern applied in practice. --- # Managed Research Communities vs. Self-Serve AI Panels vs. Causal Experiments URL: https://subconscious.ai/blog/comparisons/managed-research-communities-vs-self-serve-ai-panels-vs-causal-experiments Published: 2026-08-19 Choosing a research operating model means choosing among three different jobs: a managed insights community that runs structured studies over weeks, a self-serve AI panel tool that produces exploratory persona conversations in minutes, and a causal experimentation platform that tests which action moves an outcome. Picking the wrong one either burns weeks a fast-moving question doesn't have, or lets a directional chat transcript stand in for proof it never provided. ## What a managed insights community answers A managed community or panel platform is built for depth. A research team designs the study, recruits or maintains a panel, fields it, and analyzes results against an established methodology. One established B2B insights-community platform describes exactly this model on its own site: data aggregation, structured research programs, and dashboard delivery built for organizations with a dedicated research function. The strength is rigor: statistically framed samples, defined instruments, and integration into an existing research and analytics stack. The cost is time. Getting from a research question to a defensible answer typically runs days to weeks. ## What does a self-serve AI panel tool answer? A self-serve AI panel tool lets a team build synthetic personas and hold structured or open-ended conversations with them. It is fast: a marketing, product, or sales lead can set up personas and get an exploratory read in the same sitting, without waiting on a research team. That speed is also its limit. A persona conversation is directional and hypothesis-generating. It is not a randomized comparison, it does not produce a causal estimate of which action changes behavior, and no self-serve panel tool should be treated as settling a pricing, messaging, or launch decision on its own. ## Where does a causal experimentation platform fit? Subconscious is the causal AI company. Randomized experiments on a simulation of your market, validated against real human behavior, tell you why people choose and which action drives the outcome. That places it between "get a directional read fast" and "prove it before committing capital": run a controlled, randomized comparison for the specific product, pricing, or messaging decision, then validate the result with real human participants. It can run controlled studies against a person-level audience graph covering 800 million real people. That is an audience graph for reaching people to study, not a pre-recruited panel of respondents standing in for causal proof. ## A three-way comparison These cycle times are planning examples for comparing operating models, not fixed quotes, guarantees, or a claim about any single vendor's current pricing or delivery time. ## Using more than one model without confusing them These three models are not interchangeable, and they are not always sequential. A team might run a self-serve panel conversation to sharpen a hypothesis, then design a randomized causal experiment to test the resulting alternatives, and reserve a managed community study for the long-running relationship work a single experiment isn't built for. What breaks is treating any one model's output as another model's proof: a persona's stated preference is not a causal estimate, and a causal estimate from a simulation is not a claim about a specific recruited sample unless validated against real human behavior. ## Limitations A causal experimentation platform does not replace a managed research community for long-running relationship-based programs. Exploratory persona conversations are useful for early hypothesis generation; they are not a source of causal proof, and no accuracy percentage from any vendor should be read as settling that question. Pricing scenario testing, confidence intervals, and automated recommendation outputs are not standing claims here; treat them as decision-specific outputs that depend on how a given study is designed. ## A useful next step Start with the decision, not the tool. Name the action under consideration, the audience, and the outcome that matters, then match the model to what the decision requires: a managed community for a deep relationship program, a self-serve panel conversation to sharpen a hypothesis, or a randomized experiment when the team needs to know which action moves the outcome before committing budget. Or talk through a specific decision. [figure: Three models with output and cycle time: managed community (structured findings, days-weeks), self-serve panel (directional themes, minutes-hours), causal platform (estimate of what changes outcome, hours-days).] --- # Why Your Next Campaign Brief Needs a Test Layer Before Creative Starts URL: https://subconscious.ai/blog/use-cases/why-next-campaign-brief-needs-test-layer-before-creative-starts Published: 2026-08-19 A campaign brief reads like evidence even when it is a guess. It names an audience, states a tension, and asserts a message will land. Creative teams build against those lines as if they were tested. Catching the untested assumption is cheaper before production spend, timelines, and message credibility are already committed to it. [figure: Two-column diagram: three brief claim types, audience definition, message claim, segment differences, each paired with what a controlled test can check and what still needs real-person validation.] ## Where Does Risk Enter a Campaign Brief? Every campaign brief makes at least three claims: who the audience is, what tension motivates them, and which message resolves it. A brand strategist writes these lines from experience, prior research, or a client's stated view of their own customer. None of that is wrong on its face, but none of it is tested either. The category is not shrinking: between 2024 and 2034, the U.S. Bureau of Labor Statistics expects the market research analyst and marketing specialist workforce to keep expanding. What is changing is how early in the process that judgment gets checked. Waiting until a concept test at the end of production is the expensive place to find out the brief's core assumption was wrong. ## What a Controlled Test Adds, and What It Does Not The fix is not asking one model to guess how an audience will react and writing that guess into the brief as fact. That produces a confident paragraph with no more grounding than the strategist's own assumption did. The useful version is a controlled experiment: define the audience segment the brief targets, present the specific tension or message claim, and measure how that audience responds relative to an alternative. That test tells a strategist whether a message claim holds up against a comparison, where a segment reacts differently than the brief assumed, and which objections the brief has not accounted for. A test that hides its limits functions as marketing. This one publishes its edges. It does not tell a strategist how the campaign will perform in market, and it does not replace validation with real people once the decision is expensive or the claim goes public. A directional read that narrows which assumptions are worth defending is not the same evidence tier as a documented result a client or regulator can rely on. ## Where Does This Fit Before Creative Starts? The brief itself is the natural checkpoint, because it is the last point where a wrong assumption is still cheap to change. Running the audience, tension, and message claims through a controlled experiment before the brief goes to creative catches the same problems a post-production concept test would catch, at a stage where fixing them costs a rewrite instead of a reshoot. ## The Failure Mode to Avoid Naming this failure mode here is what lets a buyer check for it before the brief ships. The dangerous version of this practice is a brief that sounds audience-tested without being audience-tested. That happens under deadline pressure: the team wants a confident line, a tool produces a fluent paragraph, and nobody labels which sentences are backed by a test and which are still assumption. The fix is not more confidence. It is naming, in the brief itself, what was checked, what was not, and what should be validated before a claim goes external. ## A Working Sequence for This Week - Take one live brief with a real decision behind it, not a hypothetical. - Write the assumption in one sentence: which audience, which tension, which message claim. - Run that claim through a controlled experiment against at least one alternative. - Read the result yourself. Mark what changes the brief, what is inconclusive, and what still needs real-person validation before the claim ships. - Send the brief to creative with the checked assumptions labeled as checked, and the open ones labeled as open. Teams that treat this as a habit rather than a one-off end up with something more durable than a brief: a documented reason for why the brief said what it said, which is the thing that survives a client's or a creative director's first hard question. ## The Bottom Line A brief that sounds consumer-led is not the same as a brief that has been checked against a consumer. The gap between those two things is exactly where campaign spend gets wasted. Testing the audience, tension, and message claim before creative starts does not remove the strategist's judgment about what a decision requires. It gives that judgment something to stand on besides confidence. See how this plays out in practice in prior work with brand teams, or read more about the company built around this discipline. --- # How to Evaluate Synthetic-Panel Tools Before You Commit Budget URL: https://subconscious.ai/blog/use-cases/how-evaluate-synthetic-panel-tools-before-commit-budget Published: 2026-08-19 The criterion that separates synthetic-panel and AI-panel vendors is not persona count or chat interface; it is whether the tool produces a directional read or a controlled, measured comparison. An insights, marketing, or product-research leader should decide which one the pending decision requires before evaluating any vendor. [figure: Branching diagram from "pending decision" into two paths. Left: open-ended chat reaction, mixing concept and framing. Right: controlled comparison, holding population fixed with a measured, uncertain effect.] ## Why is persona count the wrong shortlist criterion? Most tools in this category work the same way: assemble calibrated personas, ask them a question, and read the spread of responses. That workflow is not built to isolate which specific change in a message, price, or concept caused a shift in stated response, because nothing in the setup is held constant across alternatives. A team procures an open-ended, multi-persona chat product, then discovers only after the contract is signed that the decision in front of them, a pricing change, a claim substantiation, a launch call, needed a defensible, measured comparison rather than a directional impression. By then the evidence already shipped the decision, and reworking it costs more than the original evaluation would have. ## The architecture question underneath the marketing language A chat-style panel tool asks personas an open question and returns free-text or rated reactions. It surfaces themes and objections quickly, but the response spread mixes reaction to the concept with reaction to how the question was framed, since nothing about the population or the alternatives is fixed. A controlled experiment instead defines a fixed population, presents two or more defined alternatives to comparable simulated respondents, and measures the difference in a specific outcome, such as choice share, between them. Holding the population and framing constant is what lets the result be attributed to the alternative being tested rather than to noise in the setup. ## What the underlying research method has to answer for Discrete-choice methodology is not new, and it has an evidence base outside marketing research. Systematic reviews of discrete choice experiments in health-related decision research have examined how well they predict real, observed choices against a human baseline, and found that predictive validity depends on study design and context rather than holding uniformly (Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysis, PMC/National Library of Medicine; How well do discrete choice experiments predict health choices? A systematic review and meta-analysis of external validity, The European Journal of Health Economics). That is the standard a buyer should hold any vendor to, not a headline accuracy figure taken at face value. "Pooled sensitivity and specificity estimates were 89% (95% CI:77–95, I 2 = 97%) and 52% (95% CI:32–72, I 2 = 95%), respectively. The area under the SROC curve (AUC) was 0.81 (95% CI:0.77–0.84)." Zhang and colleagues, eClinicalMedicine (source) Subconscious publishes its own validation methodology and study-level benchmark results on the replication leaderboard, including mid-range scores and failure modes rather than only favorable cuts. Ask any vendor under evaluation for the equivalent. ## Three approaches, one table The category collapses into three broad approaches. Buyers should evaluate which one matches the decision. ## Four questions before any vendor conversation - What decision does this inform? Open-ended exploration is fine for early concept reaction. A decision that will be defended later, a price, a regulated claim, a launch call, needs a measured comparison with reported uncertainty. - What population does the decision depend on? A consumer pricing decision needs a demographically defined population. A B2B decision needs a population defined by role, industry, and deal context. Confirm the vendor can define and hold that population constant across the alternatives being compared. - Who owns the evaluation? A single analyst running exploratory panels has different requirements than an insights team that needs role-based access, procurement review, and a documented audit trail for every result. - How will the result be validated? Ask for the benchmark, the human baseline it was measured against, and the scope where the method is known to fail. A vendor that cannot answer this is not ready for a decision that needs defensible evidence. ## Running a pilot instead of trusting a demo Vendor confidence comes fastest from piloting against a real, upcoming decision rather than a hypothetical test case: a pricing question that is actually open, a claim under consideration. Teams often budget around two weeks for a paid pilot that includes a confirmatory comparison against ground truth; treat that as a planning reference rather than a promised timeline from any specific vendor. Score the pilot on calibration against that ground truth, speed to a usable result, whether the output is a directional read or a measured comparison, and whether the audit trail meets the buyer's governance requirement. ## How does Subconscious approach the same evaluation? Where most tools in this category produce an open-ended multi-persona chat reaction, Subconscious runs a controlled discrete-choice experiment: it compares defined alternatives across a defined population and returns a measured causal effect with a confidence interval. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people when population scale is the limiting factor in a decision. That audience graph is not a recruitable panel; it defines the simulated population an experiment can be run against. Where a decision requires confirmation from actual respondents, Subconscious can validate studies with real human participants, moving from a simulated experiment to real-human validation without changing the underlying causal question being asked. ## What does a causal comparison not settle? A controlled causal experiment answers which of the tested alternatives is more likely to move the outcome; it does not itself confirm that a specific vendor's implementation, contract terms, or data-handling practice meets a specific buyer's requirements, and it does not provide the audit-trail governance that regulated procurement, pharma, financial services, or government buyers require. Governance and traceability are evaluation criteria to confirm directly with any vendor, including Subconscious, rather than assumptions to make from the existence of a causal method. Real-human validation is a distinct claim from simulated audience reach, and neither one is automatic proof of market performance. A causal experiment on a simulated population estimates a likely effect; it does not replace the pilot, the human confirmation study, or the buyer's own judgment about whether the evidence is strong enough for the decision at hand. ## Deciding what to evaluate first Define the decision before the vendor list: is it early-stage exploration, or does it need a defensible, measured comparison. Then evaluate vendors against that requirement. See how Subconscious structures a controlled comparison at /how-we-work, or walk through the process directly at /demo. --- # Simulated Experiments vs. Respondent.io: Choosing the Right Research Model URL: https://subconscious.ai/blog/comparisons/simulated-experiments-vs-respondent-io-choosing-right-research-model Published: 2026-08-19 A CMO or head of insights weighing a near-term go-to-market call, positioning, pricing, a new concept, has two different ways to get evidence before committing budget: recruit real, screened participants through a marketplace like Respondent.io for moderated interviews, or run a controlled experiment on a simulated audience first and confirm the highest-stakes findings with real people afterward. The two paths answer different questions, so the choice should follow the decision, not the category label. ## Two Different Jobs, Not Two Competing Products On Respondent.io, a team posts a study, actual people apply to take part, and the team recruits from that applicant pool for paid interviews conducted one-on-one, focus groups, or usability sessions. That workflow produces raw human texture, unscripted reactions, complex B2B screening, and usability depth, that a simulation cannot substitute for. A controlled experiment on a simulated audience answers a different question: which action moves which outcome, before a team spends time and money finding out with real people. Subconscious is a causal behavioral platform. It runs controlled, randomized experiments on a simulation of a market to estimate causal effects, then can validate the findings that matter most with real human participants. ## What Each Model Is Built to Answer ## When Should You Use Recruited Participants Instead of a Simulation? Choose a marketplace like Respondent.io when the binding constraint is the texture of a real conversation, not a directional answer: usability research on a shipping product, complex B2B screening criteria a simulation can't approximate, or a decision expensive enough that only a confirmed human response will do. Respondent.io publishes its recruiting workflow and incentive structure for teams evaluating that path (Respondent.io). ## When Should a Simulated Experiment Come First? Choose a simulated experiment when the constraint is the number of questions a team needs to narrow before it's worth paying for recruitment and calendar time. Subconscious runs these controlled studies against a person-level audience graph covering 800 million real people, distinct from a recruitable panel of participants who have agreed to be interviewed. That scale supports testing many candidate messages, concepts, or price points quickly, before deciding which ones earn a real-participant study. ## How Do You Sequence Simulated Experiments and Real Participants? The two models compose well: run the simulated experiment to find which positioning, concept, or pricing question moves the outcome, then confirm the findings that will ship with real participants through Respondent.io or an equivalent marketplace. That keeps one throughline from hypothesis to confirmed decision instead of two disconnected studies. ## Where the Simulation Stops Being Enough For decisions where raw human texture, complex B2B screening, or usability-session depth is the binding requirement, real participants remain the right tool. Simulation stays an upstream narrowing step, not a replacement for the confirmation tier. ## Next Step Teams evaluating this sequencing can see how a controlled experiment narrows a decision before it reaches recruited participants, review research methodology behind the causal estimates, or book a session to test a positioning or pricing question against a simulated market first. [figure: A five-step path: an open go-to-market question feeds a simulated audience experiment, which narrows the findings that matter, which go to recruited human participants, which confirm the decision before it ships.] --- # AI Competitive Positioning Research URL: https://subconscious.ai/blog/use-cases/ai-competitive-positioning-research Published: 2026-08-19 Competitive positioning determines who a product is for, what problem it solves, and why a buyer should choose it over an alternative. Many teams revisit that decision once a year, even while competitors and customer priorities keep changing. AI-assisted research can shorten the cycle. The useful result is not an invented customer quote. It is a smaller set of positioning hypotheses that a team can test with buyers and in market. [figure: Five-step horizontal path: map the category, compare competitor perceptions, test differentiation, rank messages, simulate a competitive decision, ending in hypotheses marked for human validation.] ## The three positioning questions Clear positioning answers three questions: - Who is this for, specifically? - What problem does it solve for them? - Why is it preferable to the alternatives they could choose? Strong positioning does not claim to be best at everything. It creates a clear reason for a defined audience to choose one offer in a real decision. ## Why is competitive positioning research difficult? Positioning depends on how customers define the category, describe the problem, and compare alternatives. Surveys often flatten that language into preset options. Interviews capture more depth but take time to recruit, conduct, and analyze, a tradeoff documented in research on qualitative interviews versus surveys. The common failure is conducting the research once and reusing it for years while the market changes. ## Use AI to narrow the field ### Map the category Define the target buyer and ask how they group the available options, which attributes matter, and what triggers evaluation. Treat the answers as hypotheses about category structure. ### Compare competitor perceptions Use public information, reviews, and approved customer evidence to describe each alternative. Probe what appears credible, confusing, risky, or distinctive. A simulation should not invent private experience with a competitor. ### Test differentiation Compare positioning statements against the same audience definition. Check whether each claim is clear, relevant, credible, and distinct. Ask which behavior the statement should change, such as consideration, trial, or willingness to evaluate. ### Establish a message hierarchy Test multiple directions and identify which messages should lead for each segment. A supporting proof point should not be mistaken for the main position. ### Simulate a competitive decision Ask the audience to compare the offer with two or three competitors and explain the criteria used at each step. The comparison is useful when the inputs are grounded and the alternatives are realistic. ## Compare several audience types A positioning panel can include an ideal customer, a current competitor customer, a category skeptic, and a lapsed customer. Running the same protocol across multiple customer types can reveal a claim that works broadly, a claim that polarizes, or a segment that needs different language. The exercise can be completed in an afternoon. It should lead to focused validation, not a claim that months of customer research have been replaced. ## How often should you monitor positioning changes? Teams can run positioning checks quarterly or even monthly rather than annually. Keep the audience definitions and questions stable enough to compare results. A sudden change in simulated response is an early signal to investigate with real customer research, not proof that the market has shifted. [figure: Four panels show the same positioning claim tested against an ideal customer, a competitor's customer, a category skeptic, and a lapsed customer, each marked as held, polarized, or needing different language.] ## How do you turn findings into action? Extract candidate language, identify the strongest differentiation claim, and map possible competitive white space. Then narrow the decision to the top one or two directions and test them in real channels. For final positioning choices, validate with customer conversations and observed behavior. A useful hybrid uses AI to reduce two or three plausible options to a testable set, then spends human research on the decision that remains. Subconscious supports decision-specific experiments on messaging and go-to-market actions, run the way described on the how we work page. The experiment should define the audience, alternative positions, outcome, and validation plan before it runs. Book a walkthrough to design one around a specific positioning decision. --- # Structured Survey Tracking vs. Upstream Causal Exploration URL: https://subconscious.ai/blog/comparisons/structured-survey-tracking-vs-upstream-causal-exploration Published: 2026-08-19 A research-ops or growth-marketing lead running a recurring satisfaction or NPS program on a real respondent list faces a narrower question than "which tool is better": should hypothesis generation stay inside the structured survey tool, or move upstream to a causal experiment platform, leaving the survey tool for what only real respondents can confirm? ## What does picking the wrong layer cost? Running every exploratory question through a real-respondent survey cycle burns days to weeks of round-trip time and caps how many questions a team can test before a launch decision. The opposite mistake: treating a directional, simulated read as a substitute for real-respondent tracking data risks reporting an unvalidated number as ground truth. Both are decision errors that show up as a missed launch window or a false confidence in a metric nobody can defend. ## What is a structured survey tool like Survio built for? Survio is a survey-builder: templates, distribution to an existing list, and tabulated analytics (Survio feature tour). Teams use it to send structured questionnaires (multiple choice, scale, short open text) to email lists, customers, or recruited respondents, and publishes tiered plans for that workflow (Survio plans and pricing). That is the right layer when: - A customer or member list already exists and needs to be tracked for satisfaction or NPS at a fixed cadence. - Results need to tabulate cleanly across hundreds or thousands of respondents. - The research methodology depends on real, recruited participants: compliance-grade confirmation, not a directional read. A structured tracking survey answers questions a simulated panel cannot: what a specific, real population actually reports, on a schedule, in a form that can be audited. ## Where does a causal experiment platform fit instead? Subconscious runs causal experiments (discrete-choice-style tests with quantified uncertainty) on a simulated population, upstream of a survey send. The fit is narrow: testing which messaging, pricing, or positioning action moves an outcome before a team commits a real-respondent fielding budget to it. When the question is which of several actions a real audience would respond to, and no fielding budget is committed yet, that's the upstream step. When the question is "how satisfied is our actual customer base right now," that's a tracking survey, full stop. ## Comparing the two layers Use the table to place a specific open question, not to rank platforms. A team with a fixed NPS cadence and a defined list has already answered the "which layer" question in Survio's favor for that program. ## From a simulated read to real-human confirmation Subconscious can test or validate studies with real human participants, so a team can move from simulated exploration to real-human validation without changing the causal question under test. That closes the gap between the two layers: the same question that narrowed candidate actions upstream can carry into a fielded, real-respondent confirmation, without reformulating it for a structured survey or recruited-respondent study. ## Limitations A causal experiment platform does not replace a structured tracking survey, an NPS program, or recruited-respondent fieldwork: those remain the compliance-grade, real-population confirmation layer. A simulated, directional read should not be reported as if it were tracking-survey ground truth. And the scale of a causal platform's underlying population data is not the same as a recruitable respondent panel or a survey tool's own list-based distribution. ## Next step For a team weighing this decision, how Subconscious runs a causal experiment and published case studies show the exploration layer in practice; the research page covers the underlying causal method. [figure: Decision path, two branches. Undecided action: causal experiment platform, then real-human confirmation. Fixed audience and question: structured survey tracking directly.] --- # Pre-Testing Out-of-Home Ad Copy Before the Production Order Locks URL: https://subconscious.ai/blog/use-cases/pre-testing-out-home-ad-copy-before-production-order-locks Published: 2026-08-19 Test out-of-home (OOH) headline and copy variants against defined audience segments before the production order locks, not after internal review alone. A billboard, transit wrap, or airport takeover has no A/B test and no rollback once it is printed or installed. ## Why OOH Punishes a Wrong Guess More Than Any Other Channel Paid social can be paused, email subject lines iterated across sends, a website headline swapped in an hour. Out-of-home copy commonly runs for six weeks or more whether it lands or not. A copy line approved in a review meeting and sent straight to the printer carries the full cost of that six-figure production and media spend with no way to course-correct mid-flight. ## What Does OOH Copy Need to Do in Two to Three Seconds? OOH copy has to be decoded inside the reader's actual attention window, not the length of time a team spends debating it in a brief. A driver moving at fifty kilometers an hour gives a highway billboard roughly two to three seconds. Someone waiting on a train platform holds a poster's attention for six to eight. A bored passenger might stretch an airport backlit panel out to fifteen. Attention windows this short are a documented constraint of the medium, not a house style choice.[^attention] Copy written like a landing page headline routinely fails this test, and the failure is invisible until it is measured against how a reader actually processes the format, not how it reads on a screen in a conference room. Design and layout compound the problem. OOH copy never runs alone: it sits inside a specific typographic hierarchy next to an image, and a line that reads well in the brief can be undermined by the art direction around it, or a flat line can be rescued by a strong photograph.[^five-second] Teams that pre-test only the words, not the words inside the finished layout, miss this until the campaign is already live. ## Why Should Reading Context, Not Purchase Intent, Define the Test Segments? OOH is an upper-funnel format, so the useful audience split is by the conditions under which someone encounters the ad, not purchase intent. The same billboard is read by a commuter half-asleep on a familiar route, a first-time visitor with no context for the brand, and a local who has seen a thousand ads in that same spot. A copy test built around segments defined by exposure pattern surfaces failures a generic focus group would miss. ## A Test Sequence Built Around the Production Timeline, Not the Brief The misses go on the record next to the hits, so a team can see both before it plans against them. The staged sequence below is inherited from prior campaign practice; the timing is a planning reference, not a claim about how quickly any test runs on any platform. - Four weeks out: test headline candidates as plain text before any art direction is locked, and check whether each reading condition can state what the ad said within its actual attention window. - Three weeks out: test the headline inside the real layout, with image and logo in position, since the layout frequently changes what the copy means. - Two weeks out: test the copy against the actual install location, such as a highway, a metro platform, an airport gate, or a specific neighborhood bus stop, because the same line can read as confident in one location and tone-deaf in another. - One week out: if the campaign runs as a family of executions across a route or a city, test the full sequence for whether it builds or contradicts itself. - Three days out: run a final check for the worst reasonable misreading: what the line could mean out of context, and what local reaction a team working from headquarters would not anticipate. Teams that ran a version of this sequence report cutting word counts by roughly thirty to forty percent before landing on a version that reads in the available time, a planning benchmark, not a guarantee for any specific campaign. ## How Does Subconscious Run This as a Causal Test? Subconscious runs randomized experiments comparing OOH copy and headline variants across defined audience segments and exposure contexts, estimating which variant drives comprehension or recall before the production order is locked. Naming what a test does not cover is what lets a buyer check it against the decision at hand. This is a decision-specific causal test of a small number of copy variants against a stated outcome, not open-ended creative feedback. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, and a team can move from a simulated study to real-human validation on the same causal question without changing what is being tested. The audience graph is a basis for controlled comparison, not a recruitable panel of real respondents held in reserve. ## Where This Method Stops Subconscious tests the causal effect of copy and creative variants on a defined outcome (comprehension, recall, message clarity) under a set of viewing conditions. It does not replace a physical mockup review or in-location context check on the finished production. It does not predict OOH media-plan return on investment, and it does not output an automated recommendation on which locations to buy. ## Next Step The same discipline applies to any high-production-cost, no-rollback asset, from a thirty-meter billboard to a four-by-six postcard mailer, including event signage, trade show booths, and packaging refreshes. The useful moment to run this test is before the headline candidates go into final art direction. See current OOH and creative research or review past causal experiments, and book time to scope a test against your next campaign brief. [figure: A five-step path from four weeks before production to three days before: bare text, then the real layout, then the install location, then the full campaign sequence, then a check for the worst plausible misreading.] [^attention]: Billboard Advertising Statistics 2025, Signs.com. [^five-second]: Billboard Design & The Five-Second Rule, Effortless Outdoor Media. --- # Routing Time-Sensitive Decisions Around the Standard Research Pipeline URL: https://subconscious.ai/blog/implementation-and-procurement/routing-time-sensitive-decisions-around-standard-research-pipeline Published: 2026-08-19 A standard custom market research project runs six to eight weeks from launch to final deliverable, and many projects run longer once scope, stakeholder review, or multi-market fieldwork are added (Drive Research; MarketResearch.com). That timeline does not match a two-week product sprint. The decision is not whether the standard pipeline is rigorous -- it is, for discovery and foundational work -- but which decisions can no longer wait for it. ## Why this decision matters When every decision routes through a six-to-twelve-week pipeline, positioning, pricing reactions, and competitive responses ship without evidence. The opposite failure is just as real: treating every question as fit for a same-cycle answer strips out the open-ended, qualitative depth that longer human fieldwork provides for foundational or high-ambiguity questions. The buyer's job is to sort decisions into the right lane, not to replace one lane with the other. ## Where does the standard research cycle spend its time? A typical custom study moves through four phases, each with a distinct, structural cause for its length. Recruitment is usually the largest single bottleneck. A study that needs 20 qualified interview participants can require screening on the order of 200 candidates and confirming roughly 25 to get 20 people who actually show up, because niche audiences, screening criteria, time-zone scheduling, and no-show rates all compound against the recruiter. Fieldwork is bounded by human bandwidth: a researcher can conduct about three to four interviews a day before quality degrades, and a focus group needs a venue, a moderator, and six to ten participants available at the same time. A study built on twenty interviews can generate fifteen to twenty hours of recordings before a single theme gets coded. ## What changes when recruitment leaves the critical path? Subconscious runs controlled, randomized experiments on a simulation of the market instead of recruiting and scheduling human participants for every study. That removes recruitment and sequential fieldwork from the critical path for decisions that need a fast, evidence-based answer. A simulation can be set up in about eight hours, and once it is live, individual experiments can run in under five minutes. When a study calls for it, the same experiment can also be validated with real human participants without changing the underlying causal question. This does not replace discovery work; it changes which decisions have to wait for a full fieldwork cycle. ## A routing framework for time-sensitive decisions Use the standard pipeline for foundational and high-ambiguity work: brand positioning at the category level, market sizing, and open-ended discovery where the value comes from letting a conversation go somewhere the researcher didn't anticipate. Route a decision toward a controlled experiment when it is time-sensitive and the question is narrow enough to state as a testable hypothesis: which positioning resonates with a specific buyer segment, whether a feature name confuses an enterprise audience, how a target segment reacts to a pricing change, or which of several campaign concepts should get built out. These are the questions that get skipped today because the standard pipeline can't turn them around before the decision window closes -- not because they matter less. Two approaches can also shorten the standard pipeline without changing its method: narrowing the scope of a single study to one hypothesis instead of a broad exploration, and running short, sprint-aligned research cycles of about one to two weeks that deliver findings in a brief working session instead of a formal report. Both reduce turnaround; neither removes recruitment and fieldwork from the critical path the way a controlled experiment on a market simulation does. ## What does this article not cover? Confidence intervals, segment-level breakdowns, and decision-ready memos are outputs that depend on how a given study is designed, not standard deliverables of every study. Continuous, always-on feedback mechanisms such as in-app prompts or support-ticket analysis can supplement either lane, but they tend to show what customers are doing more than why, which is the gap a controlled experiment is built to close. Pricing, packaging, and catalog-specific workflows are not covered by this article. [figure: A question splits on two criteria: foundational or high-ambiguity goes to the standard pipeline; time-sensitive and narrow goes to a controlled experiment on a simulation.] Before committing a decision to either lane, review current research methodology and how a study gets structured, and look at case evidence from studies that have already run through both a simulation and a human validation step. ## Sources - How Long Does Market Research Take? -- Drive Research - Considering Custom #3: Custom Market Research Project Structure & Timeline -- MarketResearch.com --- # 6 Reasons Focus Groups Give Unreliable Answers (and What to Test Instead) URL: https://subconscious.ai/blog/use-cases/6-reasons-focus-groups-give-unreliable-answers-what-test-instead Published: 2026-08-19 Focus groups give unreliable answers because groupthink, moderator framing, recruitment mismatch, social desirability, and small samples distort what a shared room reports, and controlled experiments measuring independent choices test the decision more reliably. An insights or product leader weighing a concept, message, or roadmap change has to pick a research method that won't let one dominant voice, a leading moderator, or a flattering answer stand in for what people actually do. Get that choice wrong and the team ships a roadmap built on a finding that reflects social pressure and small-sample noise, then misses the sales targets it predicted. The format dates to the 1940s and hasn't changed much since: put 6 to 10 people in one room, ask questions, and observe through a one-way mirror. It looks rigorous. It produces unreliable data anyway, and most teams never notice, because the output feels qualitative and convincing. Convincing is not accurate. [figure: Diagram of five paths (groupthink, moderator framing, recruitment mismatch, social desirability, small-sample noise) each running from an individual's real opinion to a distorted focus-group finding.] ## Six reasons focus groups misread the room ### 1. How does groupthink kill honest feedback? When several people share a room, opinions drift toward whichever one got stated first, loudest, or with the most conviction. Example: a consumer electronics brand tests a new smartwatch design. One participant opens with, "I love the rounded edges." Five of the remaining seven spend the following 20 minutes nodding along or restating a version of it. The two holdouts who dislike the design say nothing further. The research report reads "strong preference for rounded edges." Solomon Asch's conformity experiments showed that people will give an answer they know is wrong to match a group, the dynamic behind every focus-group session. ### 2. How does moderator bias shape the outcome? Everything about how a moderator asks, from tone and phrasing to body language and which follow-ups get asked, nudges what participants say, even when the moderator is trained and well-intentioned. Example: opening with "How do you feel about this product's premium price?" already frames the price as justified before anyone answers. Asking "How do you feel about paying €299 for this?" instead draws a more honest reaction from the same room. Moderators also tend to follow up eagerly on answers that confirm the client's hypothesis and move past answers that don't. It's usually unintentional, but consistent enough to be a structural bias rather than a one-off slip. ### 3. Recruitment bias means the wrong people showed up Focus group participants aren't a representative sample of a target market: they're a self-selected group who respond to a recruitment ad and are available during business hours. Example: a B2B software publisher recruits "IT decision-makers." The people who show up skew toward freelancers and consultants, not corporate CIOs, because CIOs are too busy and don't need the incentive. Repeat participants compound the problem: some take part in study after study and learn what a "good" answer sounds like, so their feedback reflects experience with focus groups, not with the product being tested. ### 4. How does social desirability bias make people perform? People want to look good in front of strangers: in a focus group, that means participants overstate the positive and understate the negative. Example: a health food brand asks participants about their eating habits. They describe diets heavier in vegetables and lighter in fast food than what they actually eat, and claim enthusiasm for a new organic snack bar. After launch, sales are flat, because stated preference never matched purchasing behavior. The distortion is strongest on socially judged topics: health, sustainability, finances, education, parenting, anywhere a "right answer" is obvious to the person giving it. The effect is documented broadly in survey methodology research on social desirability bias. ### 5. Small samples produce noise, not signal Sessions typically run 6 to 10 participants each, and most studies use 2 to 4 groups, adding up to somewhere between 12 and 40 participants total, not a sample size that supports statistical inference. Example: a retail brand runs three groups (24 people total) and reports "70% prefer packaging option A." At 24 people, the margin of error runs about 20 points in either direction, so real preference could land anywhere from 50% to 90%. That's not an actionable number. Focus groups are qualitative by design but get used to justify quantitative decisions anyway: "most participants said X" becomes a business argument, even when "most" describes 5 out of 8 people in one room. ### 6. Cost and turnaround make iteration impossible A traditional agency-run focus-group study can take 4 to 8 weeks from briefing to final report once recruitment, venue, moderation, transcription, and analysis are all included. That timeline pushes focus groups toward validation instead of exploration: teams pick a direction first, then go looking for confirmation. That's backward: the direction is exactly what should still be in question when testing starts. ## What tests the decision instead of just describing it None of the six problems above get fixed with a better moderator or a bigger incentive. They're structural, built into the shared-room format by design, so the fix is removing the room, not running it more carefully. A controlled experiment does that: each respondent decides independently, with no group in the loop to produce groupthink, no moderator framing the question live, and no audience to perform for. What gets measured is the decision itself, meaning which option someone actually picks under a randomized, controlled setup with a holdout group and a confidence interval around the result, checked where useful against a real-human baseline, rather than a stated opinion vulnerable to the say-do gap the examples above illustrate. Running many independent evaluations in parallel also removes the sample-size ceiling that caps a focus group at 12 to 40 people, making a real confidence interval possible instead of a plus-or-minus-20-point guess from three sessions. ## Where this doesn't replace real conversation This approach does not replace real customer conversations for discovery or relationship-building, and simulated respondents are not a recruited real-human validation panel. Early-stage discovery, meaning understanding a problem no one has framed yet, or building a relationship with a design partner, still needs a human on the other end of the conversation. Once the question is concrete enough to test as a choice between options, an independent, controlled comparison answers it more reliably than a room does. [figure: Chain diagram: 3 focus groups totaling 24 participants produce a reported 70% preference for option A; a margin of error of plus or minus 20 points widens that number into a true range of 50% to 90%.] ## Next step Read how a controlled experiment replaces group discussion with independent measurement, browse case studies of concept and message tests run this way, or see the underlying method on the research page. To test a specific concept, message, or roadmap decision, book a walkthrough. --- # Why Running Surveys Is No Longer Enough URL: https://subconscious.ai/blog/methods-and-validation/why-running-surveys-no-longer-enough Published: 2026-08-19 Running surveys alone stops being enough once AI can field the same instrument cheaply, leaving evidence-tier judgment and stakeholder defense as the researcher's real, remaining work. A research leader who only fields surveys is exposed the moment software can field the same instrument at lower effort. The protected work was never typing the questionnaire or formatting the deck. It is choosing which evidence tier a business decision needs, then defending that choice when a stakeholder wants an answer today. ## Why is the pressure showing up now? AI has moved from a novelty layer into daily research workflows, and teams lean on it for self-service insight, quick data prep, reporting, and analysis. That doesn't mean demand for research is disappearing: growth is still expected in the field, with the BLS occupational outlook putting market research analysts and marketing specialists on an upward path from 2024 through 2034. The real risk is narrower, not "AI replaces researchers": the researcher known mainly as the person who fields the survey gets bypassed once the mechanical parts of that job are easier to access elsewhere. When that happens, the job has to move closer to the decision: better questions, better evidence choices, better caveats, better influence on what the business does next. ## What actually changes for researchers? Research expertise used to rest partly on access - the know-how to source data, run the study, tidy up responses, read a chart correctly, and package a finding. AI is eroding that edge. More people can draft a survey, summarize a transcript, generate a persona, or ask an AI panel for a first reaction. None of that makes expertise beside the point - it puts expertise under a spotlight. When anyone can generate an answer, the person worth paying for is the one who can tell which answer holds up, and who notices when a story is generic, thinly sourced, or beside the point for the decision the business faces. ## Build an evidence system, not an AI habit In 2026, the researchers who keep their footing won't be whoever has the biggest toolkit. They'll be the ones who can state plainly what each tool is, and isn't, allowed to prove. That shows up as four layers of a working system: - Exploration. Turn to AI for hypotheses, objections, alternate routes, and other possible explanations. - Directional testing. Compare options through AI panels or simulated audiences before any budget goes toward a fielded study. - Human review. A person looks over the work: is the audience defined correctly, was the prompt free of leading bias, does the sourcing hold up, and does it fit the business context. - Validation. When a decision is costly or public, bring in data from actual respondents, behavioral signals, an expert reviewer, or a fielded study. The value is not the AI output by itself. It is the disciplined path from a question to a decision the business can act on safely. [figure: A decision path showing four evidence tiers in order: exploration, directional testing, human review, and validation, with a routing rule for which tier a business question needs before anyone can act on it.] Subconscious sits inside that same discipline at the directional-testing layer: it runs a controlled discrete choice experiment to test a concept, a message, a pricing story, or a strategic assumption before the slow or expensive part of research begins. When a decision is expensive enough to require it, the same causal question can move to real-human validation, carrying the causal question forward into the next design iteration rather than starting over (research). ## A practical workflow for using AI panels Begin from the decision itself: note what would change depending on which way the research points. From there, define the audience - segment, context, current behavior, alternatives, and the outcome that person is chasing. A panel is only ever as good as the brief it's built on. Aim the panel at exactly one thing under test - it might be a creative concept, a piece of messaging, a pricing narrative, a campaign direction, a feature concept, a moment in the customer journey, or a strategic bet the team is making. Surface confusion, pushback, side-by-side comparisons, gut reactions, and clues about what would build more trust in the idea. Never treat the first response as final: follow up, weigh different segments against each other, and watch for places where answers don't agree. After that comes the manual part: go through every response, cut what's a repeated generic theme, and separate the hypotheses worth pursuing from data that isn't yet evidence. Sort the outputs into what's safe to treat as exploration versus what still needs real validation. Label the output honestly at the end. Tags such as "directional read, panel-sourced," "AI-assisted hypothesis, not yet confirmed," and "needs validation before any external use" strengthen the method's credibility instead of undercutting it. ## Is more volume the same as validity? Piling up more responses feels like a better decision, even when it isn't - the trap. The confusion tends to show up under pressure: deadlines push the team toward speed, the tool hands back something that sounds fluent, and the deck still needs a tidy conclusion. What separates credible research is a clear line between an output and actual evidence. AI can produce something useful, but whether that output fits the decision at hand isn't something it can judge by itself. Get around that by building limits into the deliverable itself: spell out where the AI-assisted work applied, where it didn't, and what still needs validating. That researcher's confidence won't come across as weaker for it - it will read as more professional, since they can point to exactly why their certainty stops where it does. Under the ICC/ESOMAR International Code on Market, Opinion and Social Research and Data Analytics (2025), that kind of disclosure is part of what responsible research practice already expects. ## What to do this week Skip the full job rewrite. Instead, pick one visible workflow to start with: - Pick a real project with a live decision. - Write the business decision in one sentence. - Define the audience and the risk level. - Use AI or an AI panel only for the exploratory stage. - Go through the output by hand, flagging each piece as useful, weak, or unsafe. - Deliver the answer alongside an explicit caveat and a suggested next step for validation. Before drafting a single question for an upcoming survey, put in writing the three decisions that survey has to support. Keep doing that weekly across a full month. By the end, the result is a working research system, not a longer list of AI tools. If the decision is big enough, a case study showing the same causal question carried from a directional read through to validated results is worth requesting as a template. [figure: Four steps: name the decision that depends on the answer, define the audience, aim the panel at one concept under test, then manually cut generic themes and sort what's left into exploration versus real evidence.] ## The bottom line AI is changing the shape of research work: it lowers the effort for production and first-pass analysis, and gives stakeholders a way to bypass established process. Human judgment in research and strategy is still as necessary as it ever was. What shifts is the shape of the safer role: closer to the decision, fluent in AI, strict about evidence, and clear about what still needs validation before anyone acts on it. Teams that want a walkthrough of where that line sits can book time rather than guess at it. --- # AI Research for Product Teams: Testing Decisions Before You Build URL: https://subconscious.ai/blog/use-cases/ai-research-product-teams-testing-decisions-before-build Published: 2026-08-19 Product teams make dozens of small decisions a sprint. Which feature ships next. Where it sits in the queue. What gets dropped, how it gets framed, and what it ends up called. Most of them happen without customer input, because the usual research methods do not fit the pace of product work. Product managers commonly report that a large share of their time goes to work outside core product strategy, which is part of why customer input gets pushed out of the sprint. ## The gap between fast and meaningful Product research tools sit on a spectrum. On one end: analytics dashboards, session recordings, support ticket analysis. Fast and data-rich, but limited to describing history. On the other end: moderated usability studies and customer advisory boards. High-quality signal, but slow to schedule and coordinate. In between sits most of what a product team decides day to day: what goes into the next sprint, how a feature gets framed, what a pricing tier includes. These calls get made in Slack threads and design reviews, with no realistic path to a full research cycle before the decision is due. A controlled discrete-choice experiment closes part of that gap. Instead of asking a panel to describe a reaction, it puts a specific decision (this name against that name, this tier structure against that one) in front of respondents and measures which option changes their choice, with confidence intervals where the design supports them. ## When does a causal test fit in the product cycle? Not every product question needs an experiment. A causal test earns its place when a decision commits real engineering or design time and the team is choosing between distinct, nameable options rather than exploring an open-ended concept. - Feature concept, before the spec. Two or three competing framings of the same feature, tested against each other, before anyone writes a spec. - Naming and framing. A feature label or plan name where the wrong choice creates confusion that costs support tickets and churn later. - Pricing tier structure. Which combination of features and price points changes which tier a buyer picks. - Pre-ship launch flow. Whether a specific onboarding sequence changes early activation versus the alternative. Each is a comparison between concrete options already in hand, not an open-ended opinion. A discrete-choice experiment answers that shape well: it isolates one variable at a time and reports which change moved the outcome. ## What does a causal test replace, and what doesn't it replace? A causal test, run against the audience graph, tells a team which of the options in front of them changes buyer choice, and by how much. It is not a substitute for direct usability observation, moderated interviews, or in-product analytics, which surface friction a choice experiment is not designed to catch. The two are complementary: usability research finds where people get stuck; a causal test tells you which candidate fix changes what they choose. ## From simulation to real people, without changing the question Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That scale supports segment-level comparisons, such as a power user against a casual user, inside a single study, which matters when a naming or pricing decision lands differently across segments. When a decision warrants it, Subconscious can also test or validate studies with real human participants. A team can move from a simulated study to real-human validation without changing the underlying causal question, so a pricing or naming comparison run early in a sprint stays comparable to the version validated before a launch decision. ## What does skipping the check cost? The failure mode this replaces is familiar: a team builds, names, or prices a feature on internal assumptions, ships it, and finds out only afterward that real customers see it differently. That shows up as engineering and design time spent on the wrong version of a feature, time a short causal test run before the spec was finalized could have flagged. ## Where this fits, and where it doesn't This approach earns its place in decisions with distinct, testable options and real cost attached to guessing wrong. It is a poor fit for open-ended discovery, figuring out what to build rather than choosing between options already on the table. [figure: A decision path showing four points in a product sprint where a causal test applies: feature concept before the spec, naming or framing, pricing tier structure, and pre-ship launch flow.] [figure: Two-column comparison. Left, usability research: observes real use, surfaces friction. Right, causal test: compares named options, measures which change moves choice. Center note: complementary, not substitutes.] ## Next step Teams that want to see this against a live decision can review how a study runs end to end or look at worked comparisons from other product and marketing teams. For a specific upcoming naming, pricing, or launch decision, book time to scope a test against it, or start from the research methodology if the causal-inference details matter first. --- # Always-On Simulated-Customer Chat or a Causal Experiment? URL: https://subconscious.ai/blog/use-cases/always-simulated-customer-chat-causal-experiment Published: 2026-08-19 Choose an always-on simulated-customer chat when the team needs hypotheses, language, or reactions to investigate. Choose a controlled experiment when the decision is which price, message, feature, or launch action to fund. The cost of confusing the two is a confident go or no-go call that never measured the behavior at stake. [figure: Two-column chart contrasting simulated-customer chat and controlled experiment across four rows: primary question, unit of evidence, best use, uncertainty handling.] ## The purchase is really a choice of evidence An ongoing conversation interface is designed to generate responses. A team describes a concept, price, or campaign and asks a simulated customer how it might react. That can help the team surface objections, vocabulary, and questions for further research. A controlled experiment is designed to compare interventions. The team defines the action, alternative, population, outcome, and conditions held constant. The result estimates which action is likely to change the outcome and reports uncertainty where the study design supports it. A number without its limits is marketing copy. Plausible language is not causal evidence. Research on generated social data has found that fluent responses can fail to preserve important properties of human data, with failures that are difficult to detect from the text alone (Synthetic social data: trials and tribulations, arXiv). For a Q2 2026 planning cycle, the practical question is not which interface feels more responsive. It is which evidence can support the decision the team must make. ## Four questions to ask before signing ### What action will the result change? Name the actual choice. “Learn what customers think” is a research objective, not an action. “Choose message A or message B for the launch” is a decision that can be tested. ### What changes between alternatives? A useful test changes the intervention and holds the causal question steady. If the options, audience, or outcome shift during the study, the result cannot cleanly answer which action caused the difference. ### What behavior will count as the outcome? Preference, purchase choice, adoption, trust, and switching are different outcomes. Define the one that would change the business decision before evaluating the research instrument. ### What would make the result credible enough to act? Ask how the method is calibrated, what was compared, how uncertainty is handled, and where it has failed. A polished answer is not a substitute for a stated validation standard. ## The causal decision layer Subconscious is built for action questions. It uses causal experimentation and discrete-choice-style modeling to compare product, pricing, messaging, and go-to-market actions before a team commits capital. Every score here sits on the public leaderboard next to its misses. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. A live simulation can be established in about eight hours. Once live, a configured experiment can run in under five minutes. Those figures describe setup and experiment runtime. They do not include exploratory discovery, decision framing, or recruitment of real participants. ## Keep discovery and human validation in the research system A controlled experiment does not decide what the team should build from a blank page. Direct customer conversations, open-ended qualitative work, subject-matter judgment, and exploratory research still produce the alternatives worth testing. Subconscious can test or validate studies with real human participants. When the stakes warrant that check, the intervention and outcome can remain fixed while the population changes from simulated buyers to recruited people. Naming the failure mode here is what lets a buyer check it before relying on the result. It does not turn the initial experiment into automatic proof of market performance. Audience-graph reach is a targeting layer, not a pool of recruitable participants. Recruitment is a separate operational step and should be evaluated as such. ## A procurement rule that survives the demo Buy the conversational instrument when the job is to explore what a customer might say. Buy the causal instrument when the job is to choose which action to take. If both jobs matter, use each for the question it can answer and do not collapse generated reactions into behavioral proof. Review the research standard and the public benchmark record before treating any result as decision evidence. To frame a price, message, product, or launch comparison, book a working session. --- # Testing Ad Creative Before You Commit Media Budget URL: https://subconscious.ai/blog/use-cases/testing-ad-creative-before-commit-media-budget Published: 2026-08-19 A performance or brand marketing lead has three creative directions ready for a campaign. The standard playbook is to launch all three, watch the results, and cut the losers once the data comes in. That spends the losing directions' budget before anyone knows they're losing. The question worth asking earlier is which direction to fund, not which one merely got clicks after the fact. Ad effectiveness is hard to predict from a brief or a mockup. The IAB's ongoing research into digital ad spend and effectiveness tracks how much of that uncertainty follows advertisers into live campaigns, where measurement only starts once spend is already committed. ## Where the decision gets made too late In the conventional workflow, creative direction gets tested by running it. A variant either converts once it is live, or it does not, and the team finds out after the media budget behind it is spent. The campaign spends against its own answer before it has one. Moving the test earlier changes what "finding out" costs. A controlled comparison of creative directions against a defined target audience, run before any variant reaches a live auction, turns "which one performed" into "which one to fund." The decision arrives before the spend, not after it. [figure: Four labeled checkpoints in sequence: earns the first look, survives the objection, lands the right tone, makes the next step clear, each testable against a defined audience before a creative direction gets funded.] ## What a pre-spend test actually checks A creative direction fails for a specific, diagnosable reason. Four checkpoints cover most of them. ### Does it earn the first look? Before a viewer reads a word of body copy, the hook has to earn a reason to keep looking. Testing against a defined audience surfaces not just which headline or opening frame gets attention, but why: the specificity of a claim, the tension of an open question, or the credibility of who appears to be speaking. ### Does it survive the obvious objection? Attention alone does not convert. The ad also has to preempt the reason the audience would talk itself out of acting: price sensitivity, skepticism about the category, or doubt that the product does what it claims. A direction that earns attention but ignores the standing objection stalls at the same point every time. ### Does it land the intended feeling Tone is a design choice, not an accident, and it is testable. A direction meant to convey urgency can read as anxious; one meant to convey authority can read as cold. Checking the emotional register against the target audience catches that gap. ### Is the next step unambiguous An ad can capture attention, handle the objection, and land the right tone, and still fail to tell the viewer what to do next or why to do it now. This checkpoint is the simplest to test and the easiest to skip. ## Running the comparison The workflow has a consistent shape regardless of channel: - Define the audience. Build a specific, decision-relevant description of who the campaign is actually trying to reach. A vague audience produces vague feedback. - Draft distinct directions, not minor variants. Test strategically different approaches, such as one leading with price, one with speed, and one with proof, rather than small copy tweaks. - Run the controlled comparison. Evaluate each direction against the defined audience across the same checkpoints, so the result is a like-for-like comparison rather than a set of one-off impressions. - Revise, don't restart. Use the result to identify what specifically needs to change in the weaker directions rather than discarding them outright. - Confirm with a narrow live test. Launch the strongest one or two directions in market rather than the full original set, and let the live campaign settle the final call between them. When the stakes justify it, this comparison can move from a simulated experiment to real-human validation without changing the underlying question, checked against recruited respondents before the largest commitments are made. ## Where is this worth doing? Scope limits belong up front, where a buyer can check them before committing budget. Pre-spend testing pays off most clearly under specific conditions, not universally: - A test that can't move the metric that matters should say so before a team runs it. Creative is the actual constraint. If a campaign's ceiling is set by targeting or bid strategy rather than the ad itself, testing creative direction won't move the number that matters. - The audience is already defined. A well-specified audience gives a controlled test something concrete to compare against. An audience still being discovered is better served by live testing built for exploration. - Several directions are genuinely in play. The value compounds with the number of distinct directions under consideration; a two-variant decision has less to gain than a five- or ten-variant one. - The product needs explaining. Complex or considered purchases, including many B2B products and multi-stakeholder services, put more weight on the creative's ability to communicate, which is exactly what these checkpoints test. ## Beyond paid social The same logic holds outside Facebook and LinkedIn feeds: search ads, programmatic display, out-of-home, print, and broadcast all commit a media budget to an unproven creative direction unless something checks that direction first. The channel changes; the underlying question of which direction is worth funding does not. [figure: A five-step horizontal path: Define audience leads to Draft distinct directions, then Run controlled comparison, then Revise don't restart, ending at Confirm with narrow live test.] ## What this doesn't replace A method that claims to replace live testing hides its own limits. Publishing what this comparison does not do lets a buyer check it against results. A pre-spend comparison narrows which direction is worth funding. It does not replace live campaign measurement, media buying, or creative production, and it does not guarantee how any single variant performs once it reaches a real market. Teams that want to see the comparison method applied to real campaigns or that want to walk through how the controlled comparison is built can start there, or talk through a specific creative decision before the next campaign locks its budget. --- # What Is Synthetic Market Research? A Buyer's Guide to the Method and Its Limits URL: https://subconscious.ai/blog/use-cases/what-synthetic-market-research-buyers-guide-method-its-limits Published: 2026-08-19 Synthetic market research conditions AI personas on demographic and behavioral inputs, then uses them to model the way a defined consumer or B2B audience might react to a survey, a concept test, ad creative, or messaging variants. The participants are modeled rather than recruited: a team describes the audience, configures the personas, and runs the session against a language model. The method is also called AI market research, simulated market research, or virtual market research. The real decision it raises for a research or marketing leader isn't whether the method works. It's where in a research pipeline a simulated answer is good enough, and where the cost of being wrong requires a controlled experiment on real people before money moves. ## Where the method comes from The intellectual lineage is academic. In a 2023 study, Argyle et al. showed that a language model conditioned on a real survey respondent's demographic backstory could generate opinions whose spread lined up closely with how real Americans had actually answered benchmark surveys such as the ANES (Political Analysis, Cambridge University Press). That paper established silicon sampling as a viable technique, and commercial platforms packaged it into personas, panels, and workflows for marketing, product, and insight teams. ## What a synthetic-first workflow looks like A typical simulated study breaks into five steps: - Define the audience. Age range, geography, household income, job role, industry, attitudes, prior brand exposure. The more specific the definition, the more useful the simulation. - Configure personas. Assemble individual simulated participants into a research panel. Many teams run 50 to 500 personas per study, calibrated against any real-respondent data already held for that audience. - Design the research instrument. The same survey, concept-test brief, or ad-pretest stimulus a team would field traditionally. - Run the session. Each persona answers in natural language. Closed questions produce structured output; open prompts produce responses a team can read and theme. - Synthesize and decide. Compare segments, identify a leading concept or message, and decide whether the finalists need a real-respondent check before the team commits budget. A method is only as useful as the boundary published next to it. The loop produces a directional simulated result. It does not produce evidence of actual behavior. ## When directional screening is useful Five situations, drawn from how research and marketing teams use the method: - Directional narrowing. Cutting a dozen concepts down to a shortlist before commissioning real-respondent work. - Iterative exploration. Applying the same instrument to successive concepts while treating the output as directional. - Specialized audiences. Modeling hypotheses about senior B2B buyers, regulated professionals, or niche geographies before validating the decision on real people. - Cross-market comparison. Applying the same instrument across several country definitions and comparing directional differences under a consistent setup. - Sensitive topics. Exploring hypotheses about health, finance, or employment before designing research involving real participants. ## Where correlation stops answering the question A correlation score by itself is a marketing claim; the boundary around it is what makes it usable. A simulated response can track what a real audience tends to say. It does not, by itself, tell a team what happens when a specific input changes: a price, a headline, a feature. That is a causal question, and a correlation between simulated and real answers on past surveys is not evidence about a causal effect the team hasn't tested yet. Naming these failure modes is what lets a buyer check the method against them before budget moves. Three limits follow from that gap: - Statistical validation. A simulated study does not produce a population estimate with a defensible confidence interval on its own. - Genuinely novel behavior. A persona only echoes patterns already present in the model's training. It can't be trusted on a product, category, or event that has nothing comparable in that history. - Final go/no-go calls. Capital allocation, regulatory filings, and public claims should not rest on simulated data alone. The cost of skipping this step shows up later: a team that commits budget on the strength of a simulated correlation, without checking it against real holdout behavior, finds out the difference only after the launch. ## Moving from directional screening to a causal test The mature pattern is hybrid: use simulation to narrow the field, then run a randomized, controlled experiment on the audience in question for the option that will get funded. That second step is where Subconscious fits. Subconscious can test or validate studies with real human participants, and it can run controlled studies against a person-level audience graph covering 800 million real people, kept distinct from any panel a team recruits directly. The result of the controlled step is a causal effect with a confidence interval, not another correlated guess. What Subconscious claims sits on the public record next to what it declines to claim. Subconscious does not replace direct observation of real behavior, and it does not claim any specific accuracy percentage against real respondents. ## Privacy and compliance Simulated participants are generated, not recruited, so a session usually has no real personal data to process in the first place, and that removes much of the consent and data-retention complexity traditional fielding carries. For organizations with strict compliance requirements, such as healthcare, finance, or the public sector, that can make simulated screening easier to deploy early. ## Adjacent terms, clarified - Synthetic data. Artificial datasets built to train models, or to pad out a real sample that's too small. A related but different problem. - AI personas. The individual unit inside a simulated research panel. The persona is the agent; synthetic market research is the method built around it. - AI focus groups. The qualitative format of simulated research, where personas respond as a group rather than individually. - Multi-turn persona research. A newer extension where AI personas act and react to follow-up stimuli rather than only answering a fixed instrument. ## Frequently asked questions ### What is synthetic market research? A method where AI-generated personas stand in for a defined consumer or B2B audience, so a team can gauge how that audience might react to surveys, concepts, ads, or messaging. Each persona is conditioned on demographic, psychographic, and behavioral inputs and queried in natural language, without the recruitment and fielding a traditional study requires. ### When should a team use simulation versus a controlled experiment on real people? Use simulation for directional input: early concept testing, ad pretesting, messaging iteration, specialized B2B audiences, and multi-market comparisons. Use a controlled experiment on real people for the claims that carry the launch decision, where a causal effect and a confidence interval, not a correlated guess, need to back the call. ### Does simulation replace research on real people? No. It supports directional iteration, not the step that proves an effect. The pattern that holds up is hybrid: narrow the field with simulation, then validate the finalists with a controlled study on the real audience before allocating spend. [figure: Five-step path: run a synthetic panel, get a directional result, hit a limit, then run a controlled experiment on real people for a causal effect with a confidence interval.] A team that treats every stage in that path the same way can either overstate what simulation proves or apply controlled research before the question has been narrowed. Research shows how the controlled step works in practice, case studies show it applied to real launches, and a demo or a look at how Subconscious works shows where the causal step would sit in an existing pipeline. --- # AI Expert Panels as a Decision-Rehearsal Tool URL: https://subconscious.ai/blog/use-cases/ai-expert-panels-decision-rehearsal-tool Published: 2026-08-19 Teams often need a senior engineer's critique of an architecture plan, an investor's reaction to a pitch, or a marketing leader's view of a go-to-market decision. The right people may be unavailable when the question is live. Publishing what a tool cannot do is what lets a buyer trust what it can. An AI expert panel can help a team rehearse a decision. It cannot supply credentials, accountability, proprietary information, or a real expert's judgment. [figure: Five-stage chain: one question splits into distinct perspectives; disagreement points to untested assumptions; those feed an experiment; its evidence plus accountable experts produce the decision.] ## What an expert-panel exercise is The exercise runs one question through several defined perspectives. A venture investor perspective may probe market size and defensibility. A marketing perspective may probe positioning and channel fit. An engineering perspective may probe scale and technical debt. Running one prompt against a generic model is easy. Running it through five distinct perspectives can expose disagreement. Agreement is a hypothesis. Disagreement points to assumptions that need more work. Neither is evidence that the simulated experts are correct. ## Four useful applications ### How does pitch pressure testing work? A founder can use three to five investor perspectives with different stages, sectors, and theses. The goal is to surface objections before a meeting, not to predict a specific investor. One sharp objection goes straight at the math: if the deck's numbers only work when every company in the category eventually buys, the founder needs to name the narrower serviceable market a real go-to-market can reach, not the full 100% headcount. ### Marketing strategy review A B2B marketing leader, a consumer brand leader, a growth specialist, and a brand strategist will inspect different parts of the same plan. Their simulated critiques can produce questions for the real team to answer. ### Technical-plan review A distributed-systems architect, a database specialist, and a platform leader who has managed three major migrations may provide useful review lenses. ### Product strategy A B2B product leader, a platform strategist, and a product-led growth specialist can challenge a feature direction from different frames. The output should become an assumption list for real research and technical review. ## A five-step workflow - Define the perspectives with enough detail to make their incentives and constraints distinct. - State the decision, alternatives, constraints, and evidence already available. - Ask all perspectives the same first question. - Probe disagreements and request the reasoning behind each objection. - Share the transcript with the people who own the decision. The follow-up is more useful than the first response. Ask which assumption drives the concern, what evidence would change the assessment, and what failure mode deserves a test. ## Where do panels help? The method works best when a team needs multiple perspectives, is still exploring the problem, and wants to identify questions before spending more resources. It can turn one person's private preparation into a shared critique artifact. The limits belong in the open next to what the panel gets right. A $10M deal, a regulatory crisis, or an architecture decision that defines the company needs real people with relevant evidence and responsibility. Naming where the panel stops is what makes the rest of it usable. AI panels also lack social context. They cannot know what happened in a portfolio last week. They do not make introductions, advocate for a team, or accept responsibility for the result. [figure: A decision path with two branches. "Still exploring" and "low-stakes rehearsal" lead to "route to AI panel." "High-stakes deal or regulatory decision" leads to "route to real accountable people."] ## Turn critique into an experiment The most valuable output is not an answer. It is a set of competing claims that can be tested. For a product, pricing, messaging, or go-to-market decision, define the target audience, alternatives, and outcome. Compare the actions in a decision-specific experiment. Use the panel to improve the questions and expose assumptions. Use evidence from the experiment and accountable experts to make the decision, and talk to the team to scope a specific test. An AI expert panel is a thinking aid. The decision remains yours. --- # Causal AI Use Cases: Which Decisions Can Subconscious Test? URL: https://subconscious.ai/blog/use-cases/causal-ai-use-cases-which-decisions-subconscious-test Published: 2026-08-19 Subconscious helps teams test product, pricing, messaging, launch, and go-to-market actions before committing capital. The strongest use case has a defined population, plausible alternatives, a measurable behavior, and a meaningful cost if the team chooses poorly. Subconscious is the causal AI company. Teams run randomized experiments against a person-level audience graph covering 800 million real people, then validate the same study with real human participants against the causal fidelity paper behind the platform. This creates a direct path from rapid comparison to human evidence while keeping the intervention and outcome fixed. [figure: Four labeled criteria in a row: defined population, plausible alternatives, measurable behavior, and meaningful cost if wrong, as joint requirements for a good fit.] ## Commercial decisions ### Product development, innovation, and launch Compare product concepts, features, claims, packaging, and launch messages before choosing what to build or release. A decision-specific experiment shows which tested alternative produces the stronger directional response and where the response differs across defined audiences. A comparison score without its limits reads like marketing. This supports innovation and R&D when the team has concrete concepts to compare. It does not replace technical feasibility work or direct customer discovery. ### What pricing decisions can Subconscious test? Compare defined price points, packages, product alternatives, and buyer segments. A number without its limits works as marketing copy. It is not a promise of automatic price optimization, revenue forecasting, or SKU-level elasticity. The Subconscious case studies show how teams have used repeated experiments to narrow pricing and positioning decisions before rollout. ### Brand awareness, positioning, and advertising Test positioning, campaign ideas, product claims, audience definitions, channels, and creative directions. Naming what the method cannot prove is what lets a buyer check the claim. This fits a CMO choosing among concrete actions. It does not turn brand health or historical campaign correlation into causal proof. ### Market segmentation and consumer behavior Define segments before the experiment, then compare whether the tested action moves each group differently. Aggregate pattern matching is easier than individual simulation. Segment claims need enough evidence and should not be inferred from a plausible persona response. ### Customer satisfaction, retention, and customer experience Compare proposed journeys, service changes, retention offers, and experience concepts against a target behavior. A simulation can help decide which action deserves a live test. Actual churn, loyalty, and satisfaction still require observed customer data. ## Research and strategy decisions ### Market entry, risk, and go-to-market strategy Compare offers, messages, audience definitions, sequencing, and market-entry actions. Keep the experiment tied to one decision. Broad market forecasts and automated recommendations require separate evidence. ### Can Subconscious forecast market trends? This fits only when a trend creates a decision with alternatives: a team can test how defined audiences respond to actions under a stated scenario. It is not a trend oracle. Historical analysis, market data, competitive evidence, and scenario assumptions remain necessary. The causal question is which action changes the outcome under the tested conditions, not whether the model can predict every market shift. ### Market research and consumer insights teams Use causal simulation as an experimental first pass before committing a full research budget. Interviews, surveys, field experiments, and observation remain valuable for discovery and real-world confirmation. The research program explains how Subconscious uses replication and human baselines as the trust layer. ### Public opinion, polling, and policy research Compare public messages, policy choices, or interventions for a defined population and outcome. Consequential public decisions require qualified review, clear limitations, and suitable human validation. ### Sustainability and social impact Compare sustainability claims, corporate-responsibility initiatives, and behavior-change messages. Measure a defined response. Do not treat stated support as proof that behavior will change. ## Product and service experience decisions ### User experience testing Subconscious can compare product concepts, journeys, service changes, and proposed experience interventions before implementation. It does not replace watching a real person use an interface. Use direct usability research for task completion, navigation, comprehension, and accessibility. Use a causal experiment when the decision is which product or commercial action changes choice. The UserTesting, Maze, Lookback, and causal AI comparison explains the boundary. ### Can Subconscious be used for healthcare optimization? Healthcare teams can compare patient, provider, payer, communication, access, or policy interventions when the population, alternatives, and outcome are explicitly defined. Publishing where simulation stops is what makes it usable for a health decision. Simulation should not be presented as clinical evidence or a substitute for patients, clinicians, trials, safety review, or regulatory analysis. See the pharma decision lab for the current vertical framing. ### Education, media, travel, automotive, and digital services The same method can compare messages, offers, journeys, and adoption actions across education, entertainment, travel, automotive, software, and professional services. Industry context changes the study design. The causal requirement stays the same. ## Industries with repeated high-cost decisions The method is strongest where teams make repeated choices under behavioral uncertainty: - Consumer goods, retail, and ecommerce: product, price, pack, and claim decisions - Pharma and healthcare: positioning, access, and support decisions - Telecom and media: bundle, ARPU, churn, and audience decisions - Technology and software - Financial services - Government and public policy - Market research and consulting - Automotive, travel, and hospitality - Energy, utilities, logistics, and real estate - Education and learning products Industry alone does not determine fit. The team must be able to name the decision, actions being compared, people affected, and behavior that matters. [figure: A four-step horizontal path: compare alternatives against a synthetic audience, narrow to the strongest option, validate that option with human participants, then commit budget or capital to it.] ## Choose a use case worth testing Product launches, price scenarios, message choices, market-entry plans, and policy interventions fit when the decision owner must select among defined alternatives. Open-ended trend scans, generic opinion generation, automatic financial forecasts, and guaranteed outcomes are poor fits. Start with one decision. If the team can define the audience, alternatives, and target behavior, bring it to a Subconscious working session. --- # How to Test Positioning Angles Before You Commit Campaign Budget URL: https://subconscious.ai/blog/implementation-and-procurement/how-test-positioning-angles-before-commit-campaign-budget Published: 2026-08-19 The way to test positioning angles before committing campaign budget is to run a structured causal comparison, a forced ranked choice across the candidate statements against a defined buyer population, before the brief is written. Few decisions a marketing team makes carry more weight than positioning, made before a campaign brief goes out. The right angle compounds across every asset, deck, and pitch that follows it. The wrong one wastes spend at every stage of the funnel, and the workshop where most teams pick it rarely includes a real buyer signal. Positioning still gets decided this way because of timing. A traditional research study lands weeks after the workshop, after the campaign has already shipped. A controlled causal experiment compares positioning alternatives against a defined buyer population and returns a read on the winning angle, and the language that supports it, before the brief is locked. ## The decision this changes Before writing a campaign brief, a sales deck, or the site copy that follows from it, a brand or marketing leader is choosing one positioning statement over several plausible alternatives. That choice is usually made once, on instinct, in a room. If the angle is wrong, the cost isn't the workshop time, it's every dollar spent on campaigns, enablement, and copy built around a position the target buyers don't find clear, differentiated, or relevant. Subconscious.ai runs that experiment and reads out which angle wins, and why, before the spend is committed. Learn how the causal testing workflow fits a marketing team's planning cycle. ## A structured comparison, not a workshop vote ### 1. Define the positioning statements to test Before running anything, write down the angles under consideration as one-sentence positioning statements, in the same voice and structure, so what gets compared is the positioning itself, not the wording. Aim for four to six bets that differ in substance: a category framing along the lines of "X for Y," a jobs-to-be-done framing, a pain-point framing, a competitor framing, and an outcome framing. Fewer angles and the comparison isn't testing anything real; more, and a buyer population can't differentiate cleanly between them. ### What buyer population should the comparison cover? Positioning rarely lands the same way with every buyer. Before running the comparison, define the distinct buyer segments it needs to cover: the roles and contexts that will actually read the campaign, the pitch deck, or the site copy that follows from the winning angle. ### Why run the comparison as a forced choice instead of an opinion poll? Asking "do you like this?" produces noise. A structured comparison instead shows each positioning option, checks clarity by having respondents describe the company unprompted, checks for differentiation and relevance, and then forces a ranking across all the options together. Discrete choice methodology, showing alternatives together and forcing a ranked or chosen response, produces a materially stronger signal than isolated opinion questions, established in the communication-research literature on discrete choice experiments (PMC, National Library of Medicine). "The efficiency to estimate effects with small numbers of participants may be the clearest benefit of DCEs, but their flexibility also enables wide-ranging applications in message-evaluation, message-effects, and media-selection research." Reynolds and colleagues, Frontiers in Communication (source) ### 4. Read for convergence and segment splits Look for three patterns in the results. Convergence across the buyer population, most segments ranking the same option first, is the strongest positioning signal available. A segment-specific split, where one segment favors one angle and another favors a different one, points to a primary angle plus a secondary message for an adjacent segment. And the language buyers use unprompted to describe the winning angle is often sharper than the language the team originally wrote: that phrasing is worth carrying into the next round. ### 5. Lock the direction and carry it everywhere The output of a completed comparison is a winning positioning statement, the strongest supporting phrases pulled from unprompted language, and any segment-specific variants the split calls for. Hand these to the people who write the campaign brief, the pitch deck, and the site copy: positioning compounds when it stays consistent across every surface it touches. ## Iterate before you lock, not after you ship A workable pattern: test a broad set of angles and narrow to the strongest two, test sharper variants of the top angle with language pulled from the first round, then test the winner against the closest alternative with segment-specific variants and lock the direction. Several short rounds run in sequence beat one long study, both on the quality of the final positioning and on how long it takes to reach it. ## When does a simulated comparison stop being enough? A simulated positioning comparison is a decision-support signal, not a replacement for direct human research when the stakes are highest. For a category launch or a major repositioning, validate the winning angle with a small round of real-human testing before it goes public: the causal question stays the same, only the population answering it changes. See how Subconscious moves from a simulated experiment to real-human validation. A modeled buyer population and a recruited real-human audience reach different people; keep them distinct. ## What to ask for before you bring this in-house A team evaluating a causal testing workflow for positioning should be able to answer three questions before committing: What does the comparison measure, forced ranking against alternatives or isolated opinion? What happens at the edge of the method, does the vendor flag when a real-human round is warranted, or imply the simulated result is always sufficient? And does the output travel to the people who write the brief, the deck, and the copy, or does it stop at a report nobody acts on? Positioning is too consequential to leave to a single meeting and too fast-moving to wait weeks for. A structured comparison run before the brief is locked gives a marketing team a defensible answer to "why this angle," and the receipts to back it up when the campaign is questioned. Talk to the team about testing your next positioning decision. [figure: Five-step path: write positioning statements, define buyer segments, force a ranked choice across all statements, read convergence or segment splits, lock the winner into the brief.] --- # Choosing a Research Method for a Causal Decision URL: https://subconscious.ai/blog/use-cases/choosing-research-method-causal-decision Published: 2026-08-19 A causal decision calls for traditional research when the population, alternatives, or outcome remain undefined, a simulated experiment once those three exist, and real-human validation before committing budget. A head of insights, product, or growth facing a launch, pricing, or messaging call has three broad ways to gather evidence: traditional survey and qualitative research, a controlled simulated experiment, or a real-human study. Picking the wrong one is expensive: a correlational or anecdotal signal can carry a launch, price, or message decision that needed a defensible cause-and-effect estimate instead. [figure: Four-step path: traditional survey/qualitative research, to a simulated experiment once population, alternatives, and outcome are defined, to real-human validation, ending in a committed decision.] ## What the decision actually requires Traditional surveys and qualitative interviews are good at describing what buyers say they prefer. They are weaker at isolating which specific action (a price change, a feature, a message) caused a change in behavior. In a randomized controlled trial, comparing a treated group to a control group under randomization is what lets a team attribute an outcome to a cause rather than a confound. Subconscious runs this kind of controlled, randomized experiment on simulated buyers: a defined population sees one of several tested alternatives, and the platform measures which alternative moves the outcome. This is the same logic used in discrete choice experiments in health economics, applied to commercial questions like pricing, messaging, and product concepts. ## When Is a Simulated Experiment the Right Fit? A simulated experiment is the right first move when the team has: - a defined population, - two or more concrete alternatives to compare, and - a measurable outcome that matters to the decision. Subconscious can run these controlled studies against a person-level audience graph covering 800 million real people. That graph is a population for experiment design, not a recruitable panel of respondents. ## When Should You Escalate to Real-Human Validation? A simulated result is a strong basis for narrowing options, not proof by itself. Subconscious can test or validate the same study with real human participants, using the identical causal question and experimental design. That path lets a team move from a simulated comparison to human evidence without redesigning the experiment, the point at which a directional finding becomes something a team can commit budget against. Real-human validation confirms a causal comparison. It is not an observed usability session, a clinical trial, or a guarantee of market performance, and it does not replace human judgment on execution. ## When Is Traditional Research Still the Better Fit? Some questions do not yet have a defined population, concrete alternatives, or a measurable behavioral outcome. Open-ended discovery, exploratory interviews, and broad qualitative research remain the right tool there. ## Limitations [figure: Three-item checklist: a defined population, two or more concrete alternatives, and a measurable outcome, required before a simulated experiment is the right tool.] ## Next step Review the research program for how simulated experiments and human validation fit together, see how Subconscious works end to end, browse case studies of teams using this path, or book a walkthrough against a specific decision. --- # 5 Structural Signals a Research Team Will Deliver Fast, Auditable Causal Experiments URL: https://subconscious.ai/blog/company-and-trust/5-structural-signals-research-team-will-deliver-fast-auditable-causal Published: 2026-08-19 A research, product, or strategy leader picking a vendor for a high-stakes causal experiment buys more than a method: a team's ability to design the test, interpret the result, and hand over an auditable decision memo. A bureaucratic, opaque structure slows that work and hides how a conclusion was reached, even when the method is sound. Team structure is a leading indicator of that risk, and a buyer can check it before the first experiment ships: does the team operate as a flat, self-organizing group, or a hierarchy where decisions and reasoning stay locked inside management layers? [figure: A labeled list of five structural traits a buyer can check in a research vendor's team: freedom over hours worked, full transparency, autonomy and self-organization, flexible fluid roles, and coaching-style leadership.] ## Where the pattern comes from The pattern below is not unique to research vendors. It traces back to open-source teams, where unpaid contributors out-build far larger, better-funded organizations. Research on free/libre open-source software projects documents recognition and coordination structures that replace top-down management with visible, self-assigned work (Business Ethics, the Environment & Responsibility). Separate work on decision-making in self-organizing virtual teams found distributed, low-hierarchy groups can still converge on fast, coordinated decisions with clear shared goals (Syracuse University FLOSS research group). This is an organizational-design analogy, not a case study of any single company's team, compensation, or governance model. The five traits below are a checklist for evaluating a vendor's operating model, not a proprietary Subconscious methodology. ## Five traits worth checking before you commit budget ### 1. Why does freedom over hours worked matter? A team paid for output rather than hours can spend a few focused hours on high-quality work instead of stretching mediocre work across a full day. For a research vendor, this shows up as experiment designs and decision memos that read as considered, not templated: the incentive rewards getting the causal question right, not logging time. ### 2. Why does full transparency matter? A team where everyone can see the state of every active engagement catches design and interpretation errors earlier, because more people see the same information. For a buyer, the practical test: ask whether the vendor can show how an experiment's design decisions were made, not just the final slide. ### 3. Autonomy and self-organization Teams that let the person closest to a problem act on it, rather than routing decisions through a manager or committee, move faster and keep decisions with whoever has the most relevant context. Recent work on individually experienced autonomy in agile project teams frames this as a tension to manage, not an automatic win: too little autonomy slows a team down, but autonomy without shared goals can fragment it (ScienceDirect). A vendor worth trusting with a high-stakes experiment should describe how it manages that tension, not just claim to be "self-organizing." ### 4. Flexible, fluid roles When project leadership shifts based on who has the most relevant expertise, rather than a fixed org chart, a team can reorganize around a client's specific causal question instead of forcing the question into an existing structure. This matters most when a buyer's decision spans product, pricing, and messaging and no single fixed role owns all three. ### 5. Why does coaching-style leadership matter? A leadership style built around removing blockers, rather than dictating tasks and deadlines, keeps decision-making close to the people running the experiment. Worth probing directly: does the person leading the engagement make the calls themselves, or explain the reasoning and let the analyst closest to the data make the final judgment? ## Why this matters for causal experiment work specifically Subconscious.ai runs controlled experiments on simulated markets, publishes its methodology, and reports causal effects with confidence intervals rather than a single plausible-sounding answer. The same operating traits apply here: a flat, evidence-driven team can move quickly from a simulated study to real-human validation without changing the underlying causal question, and show a buyer that path rather than asserting it. That is a claim about operating model, not a substitute for checking the published methodology and results directly. ## Limitations This is a framework for evaluating a vendor's structure, not a scorecard that guarantees quality. A team can score well on every trait and still design a flawed experiment, and a more hierarchical team can still produce rigorous work. Use these traits alongside the vendor's actual methodology, published evidence, and willingness to show its reasoning, not as a replacement. [figure: Four-step chain: hierarchical decisions route through management layers, locking in the reasoning, so the buyer sees only the final slide and errors go unaudited.] ## Next step Before committing budget, ask to see how a past experiment's design decisions were made and by whom. A team that answers clearly, and that can walk a design from simulation to real-human validation without changing the question, is signaling the same structural traits described here. Learn how Subconscious's team works or see the research behind the method. --- # Counterfactual Causal Inference in PyMC URL: https://subconscious.ai/blog/methods-and-validation/counterfactual-causal-inference-pymc Published: 2026-08-19 Prediction forecasts the outcome under current conditions. Causal inference instead estimates the outcome that a different action would have produced. That distinction changes how a team measures impact. After a $250,000 advertising campaign, observed sales alone cannot show how many sales the campaign caused. The missing quantity is the outcome in the same period without the campaign. That unobserved alternative is the counterfactual. A Bayesian model can estimate the counterfactual as a distribution rather than a single guess. The difference between the observed outcome and that modeled baseline gives an estimate of the causal effect, subject to the assumptions in the model. [figure: A four-step chain: a baseline model fit on data from before the intervention, projected forward as the counterfactual, compared against the observed outcome, with the gap between them read as the causal effect.] ## Excess deaths as a counterfactual problem The same structure appears in public health. Excess deaths can be written as: Reported deaths are observed. Expected deaths in a world without COVID-19 are not. To estimate them, fit a model to the period before COVID-19, project the learned mortality pattern into the later period, and compare that projection with reported deaths. The example uses monthly deaths in England and Wales. Mortality has a clear seasonal pattern, so a credible baseline must account for that pattern before it can attribute the remaining gap to the pandemic. [figure: Monthly reported deaths in England and Wales, split into pre-COVID and post-COVID periods, with a strong seasonal pattern.] ## How do you make causal assumptions visible? A directed acyclic graph, or DAG, forces the analyst to state which variables are treated as causes, proxies, or omitted factors. The example models reported deaths using: - Time, as a proxy for demographic aging - Month, as an index for seasonal effects - Temperature, as a direct influence on mortality COVID-19 is deliberately left out of the baseline model. The purpose is to forecast a world where the pandemic did not occur. Adding a pandemic indicator would let the model explain the observed shock rather than estimate mortality in its absence. [figure: A directed acyclic graph with time, month, and temperature pointing to reported deaths.] This exclusion does not make the estimate causal by itself. The conclusion depends on assumptions: the pre-pandemic relationship remains useful in the later period, the selected predictors capture the important baseline structure, and no other unmodeled shock explains the observed difference. ## Translate the DAG into a Bayesian model Let monthly deaths be a function of time, month, and temperature: Here t runs from 1 through T. A linear specification is: ### Specify priors and likelihood The vector s contains 12 monthly effects. The original model uses these priors and likelihood: These priors translate directly into modeling choices: - β₀, Normal(40000, 10000): the intercept, centered at 40,000 monthly deaths - β₁, Normal₊(0, 50): the time trend, constrained positive for an expected upward trend in the susceptible population - β₂, Normal(0, 200): the temperature effect, which can move the estimate in either direction - σ, HalfNormal(2000): the noise scale ## Check the model before asking for an effect Counterfactual inference is only as useful as its baseline. Bayesian workflow provides several checks before the final comparison. ### Prior predictive check Draw outcomes from the model before conditioning on observed deaths. The prior predictive distribution reveals whether the model permits plausible mortality levels and seasonal variation. The example inspects 50% and 95% credible regions. If those ranges miss plausible values before the model sees data, the priors or functional form need work. [figure: Prior predictive monthly death estimates with 50% and 95% credible regions.] ### Fit only the pre-intervention period Posterior sampling uses pre-COVID observations. This prevents the pandemic shock from changing the coefficients used to define the no-pandemic baseline. ### Check posterior predictions in the fitted period Compare observed pre-COVID deaths with posterior predictive draws. A model that cannot reproduce the baseline period should not be trusted to project its counterfactual into a new period. Good fit is necessary but not sufficient. A flexible time-series model may predict well while relying on a causal story that fails under intervention. ## Generate the counterfactual with pm.set_data() After fitting the model, replace its time, month, and temperature inputs with their post-COVID values while keeping draws from the fitted parameter posterior. Then sample the posterior predictive distribution: This operation produces simulated post-COVID deaths, including observation noise, under the fitted no-pandemic model. It is often described with the do-operator: Whether that distribution identifies a causal effect still depends on the DAG and the stability assumptions. pm.set_data() is a computational mechanism, not a substitute for causal identification. ## How do you calculate and interpret the effect? For each posterior draw, subtract the sampled post-COVID deaths (which include observation noise, not just the mean) from reported deaths. The resulting distribution represents monthly excess deaths under the model. Summing across months produces cumulative excess deaths while retaining posterior uncertainty. The original analysis displays 50% and 95% credible regions for the expected and excess-death estimates. That is more informative than a single total, but the bands quantify parameter and sampling uncertainty conditional on the specification being correct; they do not capture the trend-extrapolation and misspecification risk of projecting a linear trend roughly two years past the fitting window. [figure: Observed deaths, counterfactual expected deaths, monthly excess deaths, and cumulative excess deaths with uncertainty bands.] The gap may include both direct deaths from COVID-19 and indirect effects, such as disrupted access to care. The model estimates the effect of the pandemic period as represented by the design, not every mechanism inside it. ## Apply the same structure to business decisions The mortality example maps cleanly to commercial questions: - Sales with a campaign versus expected sales without it - Conversion after a price change versus expected conversion at the prior price - Retention after a product change versus expected retention without the change - Learning after an intervention versus expected learning without it Each application needs more than a before-and-after chart. Specify the intervention, draw the causal graph, define the pre-intervention baseline, test predictive adequacy, and state the assumptions required to interpret the difference as causal. The full PyMC notebook, pm.set_data() documentation, and pm.sample_posterior_predictive() documentation provide the implementation details. ## Where does Subconscious fit in this workflow? This PyMC workflow estimates an unobserved baseline after an intervention has occurred. Subconscious applies the same causal discipline to a different moment in the decision cycle: before implementation, when a team is choosing which product, price, message, or go-to-market action to take. The team defines the actions, target population, outcome, and constraints. Subconscious runs a controlled experiment on a simulation of that market and compares the tested alternatives. The result is useful only when the design, human baseline, assumptions, and limits remain visible. That is why Subconscious treats simulation as the lab bench, not the product. The product is a tested decision. The replication leaderboard and guide to comparing simulated and human results show how that standard is applied. PyMC and Subconscious are not interchangeable tools. PyMC gives an analyst a programmable Bayesian modeling framework. Subconscious gives a decision owner a causal behavioral experiment around defined commercial actions. Counterfactual models turn a vague question into a testable one. The useful result is not merely a forecast. It is an explicit estimate of what the outcome would have been under a different action, with assumptions and uncertainty visible. --- # How to frame a causal behavioral experiment URL: https://subconscious.ai/blog/implementation-and-procurement/how-frame-causal-behavioral-experiment Published: 2026-08-19 A useful experiment starts with a decision, not a broad request for insights. Define what the team may change, whose behavior matters, which alternatives need comparison, and the context in which the decision will occur. [figure: Four ordered steps: Ask why (name the behavior and factor), Define who (the population, 2-10 traits), Define what changes (concrete alternatives), Set when and where (time and market). Ends at a review checklist.] ## Step 1: Ask why Write a causal question about human behavior. Name the behavior first, then the factor that may change it. For example: - How does fuel efficiency affect car choice? - Which product claim changes purchase intent among a defined buyer segment? - Which message changes support for a proposed policy? Avoid questions so broad that no experiment can distinguish one action from another. “What causes car buying?” may help open a discussion, but a study needs explicit alternatives and an observable outcome. ## Who should the experiment include? Specify the population whose response matters to the decision. Useful characteristics may include profession, income, age, current behavior, or another trait tied to the study. The interface accepts 2-10 traits that define the target population; this range is specific to the setup, not a universal design rule. Use only traits with a clear reason to affect the decision. Demographic detail without a research purpose adds noise and weakens the population definition. Synthetic or simulated participants compare aggregate patterns, but should not be treated as exact replicas of individuals. Human baselines and validation remain important, especially when the decision affects vulnerable groups or carries high stakes. ## What should the experiment compare? List the alternatives, attributes, or claims the experiment will compare. Each attribute needs concrete levels that participants can evaluate. For a product study, these might include: - Product concepts - Features or claims - Price points - Packaging or message options Edit or remove any attribute that does not affect the decision. The study should isolate meaningful contrasts, not collect every detail. Discrete-choice-style experiments compare behavior across alternatives, and the attribute-and-level design choices follow established good-practice standards for discrete-choice experiments (Constructing Experimental Designs for Discrete-Choice Experiments, ISPOR Task Force report). Do not assume that every study is a fully specified conjoint, MaxDiff, pricing-sensitivity, or portfolio test. The method and outputs must match the configured design. ## When and where should the experiment take place? Define the time and place that frame the audience's decision. - When: Use a specific year such as 2024 or a broader period such as the early 80s when the historical context matters. Past and present periods are easier to ground than unsupported future conditions. - Where: Choose the country, market, or other geographic scope relevant to the audience. Time and place are part of the experiment, not decorative context. A response that is plausible in one market or period may not transfer to another. [figure: Four-link chain: Name the behavior, Name the factor, State the alternatives compared, Observable outcome measured. A broad question like "what causes car buying" skips to the end, breaking the chain.] ## Review the design before running it Check that the experiment answers one decision: - Is the causal question specific? - Does the population match the people affected? - Are the alternatives concrete and meaningfully different? - Are time and place defined? - Is the outcome something the study can compare? - Are uncertainty and validation described only where the design supports them? If any answer is unclear, revise the setup before interpreting results. A precise question and controlled comparison matter more than the number of variables included. See how this framing step fits into the full testing process on How We Work, or bring a live decision to a demo to work through it directly. --- # Marketing Mix Modeling: A Complete Guide URL: https://subconscious.ai/blog/methods-and-validation/marketing-mix-modeling-complete-guide Published: 2026-08-19 Marketing mix modeling is a statistical technique that decomposes historical sales into a base level and the estimated incremental lift attributed to each marketing channel, using adstock and saturation transforms to account for carryover and diminishing returns. A marketing mix model (MMM) decomposes observed sales into a base term and the estimated incremental lift attributed to each marketing channel. Before a team reallocates budget on the strength of an MMM's channel attribution, it needs to understand what the model actually measures, and what it can't. ## What an MMM decomposes MMM starts from a simple accounting identity: Observed sales equal a base level, plus whatever lift gets attributed to TV, digital ads, promotions, and other factors. For example, TV drove some portion of sales, digital drove another, promotions another, and the remainder came from seasonality or other unexplained factors. The regression form underneath assumes each channel's effect is linear and immediate. Real marketing rarely works that way. [figure: Two columns. Left: frequentist estimation, data feeding a regression fit that outputs a coefficient with a confidence interval per channel. Right: Bayesian estimation, data plus a prior producing a posterior distribution per channel.] ## Why the basic equation isn't enough Spending on a channel doesn't translate to sales in a straight line. Zero spend gives zero impact. Some spend produces a visible lift. But spend without limit doesn't keep producing proportional sales: audiences saturate and the channel loses effectiveness. Timing matters too: a TV campaign's impact can carry over for weeks after the spend runs, while a digital ad's impact fades almost immediately. To reflect this, MMM adds two transforms before the regression step: adstock, which captures lag and carryover, and saturation, which models diminishing returns. ## What is the adstock carryover effect? An ad's effect doesn't land only in the week the money is spent. Some viewers act that week, some weeks later, some never. Adstock creates a transformed version of the spend variable that carries a fraction of each week's effect into the following weeks: The current period's adstocked value is that period's spend plus a decayed fraction of the prior period's adstock, and so on backward. λ (lambda) sets the decay rate: a channel with a high λ retains more of its modeled effect across periods, while one with a low λ retains less. As a historical planning example, $1M spent on TV in week 1 with λ = 0.8 decays to $0.8M in week 2, $0.64M in week 3, and $0.51M in week 4. ## What causes diminishing returns in marketing spend? Even after adstock, more spend doesn't produce proportionally more sales. The first dollars reach the most responsive part of the audience; later dollars reach people who are harder to convince. Before that adstocked spend goes into the regression, MMM passes it through a saturation function, and a handful of function families show up repeatedly: - Logarithmic: f(x) = log(1 + x). Steep growth at low spend, flattening as spend rises. - Hill: f(x) = x^α / (x^α + θ^α). α controls how sharply the curve bends toward saturation; θ is the half-saturation point, the spend level where the channel delivers half its total modeled impact. - Tanh: f(x) = b * tanh(x / (b * c)). b sets the ceiling on impact the channel can deliver; c governs the curve's approach to that ceiling. - Logistic: f(x) = (1 - e^(-x)) / (1 + e^(-x)). Its steepness parameter controls the curve before it flattens, with a half-saturation point near the inverse of that parameter. With both transforms in place, the flow runs: raw spend, then adstock (carryover), then saturation (diminishing returns), then regression. Each channel's estimated contribution is a coefficient applied to its saturated, adstocked spend; the base term is the model's residual level given the specified controls like price and trend, not an identified no-marketing counterfactual, and it can absorb brand equity built by prior marketing along with any omitted driver. ## Control variables: what isn't marketing Sales move for reasons that have nothing to do with ad spend: a recurring holiday, weather, added shelf space. MMM adds these as control variables alongside the media channels. Leaving them out means marketing gets credit for changes it didn't cause; including them separates real incremental lift from normal shifts in the baseline. ## Point estimate or distribution: the frequentist-Bayesian split Once the model's structure is set, its parameters (the channel coefficients and the adstock/saturation parameters) still have to be estimated from historical data. There are two common approaches. The classic route is frequentist estimation: the nonlinear adstock and saturation parameters are fit by grid search or nonlinear least squares, and the channel coefficients are then estimated by ordinary least squares (OLS) regression conditional on those fixed transform parameters, producing a best-fit number for each coefficient along with a standard error and confidence interval. Those intervals come from repeated-sampling theory rather than a probability statement about the true effect, and a team's prior knowledge from past campaigns can't be built into the estimate directly. Bayesian estimation replaces the single best-guess number with a distribution: a likely range along with the probability that the true effect falls outside it. That distinction, a point estimate versus an uncertainty-aware estimate, is the practical question a stakeholder needs answered before treating an MMM's output as the basis for a budget decision. Subconscious applies this same discipline at the level of a single decision rather than a channel coefficient: a controlled experiment estimates a causal effect and reports it with a confidence interval, so a stakeholder sees the range, not just a headline number. Read more about how Subconscious runs controlled experiments. ## What can't a marketing mix model do? MMM is backward-looking. It needs a substantial history of sales and spend data, and it works at the level of channels and weeks or months, not individual ads or individual days. It can show that TV performed well over a quarter; it can't say which specific ad or airing drove a given day's sales. Leaving out a real driver of sales, such as a competitor's move or a shelf-space change, can bias the estimated returns for every channel in the model. Because of these limits, MMM works best paired with experiments and lift tests rather than used alone. An experiment can validate or recalibrate what the model estimates; the model can extend a single experiment's findings across the full media mix. Subconscious's research applies the same standard: a reported effect always carries its confidence interval, and results replicate before they inform a decision. See the leaderboard for how those replicated results are tracked. [figure: Five boxes left to right: raw spend feeds an adstock transform carrying lagged effect forward, then a saturation transform that flattens at high spend, then regression, then each channel's contribution.] ## Further reading on the mechanics For a worked derivation of the adstock and saturation transforms with PyMC, see Media Effect Estimation with PyMC: Adstock, Saturation & Diminishing Returns. For a shorter walkthrough of the same carryover and diminishing-returns effects, see Saturation and Adstock Effects in Bayesian MMM. --- # Validate Feature Decisions Before the Spec Locks URL: https://subconscious.ai/blog/buyers/validate-feature-decisions-before-spec-locks Published: 2026-08-19 Feature decisions get validated by running a controlled behavioral experiment against a defined target population before the spec locks, replacing team votes with documented evidence. A product manager writes a PRD, hands it to engineering, and finds out six weeks later that an assumption about the target customer was wrong. The team has spent a sprint on the wrong problem, and the spec window that would have caught the error already closed. The usual fix, more user research up front, runs into its own limit: recruiting, scheduling, and running a study takes weeks the roadmap does not have. Instead, the loudest opinion in the room decides, and the PRD locks around that opinion rather than evidence. [figure: A five-step path: pick the decision; define audience, alternatives, outcome; run the comparison; add evidence to the PRD; then branch, with high-risk decisions detouring to human validation before engineering starts.] ## What causes the wrong feature to ship The failure is not a lack of customer data. It is a gap between when a PM needs an answer and when real research can produce one, and in that stretch most teams have no fast way to check a hypothesis against a target customer, so the team votes instead. Subconscious closes that gap by running a controlled behavioral experiment against a defined population before the spec locks: state the decision, the audience, the alternatives, and the outcome, then compare which action moves that outcome. That is a test, not a chatbot opinion, and it follows the same discipline as structured concept testing, which compares alternatives against a described audience instead of asking which one a team prefers (Qualtrics, "Concept Testing: Definition, Methodology & Examples"). ## What can this kind of test tell a PM, and what can't it? Treat everything a simulated experiment returns as a planning input, not a substitute for real usage data. The workflow below shows how one team sequenced this test; the counts and durations are that team's planning assumptions, not a guarantee. The team's plan compressed a roughly six-week decision cycle to about fourteen days. Read that as one workflow's plan, not a promised timeline for every team. ## When does this replace a team vote, and when doesn't it? A synthetic panel is a reasonable substitute for the low-stakes, high-frequency calls a PM currently makes by consensus: which of three feature names sets the right expectation, whether an empty state explains its value, whether admins will understand a settings trade-off. None of these individually justify a formal research project, but a wrong call on any of them still costs engineering time. It is not a substitute for the things that establish whether a feature actually works: foundational customer interviews, a beta program, in-app analytics, or watching a real user click through a live prototype. For a major pricing model change or a category expansion, the sequence is to generate and narrow hypotheses with a fast test, then validate the leading hypothesis with real people. Subconscious can test or validate studies with real human participants without changing the underlying causal question. ## A practical decision process - Pick one feature or spec decision that is currently being settled by opinion. - Define the audience, the alternatives under consideration, and the outcome that matters (adoption, comprehension, willingness to pay). - Run the alternatives through a controlled comparison and read the reasoning behind the results, not just a preference count. - Take the result into the PRD as documented evidence, alongside the open questions that still need real customer or usability confirmation. - For decisions with real budget or roadmap risk, schedule the human-validation step before committing engineering capacity. ## Where this fits and where it fails This approach fits PM-owned decisions that are too frequent and too small for a formal research project but too consequential to settle by whoever argues loudest: naming, onboarding copy, settings framing, early pricing structure reactions. It fails as a substitute for discovering problems nobody has framed yet, for usability observation on a real prototype, or for any claim about actual production behavior. ## Related reading - AI research tools compared for product managers - See a live decision walkthrough --- # AI-Simulated Panels vs. Traditional Surveys: Sequencing a Pre-Launch Research Decision URL: https://subconscious.ai/blog/comparisons/ai-simulated-panels-vs-traditional-surveys-sequencing-pre-launch-research Published: 2026-08-19 A pricing tier, a headline, or a launch concept needs a read before it ships. The real choice for a research or growth lead is rarely "AI panel or human survey." It is how much weight to put on a fast simulated result before committing budget, and which questions still need a recruited human study first. [figure: Flowchart: a launch decision enters simulated panel triage across variants. Findings split: low-cost ones go straight to the decision; costly-to-miss ones route through human validation first.] ## Two Different Sources of Answers A recruited survey draws answers from real people who were sourced, screened, and paid to respond. An AI-simulated panel draws answers from a language model conditioned on a demographic or behavioral profile, not from a person who actually experienced the product or the price. That substitution explains most of the practical differences between the two methods: what each is fast at, what each gets wrong, and what each can stand behind as evidence. ## What Each Method Answers Well Simulated panels are well suited to breadth (more questions, more variants, more cuts) and poorly suited to anything that requires a real person's physical experience, a persistent identity over time, or evidence a regulator will accept. Recruited surveys are the reverse. ## What Does the Research Support? The literature on using language models to approximate survey and choice behavior is early and mixed. One study on eliciting purchase intent from language models found that how a question is asked, and whether responses are calibrated against human baselines, materially changes how well the output reproduces real survey patterns (LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings). A separate study on using language models for discrete-choice modeling found that models can often recover plausible attribute directions and aggregate tradeoffs, but struggle more with individual-level heterogeneity and are sensitive to prompt design (Can large language models assist choice modelling? Insights into prompting strategies and current models capabilities). "SSR achieves 90% of human test-retest reliability while maintaining realistic response distributions (KS similarity > 0.85)" Maier and colleagues, "LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings" (arXiv:2510.08338) (source) The practical read: naive prompting of a language model for survey-style answers is fragile. Careful elicitation design and calibration against real human data narrow the gap but do not close it uniformly across populations or question types. ## Where Does Each Method Break Down? A simulated panel underperforms on genuinely novel behavior the underlying model has no prior exposure to, on questions where the answer changed after the model's training cutoff, and on niche populations thinly represented in training data. A recruited survey underperforms where recruitment quality is hardest to verify: low-incidence populations, fraud and professional-respondent behavior, and self-report biases such as social desirability, satisficing, and primacy effects. Neither failure mode is a reason to distrust the method generally; both are reasons to match the method to the question. ## Is It a Sequence or a Single Choice? The workable pattern is to triage broadly with a simulated panel, then decide which findings are load-bearing enough to justify a recruited human study before they change a launch, pricing, or messaging decision. Subconscious runs controlled experiments on a simulation of the market and can also test or validate the same study with real human participants, moving from a simulated result to human validation without redesigning the causal question (research methodology). That matters most for the findings where the cost of being wrong is high enough to need more than a simulated read. Simulated panels are also useful for coverage a recruited sample cannot reach at reasonable cost: Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That is a coverage graph for defining who to study, not a recruitable panel of people who have agreed to answer surveys. Audience reach describes who a study can target; real-human validation confirms a specific result against people who actually respond. ## The Buyer's Actual Trade-off The question that matters is not which method is cheaper or faster in the abstract. It is how many of the small decisions that used to get skipped, because a full recruited study felt too slow or too expensive to justify, actually get tested before they ship. A team that runs more small tests, with sharper hypotheses going into any recruited follow-up study, ends up shipping fewer unaudited guesses than a team that either tests everything with expensive recruited studies or skips testing the small decisions entirely. ## Building the Habit of Testing Before Shipping Teams that treat simulated panels and recruited human studies as a sequence, rather than a single choice made once per project, test more of their real decisions instead of a handful of the biggest ones. Reviewing how a validated study moves from a simulated result to a human-confirmed one is a reasonable next step before committing a launch, pricing, or messaging decision to a single untested read (how this works in practice). --- # AI for Market Researchers: A Professional Guide URL: https://subconscious.ai/blog/buyers/ai-market-researchers-professional-guide Published: 2026-08-19 AI is changing what market-research expertise is applied to. It can help with early exploration, instrument review, text analysis, and experiment design. Researchers still own validity, interpretation, disclosure, and the choice of method. Researchers who come out ahead will know where to deploy AI, what it produces, where it fails, and how to combine it with traditional methods. The core question: where can software reduce repetitive work without pretending simulated responses are human evidence? [figure: A four-stage horizontal path: Exploration, Validation, Analysis, Reporting. Each stage names what AI does and what task hands back to a human researcher before the next stage begins.] ## The main tool classes Simulated respondents produce directional responses for defined audience profiles. Use them for hypothesis generation and early screening, not as a substitute for validated quantitative research. Natural-language processing can organize open-ended responses and interview transcripts into provisional themes. Research-design assistance can draft questionnaires, flag possible wording problems, and propose discussion-guide structure. Analysis assistance can prepare an initial thematic pass, sentiment grouping, or draft summary. Predictive models can compare how segments may respond to an action. Their value depends on the data, design, calibration, and validation behind the prediction. Each tool changes a research task. None removes the need for method choice. ## Should synthetic respondents be treated as directional? Simulated respondents can help with hypothesis generation, instrument pretesting, early concept evaluation, and rapid mapping of a question space. They are not a final answer for high-stakes decisions. Published claims of 75 to 92 percent correlation depend on the platform, question, population, and persona definition (Psychology & Marketing reviewed the method across studies and found no single accuracy figure holds across tasks and evaluation choices). Correlation does not establish causal validity. Require the benchmark and method before using any range. Disclose when simulated respondents were used and what role they played. "Early hypothesis generation followed by human validation" is clearer than calling a simulated output market research without qualification. ## Four places to use AI in a mixed method ### Pre-study exploration Use defined audience models to surface themes, language, and possible concerns before formal fieldwork. Convert the result into sharper questions. ### Instrument pretesting Run the full questionnaire against five simulated perspectives as a planning example. Look for ambiguity, leading language, missing choices, and topics the instrument ignores. Human cognitive testing remains appropriate when the risk warrants it. ### Between-wave work Use simulation to generate explanations for a metric change between major waves. Label the result as a hypothesis and test it with real evidence. ### Segmentation exploration Compare the same stimulus across segment definitions. Use the divergence to focus later research on the differences that matter. Competitive-intelligence exercises need restraint. A simulated competitor customer cannot reveal private company plans or actual customer beliefs. ## A practical integration framework Exploration: use simulation to map the question and generate hypotheses. Validation: recruit people for the claims that matter to the decision. Analysis: use software for an initial thematic structure, then have the researcher verify and interpret it. Reporting: use software for a draft summary, then write the strategic narrative and recommendation from the evidence. The research program should record where each method enters and which claims depend on it. [figure: Five tool classes, each with one task: directional respondent answers, theme sorting from text, questionnaire drafting, initial thematic analysis, and segment response comparison.] ## What skills do researchers need to work with AI? Researchers need to design prompts and study inputs, detect model artifacts, build hybrid methods, and explain the limits to clients. Critical evaluation matters more than fluent output. Subconscious supports decision-specific causal experiments on product, pricing, messaging, and go-to-market actions. The researcher defines the decision, audience, alternatives, outcome, and validation plan. Software can increase the number of testable questions. Professional judgment determines which answers deserve trust. --- # When You Have No Co-Founder to Catch a Bad Call URL: https://subconscious.ai/blog/buyers/when-have-no-co-founder-catch-bad-call Published: 2026-08-19 A solo founder makes the pricing call, the positioning call, and the feature call alone. There is no co-founder to push back before it ships, no funded board to ask the hard question in a Tuesday meeting. The gut check a team would normally run informally never happens, so the first real test of the decision is the market itself. [figure: Five-step path: an untested decision gets framed as options, run as a test against real or modeled customers, read back as an effect with a confidence interval, then decided.] ## What it costs to skip the check A price only means something once it's set down in real numbers. Hiring that check is expensive. Budget $3,000–5,000 a month for a fractional CFO, or $200–500 an hour if you go the business-coach route instead. Most solo founders can't justify either for a single pricing or positioning question, so the decision ships untested. Research on solo-founder outcomes finds that funding and survival odds already run tighter without a co-founder to backstop a call (Greenberg and Mollick, "Sole Survivors: Solo Ventures Versus Founding Teams," SSRN), which raises the stakes on every call a solo founder makes without a second opinion. ## What is the alternative to a co-founder's gut check? The alternative to guessing isn't a roundtable of simulated expert opinions. It's a controlled test of how real people respond to the options on the table. Subconscious is a causal behavioral platform: define the decision (which price, which message, which feature framing), run it as a discrete-choice-style experiment against real target customers or a modeled audience, and get back an estimated effect with quantified uncertainty instead of a single confident take. Stating this plainly keeps a founder from confusing a second opinion with proof. A co-founder's gut check is still a guess, informed but untested. A structured test replaces the guess with evidence about what the decision actually moves. ## Putting a real decision through the test Take a concrete case: a solo SaaS founder is deciding whether to raise price or hold, after a competitor cuts theirs. The question isn't what an advisor would say. It's what happens to signups and revenue at each price point, tested against the customers who would actually buy. The same structure applies to a positioning pivot or a new feature bet: state the decision as a set of options, run the test, read the effect size and its confidence interval, then decide. ## Can you move from a modeled audience to real human participants? When a call is big enough to warrant it, the same causal question can move from a modeled audience to real human participants without changing what's being tested. The price levels, the message variants, or the feature framings stay identical; only the respondent pool changes. That gives a founder a way to raise confidence on a high-stakes call without redesigning the experiment. ## What doesn't Subconscious replace? A buyer can only check a tool's fit against a real decision if its limits are on the page. Subconscious does not simulate a panel of named advisor characters debating a decision, and it isn't a stand-in for real mentor relationships or investor diligence. It doesn't hand back an automated recommendation or a next-best-action. The founder still has to frame the decision and interpret what the estimated effect means for the business. It answers how customers would respond to a specific option, not what the founder should do with the company. ## Where to start Framing a pricing, positioning, or feature decision as a testable question is the first step. See how Subconscious tests a decision or look at applied examples. To run a specific call through a test, book time to scope it, or read more on the research behind the method. --- # Video Research or a Causal Experiment First: Choosing Between Voxpopme and Subconscious URL: https://subconscious.ai/blog/comparisons/video-research-causal-experiment-first-choosing-between-voxpopme Published: 2026-08-19 A consumer insights or CX leader deciding how to spend the next research budget faces one question: does this decision need a full video-based study with real customers, or can a controlled experiment on a simulated market screen the idea first? Get that sequencing wrong and a team either ships on a fast read that never needed real-human validation, or commits a full video research program to a hypothesis a faster experiment would have ruled out first. ## What Is Voxpopme Built to Answer? Voxpopme is a consumer intelligence platform built around video interviews with real customers, run through structured research programs and delivered into dashboards and enterprise reporting workflows (Voxpopme). That format is the right one when the question depends on facial expression, tone, spontaneous language, or the kind of open-ended discovery that only a person on camera can provide. Voxpopme has continued to build on that model, publishing structured "insight playbooks" aimed at turning video interviews into strategic decisions (Voxpopme). A video research program is a real production: define the question, recruit or field the interviews, collect and code the responses, then synthesize findings for a decision. That process is well suited to final-mile validation and to the kind of nuance a transcript can't reconstruct. ## Where Does a Causal Experiment Fit Before a Video Study? Subconscious is the causal AI company: randomized experiments on a simulation of a market, validated against real human behavior, that show which action, message, price, or feature is likely to move a specific outcome for a specific audience. That answers which action moves the outcome, not what a segment feels or says about it on camera. Used ahead of a video study, a controlled experiment on a simulated market can narrow several possible messages, prices, or feature framings down to the strongest candidates before real customers see any of them on camera. Used alongside a video study, it can test more variants than an interview budget could field, then hand the winners to a format like Voxpopme's for the qualitative depth a simulation can't produce. When a decision needs both a directional read and confirmation from real people, Subconscious can move from a simulated experiment to a study with real human participants without changing the underlying causal question being tested, checking the same experimental design twice rather than starting a second, unrelated study from scratch. Subconscious can also size a simulated population against a person-level audience graph covering 800 million real people when a study needs to model a specific market rather than a small convenience sample. Naming exactly what a data asset covers is how a buyer checks it before relying on it. That audience graph describes who the simulation can represent; it is not a recruitable panel available for video interviews. ## Matching the method to the stage of the decision ## What Does a Simulated Experiment Not Do? Publishing a method's limits alongside its results is what lets a buyer check the fit before committing budget. A controlled experiment on a simulated market does not capture facial expression, tone, or the open-ended discovery a real interview surfaces. It is not the right tool when the decision hinges on hearing customers describe a problem in their own words rather than on which of several actions is likely to move a measurable outcome. Confidence intervals, segment-level detail, and decision memos are specific to what a given study is designed to test, not a universal output attached to every result. A fidelity score published without its misses is just marketing copy. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market or a claim about matching Voxpopme's or any other vendor's specific study results. See the causal fidelity paper. ## Deciding which one to run first Start with the question, not the tool. If the open question is which of several actions is likely to move an outcome, and the team needs to compare more options than a video research budget could field, a controlled experiment on a simulated market is the place to start (how Subconscious runs a study). If the open question is why a segment feels the way it does, or a launch decision needs the final-mile confirmation only real participants can give, a video research program like Voxpopme's is the right instrument. Teams weighing both can review how comparable causal studies have been run in practice in case studies, explore the underlying method in research, or talk through a specific decision. [figure: Four stages routed to methods: screening to simulated experiment, hearing why to video, confirming before launch to real-human validation, tracking over time to repeated simulated experiments.] --- # Hire a Bayesian Expert or Buy On-Demand Access? A Buyer's Decision Framework URL: https://subconscious.ai/blog/comparisons/hire-bayesian-expert-buy-demand-access-buyers-decision-framework Published: 2026-08-19 A head of analytics who needs Bayesian or causal-modeling depth for marketing mix modeling, customer lifetime value, or causal inference is choosing between two paths: put a specialist on payroll full-time, or line up expert input for the moments it's actually needed. Hire too early and months of search time plus a full salary go to one person's narrow specialty. Defer expert input entirely and the team risks shipping unvalidated models, missing a governance deadline, or facing stakeholders who reject results it can't defend. ## What does a Bayesian expert actually do? This role builds models that express uncertainty as a range of plausible outcomes rather than a single number. Day to day, the work spans: - Designing prior distributions grounded in domain knowledge, not arbitrary guesses - Building hierarchical models that pool information across regions, products, or segments without letting any one dominate the fit - Running Markov Chain Monte Carlo (MCMC) sampling and diagnosing failures like divergent transitions and poorly mixing chains - Validating posterior outputs with prior predictive checks and unseen data - Turning probabilistic results into budget allocations, policy thresholds, and go/no-go decisions That combination of statistics, software engineering, and domain expertise defines the role. Practitioners strong across all three are uncommon; most are deep in one or two. Common applied problems teams bring to this expertise: - Marketing mix modeling (MMM). Isolating how much revenue each marketing channel drives, factoring in carryover effects, saturation curves, and diminishing returns. - Customer lifetime value (CLV). Combining purchase frequency, average order value, and churn probability into a forecast of a cohort's long-term revenue. - Causal inference. Measuring what an intervention actually caused, a price change, a launch, a policy, with no randomized experiment to lean on, through quasi-experimental designs like difference-in-differences, regression discontinuity, and matched synthetic controls. - Demand forecasting. Building time series models that carry uncertainty through, so planners work from a range of outcomes instead of a single guess. - A/B test analysis. Swapping p-value-based testing for Bayesian updating, which supports sequential analysis and states a treatment effect's probability directly. ## Why hiring is harder than it looks Few people can actually do this work. Real fluency in probabilistic programming (PyMC, Stan, NumPyro, or similar) combined with hands-on experience shipping those models to production is rare, and the people who have it cluster around research institutions and organizations with a long-running Bayesian practice. A mid-sized company running its first MMM or CLV program is competing for that talent against teams that have done this work for a decade. Hiring runs on a different clock than modeling does. Sourcing, interviewing, and onboarding a candidate takes three to six months, and even after that a new hire needs time to get up to speed on the data, model architecture, and organizational constraints before contributing. That timeline rarely fits a fixed delivery window: an annual planning cycle, a governance review, a product launch. One person cannot replicate cross-disciplinary depth. Production-quality Bayesian modeling draws on statistical rigor, domain knowledge, software engineering, causal inference, and time series methods, five distinct backgrounds at once. Taking a marketing mix model from a rough prototype to something a finance team will trust usually calls for practitioners spanning several of these backgrounds. ## What does on-demand expert access look like? On-demand Bayesian consulting gives a team a channel to experienced probabilistic modeling practitioners at specific moments rather than as permanent headcount. It differs from a scoped consulting project that ends when the deliverable ships: the relationship is ongoing, so each interaction builds on the last instead of resetting. It also differs from hiring: the team pays for access calibrated to actual need, not a full-time salary, benefits, and onboarding. Expert input matters most at five recurring moments: - Before a critical deadline. Annual planning cycles, governance reviews, and launches land on a fixed date, and a search-and-hire cycle cannot close a gap that needs solving in the next few weeks. - When model diagnostics fail. Complex hierarchical models routinely throw up divergent transitions, chains that won't mix well, and posterior shapes nobody expected. Someone who has already debugged that exact failure mode spots it far faster than someone meeting it cold. - When extending an existing model. Adding a channel, a time-varying component, or lift-test results raises structural questions about architecture, priors, and validation. A wrong early call compounds into technical debt that's costly to undo later. - When stakeholders push back on the methodology. Executives and reviewers frequently question probabilistic outputs, especially when they conflict with simpler attribution methods. - When one model needs to grow into a full measurement program. Going from a single build to a multi-channel, production-grade framework is a fundamentally different problem than the initial build. ## Hire vs. on-demand access, side by side There's no universal winner. A team that treats Bayesian modeling as a core product function, shipping weekly against a dedicated modeling roadmap, will likely need a full-time hire eventually. A team that leans on Bayesian methods for quarterly planning, is building its first MMM, or just needs expert review ahead of a governance deadline is usually better served by on-demand access. ## Where a causal experiment fits into this decision Neither staffing path replaces testing the underlying decision directly. Before committing budget to a hire or an access contract, a team can configure a causal experiment to test the specific action in question (a pricing change, a message, a launch decision) and see which outcome it moves, for which segment. Subconscious runs controlled causal experiments with quantified uncertainty on the result. Where a causal question needs confirmation with people rather than a simulated panel, Subconscious can move to real human validation without changing the causal question itself. This isn't either/or: it changes how much in-house modeling bench strength a team actually needs before scaling up. ## Limitations of this framework This comparison stays a decision framework, not a benchmarked cost-benefit analysis. There is no independent data, from Subconscious or elsewhere, on time-to-value or outcome differences between hiring a specialist and buying on-demand access. Treat the three-to-six-month hiring timeline and the five recurring on-demand moments above as structural patterns from the underlying source material, not as guarantees for any specific team or vendor. ## Next step Map your team's Bayesian modeling need against the two columns above: is it constant and core to your roadmap, or does it cluster around deadlines, diagnostics, and scaling moments? If the underlying question is really about testing a specific action before committing to either staffing model, see how Subconscious approaches causal testing or read more about the company. [figure: Two columns: full-time hiring breaks down on talent scarcity, a 3-6 month hiring cycle, and no single hire covering every discipline; on-demand access instead supplies expert input at the specific moments it's needed.] Bayesian and causal-modeling methods referenced here, including PyMC-based marketing mix modeling, follow the open-source PyMC-Marketing documentation. --- # What to Automate First in an Insights Team URL: https://subconscious.ai/blog/use-cases/what-automate-first-insights-team Published: 2026-08-19 A VP or Head of Insights facing pressure to "just use AI" is really facing a sequencing decision: which research work should move to AI-assisted exploration first, and which decisions still need real-human validation before they reach a stakeholder or an external audience. Get the order wrong and a team ships a synthetic-only finding as validated evidence, with a launch, pricing, or claims decision resting on an output nobody checked. [figure: A four-stage horizontal path: Exploration, then Directional testing, then Human review, then Validation, with the evidence standard rising at each stage before a claim can ship externally.] ## The bottleneck AI actually exposes Reports on the research industry describe AI moving into analysis, reporting, data preparation, and self-service insight work. That does not mean the demand for research judgment disappears. Federal labor-market data cuts against that narrative: the BLS occupational outlook for market research analysts still calls for the analyst and marketing-specialist workforce to expand across the 2024-2034 decade. The real risk is narrower: spending adoption energy on the most visible workflow instead of the safest high-volume one. When automation takes over a job's mechanical parts, whoever held that job needs to sit nearer the actual decision. That means sharper questions, better evidence choices, clearer caveats, and more influence over what a stakeholder does next. ## What does the role stop rewarding? The old advantage in insights leadership was partly about access: knowing how to field a study, clean the responses, and package the finding. AI narrows that advantage. More people can now draft a survey, summarize a transcript, or ask a synthetic audience for a first reaction. That does not make research expertise less valuable, but it does make that expertise easier to verify. Once anyone can generate an answer, the worth shifts to whoever can tell a trustworthy one from a merely fluent one, and can flag when an output is generic, badly grounded, or beside the point for the decision on the table. ## Build a sequencing rule, not an AI habit A workable rule has four layers: what AI is allowed to touch first, what a synthetic pass can compare directionally, what still requires a human check, and what still requires real validation. - Exploration. Use AI or a synthetic panel to generate hypotheses, objections, and alternative framings before committing research budget. - Directional testing. Compare concepts, messages, pricing stories, or journey moments quickly, across defined audience segments. - Human review. Before anyone treats an output as a finding, confirm the audience was defined correctly, the prompts stayed neutral, sources are traceable, and the business context holds up. - Validation. Move to real respondent data, behavioral data, or fielded research once the decision is expensive or the result will reach an external audience. This is the same separation Subconscious's own workflow is built around: run a controlled, decision-specific experiment against a person-level audience graph covering 800 million real people, then test or validate the same study with real human participants when the decision warrants it. The audience graph is a modeling resource, not a recruitable panel, and moving from simulation to real-human validation does not require changing the causal question. The Subconscious research program covers how that validation layer works, and the case studies show teams applying it to specific launch and pricing decisions. ## Where do teams get this wrong? Teams often default to whichever workflow draws the most internal attention, when the safer choice is the one with the highest volume and the lowest stakes. That default usually traces back to pressure: a deck needs a conclusion and a fluent AI answer is sitting right there. A useful draft is not automatically a valid answer to the decision facing a stakeholder. The fix is to make the limit part of the deliverable: spell out where the AI-assisted work applied, where it stopped, and which validation step still has to happen before anyone repeats it outside the team. That framing reads as more disciplined, not less confident. ## A sequencing checklist for one workflow Rather than automating the whole job at once, pick one live project and run it through a single pass: - Write the business decision in one sentence. - Define the audience and how much is riding on the answer. - Limit AI or a synthetic panel to the exploratory and directional stages, nothing further. - Go through the output by hand and flag what holds up, what's shaky, and what shouldn't be repeated. - Present the answer with a clear caveat and a named next validation step. Repeat that once a week for a month on the same workflow before adding a second one. The repetition builds a working system, not just a list of tools, and shows speed, judgment, and where the evidence still needs to catch up. Teams weighing where this kind of experiment-then-validate approach fits their own stack can review how Subconscious's process works end to end. [figure: Two-column comparison: the workflow picked for drawing attention and answering deck pressure, versus the workflow actually safe to automate first, high-volume and low-stakes.] ## What limit is worth stating plainly? None of this replaces researcher judgment, produces an automated recommendation, or models financial return on its own. A synthetic or AI-assisted pass answers a narrower question: which tested action moved a defined audience, and by how much, under the conditions in the experiment. Treat that as a reason to run a faster first pass and a clearer validation step, not as a substitute for either. --- # Why Release Notes Need a Pre-Publish Read Test URL: https://subconscious.ai/blog/use-cases/why-release-notes-need-pre-publish-read-test Published: 2026-08-19 Release notes need a pre-publish read test because a single draft is written from the product's perspective and fails discovery, adoption, trust, or archive value for at least one reader segment. Release notes go out more often than anything else a product team writes: every sprint, every launch, even the quiet patches. The process rarely changes: someone squeezes in half an hour on Thursday, the only reviewer is another engineer, and it publishes without anyone checking whether a reader would follow it. ## The decision: test the draft or ship it cold The Head of Product or Product Marketing Manager who owns the changelog faces a repeated choice: run a structured comprehension check against defined reader segments before every release, or keep relying on ad hoc internal review. A note that reads clearly to the person who wrote it can still confuse its intended reader. ## Why do release notes carry more risk than they look? Release notes do several jobs at once, and a single draft rarely does all of them well. - Discovery: a reader scanning to find out whether anything they care about changed, wants scannable headers and a clear signal of new versus updated versus fixed. - Adoption: a reader deciding whether a new feature is worth trying. "Add tags to messages" describes a feature; "organize conversations by project to find threads faster" describes a reason to use it. A note that only does the first leaves adoption flat. - Trust: a reader checking whether the team is shipping and whether reported issues get fixed. A vague fix list erodes trust; a specific one builds it. - Archive: a reader months or years later, support staff, a new hire, someone tracing when a security behavior changed. A note clear on launch day can become unusable after six months if it leaned on jargon from a forgotten release. A peer-reviewed large-scale study of Google Play apps found that more informative release notes are associated with higher average user ratings (Empirical Software Engineering, Springer Nature). ## Reader segments worth testing separately The most common failure is writing from the product's perspective instead of the reader's: "added support for nested tags" instead of "organize tags into folders for cleaner navigation." That failure shows up differently depending on who is reading, which is why one draft misses more than one reader type. ## A pre-publish sequence, not a single review pass Releases that hold up under read testing go through a few distinct checks rather than one editorial pass: - Prioritization, before the draft. Show the reader segments the unedited list of what shipped and ask which items they'd stop to read and which they'd skip. This determines which item gets a paragraph and which goes in the fine print. - Scan test, on the first draft. Ask what the single most important thing in the release is, on a short skim. If different reader types land on different answers, that's expected. If a segment can't identify anything, the headers and lead lines need rework. - Comprehension test, on a fuller draft. For each item: do I understand what changed, why it matters, and what to do next. - Support-load test, before publish. Ask which items are most likely to generate a support ticket, so the note can answer those questions preemptively or support can prepare before the email goes out. - Archive test, after publish. Read the notes as a reader arriving months later searching for a specific change. Notes clear at launch are often opaque without edits for self-containment. ## Where does a causal test fit into this decision? Subconscious runs controlled experiments against defined buyer or reader segments to compare which draft of a decision-facing document produces the clearest, most action-enabling response before it ships. Applied to release notes, that means comparing wording for the same change across the reader types above and measuring which version produces comprehension and the intended next action. This is testing a messaging or communications action before committing to it, not an automated copywriting tool. Where the question turns on something a simulated read can't fully settle, an ambiguous security or billing change, for instance, a team can move from a simulated comparison to real-human validation without changing the underlying question. ## What does this not replace? A structured pre-publish test does not turn release notes into a support triage system, and it doesn't substitute for the engineering work of writing an accurate changelog. It narrows the gap between what the team believes it wrote and what the reader takes away, before that gap becomes a ticket queue. It's a way to catch a miss before the send button, not a promise every release reads perfectly. ## Next step Release notes are a recurring, low-stakes-per-instance decision that compounds across a year of shipping. Teams can review examples from other launch and messaging decisions or set up a scoped test against an upcoming release. [figure: Five steps in order: Prioritization before drafting, Scan test on the first draft, Comprehension test on a fuller draft, Support-load test before publish, Archive test after publish.] --- # Estimating a Private-Market Benchmark When There Is No Public Price URL: https://subconscious.ai/blog/methods-and-validation/estimating-private-market-benchmark-when-there-no-public-price Published: 2026-08-19 A public equity index is built from transaction prices that happen constantly: every trade updates the number. A private-market portfolio has no daily price tick, no closing quote. GPs do report quarterly appraisal-based NAVs, but those marks are stale and smoothed, which is why the benchmark is inferred instead from the capital calls a fund draws down and the distributions it eventually pays out. A quant or risk team that needs a benchmark for that portfolio has to infer one, and the inference method determines whether the resulting number is honest about how uncertain it is given the model it conditions on. ## The two ways to turn cash flows into an index One published approach, described in an NBER working paper on estimating private equity returns from limited partner cash flows and its companion paper, treats the return series as a latent variable: a hidden path that is never observed directly, only inferred from the pattern of calls and distributions over time. A Bayesian implementation of that model uses priors (starting assumptions about plausible ranges for the hidden path) to constrain the inference, then reports a full posterior distribution rather than a single number, giving the resulting index its uncertainty band. A standard machine-learning fit on the same cash-flow data can also produce an interval, through bootstrapping, quantile regression, or conformal prediction, but that interval doesn't condition on the same prior assumptions the Bayesian posterior does, and isn't interpreted as a probability statement about the hidden path the way a posterior is. ## What changed when the model was rebuilt with modern samplers? A documented applied engagement brought that published model into production for a risk-workflow platform serving multi-asset portfolios. The original paper used first-generation samplers, slow enough to limit how much the model could be explored or iterated on. Rebuilding it on modern samplers sped up each run, giving more chances to check the model against domain knowledge, adjust it, and refit before committing to an estimate. The extra work was not about chasing a better-looking number or predicting next quarter's return. The client wanted an explanation of why the inferred trend line behaved the way it did, which comes from the model's structural specification (latent returns, factor loadings, cash-flow mapping) rather than from Bayesian estimation itself; a maximum-likelihood fit of that same structural model would be equally interpretable, while a black-box fit optimized only for a good fit to the data would not. ## How was the result read against a known benchmark? The output was a cumulative-return index compared against two references: the public US stock market and the venture capital index published by Cambridge Associates, an industry-standard performance benchmark for private funds. The inferred index tracked its own distinct path relative to both. Cambridge Associates is NAV- and appraisal-based, so it is smoothed and beta-suppressed relative to a series built from the portfolio's own cash flows, which is the substantive reason to expect divergence between the two. The value of the comparison was not that the inferred index matched Cambridge Associates, but that it could be checked against an external reference at all, because it came from a documented, repeatable model rather than an unexplained trend line asserted without a method behind it. [figure: Three cumulative-return lines: the public stock market, the Cambridge Associates VC benchmark, and the inferred private-equity index built from the portfolio's own cash flows, each taking a visibly different path.] ## Where does this pattern apply, and where does it not? The lesson extends past private equity: any time the number a team wants does not exist as a directly observed transaction and has to be inferred from indirect signals, the choice between a point-estimate model and an uncertainty-aware Bayesian model is a choice about how honestly the result represents its own confidence. A risk report built on a point estimate with no band invites the reader to treat an inference as an observation; one built on a posterior distribution keeps that distinction visible. Subconscious is built on the same family of uncertainty-aware inference this case illustrates, latent-structure estimation with quantified uncertainty, though Subconscious's own effects are identified causally through randomized experimental design rather than inferred from a latent state-space model. That is a methodological parallel worth naming, not a claim that Subconscious builds private-market return models. Subconscious does not ingest fund cash-flow data and does not produce private-equity benchmarks; the research program applies the same discipline of a stated uncertainty band to a different question: what a market will actually do in response to a tested decision, not what a private fund's unobserved return path looked like historically. Teams evaluating either kind of model should ask the same question before trusting a number: does this range state what it conditions on, or is that left unsaid? How Subconscious runs that kind of test and what a first test looks like start from that same requirement. [figure: Branching diagram: cash-flow data feeds two methods. The ML fit ends in one trend line with no band. The Bayesian model ends in a trend line with a shaded credible band, both checked against an external benchmark.] --- # A Triage Rule for the Solo Consumer Insights Manager URL: https://subconscious.ai/blog/implementation-and-procurement/triage-rule-solo-consumer-insights-manager Published: 2026-08-19 A solo insights manager cannot run a full recruited-participant study for every request that lands on their desk. Product wants a reaction to three onboarding flows, marketing wants a gut check on ad copy, sales wants a positioning read before Friday. Say yes to all of it and the researcher becomes the bottleneck holding up every launch. Say no reflexively and stakeholders make pricing and positioning calls on gut feeling alone. The fix is not more headcount. It is an explicit rule, applied before any fieldwork starts, for which requests earn a fast directional pass and which earn a fully validated study with real participants. ## Why can't every request get the same treatment? A one-person research function has a fixed budget and a fixed calendar, and every request competes for both. Treating a minor packaging tweak the same as a final pricing decision means either the pricing decision gets shortchanged or the packaging tweak eats weeks it does not need. The U.S. Bureau of Labor Statistics tracks market research analyst work broadly; the tension of turning limited capacity into decisions the business can trust holds whether the analyst sits on a team of twenty or a team of one (U.S. Bureau of Labor Statistics). The rule that resolves this is a triage decision, not a research methodology: sort every incoming request along two axes before choosing a validation path. - Financial risk. How much money moves if the decision is wrong, comparing an ad headline against a final price point. - Strategic stakes. How hard the decision is to reverse once shipped, comparing a copy edit against a product pivot. Low-risk, low-stakes requests do not need a recruited human panel. High-risk, high-stakes requests should never skip one. ## A four-step workflow for running the rule ### Step 1: Assess the decision risk at intake When a stakeholder submits a request, ask what breaks if the answer is wrong. A minor creative tweak or a copy variant is a candidate for a fast simulated pass. A pricing model or a major positioning shift is not; route it toward recruited human validation from the start. Ask the requester for a concrete artifact, such as a concept, a claim, a landing page, or a specific question, rather than open-ended strategy language. ### Step 2: Run a controlled simulated pass on the low-stakes requests For anything scored low or medium risk, run a controlled experiment against a simulated population before spending recruitment budget. Subconscious runs this as a causal test: the concept, claim, or variant is treated as a controlled intervention, and the platform reports a causal effect with a confidence interval rather than a raw sentiment score. That distinction matters for a solo researcher defending a recommendation. "This variant caused a measurable shift in stated preference" holds up in a stakeholder meeting in a way "people seemed to like it more" does not. ### Step 3: Refine before you spend recruitment budget Read what the simulated pass surfaces, including which objections came up and which claims did not land, and revise the concept or the questions before running it again. This loop is cheap enough to repeat multiple times, so the version that eventually reaches a recruited human study is already the strongest candidate, not the first draft. ### Step 4: Reserve recruited human fieldwork for the requests that earned it Once a request scores high on financial risk or strategic stakes, move it to validation with real participants. By the time a pricing or positioning decision reaches this stage, the obvious flaws are already gone, so the recruited study tests a refined concept instead of a first draft. Subconscious can carry the same causal question from a simulated pass into real-human validation without redesigning the study. ## What is the simulation actually testing? The causal register is what separates a directional read from a guess. A controlled experiment run against a simulated population isolates one variable, such as a price point, a headline, or a feature name, and reports how much it moved a stated preference or choice, with a confidence interval attached. That is different from asking a model what it thinks of a concept and treating the reply as data. Case studies that pair a simulated first pass with recruited human validation show the same causal question answered twice, once fast and once at full rigor, rather than two different questions being compared. ## Where simulation stops being enough A simulated population is built on historical and established behavioral patterns. As independent validation work, including commercial pilot studies run by outside firms, has found, correlation between simulated and real human responses on directional questions has landed in the 80 to 90 percent range as a historical benchmark, and one commercial platform's own benchmarking has reported a range as high as 80 to 95 percent against historical human data. Those figures describe past benchmarking exercises, not a guarantee for any specific study, and they do not extend to every use case: - A simulated population cannot predict novel behavior in a category with no historical analog, such as a genuinely new product category, a sudden macroeconomic shift, or an unprecedented event. - A simulated population is not built for statistical population estimates with defined confidence intervals suitable for regulatory or audit purposes. That kind of proof requires recruited human respondents. - A simulated population does not experience physical reality: shipping delays, real financial transactions, the physical feel of packaging. Longitudinal tracking of real customer cohorts still needs real-world behavioral data. [figure: Two columns. Causal test: isolate one variable like price or headline, report how much it moved stated preference, with a confidence interval. Opinion ask: ask a model what it thinks, treat the reply as data.] ## Putting the rule to work on the next request The next time a stakeholder request lands, the question is where it falls on risk and stakes, not whether there is time for it. Testing the framework on a live request is the fastest way to see whether the routing holds: book a walkthrough with a specific request already in hand. [figure: Four-step path. Step 1 scores risk and stakes. Low/medium risk goes to step 2, a simulated pass. Step 3 refines results. Only high-risk, high-stakes requests reach step 4, recruited fieldwork.] --- # Conversation-Led Exploration vs. Causal Experiments: Which Evidence Can Support a Market Decision? URL: https://subconscious.ai/blog/comparisons/conversation-led-exploration-vs-causal-experiments-which-evidence-support Published: 2026-08-19 A fluent conversation can help a team explore an idea. It cannot, by itself, show whether changing a price, message, feature, or launch plan will change customer behavior. A consequential market decision needs a comparison across defined actions, for a defined audience, against a defined outcome. The buying question is not which conversation feels most human. It is whether the evidence can identify the action most likely to move the outcome that matters. [figure: Two columns: conversation (no comparison, one viewpoint, themes, fits ideation) vs. causal experiment (alternatives compared, defined audience, estimate with uncertainty, fits launch decisions).] ## Decide what the evidence must support Conversation-led exploration and causal research serve different jobs. The first can generate possibilities, expose assumptions, and help a team sharpen its questions. The second is designed to compare actions before budget, roadmap capacity, or sales effort is committed. That boundary matters: plausible language is not behavioral evidence, even when a response is coherent and useful. ## Two methods, two standards of evidence The table compares evidence methods, not competing products. One prominent consumer character-chat service describes its own experience around chat, roleplay, and character creation, relevant to entertainment and open-ended interaction, not a substitute for a controlled market test. ## The costly error is confusing engagement with causality A satisfying exchange answers whether the interaction can continue coherently. It does not create the counterfactual a buyer needs: what would have happened if the same audience had encountered action B instead of action A? Without that comparison, a team can mistake a persuasive response for evidence that a decision will work. That is a familiar say-do gap in a new interface: capital, roadmap capacity, and commercial effort can all follow an action that was never tested against a credible alternative. ## How should research be built around the action? Subconscious runs controlled causal experiments around four explicit elements: the decision, the audience, the alternatives, and the outcome. The result estimates which action moves the specified outcome, with uncertainty stated where the evidence supports it. A team can then validate the same causal question with real human participants. That check evaluates whether the result holds against real behavior, not whether it becomes a different study. Audience reach is a separate capability. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. A number earns trust only when its scope sits right next to it. That figure describes the graph available for audience definition and experimental reach, not the number of recruited participants in real-human validation. ## Know what the result cannot settle Naming where a method can go wrong is what lets a buyer trust the result. A controlled experiment is only as useful as its specification. The wrong audience, alternatives, or outcome can produce a precise answer to the wrong business question. Uncertainty must remain visible where the evidence does not support a stronger conclusion. A limit stated plainly is what turns a claim into something a buyer can verify. Real-human validation has a clear boundary. It does not turn a causal action test into an observed usability session, a clinical trial, or an automatic guarantee of market performance. Qualitative conversation remains valuable when a team needs to generate ideas, explore language, or identify questions worth testing. ## What decision rule should you use before choosing a method? If the cost of being wrong is low and the goal is exploration, a conversation may be enough. If the decision commits money, roadmap capacity, or a market position, require a defined audience, explicit alternatives, a measurable outcome, and a comparison that can distinguish one action from another. Talk to the Subconscious team when a pricing, messaging, product, or launch decision needs that standard of evidence. --- # How Government Communications Teams Can Pre-Test Public Messaging Before Launch URL: https://subconscious.ai/blog/industries/how-government-communications-teams-pre-test-public-messaging-before-launch Published: 2026-08-19 A government communications team can compare draft public messages against a defined citizen population before launch, using a randomized experiment on a simulation, validated with real human participants where warranted. This does not replace statistically valid polling, protected-group research safeguards, legal review, or accountable public decision-making. [figure: Five-step path: concept, copy, channel, and final assets each run a causal experiment on a simulated population; human validation is set apart, applying only to the highest-exposure decision.] ## Why Does This Decision Carry More Risk in the Public Sector? When a private brand ships weak messaging, the cost is a poor campaign and lost revenue. When a ministry, public health authority, or municipal communications team ships weak messaging, the cost can be a news cycle, a parliamentary inquiry, or a breakdown in public trust that outlasts the campaign itself. The evidence available to public sector communicators before launch is often thinner than what a private brand would use for a comparable decision. ## What Causes Public Messaging to Ship Untested Three structural pressures explain why: Procurement and timeline mismatch. Government research procurement is slow relative to campaign cycles, and market research documentation practices at federal agencies have drawn their own scrutiny (GAO, Market Research: Better Documentation Needed to Inform Future Procurements at Selected Agencies). By the time a research contract clears procurement, the campaign has often already launched, so the evidence arrives too late to change the work. Citizen sensitivity. Citizens are not customers, and asking them about a government message risks turning the question into a political story. Research on public messaging has to be carefully scoped, adding time before a single question is drafted. Topic sensitivity. Public health, immigration, taxation, family policy, and energy transition are all politically charged. Internal stakeholders often disagree about what to test and how to interpret results, consuming time that should go to testing the message. ## Where Does Subconscious Fit In? Subconscious is the causal AI company. A communications team can run randomized experiments on a simulation of a defined citizen population, compare message or intervention alternatives, and see which one is more likely to move a stated behavioral outcome: comprehension, trust, or intent to act. Two grounding claims matter for this use case: - Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, for population-level reach rather than a small convenience sample. This is a modeled audience graph, not a recruitable panel of 800 million people who answer questions on demand. - Subconscious can test or validate a study with real human participants. A team can move from a simulated comparison to a real-human check without changing the underlying decision or population. The practical advantage: instead of one research round per campaign, a team can run a comparison at each message, headline, or channel change, and reserve real-human validation and statistically valid polling for the decisions that carry the most political or legal exposure. ## What a Faster Testing Cycle Looks Like Costs and timelines for traditional public sector research vary widely by country, procurement rules, and study design, so specific figures are omitted; the structural difference is in when evidence arrives relative to the launch decision. ## An Illustrative Six-Week Workflow This is a generic illustration of how a communications team could sequence testing across a campaign, not a record of a specific engagement. Assume a ministry is rolling out public messaging for a newly expanded parental leave entitlement, with three goals: build awareness, get more qualifying families to actually use it, and land the policy as supportive rather than paternalistic. - Week 1: define the population and validate the brief. The team specifies the eligible population and broader audience, then compares the brief's core assumptions to check whether citizens understand the current system well enough for new messaging to land. - Week 2: compare concepts. Three message concepts are compared: the practical entitlement (how much leave, at what rate), the family benefit (more time with a child), and shared parental responsibility. The comparison shows which concept moves stated intent to use the entitlement and where the effect differs by segment, for example if a family-benefit framing under-performs with one parent in a two-parent household. - Week 3: test copy and calls to action. Draft headlines, hero copy, and the call to action are compared for confusion and unintended readings before any final creative is produced. - Week 4: check channel-specific treatments. The same core message, adapted for digital, broadcast, print, and transit placements, is compared to catch a treatment that reads as patronizing in one channel but not another. - Week 5: brief decision-makers with evidence. The campaign team walks into the approval meeting with a specific comparison result behind the recommended concept, rather than an unvalidated preference. - Week 6: final check before launch. A last comparison on the polished assets catches remaining copy issues before the campaign goes live. ## Where a Causal Experiment Is Not the Right Tool ## Limitations and Failure Conditions - A comparison result describes a modeled population's likely response to a specific alternative. It is not a statistically valid estimate of public opinion and should not be cited as one. - Legally protected categories in the target population require the same data protection and research safeguards as any other public research involving those groups. - A crisis unfolding in real time needs live social listening and polling, not a pre-launch comparison run before the crisis existed. - The method depends on the team being able to name the population, the specific alternatives, and the outcome in advance. An open-ended question ("what do citizens think of us") is a poor fit. ## How Should a Team Get Started? A team new to this approach usually starts with one campaign: define the population and outcome that matters, compare a small number of message alternatives, and read the result before deciding whether to expand the practice to every campaign asset. The use cases overview covers how the same method applies to public opinion, polling, and policy research alongside commercial decisions, and the research program explains how replication against human baselines works as the trust layer behind any comparison result. Teams ready to scope a specific campaign decision can bring it to a working session. --- # What Agent-Run Market Research Changes, and Where It Still Needs a Human URL: https://subconscious.ai/blog/use-cases/what-agentic-market-research Published: 2026-08-19 An AI agent can now take a research brief, choose an audience, run a synthetic panel against that audience, and hand back a summary of findings, with no person touching any step in between. That capability puts a decision in front of marketing and product leaders: let the agent run the whole loop unattended, or hold a checkpoint inside it. The right answer depends on what the finding is about to pay for, not on how much of the loop the agent handles. ## How the loop compresses Conventional research moves through separate stages: someone writes the brief, recruits respondents, fields the questions, analyzes the responses, and writes up a report. Each stage waits on the one before it, with its own lead time. An agent-run version folds those stages into a single pass. It takes the brief, selects an audience it judges to match the request, asks the questions of a synthetic panel, and returns clustered themes, tensions, and a recommendation. Three weaknesses of the conventional pipeline explain why agents took hold first. Recruiting real respondents is slow and costs money before an answer comes back. Reading and clustering hundreds of open-ended responses is repetitive work that a model handles well. And because each study is expensive to run, most teams field it once, write the report, and move on, even when the underlying question deserves five more angles. Removing those frictions turns research from a scheduled project into a check a team can run as often as a decision comes up. ## What breaks when nobody checks the result? Compressing the loop does not make a bad read less risky; it removes the pause that might have caught it before anyone acted. A pricing, launch, or messaging call made off an ungrounded synthetic-panel finding does not fail in the research report. It fails downstream, in the campaign spend, the sales pitch, or the product build that decision funded. By the time the gap between the panel's answer and how real buyers respond shows up, the budget behind it is already spent. ## The layers underneath an agent-run study Three components have to work together for an agent to run a study. An agent receives the brief and decides how to act on it. A protocol lets that agent discover and call research tools without a custom integration for each one; the Model Context Protocol, published by Anthropic, is the standard most agent tooling has converged on. And a research platform has to expose panels, audience definitions, or survey infrastructure as something the agent can call, rather than just a dashboard a person reads. That stack is still young. Most of what exists today handles the ask-a-question, get-an-answer pattern well. Turning a returned finding into a decision safe to act on is a separate problem, and it is the one a checkpoint solves. [figure: A left-to-right decision path with five stages: a written brief feeds an agent-run synthetic panel, which passes through a human validation gate, producing a confirmed causal effect that becomes a shipped decision.] ## Where does the checkpoint have to sit? The checkpoint does not need to slow down every study an agent runs. It needs to sit between a synthetic-panel finding and the point where that finding starts spending budget: a launch date, an ad flight, a pricing change, a positioning line. Subconscious is built around controlled experiments that estimate which action is likely to move a specific behavioral outcome, with uncertainty reported where the study design supports it, rather than a single confidence score standing in for the whole read. Subconscious can also test or validate a study with real human participants, so a team can move from a simulated read to a real-human check without changing the underlying causal question. Explore the research behind that method, or see how the two-step process works in practice. ## What an agent cannot shortcut Three constraints hold regardless of how much of the loop an agent owns. The brief still has to be specific: a vague question produces a vague answer no matter how the panel is run. A synthetic read still has to be checked against something real before it carries weight, on a cadence the team sets on purpose, not skips because the study got easy to repeat. And strategy still needs a person: an agent can run the experiment, but deciding what the business does with the result is not a step to hand off. [figure: Three stacked layers an agent-run study needs: an agent interpreting the brief, a protocol like MCP calling research tools, and a platform exposing panels to call, not just a dashboard to read.] ## Next step Teams adopting agent-run research get more value from deciding up front which decision classes require a human checkpoint than from trying to bolt one on after a launch goes wrong. Review a few case studies of decisions taken through that process, or book a walkthrough to map where a checkpoint belongs in your own research loop. --- # Why a 20% Retention Rate Means Different Things for a 10-User Cohort and a Million-User Cohort URL: https://subconscious.ai/blog/methods-and-validation/why-20-retention-rate-means-different-things-10-user-cohort-million-user Published: 2026-08-19 A marketing analytics lead looks at a dashboard and sees a young cohort retaining at 20%. The number reads the same whether it came from 10 users or 1 million. It isn't the same number. A ratio hides how much confidence you should place in it, and that gap is where teams greenlight a channel or a campaign off a signal that was never statistically load-bearing, then reallocate budget when the next cohort regresses to a different rate. ## Why a retention ratio is the wrong thing to model Retention is always between zero and one, and it's a quotient: active users divided by cohort size. Two cohorts can post the same 20% and carry completely different amounts of information. Treating those as interchangeable collapses the uncertainty that separates them. The fix is to stop modeling retention directly and instead model the count of active users with a likelihood, so retention becomes a scored estimate rather than a raw division. ## How do you model active users instead of a ratio? A binomial likelihood ties the number of active users to cohort size and an underlying retention probability: where N is cohort size and p is the retention probability the model estimates rather than assumes. A link function then relates p to cohort age, elapsed time, and seasonality. This structure produces a credible interval around every retention estimate, not just a point value, so a 20% rate from 10 users comes back with a wide interval and a 20% rate from a million users comes back tight. A companion revenue model follows the same logic: revenue per cohort-period is modeled with a distribution tied to the number of active users and an average-revenue-per-user parameter, so retention and revenue share the same cohort-level uncertainty rather than being reconciled after the fact. ## What this buys forecasting, specifically Juan Orduz's Bayesian treatment of this approach, built on a binomial likelihood for retention paired with Bayesian Additive Regression Trees (BART) for the non-linear feature interactions, reports that credible intervals widen for smaller and younger cohorts while the same pooled structure still produces forecasts for very young cohorts with little data of their own (Orduz, "Cohort Revenue & Retention Analysis: A Bayesian Approach," arXiv:2504.16216). That's the practical payoff: a model that tells you when to trust a young cohort's number and when to wait for more data, instead of a dashboard that reports every ratio with equal confidence. ## Where does this discipline generalize? Quantifying uncertainty explicitly, rather than reporting a point estimate and moving on, is the same standard Subconscious applies before treating a result as a decision input: an estimate is only as useful as the confidence interval attached to it. It isn't a claim that any specific cohort pipeline runs inside Subconscious's product. ## What are the limitations, and what doesn't this prove? This is a modeling-methods explainer, demonstrated on a synthetic dataset with known ground truth so the model's accuracy could be checked against it. It's a useful pattern for anyone building cohort-level CLV forecasts, not a validated production benchmark, and it doesn't establish results in a live, messy revenue dataset. Extending it, by layering in acquisition channel as a covariate, pooling across hierarchical markets, or swapping BART for a neural-network component, is a reasonable next step for a team with the engineering capacity to build and maintain it, not a guarantee of the same interval widths or forecast accuracy on a different business. ## Next step If the open question is whether a channel or campaign result is real or just a young cohort's noise, Subconscious's leaderboard shows how causal estimates get reported with their own uncertainty, and how Subconscious works walks through the same estimate-with-confidence-interval discipline applied to a decision before it ships. [figure: A causal-chain diagram showing a raw 20% retention ratio moving through cohort size and a binomial likelihood to a credible interval and a forecast decision] --- # Why Synthetic Consumer Ratings Need Better Elicitation URL: https://subconscious.ai/blog/methods-and-validation/why-synthetic-consumer-ratings-need-better-elicitation Published: 2026-08-19 Synthetic consumers are easy to generate and hard to trust. Ask a language model for a score on a 1-5 purchase-intent scale and it may overuse the middle, avoid extreme answers, and produce a distribution unlike a human panel. The problem isn't just which model answers the question; the method used to elicit and score the answer can change the result. A 2025 preprint tested a method called Semantic Similarity Rating, or SSR. Instead of demanding a number from the model, SSR lets the model answer in natural language. It then compares that answer with reference statements for each point on the rating scale and converts the similarities into a probability distribution. The distinction matters for any team using simulated buyers to test a product, price, or message: a familiar survey format does not guarantee familiar response behavior. [figure: Four steps: model gives a natural-language purchase-intent answer, it's compared with reference statements per scale point, that becomes a probability distribution, then checked against a human survey baseline.] ## Why direct numerical ratings break down A prompt that asks a language model for a single rating forces it into a narrow response format, without checking whether the resulting distribution behaves like human survey data. Direct ratings can cluster around 3 on a 1-5 scale and suppress disagreement, making concepts look more similar than they are. A team may get a clean table of scores while losing the variation needed to rank ideas or spot a weak concept. SSR separates two tasks: - The model explains its purchase intent in ordinary language. - A semantic mapping step compares that answer with reference anchors and assigns probabilities across the rating scale. ## What the evaluation found The evaluation drew on 57 real consumer surveys (9,300 human responses) fielded by a large consumer products company; method-level comparisons are made at the survey level. Its reported results included: - Product rankings that matched human survey rankings 90% of the time - Response distributions that lined up with actual survey results by more than 85% - Wider, more discriminating response patterns than direct numerical prompting The evaluation also compared GPT-4o with Gemini 2.0 Flash and tested whether age, income, and product category changed the synthetic responses in plausible ways. The method did not require task-specific training data or fine-tuning. Those results support a narrow conclusion: text elicitation followed by semantic mapping reproduces these aggregate survey patterns better than asking a language model for a rating directly. They do not establish person-level fidelity, universal validity across categories, or a replacement for human research. See Subconscious's research for how elicitation and mapping choices are validated before a study result is used to support a decision. ## What a buyer should ask before using the result An aggregate match can still hide errors that matter to a decision: before using a synthetic panel, inspect the validation at the level where the business choice will be made. ### Does the method preserve distributions? A matching average can conceal variance collapse, so compare the full distribution, not just the mean. Look for missing extremes, excessive neutral responses, and concepts that the synthetic sample fails to distinguish. ### Does it preserve rankings? Early product screening often depends on ordering concepts rather than predicting an exact score. Ranking agreement should be reported separately from distributional similarity because the two measures answer different questions. ### Does it work by segment? Aggregate prediction is usually easier than individual or subgroup simulation. A method can match the total sample while flattening differences by age, income, culture, or category experience; segment claims need their own evaluation. ### Does the validation match the intended use? Purchase intent is not the same as satisfaction, trust, relevance, or actual purchase behavior. A method validated on one response type should not be assumed to transfer to another: new questions, categories, scales, and populations need fresh checks. [figure: Four-item checklist: distributions preserved, rankings preserved, works by segment, matches the intended question type.] ## A practical research sequence Synthetic evidence is most useful when it narrows a decision before a more costly commitment. A disciplined sequence is: - Define the exact product, pricing, messaging, or launch choice. - Generate natural-language responses from a specified target population. - Map those responses into a measure with stated reference anchors. - Compare distributions, rankings, and segment behavior against a human baseline. - Use the synthetic result to screen scenarios, not to claim certainty. - Reserve human research for validation where the decision risk warrants it. This workflow treats calibration and validation as part of the product, not as a footnote. The full preprint provides the mathematical details and experiments. Subconscious's how we work page describes how a synthetic study like this one can move to real-human validation without changing the underlying question. The useful lesson is simple: better prompts alone do not make synthetic research reliable. The elicitation method, mapping procedure, and human baseline determine whether the output can support a decision. --- # Testing a New Offer on a Limited Runway, Before You Build It URL: https://subconscious.ai/blog/buyers/testing-new-offer-limited-runway-before-build Published: 2026-08-19 A founder or small-business owner with limited time and capital has one real decision on the table: which product, offer, message, audience, or price deserves the next few weeks of work. Getting it wrong costs runway that does not come back, and can leave the founder confident in the wrong idea for the wrong reason. Three friends said it sounded good. Subconscious can compare a small set of plausible commercial actions for a defined buyer before the founder builds, buys media, or commits to a price. A test result carries meaning only when its limits are published alongside it. This test does not replace shipping, selling, or watching what real customers do. It narrows the range of expensive mistakes before the founder makes them. [figure: A four-step horizontal path: Define the buyer, then Test the offer as written, then Compare price points, then Compare messaging variants, each step feeding the next before the founder builds anything.] ## Why doesn't asking five friends work? Feedback from a personal network is a convenience sample, not a market signal. Friends are not the target buyer, a Slack group is an echo chamber, and neither will tell a founder that the price is wrong or the headline is confusing. A traditional user-research firm, as a planning example, has been quoted around ten thousand dollars for a single engagement, also out of reach. Confidence built on a handful of friendly responses leads a founder to spend weeks building something the actual market will not pay for. The cost of skipping validation is not usually a bad product. It is a good product with the wrong price, the wrong headline, or the wrong first audience. A small number of structured comparisons could have caught those mistakes before the work started. ## What a resource-limited founder actually needs to test A founder does not need a research department. The recurring questions, asked at every stage of the business, are: - Is the offer wanted by the intended buyer? - Does the message make that buyer care enough to act? - Will they pay the price being considered? A traditional research firm typically covers this through three separate engagements: a focus group, a survey, and a pricing study. As a planning example only, not a current Subconscious price, delivery time, or guarantee, that combination has been quoted at around $25,000 and six weeks. That puts it out of reach for most solo founders and small teams. The practical need is a faster, cheaper first pass that still produces reasoned answers rather than a raw vote count. ## A worked example: pricing a productized service The case below is a planning example. The dollar figures, timing, and study sizes describe one hypothetical setup, not a current Subconscious deliverable, price, or guaranteed outcome. A freelance designer is considering a productized offer: brand identity packages for early-stage SaaS companies at a flat price of $2,500. The designer tests the decision before building the funnel: - Define the buyer. Describe the audience precisely: founders of bootstrapped SaaS companies with 1 to 10 employees who currently do their own design or use cheap templates. One planning setup defines 30 simulated buyers, within a stated range of 25 to 50. - Test the offer as written. State the offer in plain language: "brand identity packages, flat fee $2,500, delivered in two weeks." Ask whether the defined buyer would choose it, and why or why not. - Read the reasoning, not just the count. A response like "interesting, but $2,500 feels high for a SaaS company at $100K ARR" is more useful than a yes/no tally, because it names the specific objection to fix. - Compare price points. A three-point planning comparison of $1,200, $1,800, and $2,500 showed $1,200 as a clear choice for a lighter starter tier, $1,800 as a stretch, and $2,500 as a premium full package. The founder gets this comparison before committing to one number. - Compare messaging variants. Testing headline variants against the same buyer surfaces which version is understood fastest and which one motivates action, rather than which one the founder personally likes best. In this planning example, the full pass across offer, pricing, and headline testing took about six hours over two evenings. The output is a narrower, better-reasoned starting point, not proof that the offer will sell. ## Cost comparison as planning context The figures below are planning examples describing traditional and self-serve research options in general, not Subconscious pricing: These cost and timing figures are in line with published market-research pricing guidance (Drive Research's 2026 market research cost guide), not current Subconscious pricing. The gap between the two is commonly described as not a 10x cost difference but closer to 100x, and not a 2x speed difference but closer to 20x. That framing describes the category generally; it is not a Subconscious figure. ## Where this method helps most - Naming. Comparing name candidates against the target buyer surfaces what each name signals, not just which one gets more votes. - Landing-page copy. Comparing headline and subhead variants against the same defined buyer shows which version is understood fastest. - Pricing. Comparing price points against a defined buyer shows whether the buyer is likely to perceive the offer as cheap, fair, or premium. - Feature priorities. Comparing backlog candidates against the same buyer definition helps separate noise from features buyers would actually pay for. - Audience selection. Comparing the same offer against two or three candidate audience definitions can show which population responds most strongly. That is a useful signal for choosing a first beachhead. ## What does this replace, and what doesn't it replace? This approach is a reasonable substitute for guessing based on a personal network, setting price by copying a competitor, or writing copy in a vacuum. Publishing what a test cannot do lets a buyer verify the claim before acting on it. It does not replace actually shipping the product, talking directly to people who have already paid for it, watching how they actually use it, or reading live analytics. A pre-launch test does not prove demand. A small starting point is deliberate: pick the one decision that has been unresolved the longest, define roughly 25 buyers matching the target profile, ask the question in plain language, and read the reasoning before deciding. ## Limitations The limits of a method belong in the open, next to its results. A structured pre-launch test does not replace selling, does not replace observed customer behavior, and does not guarantee any particular price or conversion outcome. Treat every dollar figure and timing note above as illustrative, not as a Subconscious price, delivery time, or performance guarantee. Founder judgment about the specific business still carries the decision. The Subconscious research program explains how validation against human behavioral studies supports this kind of comparison. When the decision is high enough stakes to warrant it, a team can move from a simulated comparison to a real-human validation pass without changing the underlying question being tested. The use cases overview shows how that step fits into a broader decision workflow. If there is one decision on the desk right now, such as an offer, a price, or a headline, bring it to a Subconscious working session before spending the next few weeks building around a guess. --- # Radar Plots Must Die URL: https://subconscious.ai/blog/use-cases/radar-plots-must-die Published: 2026-08-19 Radar charts must die because a polygon's area encodes arbitrary choices like spoke order and angular spacing, so a linear bar or distribution chart on a shared axis reports the underlying percentiles more directly. A radar chart draws each of several independent metrics as a spoke and closes the points into a polygon. It compresses many numbers into one memorable shape, which is why it shows up whenever a team hands multi-attribute results to an executive: a causal experiment with several attributes, claims, or segments tested at once. The problem: the shape a reader perceives depends more on the analyst's arbitrary choices, spoke order, angular spacing, than on the numbers themselves. For a buyer reporting causal effects across several attributes, the choice comes down to two chart families: a compressed polygon, or a linear, order-independent encoding such as bars or distributions on a shared scale. This isn't cosmetic. A polygon can amplify or suppress an effect purely through chart geometry, not the underlying estimate, and a leadership team can greenlight a claim, a price, or an audience based on a distorted read rather than the causal result. ## What does the polygon actually encode? A radar plot maps a vector of percentiles onto a closed polygon in two dimensions. What the reader actually takes in, area, aspect ratio, pointiness, balance, maps back to nothing true about the underlying measurements. Two problems compound: - Spoke order carries no inherent meaning. Independent metrics don't come with a built-in sequence, yet a radar is inherently spatial, and spatial layouts push readers toward spatial inference: two adjacent spokes read as related, even though that proximity is only a layout choice. - The radial encoding distorts magnitude through the polygon's own geometry. The radial axis encodes each percentile linearly, so an 80th-percentile spoke runs twice as long as a 40th-percentile spoke. But the polygon's area, what a reader actually perceives, follows a different rule. For a polygon with n spokes at equal angular steps: Every term in that sum is a product of adjacent spoke values: each spoke's visual weight is proportional to its own percentile times the sum of its neighbors' percentiles, not to its percentile alone. A high value flanked by other high values gets visually amplified; a low value flanked by other low values gets visually suppressed beyond what its percentile would suggest. In one worked example from the source dataset, two metrics landed at nearly identical percentiles, but the polygon suppressed one of them by 11.6 percentage points of area relative to the other, purely because of which spokes sat next to it. A third cost stacks on top: comparing two non-adjacent spokes requires mentally rotating one value onto the other before judging which is larger, and that gets more expensive as the spokes sit farther apart on the wheel. ## Why linear position beats angle and area None of this argues against polar coordinates generally. A wind rose, an hourly clock-face plot, a shot map broken out by attack angle: each belongs on a circle, since the radial axis stands for an actual direction or time, so adjacency on that circle reflects a real relationship in the data. The problem is specific to closing a polygon over independent metrics with no natural angular structure. The visual-perception literature is direct on what to use instead: positions along a single quantitative scale, laid out linearly on one axis, are the easiest encoding for people to compare accurately. Length on a shared baseline outperforms angle, area, and radial position, which rank near the bottom of the same accuracy scale (Cleveland & McGill, Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods, Journal of the American Statistical Association). A percentile bar strip follows directly: each metric gets a horizontal bar on a shared axis, named on the left, extending to its percentile on the right. Run the radar's failure modes back through this design and each one disappears. Reordering the rows changes nothing, since each row's encoding is (label, length), and its apparent size has nothing to do with its neighbors. Length scales linearly: a bar at the 90th percentile runs exactly double one at the 45th, rather than reading as four times heavier because of what sits beside it. The same test explains why other visualizations hold up: shot charts, where the 2D space is the physical court, so coordinates don't need reordering, and area deliberately encodes density rather than emerging incidentally from a closed shape; heatmaps, where rink or field coordinates are the chart's real space. The rule: a chart's axes have to mean something real. Field coordinates and literal physical quantities work. An arbitrary angular assignment connected into a polygon does not. ## How do you read grouped estimates without losing distribution? A bar strip solves ordering and area distortion, but still reduces each metric to a single point estimate. A grouped percentile-ridge layout carries that further: instead of one bar per metric, each row shows the full peer distribution as a shape, with the specific value marked as a dot inside it. Groups of related metrics are labeled, with each group's mean percentile shown at the header. This keeps everything that makes the bar strip work, no polygon, no adjacency effect, linear encoding, while adding what a single-number chart drops: whether a percentile gap is genuinely extreme inside a long-tailed distribution, or nominal inside a distribution with little real spread. ## Where does this apply beyond sports analytics? The design problem in football radar charts, several independent quantitative dimensions, no natural ordering, no angular interpretation, is the same problem a causal experiment produces once it estimates effects across several attributes. Subconscious's causal experiments return per-attribute effect estimates with uncertainty across many independent dimensions: which claim moved intent, which price point changed conversion, which audience segment responded differently. Handing those estimates to a decision-maker as a compressed polygon risks the same failure: an attribute's visual weight depends on its neighbors, not its estimated effect. Reporting effects on a shared linear scale, with confidence intervals attached to each bar, keeps the chart's geometry from editorializing a result the estimate doesn't support. This is a data-visualization design principle, not a Subconscious product or shipped feature. The specific case above, and its 11.6-percentage-point distortion, comes from sports-analytics data and generalizes as a chart-design argument. Applying it to causal effect estimates means keeping the display linear and order-independent; it does not mean the underlying experiment design or effect estimates changed. How a result is estimated and how it is displayed are separate problems, and treating a distorted chart as new evidence about the underlying effect is exactly the mistake this argument is against. ## The practical takeaway for reporting causal results Before a team sends multi-attribute results into a deck, ask the same questions that apply to a polygon: does the chart's shape depend on attribute order, does one attribute's visual size depend on its neighbors rather than its own estimate, and can a reader compare any two attributes without mentally rotating between them. A bar strip or a grouped-ridge layout on a shared axis answers yes to none of those. A closed polygon over independent attributes answers yes to all three. Teams that want a closer look at how these estimates get produced can review the method Subconscious uses to run its experiments. ## Limitations This argument is drawn from sports-analytics visualization work, not a Subconscious case study or benchmark. Nothing here implies Subconscious sells a composite-score or player-card product, and no accuracy or speed claim attaches to Subconscious from this material. The principle, use a linear, order-independent encoding for independent multi-attribute estimates, applies to how results are displayed; it says nothing about how those estimates were produced or how accurate any specific model is. [figure: Left: a radar polygon with arrows showing a spoke's area shifting with its neighbors' values. Right: the same metrics as bars on one shared axis, each length reading directly off the scale.] --- # Audience Profiling or Causal Testing: Choosing Between MRI-Simmons Catalyst and a Causal Behavioral Platform URL: https://subconscious.ai/blog/comparisons/audience-profiling-causal-testing-choosing-between-mri-simmons-catalyst Published: 2026-08-19 A consumer insights or brand leader choosing where to spend the next research dollar is solving two problems at once: who is the audience, and which action will change what that audience does. MRI-Simmons Catalyst answers the first question. A causal behavioral platform like Subconscious answers the second. Treating them as competitors instead of a sequence wastes a media budget. ## Two questions, not one platform category "Who should we target?" and "which message, price, or offer moves them?" call for different evidence. Audience-profiling tools describe a population using panel and licensed data. Experimentation platforms compare alternatives under controlled conditions and estimate which one changes a specific outcome. ## What is MRI-Simmons Catalyst built to do? MRI-Simmons positions Catalyst as a consumer audience-profiling and activation platform: it segments and analyzes panel-based consumer data, builds standardized audience definitions, and pushes those audiences directly into advertising and media-activation systems (Catalyst Platform, MRI-Simmons). Its value is the direct path from a defined audience to a running campaign: dashboards, geo mapping, and activation live inside one workflow. What Catalyst is not built to answer is whether a given message, price, or offer causes a better outcome than the alternative. Profiling tells a team who is in the room. It does not run the experiment that tells them which door they'll walk through. ## What is a causal experimentation platform built to do? Subconscious is a causal behavioral platform: controlled experiments run on a simulation of a market, validated against real human behavior, estimate which action, such as a message, a price point, a bundle, or an audience, is likely to move a defined outcome, with uncertainty reported where the study design supports it. The starting point is a decision, not a demographic profile: which of these actions should we take, and why would it work? That distinction matters because a profiled audience can still respond in an unexpected way to a specific claim or price. Correlational data describes what a segment looks like; a controlled experiment tests what changes its behavior. See how the method works. ## Where the two approaches differ ## Why isn't this a substitution decision? Neither tool replaces the other's job. A team that only profiles skips the test of whether its chosen claim works. A team that only tests causal effects without profiling still needs a way to reach the audience it validated. The workflow: define the decision and test the candidate actions on a simulated market first, then take the validated action to an audience-profiling and activation platform to reach the people who matter. That order reduces the odds a campaign spends against an assumption nobody checked. ## The evidence behind the causal claim Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, drawn from a validation corpus of roughly 300 replicated studies across 9 domains (the causal fidelity paper). A fidelity score without its limits is marketing, so the limit gets published next to the number. It is a validation result, not a guarantee for a new market. The company is also SOC 2 Type I certified. Publicly, a market simulation can be stood up in roughly eight hours, with individual experiments running in under five minutes once the simulation is live. That is a workflow claim about Subconscious's own setup time, not a comparison to any other platform's timeline. ## Where this comparison has limits A platform that lists what it cannot do gives a buyer something concrete to check against their own workflow. Subconscious does not provide the audience-profiling, standardized reporting, or direct media-activation capability that MRI-Simmons Catalyst is built around, and it has no equivalent to Catalyst's downstream integration into advertising systems. Subconscious's public site does not currently publish a self-service price list, so cost comparisons require a direct conversation with each vendor. And any regulated or high-stakes decision that depends on formal population statistics still needs a data source built and audited for that purpose; a controlled behavioral experiment is not a substitute for it. ## Questions worth asking before you choose - Do we already know who the audience is, or are we still deciding which claim will work on them? - Is the immediate need a media-ready audience segment, or evidence for a decision that hasn't shipped yet? - Would a controlled test of the message or offer change what we plan to activate? - Does the decision require regulator-grade population statistics, or directional evidence to move faster on an internal call? See how a Subconscious engagement is structured, or review published case evidence before deciding where to start. [figure: Five-step left-to-right path: define the decision, test candidate actions on a causal platform, validate which action wins, profile the audience, then activate the campaign to that audience.] --- # Customer Insight Platforms: Matching the Evidence Tier to the Decision URL: https://subconscious.ai/blog/use-cases/customer-insight-platforms-matching-evidence-tier-decision Published: 2026-08-20 [figure: Three rows: open-ended conversation gives an impression, no alternatives compared; discrete-choice experiment gives a measured effect, alternatives compared; fielded study gives real responses, alternatives compared.] A team evaluating a customer insight platform usually starts by comparing feature lists. The more useful question is narrower: what does the next decision actually need as proof? A fast, open-ended conversation with a synthetic stand-in for a customer type produces a directional impression. A controlled experiment that puts defined alternatives in front of a defined population produces a measured effect. Those are not interchangeable: treating the first as if it were the second is where budget gets spent on the strength of an answer that was never tested. ## The Gap a Conversation Cannot Close A number without its limits is marketing. This article states plainly where an open-ended AI persona session stops: it can explore a topic, surface language, and generate a hypothesis. It cannot tell a buyer why customers would choose one option over another, because it never puts two or more defined alternatives in front of a defined population and measures which one moves an outcome. The response is fluent, but it was never compared to anything. That gap is part of why adoption of AI-generated research participants remains uneven: most researchers already use AI somewhere in their process, but a much smaller share say they trust AI-generated participants as a stand-in for real customers (Development Corporate, 2026). The trust gap tracks the evidence gap. ## What Are the Three Tiers of Customer Insight Evidence? Most customer-understanding work sorts into three tiers, each legitimate for a different kind of call, and none replacing the others. Traditional insight platforms that aggregate CRM data, survey responses, support tickets, and behavioral analytics sit outside this table entirely. They describe what customers already did; they cannot test what customers would do in response to an alternative that does not exist yet, and they cannot speak to customers a company has not yet acquired. ## What Does a Discrete-Choice Design Add? A discrete-choice experiment defines the alternatives (two prices, two messages, two feature sets), assigns a population to see them, and measures which alternative moves the outcome, with a confidence interval attached to the answer. The method has a long research record outside marketing, most visibly in health economics, where it is used to predict how patients would choose between treatment options before a treatment exists to observe directly (PMC / Frontiers in Communication, 2026). The design's external validity against later real-world choices has been studied and holds up reasonably well across health-choice contexts, though not perfectly, which is one reason the method sits as a middle tier rather than a replacement for real-world confirmation (The European Journal of Health Economics, 2018). ## Where the Tiers Actually Get Used Teams that use more than one tier route each recurring decision to a fixed evidence level rather than deciding case by case. Illustrative, not prescriptive, examples: - A product team runs a 30-minute, open-ended AI persona session before writing sprint specs, purely to sanity-check an assumption before deeper work starts. - A marketing team tests a campaign message with an open-ended session before a brief is finalized, then escalates to a controlled comparison of the finalists once the brief narrows to two or three real options. - A strategy team uses a multi-persona panel to surface positioning risk ahead of a quarterly review, then commissions a controlled experiment only where the review surfaces a genuine fork in direction. ## Matching Tier to Stakes The right tier follows the cost of being wrong, not convenience: - Exploratory questions (testing language, surfacing a hypothesis, narrowing a long list) are proportionate to a fast, open-ended conversation. - Budget- or engineering-committing questions (a price change, a positioning claim, a feature trade-off, a launch call) need a controlled experiment that compares the actual alternatives on the table and returns a measured effect. - Decisions high-stakes enough to require recruited human validation before shipping need a path to a fully fielded human study. ## How Does Subconscious Fit the Middle Tier? Subconscious runs a controlled discrete-choice experiment: it compares the defined alternatives across a defined population and returns a measured causal effect with a confidence interval. That is the middle tier, built for calls that need defensible evidence rather than a plausible-sounding reaction. Subconscious can also test or validate a study with real human participants, so a team can move from a simulated experiment to real-human validation without changing the underlying causal question. See how Subconscious structures and validates these experiments and how the process runs end to end. ## What This Method Does Not Cover Naming a failure mode is what lets a buyer check it before relying on it. A controlled causal experiment is not a substitute for real customer interviews, support-ticket or CRM analysis, direct usability research, or a researcher's own judgment about which decisions are worth the extra rigor. The misses belong on the public record next to the hits. Audience reach and recruited human validation are distinct claims and should not be conflated: the size of a simulated population is not the number of real people recruited for a fielded study. Continuous, always-on customer monitoring is not a current capability described here. Ready to see which tier fits an upcoming decision? Book time to walk through it, or review case evidence from completed studies. --- # Virtual Advisory Board: Using AI to Stress-Test a Decision Before It Ships URL: https://subconscious.ai/blog/use-cases/virtual-advisory-board-using-ai-stress-test-decision-before-ships Published: 2026-08-20 A founder or GTM leader with a pricing change, a positioning shift, or a market-entry call due this week rarely has an advisor free to sanity-check it. The alternative isn't a chat persona standing in for a board member. It's a controlled test of the decision itself, against a real audience, before the budget is spent. [figure: Four-step path: define the decision as a testable choice, test the action against a real audience, estimate the causal effect, validate with real humans before shipping.] ## Why the advisory gap exists Founders and leaders cannot know everything, and advisory boards exist to close that gap: an investor with a trained eye for quality, an operator who has sat in the same seat before, an industry veteran who spots a recurring pattern instantly. Building that board takes months of networking, the right introductions, a negotiation about equity or cash, and a company stage attractive enough to draw senior people in. Most early-stage teams operate without it, and they don't always notice what they're missing: - Access to networks. With no industry connections already in place, getting in front of the right advisor is a chicken-and-egg problem. - Stage attractiveness. Top advisors pick and choose, favoring companies that can already point to traction. - Availability. Monthly calls slip. The answer that was needed on Tuesday arrives two weeks later. - Geographic distance. An advisor in a different city or time zone adds friction to every touchpoint. ## What does a simulated panel actually replace, and what doesn't it replace? A panel of simulated respondents, calibrated to a target audience, can stand in for the availability problem: it's there the day a decision needs pressure-testing, not the day a calendar opens up. Publishing what a simulated panel can't do is what lets a buyer check the fit before relying on it. It does not replace what a real advisor uniquely provides. What it can do: - Surface a decision for testing the same day it's written, without a scheduling delay. - Run the same question against many audience segments at once, so a pricing or positioning question gets tested from more than one buyer's perspective. - Let a team iterate on a question, narrowing from a general concern to a specific one, without consuming anyone's limited time. The misses go on this list right next to the hits above. What it can't do: - Make an introduction or open a door. A simulated panel has no network. - Carry personal accountability. A real advisor feels responsible for advice given; a test result doesn't. - Supply proprietary information a real operator picked up from a live deal or a competitor's board meeting. - Substitute for the credibility of naming real advisors during fundraising or partnership conversations. ## Reframe the question: not "what would an advisor say," but "what does the evidence say" Asking a chat interface to role-play a VC or a CMO produces a plausible-sounding opinion. It is still an opinion, generated from patterns in training data, not evidence about how a target audience would actually respond to a specific choice. The more useful framing treats the pending decision as a causal question: which version of the pricing page, the positioning line, or the roadmap message actually changes buyer behavior, and by how much. Subconscious runs controlled experiments against a person-level audience graph covering 800 million real people, structured to isolate which specific action drives the outcome (arXiv, 2025). That answers a narrower, sharper question: does this specific change move the number that matters. "Our findings reveal that proprietary LLMs can generate valid and behaviourally sound utility specifications, particularly when guided by structured prompts (CoT)." Sfeir, Nova, Hess, and van Cranenburgh, Journal of Choice Modelling (source) ## A four-step path from open question to a decision you can defend - Define the decision. Not "what do you think of our roadmap" but "does leading with Feature A over Feature B change signup intent." A specific, testable question is what makes the exercise falsifiable rather than a conversation. - Test the action against a real audience. Run the specific choice, such as a price point, a headline, or a feature framing, as a controlled experiment rather than a general discussion prompt. - Estimate the causal effect. The output is a measured difference between options, not a summary of what a simulated persona said it liked. - Validate with real humans before the decision ships. A team can move from the simulated test to real-human validation without changing the underlying causal question, which is the step that turns a modeled estimate into evidence a leadership team can act on. [figure: Two columns: left lists what a simulated panel can do (same-day testing, many audience segments at once); right lists what only a real advisor provides (introductions, accountability, proprietary insider information).] ## Where does this approach fit, and where doesn't it fit? This approach is most useful before a decision is final and no advisor is available this week: a pricing change, a positioning test, a market-entry call, a roadmap trade-off. It is not a substitute for a real advisory board, and it is not a market-performance guarantee: a causal test of one decision is not a clinical trial or a usability study, and it doesn't predict every downstream outcome in market. Teams that already have advisors can still use this to prepare: arrive at the next advisory conversation with a tested question instead of an open one, so the limited time with a real advisor goes toward judgment calls a test can't answer. Teams that lack an advisory board can use it to avoid shipping a decision that was never pressure-tested against anyone. See how the audience graph and study methods work, what a decision-testing engagement looks like end to end, or book a walkthrough against a specific pending decision. --- # Audience Research for Marketing Managers URL: https://subconscious.ai/blog/buyers/audience-research-marketing-managers Published: 2026-08-20 Marketing managers make decisions about audiences, messages, campaigns, content, and launches. Formal research budgets do not always follow those decisions: teams use internal opinion to choose an action, then use campaign performance to learn whether the choice was right. Subconscious gives marketing teams a way to test product, pricing, messaging, and go-to-market actions before committing production or media spend. The work is strongest when it compares clear alternatives for a defined audience and measures how those alternatives change a stated choice. [figure: Five boxes, one per marketing decision, each with its own arrow to a separate experiment rather than a single shared box.] ## What marketing teams need from research The decision changes from project to project: - Audience understanding asks which segments matter and how their needs differ. - Message testing asks which claim changes preference or intent. - Campaign concept testing asks which direction deserves production. - Competitive positioning asks why a buyer would choose one offer over another. - Content planning asks which subjects and formats support a specific audience decision. - Launch planning asks how defined segments respond to an announcement or offer. Each question needs its own experiment. A broad conversation about what an audience likes may generate hypotheses, but it does not prove which marketing action caused a difference. ## Turn an audience question into an experiment Begin with the choice the marketing manager must make. Name the audience, the alternatives, the expected behavior, and the evidence needed to act. For example, “Which message is best?” is too vague. “Which of these messages changes stated choice among procurement leaders evaluating the same offer?” identifies an audience, a controlled contrast, and an outcome. ### Define audiences from evidence Audience definitions can include buyer role, category experience, current solution, needs, and decision constraints. Demographic detail alone does not establish how someone will behave. Use customer data or approved research when available, and do not treat a richly written persona as validation. A 45-minute planning conversation may help a team surface assumptions about an audience. It should be used to form hypotheses, not presented as a substitute for interviews, ethnography, or observed customer behavior. ### Test messages as controlled alternatives Keep the offer and context stable while changing the message. Present three to five clear directions, then compare the resulting choices across the same audience definition. Questions about clarity, credibility, and relevance can explain a result. They should follow the choice rather than replace it. ### Review campaign concepts before production Campaign testing is most useful before the team has invested in finished creative. Compare the core idea, message, and intended response while the work can still change. Visual, video, and experiential campaigns require care. A text description cannot reproduce the emotional impact of finished creative. Use real audience testing when the decision depends on that experience. ### Test competitive position in context An offer rarely appears alone. Put it beside the alternatives buyers already consider. Ask what changes preference, what remains unclear, and which segment responds differently. Competitive comparison should use accurate descriptions. It should not invent a rival’s weakness or assume that simulated responses represent the market. ### Use audience research to guide content Content strategy becomes more useful when tied to a buyer decision. Instead of asking which topics an audience likes, test which subject, proof point, or format helps a defined buyer understand or choose an offer. The output can inform editorial priorities. It does not establish channel performance, reach, or conversion without real campaign data. ## Research across a buying committee Account-based marketing often needs one offer to make sense to several roles. A CFO, IT director, end user, and procurement lead may evaluate the same purchase through different constraints. A multi-segment experiment can compare the same message across three to five audience definitions. The result may show where one message travels across the committee and where a role needs different evidence. Publishing where synthetic results diverge from individual behavior is what lets a team judge the comparison instead of just trusting it. Keep the comparison honest. Synthetic research tends to reproduce aggregate patterns better than individual behavior. It may also flatten cultural differences, reduce variance, or reflect prompt wording. Those limits matter more when a segment is small, specialized, or poorly represented in the grounding data. ## Put testing into the marketing workflow Research has more value when it happens before a commitment: ### Before the brief Use a 30-minute session to identify the decision, audience, alternatives, and assumptions. The session creates an experiment brief, not audience truth by itself. ### Before production Test the core message or concept while changing direction is still inexpensive. Preserve the same conditions across alternatives so the result can support a comparison. ### Before launch Check for objections, ambiguity, and segment differences. Use real audience validation when the launch risk or creative format requires it. ### After launch Treat synthetic findings as hypotheses about performance. Use analytics, conversion data, interviews, and other observed evidence to determine what happened. ### During content planning Run a quarterly audience intelligence session to revisit audience priorities and language as market and campaign evidence change. A saved persona should not become a permanent substitute for current research. [figure: One message box with arrows to four role boxes: CFO, IT Director, End User, Procurement Lead. Two boxes show accepted, two show needs different evidence.] ## Know what the evidence can support Synthetic audience research is useful for exploration, scenario comparison, and early iteration. It is not a blanket replacement for customer research. The known gap goes in the open so a marketing team can judge how much weight to give a synthetic result before spending real budget. Purchase behavior is especially easy to overstate. What someone says they would do can differ from what they do when money, time, reputation, or organizational approval is at stake, the same say-do gap that shows up when stated survey responses are compared against measured choice (LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings, arXiv). Use observed conversion data for behavioral validation, and see a worked example in a published case study. "SSR achieves 90% of human test-retest reliability while maintaining realistic response distributions (KS similarity > 0.85)" Maier and colleagues, arXiv preprint "LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings" (source) This limit is named here so a team validating a specialized audience knows exactly where to add real evidence. Cultural nuance also requires direct evidence. Validate work involving a specialized community with people from that community. Do not assume a model can represent identity or context reliably because its answer sounds specific. Use an early experiment to improve the decision before production. Match the final validation to the cost of being wrong. --- # A Single Buyer Profile Is Not a Controlled Comparison URL: https://subconscious.ai/blog/use-cases/single-buyer-profile-not-controlled-comparison Published: 2026-08-20 Naming what a method cannot do is what lets a buyer check the claim before acting on it. A single buyer profile is not enough evidence to greenlight a positioning line, pricing move, pitch, or roadmap feature. It organizes what a team already believes about a buyer. It cannot show which action will change behavior across a market. That distinction matters before a launch, sales push, or engineering quarter. The cost of choosing poorly appears later as a missed message, a weak sales cycle, or roadmap capacity committed to the wrong action. ## One coherent answer can hide buyer disagreement The gaps in a method belong in the open next to what it does well. A conversational buyer stand-in produces one plausible response from the interviews, CRM notes, and assumptions supplied. There is no control, no baseline, and no distribution of responses. The team cannot tell whether the answer reflects market disagreement or its own inputs. Research on demographic role prompting finds that model outputs can collapse toward a flattened group viewpoint instead of preserving variation within that group (research on demographic role prompting). Choice-modeling research finds that prompting strategy and model choice affect how closely simulated choices match real preference structures (choice-modeling research). "Furthermore, item-level diagnostics reveal that behavioral variation tracks coarse demographic stereotypes rather than the fine-grained individual differences specified in each persona." Xiao and colleagues, arXiv preprint "The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models" (source) The methods answer different questions: [figure: Four steps: name alternative actions, define audience and outcome, run a controlled comparison across a modeled population, then validate with human participants for high-stakes calls.] ## How do you turn a launch choice into treatments? The useful unit of work is not a conversation. It is a decision-specific experiment. Name the actions: a positioning test might compare two value propositions, a roadmap test two feature descriptions. Then define the audience, the behavior that matters, and what stays fixed across the comparison. A directional estimate is only useful once its limits are stated plainly. Subconscious runs controlled, discrete-choice-style experiments across a modeled population. It estimates the directional difference between the actions under test and reports uncertainty where the study design supports it. It does not eliminate modeling error. ## How do you match the audience to the decision? Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That number describes modeled audience reach, not recruited human participants. The graph makes it possible to define a buyer population for the decision instead of treating one profile as the market. If a buying committee contains different roles, the experiment should preserve those distinctions rather than average them into one voice. This method is useful when the team can state the alternative actions: - Compare value propositions before selecting a positioning line. - Compare pitch framings or objection responses before a sales push. - Compare candidate headlines or claims against a defined buyer population. - Compare feature descriptions before committing roadmap capacity. - Compare configured pricing scenarios without implying automatic price optimization. ## Carry the same causal question into human validation Naming what a modeled population cannot prove is what lets a team check the claim before committing budget. A modeled population is not proof of individual-level behavior and does not guarantee launch performance. Interviews, CRM evidence, and win/loss calls remain useful inputs to test, not evidence that one action caused a better outcome. For a high-stakes decision, Subconscious can test or validate the study with real human participants. The alternatives, audience, and outcome stay fixed, so the team can move to real-human validation without changing the causal question. ## What should you define before committing the budget? Write down the action, alternative, audience, and outcome before asking for an answer. If the team cannot define those elements, more open-ended buyer discovery may help. If it can, a controlled comparison gives a stronger basis than one agreeable conversation. See how a study is structured, then use the live walkthrough to frame the first comparison. --- # What AI Can Draft in Market Research, and What a Researcher Still Owns URL: https://subconscious.ai/blog/use-cases/what-ai-draft-market-research-what-researcher-still-owns Published: 2026-08-20 A VP of Insights does not have to decide whether AI belongs in the research pipeline. It is already there, drafting guides, coding verbatims, and producing first-pass synthesis before anyone asks for it. The decision that remains is narrower: which of those outputs a researcher can hand to a stakeholder unreviewed, and which ones still need a person who can defend the method behind the number. Get that split wrong and the cost shows up later. A team that lets AI own framing and interpretation, not just drafting, ships a launch or pricing call built on an unvalidated read of open-ended text. When the result does not hold in market, there is no defensible chain of reasoning to point to, only a plausible-sounding summary. ## What Parts of Market Research Actually Got Faster With AI? The mechanical parts of research work, drafting a discussion guide, coding verbatims, producing a first-pass synthesis, moved from slow and expensive to fast and cheap. That did not eliminate research demand. Growth projections for market research analysts from 2024 to 2034 still hold (Forbes Technology Council, 7 Market Research Trends To Watch For In 2026), because the work that survives is not the part AI automated. What moved is access. Producing a draft survey, summarizing a transcript, or generating a first reaction from a synthetic audience no longer requires specialized tooling. That makes the mechanical layer worth less and makes the judgment layer, the part that decides which draft deserves trust, worth more. ## What Research Judgment Stays With a Person, Not AI? Four decisions stay with a person regardless of which tool drafts the first pass: - Problem framing. What decision is actually being made, and what evidence would change it. - Audience definition. Who the tested population is, and whether that population maps to the real buyer. - Methodology judgment. Which comparisons are valid, and which conclusions the design cannot support. - Validation routing. Which findings can stay directional, and which ones are expensive or public enough to require confirmation against real behavior. A useful discipline for keeping the split honest is a four-layer evidence system: use AI for exploration and hypothesis generation, use a directional comparison to narrow options quickly, put a person between that comparison and any stakeholder-facing claim, and reserve real respondent or behavioral data for decisions where being wrong is costly. The output of the fast layer becomes evidence only once someone checks the audience definition, the framing, and the source grounding behind it. ## Where a causal test fits this split This is the same split a controlled discrete choice experiment sits inside. Subconscious runs causal experiments against a person-level audience graph covering 800 million real people, and a team can move from that simulated comparison to a study with real human participants without changing the underlying causal question. The simulated pass narrows which alternatives are worth testing further; the person still frames the tested decision, defines the compared alternatives, and reads the confidence interval before it reaches a stakeholder. A causal effect with a confidence interval is a different kind of output than a correlational summary of open-ended text: one tells a team how a tested audience responded to a defined choice, with a bound on how confident that estimate is; the other tells a team what a document said. Confusing the two is the failure mode from the opening: treating a fluent output as validated evidence. See how this comparison-to-validation path has played out across other decisions; the workflow for structuring the comparison is in how Subconscious runs a study. ## What Does a Causal Experiment Not Replace? A causal experiment does not replace the researcher work: framing the business question, defending the method to an executive, and deciding when a directional read is not enough. It makes no speed or cost claim against any other research tool, and a simulated comparison is not a substitute for observed customer behavior once a decision is expensive or public. The audience graph is a population to test against, not a panel of recruited participants standing in for fielded research. [figure: Four-step chain: AI exploration feeds a directional comparison that narrows options, a person checks it before it becomes a stakeholder claim, then costly or public decisions continue to real respondent data.] ## A concrete next step Before running a comparison, put the business decision into a single sentence, name who it affects, and set how much risk is on the table, then decide in advance which outputs from the fast layer are allowed to reach a stakeholder unreviewed. Keep the framing, the audience definition, and the read of the research with the person who has to answer for it. [figure: A left-to-right path: an AI-drafted output moves into a directional comparison, then a person checks framing, audience, and method, before high-stakes claims get validated against real behavior.] --- # 8 Questions to Ask Before You Trust a Segment Estimate URL: https://subconscious.ai/blog/industries/8-questions-ask-before-trust-segment-estimate Published: 2026-08-20 A quant researcher who owns a firm-characteristic return model has to choose every time a new segment shows up in the data: report one pooled coefficient for every sector, split the model so each sector gets its own independent fit, or let the data decide how much a sector is allowed to differ from the rest. Get that choice wrong and the model either erases a real effect or mistakes noise for one, and either error changes which action gets taken on which segment. A applied analysis worked through this problem for cross-sectional US equity returns, using the firm-characteristic framework from Freyberger, Neuhierl, and Weber (2020, The Review of Financial Studies). Its finding: a single LASSO coefficient applied to every sector can hide a variable whose effect reverses sign from one sector to the next, while fitting each sector in total isolation can turn a 50-observation sample into a confident-looking but noisy estimate. A Bayesian hierarchical model, using partial pooling, sits between those two failure modes, and it shrinks each sector's estimate toward a learned group mean instead of toward zero. Any team that reports a segment-level number (by industry, by cohort, by region) faces the same pooling decision. Eight questions worth asking before trusting one. ### 1. Does one number really apply to every segment? Two common default regularization methods, LASSO and Ridge, produce a single coefficient per variable and apply it uniformly across every group in the data. Ridge shrinks coefficients multiplicatively, by proportionally more in low-variance directions; LASSO subtracts a constant amount via soft thresholding, which is why it forces small coefficients to exactly zero. Neither method can recognize that momentum, say, might carry one meaning at a tech firm and a different one at a utility: it can only say the variable matters or it doesn't. ### 2. What happens if you fit every segment in total isolation? The opposite failure mode is no pooling: fit each segment's coefficient independently, with no information shared across groups. In the source simulation, per-sector OLS produced visibly noisy, unstable estimates for the weaker signal even though the true global effect was the same size across sectors. ### 3. Does a middle option exist? Partial pooling is that middle option. Each segment still gets its own coefficient, but that coefficient comes from a common distribution built around a mean the model learns from every group at once. A segment with limited data gets pulled toward the mean; a segment with a distinctive, well-supported pattern is allowed to deviate. The regularization target is the group mean, not zero. ### 4. Can the model tell you how confident it is? A point estimate hides its own uncertainty. The hierarchical model's posterior distribution over each segment coefficient carries an explicit uncertainty band: it is appropriately confident about strong signals and appropriately uncertain about weak ones. In the real-data application, the posterior for a strongly predictive characteristic like LME could show genuine sector heterogeneity that a single pooled coefficient would mask entirely. ### 5. What does the pooling structure actually look like in code? The hierarchical model's structure maps closely onto its mathematical form: sector intercepts and sector-specific coefficients are drawn from group-level hyperpriors, using a non-centered parameterization to keep posterior sampling stable when the cross-sector spread is small. The posterior itself has no closed-form solution, so it is approximated by sampling, commonly with the No-U-Turn Sampler, an adaptive Hamiltonian Monte Carlo method that builds momentum toward informative regions of the parameter space and avoids retracing its own path (Hoffman & Gelman, 2014, Journal of Machine Learning Research). ### 6. Does the model's own prior make sense before you look at data? Before fitting anything, a prior predictive check draws samples from the model's priors alone and asks whether the simulated data looks plausible. If simulated returns imply implausible monthly swings, the priors are too loose; if they cluster tightly near zero, the priors are too tight and will overwhelm a real but modest signal. This step catches a badly specified model before it ever touches real observations. ### 7. Does the fitted model actually match the data it was trained on? A posterior predictive check runs the reverse direction: simulate data from the fitted model and compare it to what was actually observed. In the source analysis, the simulated return distribution matched the real one closely, with no severe misspecification, evidence that the model's likelihood and priors were consistent with the data, not just a good fit by construction. ### 8. Where does the pooling choice stop mattering? Partial pooling only helps when segment differences are real and worth preserving. If every segment genuinely shares the same effect, full pooling is the right answer and hierarchical modeling adds complexity without benefit. If segments are large enough and different enough, no pooling loses little to noise. That judgment call sits behind any report that breaks a metric out by cohort, region, or account tier without stating how much of the split is signal and how much is sample size. ## Where this reasoning applies beyond return models Subconscious tests actions and estimates causal effects with confidence intervals rather than a single point prediction. The lesson from the source analysis is the same check a buyer should run before trusting any segment-level result from a causal study, whether that segment is a customer cohort, a region, or a product line. This does not mean Subconscious runs Bayesian hierarchical return models, financial backtests, or portfolio construction. It means the underlying discipline, reporting an effect with its uncertainty and being honest about whether a subgroup difference is supported by the data or just by a small sample, is the same discipline Subconscious applies to causal experiments. Teams that need to move a causal question from a simulated study to validation with real human participants can do so without changing the question asked; see current experiment results and worked examples. [figure: A four-step path: prior predictive check, fit hierarchical model, posterior predictive check, report the estimate. The two check steps are marked as gates the model must pass before moving on.] ## Limitations The source analysis carries its own limitations worth naming: the training data used a balanced-panel requirement that tilts the sample toward larger, continuously traded firms, and its backtest performance figures reflect a specific historical window and cost assumption. None of those figures are Subconscious performance claims: they describe a financial modeling exercise, not a Subconscious product result. The pooling question they illustrate is real and reusable; the trading returns are not. [figure: Three columns: full pooling gives one coefficient for every segment; no pooling fits each segment alone, noisy on small samples; partial pooling pulls each estimate toward a group mean, with an uncertainty band.] --- # What Is a Simulated Buyer, and When Should You Trust One? URL: https://subconscious.ai/blog/methods-and-validation/what-simulated-buyer-when-should-trust-one Published: 2026-08-20 A simulated buyer is a model of a real audience member, built from demographic, behavioral, and prior-response data, that answers a research question the way that audience actually would. It is not a chatbot improvising an opinion. It is the unit a controlled behavioral experiment runs against before a team commits budget to a launch, price, or message. The decision this article answers: which research questions can move through simulation, and which ones still need real people. Route the wrong question to the wrong method and a team either ships on a guess it can't defend, or spends a field budget answering a directional question that simulation could have settled. [figure: Decision path from "research question," splitting on whether it needs a defensible population estimate: concept/message/segment questions go to a simulated buyer; population-estimate questions go to real-human fielding.] ## What makes a simulation trustworthy rather than a guess Three layers determine whether a simulated buyer's answer means anything: A frontier reasoning model. GPT-class, Claude-class, Gemini-class systems supply the general language and reasoning ability underneath every answer. Audience conditioning. Demographic and behavioral inputs, such as age, geography, household income, occupation, attitudes, and prior brand exposure, bind the model to a specific audience segment rather than a generic voice. Conditioning on real prior response data for that audience is what separates a calibrated simulation from a costume. A response protocol. Rules for how the model answers: question format, scale, and follow-up handling. A protocol that accepts every answer at face value produces a chatbot. A protocol that scores disagreement, flags low-confidence answers, and reproduces similar results on a repeat run produces something you can audit. Four properties separate an audited simulation from a thin prompt wrapper: - Fidelity to a real audience. The model is calibrated against real prior data for that segment, not just a job title and an age typed into a prompt. - Disagreement and pushback. A real respondent says "I would not buy this," misreads the question, or changes position under a follow-up. A model that agrees with everything is answering as a chatbot, not as the audience. - A confidence signal. Every answer should carry an estimate of how reliable it is, so a team can flag the low-confidence ones instead of treating every output as settled. - Reproducibility. Run the same setup against the same stimulus again and the result should land in the same range, not swing wildly. Subconscious tests this property against real-human studies rather than assuming it. ## Which questions reward simulation, and which don't? The dividing line is whether the question rewards general reasoning about preferences or demands unique lived experience the model cannot have had. Simulation handles these well: - "Would you buy any of these product concepts?" - "What's off-putting about this messaging?" - "Describe your process for evaluating a vendor switch." - "What would push you from your current vendor to a competitor?" - "Is this ad creative confusing in any way?" Simulation handles these poorly: - "Describe the specific moment last summer when you switched insurance providers." Each calls for reasoning about preference, reaction, or evaluation criteria, something a well-conditioned model can approximate. The insurance-switch question instead demands invented autobiographical detail, and a model will fabricate specifics rather than admit it has none. ## Simulated testing versus real-human fielding A number without its limits is marketing copy. The timing and cost figures are inherited historical planning examples. They are not current Subconscious or vendor prices, delivery estimates, guarantees, or service levels. The pattern that holds up: route concept screening, message iteration, segment exploration, and multi-market comparison to simulation first. Reserve real-human fielding for population-level validation, hero claims, and any number that has to survive regulatory or PR scrutiny. Treating that split as roughly 80 percent simulation-first work and 20 percent real-fielding work is a reasonable planning heuristic for a research queue, not a fixed ratio every program will hit. ## What a simulated study group looks like Most teams run simulated buyers in groups rather than one at a time: - Naming this limit is what lets a buyer check the number against the method. 50 to 500 modeled respondents (a nominal count, not an effective sample size: responses share a common model error source rather than being independent draws) - Stratified across the demographic and behavioral dimensions that matter to the decision - Calibrated against real prior data for that audience when it's available - Run against a defined instrument: a concept test, an ad pretest, or a structured comparison - Output as structured comparison data alongside open-ended qualitative response ## Where simulation is the wrong tool Three situations call for real fielding instead: Statistically validated population claims. Anything a team needs to defend as "X percent of the target population thinks Y" requires a real study designed to produce that estimate. Genuinely novel categories. Products or events with no analog the model has seen before produce plausible-sounding guesses with no signal behind them. Sensory or emotional response. Reactions to a physical product, a package design, or a video ad require real human perception. A model can reason about the likely reaction; it cannot feel one. ## How do you move from a simulated result to a validated one? The useful capability is testing the same causal question, does this concept, price, or message change buyer behavior, through simulation first, then real-human validation when the decision calls for it, without changing what the study measures. Subconscious can test or validate a study with real human participants, so a team that started with a simulated screen can move to real-human validation on the same causal question rather than starting over. Subconscious's method is a controlled causal experiment on a simulated market, benchmarked for replication accuracy against real human studies. The misses go on the public leaderboard next to the hits. Subconscious tests this against roughly 300 replicated human studies across 9 domains. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. ## Where the term comes from, and its limits The academic backbone is Argyle and colleagues' 2023 paper on conditioning a frontier model on a real respondent's demographic background to produce opinion distributions that match benchmark surveys, an approach the literature calls silicon sampling (Cambridge University Press, Political Analysis). The commercial category built products around that idea afterward. The boundary that matters for a buyer is not whether the underlying model is impressive. A study design that tracks that boundary, and that can hand off a directional finding to a validated one without redefining the question, is what separates a research program from a stack of plausible-sounding guesses. Next step: check the leaderboard for how simulated results have tracked against real-human studies, or see how Subconscious runs a study end to end. --- # How to Pre-Test a Pricing Increase Announcement Before It Sends URL: https://subconscious.ai/blog/use-cases/how-pre-test-pricing-increase-announcement-before-sends Published: 2026-08-20 [figure: Five customer segments a pricing letter reaches, each with what they check for and what breaks it: grandfathering buried, bait-and-switch feel, no downgrade path, no upgrade story, nothing to forward internally.] A pricing increase letter is the single email a subscription business cannot take back. Every paying customer gets the identical version in the identical moment, and the wording decides whether next quarter's revenue holds or slides after churn. There is no preview send, no rollback: the customer reads it once and stays, downgrades, or cancels. ## Why this decision is harder than it looks Most founder communication is asymmetric in the founder's favor. A weak blog post costs a week of reach. A weak pricing letter costs revenue directly, for three reasons. The first is that the letter is read differently than any other email a company sends: the customer is weighing, sentence by sentence, whether the new price is worth what they're getting for it. The second is that the customer base is not one reader. A four-year subscriber filters the letter through loyalty; a signup from the last sixty days filters the same words through suspicion of a bait-and-switch. A customer paying twice the account average reads it looking for what they're getting for the money; a customer paying half the average reads it braced for the increase to land hardest on them. The third is that the reader has somewhere else to go. The email competes with the customer's impulse to open a competitor's page before finishing the paragraph. All three are reader-modeling problems that can be tested before the email leaves drafts, rather than discovered in the support queue over the following forty-eight hours. ## The decision this article is about The choice a founder or head of growth has to make is not whether to raise the price. It is which draft of the letter (which rationale, which tone, which options offered) goes out to the entire base. A generic rationale, a grandfathering offer buried past the first paragraph, or no visible downgrade path are drafting choices, and those can be checked against how different customer segments actually react before the send, not after. ## What does a pre-send pricing test measure? A useful test of a pricing letter is not a style read. It is a controlled comparison of draft variants (different rationale framings, different tones, different sets of options offered) run against a simulated population built to reflect the base by tenure and price band, not by industry. Five segments cover most of the variance in how a pricing letter lands: Testing the letter against this segmentation, rather than one internal reviewer, surfaces whether the rationale paragraph reads as one consistent reason or several different stories depending on who's reading it. ## What does this kind of test typically surface? Across pricing-letter reviews built this way, a handful of failure patterns repeat: - The rationale is generic. Citing "inflation, growth, and investment in the product" lands as boilerplate for nearly every segment. A rationale specific enough to argue with is what gets a nod instead of a complaint. - Grandfathering is under-communicated. Founders often plan to grandfather early customers but describe it in the fourth paragraph. A customer who's been paying since day one usually stops reading at the price line before reaching it. - The upgrade story is missing. A letter that raises the price without saying what the increase funds reads as a tax to the highest-paying, heaviest-usage customers. - The downgrade path is treated as an afterthought. Founders tend to mention the lower tier in passing so as not to encourage downgrades. A visible downgrade path is what keeps a price-sensitive customer on some plan instead of leaving entirely. - Send timing changes the read. The same letter lands differently at six in the morning than at three in the afternoon; a morning send reads as urgent, an afternoon send reads as considered. It requires putting the actual draft in front of the actual segments before the decision to send is final. ## A pre-send sequence, as a working example One way to sequence this is to test one question at a time as the letter tightens: - Two weeks out: the rationale check. Show the letter without the price visible. Ask why the company is raising the price. If segments disagree on the reason, the rationale paragraph is ambiguous. - Ten days out: the reaction check. Show the letter with the price visible. Ask how it makes the reader feel about the company. This is where "they thought about me" separates from "they sent me a bill." - One week out: the options check. Ask the price-sensitive and procurement-routed segments which option, if any, they'd take. If neither finds one, the letter offers only accept-or-churn. - Four days out: the forwarding check. Ask the procurement-routed segment what their own approval chain would say if this were forwarded to them. - Two days out: the headline check. Ask which line would get screenshotted if the letter leaked, and whether that line is fair on its own. ## Where does this fit with real customer testing? A test built this way answers a causal question: which draft changes the reader's stated intent to stay or leave, using a simulated population standing in for the base. It does not replace talking to real customers where the decision warrants it: Subconscious can test or validate a study with real human participants, moving from a simulated read of the letter to a real-human check on the same question before the send. Subscription-business churn research treats the moment a price change is communicated as its own measurable event, not background attrition folded into a general churn number (Paddle: The World's Largest Study on Churn). ## What this doesn't cover Publishing what a method can't do is what separates a real result from marketing copy. This kind of test does not replace legal or compliance review of the letter's language, particularly around price-change notice requirements in a customer's contract. It does not guarantee a real-world retention outcome or forecast the resulting churn rate; a simulated read of a draft catches an obviously weak version before it ships, nothing more. And it does not substitute for the founder's judgment about which rationale and which options actually make sense to offer; it only tells them how a draft they've already chosen is likely to land. ## Beyond the pricing letter The same five-segment approach, lightly adjusted, applies anywhere a customer has to absorb a change management is imposing rather than one they chose to opt into: terms-of-service updates, plan migrations, or the retirement of a free tier. Nearly every subscription business raises prices annually or more often. Get the letter wrong and the price shows up as churned revenue the following quarter; testing a draft against the segments who will read it costs a fraction of that. The method behind these tests and how a study is set up are worth reviewing before the next price change goes out. A demo walks through running one against your own base, and past studies show what the comparison looks like end to end. --- # Likelihood Approximations Through Neural Networks: A Validation Checklist Before You Trust the Output URL: https://subconscious.ai/blog/methods-and-validation/likelihood-approximations-through-neural-networks-validation-checklist Published: 2026-08-20 A behavioral model that reports a choice probability or a causal effect is only as trustworthy as the likelihood behind it. When that likelihood has no closed form, a team has to decide whether to accept the model's output on faith or check it against the simulator that generated its training data first. The safer answer is: check it. This walkthrough, adapted from a PyMC tutorial on likelihood-free inference, shows what that check looks like for a model with an intractable likelihood, and the Bayesian workflow that follows once it passes. ## Why some models have no likelihood to evaluate Standard Bayesian inference needs a likelihood function that scores how probable the observed data are under a given set of parameters. Many models used to analyze reaction times and choices don't have one in closed form. The example model is a Drift Diffusion Model (DDM), a common framework for joint reaction-time-and-choice data. A DDM describes a decision as a random walk: evidence accumulates at a drift rate toward one of two boundaries, and whichever boundary is crossed first determines both the choice and the reaction time. The standard DDM has a known analytical likelihood, but realistic variants do not. One example is a variant where the decision boundary narrows as a deadline approaches. Those variants remain easy to simulate; you can generate synthetic trials from them in a few lines of code. They're just hard to score directly. That gap between "easy to simulate" and "hard to score" is what simulation-based inference (SBI) is built to close. ## What is the three-step pipeline for simulation-based inference? Simulation-based inference replaces the missing analytical likelihood with a learned approximation, trained entirely on simulated data: - Simulate. Generate synthetic trials across many parameter settings using the model's simulator. The walkthrough this is adapted from draws 1000 trajectories at a time from a DDM simulator to build training data. - Train an approximator. Fit a neural network on the simulated data so it learns to score data the same way a true likelihood would. The source example trains two specialized networks, one scoring the reaction-time density for observed responses and one scoring how likely a response was withheld, and combined the pair spans the full likelihood. - Validate against the simulator. Before the trained network goes anywhere near a real inference run, its output has to be checked against the simulator that generated its training data. Step three is where most of the risk in this workflow lives, and the easiest step to skip under time pressure. ## What does "validate against the simulator" actually mean? Validation here is not a summary statistic. It's a direct visual and numerical comparison between what the trained network predicts and what the simulator itself produces when run at the same parameter values. For the reaction-time network, that means plotting the network's predicted likelihood curve directly against a histogram of reaction times the simulator generated at the same parameter setting, for several settings across the plausible parameter range. For the choice-probability network, it means overlaying the network's predicted probability against the simulator's own observed choice frequencies. Both checks are described in Fengler et al.'s work on likelihood approximation networks for cognitive neuroscience, the peer-reviewed source for this class of approximation. This is the checkpoint that separates "the network converged during training" from "the network's output means what I think it means." A network can report a low training loss and still produce a likelihood surface that diverges from the simulator in the regions of parameter space that matter for a specific dataset. The broader case for this approach, using a trained approximator instead of an intractable likelihood and treating the approximator as something to validate rather than trust by default, is laid out in Cranmer, Brehmer, and Louppe's survey of simulation-based inference. ## How is the validated network wired into a Bayesian model? Once a network's output has been checked against the simulator, it can be dropped into a probabilistic model. In the source workflow, the trained networks are wrapped as custom PyTensor operations and injected into a PyMC model as potential functions, a mechanism for adding a custom log-likelihood term rather than using a built-in distribution. From there, standard Bayesian machinery applies: priors go on the model's parameters, and Markov chain Monte Carlo sampling produces a posterior distribution rather than a single point estimate. The example model draws 500 trials under one experimental condition and 500 under the other and trains its two networks for 10 and 20 epochs respectively. Those specific counts are the tutorial's own settings for building training data quickly on a local machine, not a floor or ceiling for how much simulation or training a given problem needs. A production model on a wider parameter range would generally need more of both. The point is the pipeline's shape, simulate, approximate, validate, then infer, not the specific trial or epoch counts. Recovering the true generating parameters from synthetic data, checking the posterior lands where you know the answer because you generated the data yourself, is the same closing step. It's a necessary check before the pipeline runs on data whose true answer is unknown, though recovering parameters from a single synthetic dataset doesn't by itself confirm the pipeline is correct across the board. ## Where this discipline matters beyond one tutorial The specific failure mode this validation step guards against is general: any model whose output feeds a business decision but whose fit was never checked against the simulator or dataset that generated it is a black box wearing a probability distribution. The number in the answer is real; whether it means what the reader thinks it means is not. A related discipline applies to how Subconscious reports results: causal effects come from the randomized design of the experiment, and Subconscious reports them with confidence intervals rather than a single simulated prediction. A confidence interval on its own doesn't validate the model, but an effect checked against structure the model didn't just invent, paired with that interval, holds up to scrutiny that a bare point estimate doesn't. A team evaluating how a decision workflow gets from a simulated estimate to something checked against the simulator or process that generated it should expect that checkpoint to exist regardless of which modeling approach produced the first number. [figure: Four-step path: simulate synthetic trials, train a network to approximate the likelihood, validate its predictions against the simulator's own output, then wire the network into a Bayesian model as a log-likelihood term.] [figure: Five-step chain: a validated network is wrapped as a PyTensor operation, injected into PyMC as a potential function, combined with priors, sampled with MCMC, producing a posterior distribution.] ## Limitations This is a methods walkthrough on likelihood-free Bayesian inference for drift diffusion models, built around open-source PyMC tooling. It's not a description of a Subconscious product pipeline, and Subconscious does not ship this exact neural-network-likelihood workflow or DDM-specific tooling. The general lesson, approximate a missing likelihood and then validate that approximation against the simulator's own output before it informs a decision, is the part worth carrying forward regardless of which models or software a team uses. --- # Ipsos Synthesio vs. a Pre-Launch Causal Test: Two Different Questions URL: https://subconscious.ai/blog/comparisons/ipsos-synthesio-vs-pre-launch-causal-test-two-different-questions Published: 2026-08-20 A brand team that already runs Ipsos Synthesio for always-on monitoring is not choosing whether to replace it. The real question: does listening to what people already said tell a team whether a message, claim, or positioning choice that has not gone live yet will work? It does not, because listening only reports on conversation that already happened. A message that backfires is only visible after it is published, spending, and shaping how the market reads the brand. ## What Ipsos Synthesio answers Ipsos Synthesio aggregates social, search, news, and forum data into dashboards and reports for consumer-intelligence and market-research teams. It tracks sentiment, share of voice, and emerging conversation across a category, and it plugs into the marketing and analytics stack that enterprise research functions already run. What it cannot do is tell a team how an audience will respond to a message, claim, or positioning line that has not shipped. Listening data is observational; there is no experiment behind it, because nobody has been shown the thing that has not been said yet. ## Where does the gap show up before a launch? Treating a monitoring read as a substitute for a pre-launch test is how a message ships on evidence that was never designed to answer the question being asked. ## Testing the message before it goes live Subconscious runs controlled experiments against a person-level audience graph covering 800 million real people. Instead of waiting to see how a market reacts to a message that has already gone out, a team defines two or more candidate messages, claims, or positioning lines and tests which one is more likely to change behavior, before committing media spend. This is a causal test of a proposed action, not a listening or monitoring product. When a decision calls for it, Subconscious can validate a study with real human participants, moving from a simulated experiment to real-human validation without changing the causal question being asked. ## Comparing what each method answers ## How do listening and causal testing work together? The two categories are complementary, not competing. Listening surfaces what is already happening in a category: which themes are rising, where sentiment is turning, what a competitor's claim triggered. Those signals are useful raw material for shaping which candidate messages are worth testing next. A pre-launch causal test then evaluates those specific candidates before they ship. For a brand team running both, the sequence is: listen to know what to test, then test before committing budget to what you publish. ## What does Subconscious not cover? Subconscious does not perform social listening, sentiment monitoring, or real-time conversation tracking; that remains listening's job, and a platform built for it should stay in the workflow for ongoing brand-health monitoring and crisis detection. Audience-graph reach is not the same as a recruited panel. Confidence intervals and other quantitative outputs are only available when the specific study design supports them, not as a universal feature of every test. And a simulated result should be treated as a first pass, not a final answer, when the decision is high enough stakes to warrant recruiting real participants. [figure: Four stages: listening surfaces rising themes, those become candidate messages, a causal test measures which candidate is more likely to change behavior, then an optional step validates high-stakes results.] To see how a pre-launch causal test applies to a specific message or claim, talk to the team or read more about how Subconscious works. --- # AI Research for Advertising Agencies: Test Creative Before the Client Commits Budget URL: https://subconscious.ai/blog/buyers/ai-research-advertising-agencies-test-creative-before-client-commits-budget Published: 2026-08-20 An agency pitch lands in two weeks. A campaign has to launch next month. The client wants evidence the concept will work before it commits media budget. Traditional pre-launch research does not move at that speed, so most creative decisions ship on instinct alone. Subconscious gives an agency a way to answer the question that actually matters: which creative concept, tagline, or message hierarchy causes the stronger response from the client's target buyer, before the agency recommends one to the client. [figure: Three rows: creative instinct gives an opinion, not measured response; traditional research needs a separate budget and fieldwork window; a causal experiment is scoped per study and returns a measured effect on response.] ## The research gap in agency workflows The client briefs the campaign. The creative team develops two to four concepts. Someone proposes testing them with real consumers. The research team, or a partner research agency, scopes recruitment, methodology, timing, and cost, and published market-research pricing guidance shows that scoping work routinely lands well past what a single campaign has set aside (Drive Research's 2026 market research cost guide). If the proposed study exceeds the campaign's budget or schedule, the creative director's preferred concept ships instead. That gap leaves a large share of creative decisions (taglines, visual concepts, message hierarchies, campaign themes, brand positioning) without any structured read on how the target audience actually responds. Nobody can tell whether the creative director's judgment was right until the campaign is already in market. ## How the decision compares across methods The third row is the option Subconscious adds: a controlled discrete-choice experiment that compares the concept alternatives directly, rather than collecting an open-ended reaction from a recruited group. ## What the experiment actually measures The agency defines the concepts or message alternatives to compare and the audience segments they need to hold up in market. Subconscious runs a controlled experiment across those segments and returns a measured causal effect on response for each alternative, not a simulated persona's free-text reaction. The agency can compare which concept moves the target buyer's stated choice. /research and /case-studies describe how Subconscious structures and validates these experiments. When the decision calls for it, a team can move from this kind of simulated experiment to a study recruiting real human participants, without changing the causal question being tested. Stating plainly when a step is optional helps the agency plan budget and timeline around it. That step is not automatic and should be reserved for decisions where the extra evidence changes the recommendation the agency makes to the client; most creative-concept comparisons do not need it. ## Where this fits into agency work Pitch preparation. A pitch team rarely has the research budget or runway for a traditional study. Arriving with a measured comparison between strategic positioning options, even from a controlled experiment rather than in-market data, differentiates from competitors who present creative instinct alone. Message hierarchy. Which line should lead the campaign? Comparing message alternatives head-to-head against the same defined audience gives a measured answer to that question, rather than commissioning a full study for each candidate line. Multi-market adaptation. Comparing a concept against distinct market-defined segments tests whether it holds up before the agency invests in market-specific adaptation. Creative refinement. Once a broad concept is approved, the same approach can compare narrower execution choices (tone of voice, visual direction, call-to-action wording) against the client's audience. ## What this does not replace A measured effect is only useful with its limits stated plainly. A controlled comparison of creative concepts does not measure emotional response: it returns a measured preference signal. It does not replace client discovery work, moderated qualitative research with real participants, or observed in-market campaign performance. And a simulated experiment is not itself real-human validation: the audience-segment comparison and a study recruiting real participants are two different steps, and an agency should keep them distinct. ## A useful next step Pick one project already on the agency's calendar where a research decision would otherwise get skipped for budget or timeline reasons. Define the concept alternatives and the client's target segments, then compare them before the recommendation goes to the client rather than after the campaign launches. Book a demo to see how Subconscious structures that comparison, or read how Subconscious works for the underlying method. --- # Cultural-Data APIs vs. AI Panels vs. Causal Experiments: Which Research Tool Fits Your Decision URL: https://subconscious.ai/blog/comparisons/cultural-data-apis-vs-ai-panels-vs-causal-experiments-which-research-tool Published: 2026-08-20 A CMO or VP of insights choosing how to justify a launch, message, or positioning decision usually has three kinds of tools on the table, and they answer three different questions. A taste and preference data API tells you what correlates with what. An AI panel tool lets you interview simulated stakeholders and hear what they might say. A causal experimentation platform tells you what happens to real behavior if you take a specific action. Mixing these up is the expensive mistake: treating a correlation or a simulated opinion as proof that a message or launch will move the outcome, then finding out only after the spend is gone. ## What a cultural-data API answers Qloo positions itself as a cultural intelligence layer for AI products. According to its API product page, the platform connects taste and preference signals across categories such as music, film, dining, travel, and brands, and makes that data available through APIs, reports, and data feeds for integration into recommendation engines, personalization systems, and language models. This kind of tool answers "what tends to go with what" for a given audience. It is strongest as an input to a product or model, not as a way to test whether a specific message or launch action will change what someone actually does. ## What does an AI panel tool answer? A separate category of research tool builds interactive AI-generated stakeholder panels: you interview them, run them through surveys, or convene a group discussion to explore reactions to a concept. These tools are useful for early, open-ended exploration, fast directional feedback, and surfacing objections before a message or concept is fully formed. What they cannot do is establish that a specific action caused a specific outcome: a plausible-sounding answer from an AI panel is still "what might someone say," not evidence of what a market will do under a controlled alternative. ## Where does a causal experimentation platform fit? Subconscious.ai sits in a third category. It runs randomized experiments on a simulated population to estimate which action drives an outcome for a defined decision, such as a message, a price, a product change, or a launch: if we take this action, what changes, and how confident are we. The platform can run controlled studies against a person-level audience graph covering 800 million real people, a scale question that is distinct from a recruitable panel. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. A fidelity number by itself is marketing until its limits are published alongside it. It is a validation result, not a guarantee for a new market; read the methodology at the causal fidelity paper. When a decision needs it, Subconscious can also test or validate a study with real human participants, moving from a simulated experiment to real-human validation without changing the underlying causal question. Details on how a study is scoped and run live at /how-we-work, and published methodology and results live at /research. ## Comparing the three categories ## Limitations to plan around A published limit is what lets a buyer check the tool before they commit budget. Subconscious does not license cultural taste or preference data for personalization or recommendation systems, and it is not a substitute for a preference-data API feeding a model or a recommender. It is also not an interactive interview or panel product built for open-ended qualitative exploration. A causal study needs a defined decision and specific actions to compare against each other, so it is not the right tool for broad, undirected cultural or taste mapping. Confidence intervals, segment breakdowns, and other statistical outputs should only be expected where the specific study design supports them. ## How do you choose among the three? If the job is enriching a product or model with cultural context, a preference-data API is the right layer. If the job is fast, open-ended exploration of a concept before it is fully formed, an AI panel tool has a place. If the job is deciding whether a specific message, price, or launch action will change real behavior before committing budget to it, that is a causal experiment. Teams that need to make that last call can see how a study gets scoped and run at /demo, or read more about the company at /about. [figure: Three research tool categories, each labeled with the question it answers: cultural-data API (correlation), AI panel (simulated opinion), causal experimentation (causal effect), with optional real-human validation.] --- # Concept Testing Before You Build URL: https://subconscious.ai/blog/use-cases/concept-testing-before-build Published: 2026-08-20 Concept testing works best while an idea can still change. Once a team has spent weeks refining a concept, internal agreement can turn research into a request for approval. The better question comes earlier: which version changes the behavior that matters, for which buyers? Subconscious helps teams compare product, pricing, messaging, and go-to-market actions through decision-specific causal experiments on simulated buyers. The aim is not to collect another set of opinions. It is to test alternatives before a team commits capital, then validate important decisions against real customer evidence. [figure: A five-step horizontal path: name the decision and alternatives, build comparable alternatives, check clarity with five prompts, revise one variable and document it, validate against real customer evidence.] ## Why timing changes the value of a concept test A traditional study may require a developed concept, finished stimulus, an agency brief, respondent recruitment, fieldwork, and analysis. Example planning constraints: - Fieldwork alone can run 6-8 weeks before a team sees a result. - Budget for a single round often lands between €15,000 and €30,000. Those constraints create a familiar incentive: expensive tests tend to be reserved for ideas the team already favors. At 2-3 concepts per study and only 2-4 studies fielded each year, a team lands somewhere around ten tested concepts annually, at best. Early concept testing changes the sequence. Teams can compare rough alternatives, learn which differences matter, and carry fewer, stronger concepts into customer research. ## Start with the decision A useful concept test begins with one choice the team must make. Examples include: - Which value proposition should lead the launch? - Which product configuration should move forward? - Which message changes stated choice among the target audience? - Which price scenario deserves further study? - Which concept works across segments, and which needs a separate treatment? Write down the action, the audience, the alternatives, and the behavior to observe. If the result cannot change a decision, the test is not ready. ## Compare actions, not descriptions Concept teams often ask whether people like an idea. A stronger design asks buyers to choose between clear alternatives under the same conditions. ### Value proposition Describe the problem, the proposed solution, and the expected benefit in plain language. Compare distinct versions rather than polishing one version in isolation. ### Positioning Hold the offer steady and vary the frame. A time-saving message, a risk-reduction message, and a growth message imply different reasons to act. The experiment should reveal how those alternatives change choice across defined segments. ### Feature priorities Long wish lists hide tradeoffs. Say the roadmap holds ten features: the real question is which three earn their place, and for whom. Present concrete configurations and ask buyers to choose. A decision-specific comparison produces more useful evidence than asking whether every feature sounds valuable. ### How should pricing scenarios be tested? Pricing should be treated as scenario testing, not an automatic optimization claim. Compare explicit alternatives when the study supports them. Do not infer elasticity, willingness to pay, margin effects, or a revenue outcome unless the study was designed to estimate those quantities. ### Competitive position Place the concept beside real alternatives. If buyers cannot distinguish the offer or explain why they would switch, the team has learned something specific enough to change the concept. ### Naming and language Names, taglines, and feature descriptions can be tested as parts of a decision. Keep the underlying offer fixed when the goal is to isolate the effect of language. ## Iterate without losing the experiment The reaction to Version 1 may suggest a better Version 2. Iteration is useful, but each round still needs a clear comparison. Record what changed, keep the audience definition stable, and avoid changing the offer, message, and price at the same time. This example workflow takes 2-3 hours. Actual delivery depends on the study scope and validation requirements. ### Step 1 (30 min): Build three personas Define the core buyer, an adjacent segment, and a skeptical segment. Use available customer evidence to ground those definitions. Treat each persona as a hypothesis about an audience, not as proof of behavior. Then write the concepts in comparable form. ### Step 2: Test the decision logic (60 min) Use the same five prompts to check whether each concept is clear enough for an experiment: - What is the first reaction? - Who does the buyer think the concept is for? - What would stop the buyer from choosing it? - How does it compare with the current alternative? - What would make it necessary rather than merely useful? These prompts help expose missing assumptions. They do not by themselves establish causal effects or predict purchase behavior. ### Step 3: Revise one variable at a time (30 min) Use the response to Version 1 to create a more precise Version 3, but document the change. A clean record keeps iteration from becoming a sequence of unrelated prompts. ### Step 4: Record the decision evidence (30 min) Capture the alternatives, audience definitions, observed differences, consistent objections, and remaining uncertainty. State what the evidence supports and what still needs real-world validation. [figure: A five-step loop for iterating on a concept test: observe the reaction to the current version, change exactly one variable, keep the audience definition fixed, record what changed, then test the next version and repeat.] ## Where does early concept testing help most? Early-stage ideation earns its keep at the top of the funnel, where a long list of twenty candidates has to shrink down to the five worth building. Iterative refinement helps when a promising concept has unclear positioning. Cross-segment testing helps when the same offer must work across distinct buyer groups. Standing up five personas takes far less time than fielding five separate respondent groups. That comparison applies to exploratory hypothesis work, not final validation. The method is less useful when the stimulus depends on a finished sensory experience, when the target community cannot be represented with care, or when the decision requires observed purchase behavior. Aggregate patterns are generally easier to approximate than individual behavior, and prompt sensitivity, variance collapse, and demographic flattening remain real risks in synthetic research (Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents, ACM Web Conference 2026 Companion Proceedings). Concept testing should compress the path to a better hypothesis. It should not be used to claim that someone will buy. For large investments, use the early experiment to choose what deserves deeper customer research, then validate the decision with evidence appropriate to the risk. --- # Where AI Belongs in the Research Process: A Buyer's Decision Guide URL: https://subconscious.ai/blog/buyers/where-ai-belongs-research-process-buyers-decision-guide Published: 2026-08-20 A research or insights leader who folds AI-assisted methods into an existing practice faces one decision repeatedly: which stage of the process gets a fast AI-assisted read, and which stage still needs a fielded study with real respondents. Get that wrong and a directional exploration gets presented as validated evidence. That mistake costs the researcher's credibility: the next study they present gets read with more suspicion, not less. [figure: Path diagram: AI-assisted exploration feeds a controlled experiment on a simulated market, which branches on stakes, low stops there, high continues to a fielded study.] ## Why This Decision Is Getting Harder AI has moved from a novelty layer into daily research workflows: drafting surveys, summarizing transcripts, generating first-pass reactions to a concept. That does not eliminate the need for a researcher; it removes the advantage that used to come from access alone: knowing how to field a study, clean the data, and produce a chart. When more people can generate a plausible-sounding answer, the valuable skill becomes knowing which answer deserves trust, and why. AI-assisted exploration is fast and cheap because it is not measuring real behavior. A fielded, human-respondent study is slower and more expensive because it is (Drive Research's 2026 market research cost guide). Treating the first as a substitute for the second, when the decision is expensive or public, is the actual failure mode. ## The Method Boundary That Matters Three categories of method sit on this continuum. A research team benefits from naming which one it is using before presenting a result. Subconscious sits in the middle row: the controlled, causal step between open-ended AI exploration and a fielded human study. The replication leaderboard describes how these experiments are structured and validated. ## What Causes the Mislabeling Problem? The failure is rarely a bad tool. It is a missing label. A team under deadline pressure runs an AI-assisted exploration, gets a fluent answer, and drops it into a deck without stating what stage of evidence it represents. The fix is making the evidence stage part of the deliverable: what was tested, at what stage, and what still needs checking before an expensive commitment rides on it. ## A Practical Evaluation Framework Before running any test, a research lead can walk through four questions: - What decision does this test inform, and what does the business do differently depending on the result? - How expensive or public is the decision if it turns out wrong? Low-stakes, reversible decisions can move on an AI-assisted exploratory read. High-stakes, public, or regulated decisions need a controlled experiment, and often a fielded human study. - What alternatives is the team actually comparing? A vague "get reactions to this idea" question produces a vague answer. A defined set of alternatives, audience, and outcome produce a testable one. - What would change the answer? If nothing found in testing could shift the recommendation, the test was not designed to inform the decision. ## Where Does Real-Human Validation Fit? Subconscious can test or validate studies with real human participants. A team can move from a structured, simulated experiment to real-human validation without changing the underlying causal question: the same alternatives, population definition, and outcome carry through, so validation checks the same claim the experiment made. Audience reach and recruited validation are distinct claims. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people; that scale describes the simulated market, not a recruitable panel of human respondents available for every study. ## Limitations and Failure Conditions A controlled experiment is not a substitute for the researcher's own judgment, the stakeholder framing around a decision, or fielded human validation when the decision is expensive or public. Real-human validation checks a causal claim; it does not turn that claim into an observed usability session, a clinical trial, or automatic proof of market performance. Keep audience-scale claims, simulated experiment results, and recruited real-human validation labeled separately in any deliverable that leaves the research team. Treating an exploratory, low-cost read as if it carried the confidence of a validated study is what damages credibility when the underlying decision turns out wrong. ## Where Should a Research Team Start? Pick one active project with a real, live decision behind it. Use a fast AI-assisted or synthetic exploratory pass only for the discovery stage: narrowing hypotheses, not confirming an answer. Move anything supporting an expensive or public decision into a controlled, causal experiment and, where the stakes justify it, real-human validation. Book a working session to walk through where a specific decision falls on that continuum, or see how the platform structures a study end to end. --- # What a Live Buyer-Reaction Demo Can and Can't Prove in an Agency Pitch URL: https://subconscious.ai/blog/buyers/how-agencies-use-live-buyer-panels-pitches Published: 2026-08-20 An agency preparing a new-business pitch must decide whether to open with a live, in-room demo of how a simulated audience reacts to the prospect's brand, or hold that moment back. The harder question: what can the agency honestly tell the prospect that reaction proves? Get that wrong with the wrong prospect and the tactic backfires on the account it was meant to win. ## What a live reaction demo shows, and what it doesn't The mechanics are simple: most research-forward agencies can build one before a first meeting. Someone assembles a small simulated audience matched to the prospect's buyer profile, often eight to twelve profiles for a session, then shares a screen and types a live question: what does this audience think of the brand, the messaging, a competitor. The room watches text responses appear in real time; the session typically runs fifteen to twenty minutes before the conversation shifts back to next steps. What the room is watching is open-ended reaction to a single prompt. Nobody defined two or more specific options in advance. Nobody varied one thing at a time to isolate what moved the response. Nobody attached a confidence interval to what came back. It reads as evidence because it's live and the output looks specific, but a single open reaction to one prompt is not a comparison, and does not tell the prospect which of two messaging directions would actually change a buyer's choice. ## Where does this break with the prospect an agency wants most? The prospects worth winning often have their own research or data function, and know the difference between a live reaction and a tested claim. If that prospect asks how the audience was built, what the sample represented, or what confidence interval sits behind the answer, and the agency has no response beyond "watch what it says," the moment built to demonstrate research capability shows the opposite: that nothing was tested. That failure lands hardest with the buyers who write procurement requirements to screen it out. Procurement at larger accounts increasingly probes sample construction and confidence levels rather than accepting a demo at face value (GreenBook, GRIT Report 2025: Insights Industry Trends). ## A pitch moment built around one completed comparison instead The alternative isn't abandoning a live moment; it's changing what that moment claims to show. Subconscious can run a controlled discrete-choice experiment that compares a prospect's actual message or positioning options against a defined audience and returns a causal effect with a confidence interval attached, a result the research methodology documents. That's a different claim than an open reaction to a single prompt: instead of "here is what this audience says," the pitch becomes "here is which of your two options this audience actually preferred, and how confident we are in that difference." An agency can walk a prospect through a comparison finished before the meeting rather than improvising live, drawing on the same case study format it would use with a paying client. The audience behind the comparison is also worth describing accurately: a person-level graph built for study design, covering 800 million real people, not a panel recruited and scheduled for a session. [figure: Four-stage path: a live reaction demo, a prospect asking about sample and confidence, a prepared comparison with a confidence interval, and an optional real-human validation stage.] ## What does a completed comparison still not prove? A single controlled comparison run before a pitch is not proof of what happens after launch. The result carries a stated confidence interval, not a guarantee, and that distinction is worth saying out loud rather than leaving it implied. Moving from a simulated comparison to real-human validation is a separate, explicit next step, not something bundled automatically behind every simulated result: a team can test the same study with real human participants without changing the underlying causal question. [figure: Four-step path: a completed simulated comparison, its confidence interval, a note that the interval is not a guarantee, and a separate optional step validating the same study with real human participants.] ## What should an agency decide before the next pitch? The decision an agency needs to make before a new-business meeting is narrower than "should we do a live demo." It is: does the planned moment answer the question this specific prospect will ask? A research-sophisticated prospect wants a defined comparison and a confidence interval. A less sophisticated one may respond just as well to a live reaction, provided the agency doesn't overstate what it means. Matching the pitch moment to the prospect, and being precise about what it proves, is what keeps the tactic from turning into the exposure it was meant to prevent. Agencies weighing this can see how a comparison study for a specific prospect audience gets scoped by booking time to walk through it, or read how the underlying experiment process runs end to end. --- # How to Use Synthetic Audiences Without Losing Credibility URL: https://subconscious.ai/blog/implementation-and-procurement/how-use-synthetic-audiences-without-losing-credibility Published: 2026-08-20 Synthetic audiences keep credibility when a team decides, before a study runs, which findings stay directional and which must clear real-human validation before backing a launch or a public claim. A consumer insights lead who ships a synthetic-audience finding and gets it wrong in front of leadership does not just lose one argument. The team's evidence gets treated as unreliable going forward, and every future finding has to fight that history. The risk is not that synthetic exploration is unreliable by design. It is that an undisclosed or unvalidated finding gets treated as proof when it was only ever a directional read. The fix is not avoiding synthetic audiences. It is building a repeatable gate that decides, before a study starts, which questions can stay simulation-only and which must move to real-human validation before anyone uses the result to justify a launch, a price, or a claim. [figure: Decision path: exploration and directional-comparison questions need no gate; a question tied to a launch, price, or public claim passes human review, then must pass real-human validation before shipping.] ## Why the disclosure question is not optional Synthetic and AI-assisted methods now sit inside ordinary research workflows: hypothesis generation, first-pass analysis, message screening. That shift raises the exact question a compliance-minded buyer has to answer before adopting any of it: what gets disclosed, and to whom. The updated ICC/ESOMAR code addresses how AI and synthetic methods should be disclosed and governed in commercial research, and rising use of AI in complex data practices is part of why national research bodies are adopting it (Research World, ESOMAR; BestMediaInfo on MRSI's ICC-ESOMAR 2025 Code adoption). A team that cannot say what a synthetic finding was used for, and what it was not used for, is out of step with where the standard is heading, not just exposed to an internal credibility risk. ## Should the gate be set before the study or after the finding? The gate is not a judgment call made once a result looks convincing. It is a decision made in advance, based on what the finding will be used for: - Exploration only. Hypothesis generation, objection mapping, early message screening. No launch, pricing, or public claim depends on the result alone. - Directional comparison. Ranking concepts, framing options, or positioning statements against each other to narrow a shortlist before a more expensive study. - Validation-gated. Any claim that will justify a launch, a price change, a regulatory position, or an external statement. This tier requires real-human confirmation of the same question before it ships. Write the tier down before the study runs. A team that decides the tier after seeing the result will tend to upgrade a convenient finding to "good enough to act on." ## How do you run a directional comparison? Once a question is scoped to exploration or directional comparison, the workflow is a controlled experiment, not an open-ended chat. Start with the decision the business needs to make and what would change it. Define the audience the comparison represents, including its segment, context, and current behavior, so the result can be traced back to who it describes. Hold the stimulus and question constant across the comparison so any difference in response reflects the variable under test, not a change in the prompt. This is where Subconscious fits the tier: causal experiments run against a simulated market let a team compare which message, price point, or feature framing moves a stated outcome, with confidence attached to the comparison rather than a single fluent-sounding answer. ## Why put a human between the output and the claim? A synthetic comparison produces output. Deciding whether that output is valid evidence for the decision in front of the team is a human step, not an automatic one. The review checks three things: whether the audience definition matches who the business needs to understand, whether the stimulus or prompt introduced bias toward one answer, and whether the grounding behind the audience is current and relevant to the market in question. ## Validate before the claim goes public Naming where a method stops is what lets a buyer check it against the claim. Some questions cannot be answered by simulation alone, no matter how well the comparison was run: anything that depends on observed behavior, anything that will justify a launch or pricing commitment, and anything a regulator or an external audience could challenge. For this tier, Subconscious can test or validate studies with real human participants, so a team can move from the simulated comparison to real-human validation without changing the causal question. The audience, stimulus, and outcome measure stay the same; only the source of the response changes, which is what makes the human result a genuine check on the simulated one rather than a separate study answering a separate question. The misses belong on the record next to the hits, or the record is not worth trusting. Real-human validation confirms or corrects the simulated comparison. It does not turn the exercise into an observed usability session, a clinical trial, or automatic proof that the finding will hold in market. ## Disclose the tier in the deliverable The labeling problem behind most synthetic research failures is not that the method was used. It is that the tier was hidden. State plainly, in the deliverable itself, which tier a finding belongs to, the purpose it was applied to, the purpose it was withheld from, and what validation step comes next if the decision escalates. A finding labeled "directional, exploration only, not yet validated" is more useful to a stakeholder than one that reads as settled and turns out not to be. The first version survives scrutiny; the second is the failure mode that costs a research function its credibility. [figure: Three checks before a synthetic comparison counts as evidence: audience definition matches who the business needs, stimulus or prompt didn't bias the answer, and grounding is current for the market.] ## A starting checklist Before a synthetic-audience study begins, confirm: - The business decision the finding will inform, written in one sentence. - The tier: exploration, directional comparison, or validation-gated. - The audience definition, including segment, context, and what would make the comparison invalid. - Who reviews the output for bias and grounding before it moves forward. - Whether a real-human validation step is required before the finding can support a launch, pricing, or public claim, and who owns scheduling it. Teams that answer these five questions before running a study spend less time defending a finding after the fact. Review current research methodology and worked case studies to see how the same decision-gate structure plays out end to end, or see how we work with insights teams building this into a standing process. To scope a validation-gated study, book a session. --- # AI Consumer Behavior Analysis: Move from Events to Causes URL: https://subconscious.ai/blog/use-cases/ai-consumer-behavior-analysis-move-events-causes Published: 2026-08-20 An analytics dashboard can show 34% of users leave after the third screen; churn data can show a spike in month four. Neither explains what changed between month three and month four. Behavioral data records events. Consumer behavior analysis studies the triggers, tradeoffs, habits, and context that produced them. Causal experiments test whether a specific action changes an outcome. [figure: Four-stage chain: analytics shows an unexplained drop, simulation probes a modeled audience for triggers, an experiment tests one specific action, and the result is a decision with a measured effect.] ## What does consumer behavior analysis study? Consumer insights cover attitudes, preferences, and perceptions. Behavior analysis focuses on the decision process: triggers, heuristics, emotional changes, and post-purchase rationalization. The discipline draws from behavioral psychology, cognitive science, and decision theory. Useful questions include: - What starts an evaluation? - Which heuristics dominate the category? - When does habit replace deliberate choice? - What causes a loyal customer to switch? - How does a post-purchase explanation affect repeat behavior? ## Why can't analytics alone identify the mechanism? Customers who use a feature may retain better because the feature creates value, or because they belong to a segment already more committed. A discount in month two may increase renewal because it changed perceived value. It may delay churn by one cycle. Event data cannot distinguish them without a design that tests the intervention. Surveys also have limits. People often cannot accurately report the mental process behind a choice, and instead construct a plausible-sounding account after the fact (Nisbett & Wilson, "Telling More Than We Can Know," Psychological Review). Treat stated reasons as evidence to compare with behavior, not as the cause by default. ## Use simulations to form behavioral hypotheses Build the audience profile around how a person decides, not just who they are: the habits that govern the category, how much risk they carry into the choice, how they hunt for information, and the brand history they bring. Causal experiments can then test the resulting hypotheses against a specific choice. Probe the search trigger, first criterion, information source, moment of friction, and final tradeoff. Map when a repeated behavior shifts from deliberate choice to habit and what might interrupt it. A researcher can run probes across dozens of audience configurations in an afternoon; comparable ethnographic work may take months. Breadth does not guarantee fidelity. ## Decisions that benefit ### Product design Study the habits a product must fit or displace: one requiring a new routine needs a different adoption plan than one that fits an existing workflow. ### Retention Map the sequence that precedes cancellation. Identify when value erodes, when switching feels easier than staying, and which intervention might alter the path. ### Messaging Compare messages against the decision mechanism. A segment motivated by regret avoidance may respond differently from one motivated by aspiration. ### Competitive strategy Model what keeps a competitor's customers and what might prompt reconsideration. Use public evidence and validate the hypotheses. ### Pricing Compare how segments interpret price points and value arguments. Frame the output as decision-specific scenario testing, not a live automated price optimizer. [figure: Five steps: pick one decision, define audience and outcome, probe trigger to commitment, repeat across 3-5 audience variants, compare to find where a policy is too broad.] ## Start with one decision A team does not need a six-figure research budget to frame a behavioral question. Pick one decision based on an unexplained pattern. Define the audience, alternatives, and outcome. Spend 30 minutes probing the path from trigger to commitment. Then run the same protocol across three to five audience variants. Differences between segments can show where one policy or message is too broad. The simulation supplies hypotheses. Analytics supplies observed patterns. A causal experiment tests the proposed action. Human research remains essential when the decision requires individual or cultural fidelity the model lacks; see how we work for that validation step. --- # AI for New Product Development Research: Test Each Decision URL: https://subconscious.ai/blog/use-cases/ai-new-product-development-research-test-each-decision Published: 2026-08-20 New product development contains decisions about opportunity, concept, positioning, pricing, and launch. If research appears only after months of development, the team learns when changing course is most expensive. The often-cited 70-90% new-product failure range is not a Subconscious benchmark. The structural point: concentrating research into one or two checkpoints leaves long stretches of product work driven by assumptions. [figure: Five-stage path from opportunity and ideation through concept screening, concept validation, positioning and pricing, to launch planning, each marked as its own decision point needing a separate customer test.] ## Find the missing decisions Map where customer evidence enters the process: - opportunity identification; - concept generation and screening; - concept validation; - positioning and pricing; - launch planning. In many workflows, formal concept testing begins 3-6 months into the process. Early opportunity choices and late launch choices receive less direct customer evidence. ## Test actions at each stage ### Opportunity and ideation Explore customer problems, workarounds, and unmet needs before generating solutions. Use simulation to form hypotheses, then validate the important ones with people and observed behavior. ### What happens during concept screening? A planning exercise might reduce twenty rough ideas to five candidates. Compare the same dimensions across each concept. Survival means a concept earned more research, not that the market validated it. ### Concept validation Compare shortlisted concepts on comprehension, relevance, objections, and directional preference. ### How should positioning and pricing work? Compare “save time,” “reduce risk,” and “increase revenue” as distinct frames. Pricing work should remain decision-specific scenario testing unless a current study supports a stronger output. ### Launch planning Test messages, audience definitions, and adoption triggers before day one. Ask how a buyer would describe the product and what objection would stop the next action. [figure: A four-step chain: twenty rough concepts feed a comparison on the same dimensions, narrowing to five candidates, ending with a label stating survival means the concept earned more research, not market validation.] ## How do you use repeated small studies carefully? One workflow uses three capabilities: audience definitions for each phase, rapid iteration, and comparison across five customer types. These are study-design examples, not universal product limits. For CPG, the decision may involve a concept, claim, or package. For software, it may involve feature bundles or onboarding. For services, it may involve packaging and buyer language. In every case, the experiment should name the action and outcome. Start with the first live decision where customer input is missing. Use simulated evidence to narrow options, then spend human-research budget on the questions that require real-world validation, the split that matters because model outputs on choice-style tasks stay sensitive to prompting and struggle with segmentation and individual-level heterogeneity (Can large language models assist choice modelling? Insights into prompting strategies and current models capabilities, arXiv, checked 2026-07-27). See a worked example in a published case study. --- # AI Panels for Creative Agencies: Compare Concepts Before the Client Review URL: https://subconscious.ai/blog/buyers/ai-panels-creative-agencies-compare-concepts-before-client-review Published: 2026-08-20 AI panels let agencies run causal behavioral comparisons of campaign directions, headlines, tone, and calls to action across defined audience segments before a client ever sees the work. An agency can spend three weeks on a campaign concept, lose the client review, then spend three more weeks on revisions. Early behavioral comparison cannot guarantee approval, but it can expose weak messages and unsupported audience assumptions before production. [figure: Five-step horizontal path: develop three to five directions, narrow to three, run a comparison session across audience segments, revise, then present the chosen direction with its evidence and limits.] ## Test the decision, not the client A conference room may include a CMO, brand manager, and legal team. A 52-year-old executive may react differently from the 25-year-old consumers the work addresses. Define the campaign action and the behavior it should change. Compare the same stimulus across relevant audience definitions. ## Concept and copy comparisons For a sportswear campaign aimed at Gen Z runners, a planning exercise might test: - the core idea; - the headline or tagline; - the campaign tone; - the call to action. Ask which concept is understood, what feels credible, and what creates confusion. Do not turn simulated quotes into real-customer testimonials. One planning example cuts twenty copy variants down to five finalists, backed by data on why those five outperformed the rest. Comparing alternatives against a defined audience before production is a documented concept-testing practice (Qualtrics, "Concept Testing: Definition, Methodology & Examples"). Brand-voice work can compare the same message in three voices, such as playful, authoritative, and conversational. ## How do you define audiences beyond age? A label such as “women 25-34” is less useful than a decision-relevant audience definition that includes context, goals, category behavior, and constraints. Compare separate segments rather than averaging away disagreement. ## An example agency workflow One planning workflow is: - use early research to understand audience language; - develop 3-5 directions; - narrow to the strongest 3; - compare the top 3 in a 2-3 hour session; - revise; - present the alternatives and the limits of the evidence. The example adds half a day. It is not a Subconscious delivery-time or outcome guarantee. The method behind that comparison step is described on our research page. [figure: Diagram contrasting two readings of one concept test across three audience segments: an averaged score that looks acceptable, versus the same three segments shown separately, revealing one rejects the concept.] ## Who stays accountable for creative decisions? Audience evidence should sharpen an idea, not flatten it. Creative directors still decide what the brand should say. Clients still decide what to approve. Real experiments still determine how the work performs. Start with one project and one clear decision. Use causal behavioral experiments to compare actions before spending on production, then validate the selected direction with human research or live behavior. Agencies ready to scope a pilot can book a working session. --- # Before You Trust a Simulated Buyer Study, Ask What Validates It URL: https://subconscious.ai/blog/use-cases/before-trust-simulated-buyer-study-ask-what-validates Published: 2026-08-20 A simulated buyer study earns trust when it runs as a controlled, causal experiment against a defined population and its findings are checked against real human participants using that same design. A CMO deciding whether to greenlight a message before production and media dollars commit needs proof a modeled-buyer study tracks how real buyers respond, not a confident-sounding transcript. A free-form conversation with an AI stand-in generates plausible reactions on demand. Plausible is not predictive, and the gap between the two is where launch budgets go to die. ## The real choice: prompt-based reactions or a validated experiment The question for a CMO or VP of Insights is rarely whether to test messaging before it ships. It is whether a prompt-based reaction from a language model is sufficient evidence to greenlight that message, or whether the decision needs a controlled, causal experiment validated against real human behavior. A modeled panel that reads as confident and directionally positive, but has quietly collapsed the variance and reasoning diversity a real audience would show, sends a campaign into production on a false signal. The message ships, real buyers don't move, and the miss stays invisible until the media budget is spent and the launch window has closed. ## Why can fluent output still mislead? Peer-reviewed work on large language models as stand-ins for real survey respondents documents specific, repeatable failure modes, not random noise. Models conditioned on demographic or attitudinal backstories can flatten the diversity of real opinion, understate disagreement, and produce answers that look more internally consistent than any real population is (Cambridge University Press, Political Analysis). A related study on digital personas approximating human survey findings concludes reliability is conditional: it depends on question type and calibration method, not the underlying model alone (arXiv). Neither paper concludes that modeled respondents are useless. Both put the burden of proof on whoever ran the simulation: an ungrounded conversation is a hypothesis, not evidence. ## What does a controlled experiment change? Subconscious runs controlled experiments on a simulation of the market rather than open-ended interviews with stand-ins. That distinction addresses the failure modes above: a controlled design compares defined alternatives against a defined population under a fixed decision, instead of letting a model free-associate a plausible answer to an open prompt. The question is not what an AI thinks buyers would say. It is which of the tested alternatives moves the outcome, for which segment. Subconscious can run those studies against a person-level audience graph covering 800 million real people. That graph defines who a study represents, not a recruitable panel of 800 million people standing by to answer questions. ## The step most teams skip: closing the loop with real people Subconscious can test or validate the same study with real human participants, without changing the underlying causal question the study was designed to answer. A team can run the comparison against the simulated market, then move the identical design to a real-human sample to confirm the direction holds, instead of switching methods midstream and hoping the two agree. This turns "the model said message A wins" into "the model said message A wins, and a real-human replication checked whether it held." An ungrounded conversation offers no equivalent second step. ## Where does the method still fall short? A controlled simulation, validated or not, has real limits a CMO should weigh before treating any result as final: - It does not certify a market-share or adoption number. A comparison showing message A outperforming message B is not a forecast of real market share. - It cannot model behavior with no precedent. A novel product category, crisis response, or cultural moment has no comparable pattern in the data, and extrapolating one is speculation dressed as output. - It does not substitute for regulated or safety-relevant human-subject research. Contexts carrying legal, medical, or safety weight require real participants under the applicable protocol. - Real-human validation checks one study, not the method generally. Confirming one finding with real participants does not certify every other simulated result the team has run. ## Sequencing the test before the budget commits - Define the decision, the alternatives, and the population before touching any tool. Vague segments and vague messages produce vague comparisons. - Run the controlled comparison against the simulated population and look at which alternative moves the outcome, not which one sounds most persuasive. - Where the decision carries real budget risk, validate the finding against real human participants using the same causal design. - Treat an unvalidated conversation with an AI stand-in as a starting hypothesis for a real study, not the study itself. Teams evaluating this category can read Subconscious's replication research and the current leaderboard of validated studies, review how the experiment workflow runs, and scope a specific decision once the alternatives and population are defined. [figure: Branching path: one branch, an open-ended AI conversation, dead-ends unvalidated. The other runs a controlled experiment on a simulated market, then real-human validation, ending confirmed or contradicted.] --- # AI Market Segmentation: Test Behavior, Not Demographic Labels URL: https://subconscious.ai/blog/use-cases/ai-market-segmentation-test-behavior-not-demographic-labels Published: 2026-08-20 Segmentation fails when a label looks useful in a deck but does not predict response to a product, price, or message. Two 35-year-old marketing directors in the same industry can have different constraints, risk tolerance, and buying behavior. Behavioral experimentation gives teams a better question: which action changes which outcome for which defined audience? Subconscious research answers that question directly. [figure: A five-step path: name the behavior to change, define two contrasting audiences by differences that matter, present both the same message, compare response, validate the causal question with human research.] ## Why demographics are not enough Age, income, geography, company size, and industry are easy to collect. They rarely explain a decision on their own: demographic variables are, in general, poor predictors of behavior and less-than-optimal bases for a segmentation strategy (Circana, "Demographic vs. Behavioral Segmentation: Which Offers More Marketing Precision?"). Psychographic surveys can add depth, but a survey of a thousand people cannot provide a deep follow-up conversation with every respondent. Synthetic audiences can help teams explore how context and attitudes may interact. They do not establish that a fixed persona represents every real person in a segment. ## How do you build segments around decisions? Start with the behavior the team needs to change, such as purchase, adoption, renewal, or response to a message. Then define the relevant audience differences and compare the same alternatives across them. A useful experiment might compare a concept across five audience definitions and inspect where the directional response differs. The study should preserve the same stimulus and outcome so differences can be interpreted. ## Practical uses ### Product strategy Compare feature concepts before development. Ask which alternative changes adoption intent and what assumptions need human validation. ### GTM and messaging Test the same value proposition across roles or contexts. Avoid claiming that one phrase “hooks” an entire demographic. Use the result to choose a stronger real-world test. ### Pricing scenarios Different audiences may respond differently to price points or packaging. Frame this as decision-specific scenario testing. Do not present it as automated price optimization or a universal measure of willingness to pay. [figure: A list of three decision areas, product strategy, GTM and messaging, and pricing, each paired with the specific behavior a team would test before acting on it.] ## What is a simple starting exercise for segmentation? Define your first two contrasting customer types using observed differences that matter to the decision, not stereotypes. Present the same message or concept to both. Compare the response, identify the causal question behind the difference, and validate that question with human research or behavioral data. Good segmentation does not sort people into clever names. It makes a decision testable. --- # AI Legal Triage: Prepare for Counsel Without Mistaking It for Advice URL: https://subconscious.ai/blog/use-cases/ai-legal-triage-prepare-counsel-without-mistaking-advice Published: 2026-08-20 Example attorney rates of $300-500 per hour can turn a simple question into a $1,000+ conversation, and lawyers' median pay reflects that hourly value.[^bls] AI can help organize facts and questions, but it is not an attorney, does not give legal advice, and cannot represent anyone. Subconscious is not a legal-advice product. Its role is testing product, pricing, messaging, and GTM actions, the kind of work covered in how Subconscious works. Legal analysis belongs with qualified counsel. [^bls]: Lawyers: Occupational Outlook Handbook, U.S. Bureau of Labor Statistics. [figure: A five-step path: write the facts and deadline; draft questions with AI from notes, not confidential material; check litigation or IP stakes; bring the list to counsel; counsel gives the judgment AI cannot.] ## What useful legal triage looks like Before a consultation, write down: - the decision and deadline; - the parties, jurisdiction, and relevant documents; - what has already happened; - the financial or operational exposure; - the questions only counsel can answer. An AI assistant may help explain general terms or turn notes into a question list. Do not give it confidential material unless its handling, retention, and access rules have been approved. ## Which legal questions require careful routing? Common topics include entity structure, contracts, intellectual property, and employment. Examples range from choosing an LLC or C-Corp with two co-founders to understanding the difference between 1099 contractors and W-2 employees. The facts and jurisdiction can change the answer, so an apparently routine question may still need a lawyer. A blog post from 2019 is not reliable evidence that current law is unchanged. Check the date, jurisdiction, and authority behind any legal information. [figure: Five-step path: state the question; check jurisdiction and facts; check litigation, IP, or employment stakes; check that any cited source is current, not an old blog post; then route to counsel or proceed.] ## How much do legal consultations cost? A small business fielding three to five legal questions in an average quarter, at $400 an hour with a one-hour minimum per call, faces a quarterly bill of $1,200-2,000 for basic consultations. These figures are not a current price quote or a reason to avoid counsel. The sound workflow is simple: organize the question, identify the right type of lawyer, and use the paid meeting for the facts and decisions that require professional judgment. A demo shows how that same discipline, separating the question worth testing from the question that needs a lawyer, applies to GTM decisions. If litigation, regulation, employment rights, intellectual property ownership, or a high-stakes contract is involved, get qualified legal advice. --- # Real Estate and Financial Services: Test Messaging Before It Ships URL: https://subconscious.ai/blog/industries/real-estate-financial-services-test-messaging-before-ships Published: 2026-08-20 Real estate and financial services share a research problem: high-stakes decisions, long buyer journeys, and hard-to-recruit audiences. Positioning, client letters, and feature messages often ship on instinct because pre-testing is slow. A controlled behavioral experiment against a defined buyer population is a faster check before the expensive step: a clinic, a compliance review, a campaign. [figure: Four-step decision path: define a narrow buyer segment, vary one commercial element like positioning or price, compare how the segment responds, then route the result to sales, compliance, or a live campaign.] ## Which decisions in these categories are suitable for a controlled experiment? The strongest candidates hold the buyer context stable and vary one commercial action: positioning, price, message, or script. This is concept testing: comparing alternatives against a defined audience, not asking which one the team likes (Qualtrics, "Concept Testing: Definition, Methodology & Examples"). ## Real estate: buyer segments and listing-level use cases Real estate messaging speaks to a specific buyer, neighborhood, and moment, so a launch experiment only works with a narrowly defined buyer population. One planning scenario used four segments: - First-time buyers, ages 30–38, household income €120K–€200K, currently renting, considering ownership in 12–24 months. - Move-up buyers, ages 40–55, household income €250K–€500K, moving from a starter home to a suburb or peri-urban residence. - Investment buyers, age 45+, accredited-investor profile, seeking cash-flow assets with light personal involvement. - Downsizer buyers, age 60+, equity-rich, considering a smaller residence with amenities. This is an illustrative planning scenario, not a Subconscious customer result. A matched panel pre-tested launch positioning, listing copy, imagery, and open-house language. It found three fixable issues: a "luxury living" framing that read generic to first-time buyers, an amenity list burying the most-cited feature (workspace flexibility), and open-house language that read as pressuring. The misses go on the record next to the hits, which is why this gets said plainly. The comparison catches findings like these but does not prove a launch outcome; treat any conversion result as unverified until sales data confirms it. Beyond launch positioning, individual listings raise their own comparisons: ## Financial services use cases Financial services messaging carries the same structural problem as real estate, plus a regulatory constraint real estate does not have. ### Wealth management and private banking A private bank's client base is usually too small and sensitive to A/B test directly, and HNW or UHNW prospects rarely respond to research outreach. Build a panel matched on geography, wealth source, and asset preferences, then test new positioning, offerings, and advisor messaging before a client sees it, to find where it reads as appropriate rather than generic or paternalistic. ### Retail banking and neobanks A neobank has a real user base and can pre-test in production through feature flags, staged rollouts, and in-app A/B, but regulated disclosure and pricing copy still carry the cost of learning only after launch. A matched panel can pre-flight feature naming, pricing-structure explanation, and onboarding copy before deployment. ### Insurance Insurance marketing has to carry trust signals and regulatory constraints together. A panel can compare framings of a new feature, such as a cyber-insurance rider, climate-resilience add-on, or parametric product, to find which explanation lands and which reads as jargon, ahead of compliance review. ### Investment products Investment marketing has to satisfy regulatory clarity and emotional resonance together; many drafts fail on dry, feature-list language. A panel matched to the target retail-investor profile can compare framings of the same regulatory information to find which respects the reader's intelligence without changing the substance a compliance reviewer checks. ### Mortgage and lending A mortgage message that works for a first-time buyer does not work for someone refinancing or drawing equity. A panel can compare messaging variants against buyers matched to each life stage, tailoring messages to moment rather than product line. ## A worked planning example: wealth-advisor communication refresh The following is an illustrative planning scenario, not a Subconscious customer result. A boutique wealth-advisory firm with 60 clients and assets in the mid-hundreds of millions wants to refresh its brand and client communications after five years without an update. It cannot research its own clients without turning research into a brand event, and cannot easily reach prospective first-generation HNW clients (entrepreneurs, founders, recently liquid) through ordinary recruiting. A matched panel of that profile (ages 35–55, mixed sophistication, a forming portfolio) supports a five-phase sequence. ## Limits and complements A number without its limits is marketing, and this method has real edges. A controlled experiment does not replace the relationship work that closes these deals; it sharpens the messaging that creates those conditions. It does not replace regulatory and legal review: every output is a draft, and compliance still has to clear it before it ships. Nor does it replace local-market or advisor expertise: a broker who knows the neighborhood, or an advisor who has built the client relationship, brings knowledge no panel can produce. Subconscious can test or validate studies with real human participants. When a decision needs it, a team moves from the simulated comparison to real-human validation without changing the underlying causal question: the same intervention, population, and outcome are checked against real behavior. ## Where does this fit in the research stack? ## Getting started Pick one upcoming decision with real cost, such as a listing launch, pitch refresh, or product update, and run a matched panel of the target audience first. Then scope a controlled experiment against a tightly defined buyer segment before it ships. Review the approved case studies for evidence that has passed the claim gate, and route anything customer-facing through compliance before it goes live. --- # Validating a Deprecation Announcement Before It Ships URL: https://subconscious.ai/blog/use-cases/validating-deprecation-announcement-before-ships Published: 2026-08-20 A product manager sunsetting a feature has one real question before it goes out: will the affected customers read this as a reasonable trade-off, or a reason to distrust the vendor. ## Why does the announcement, not the deprecation, do the damage? Users absorb a feature being removed. What they react badly to is how it is communicated: no warning, no migration path, a tone that reads as dismissive of the people who relied on it, or feedback that appears to have been ignored for years. The announcement is often finished the week it ships, reviewed by people who never used the feature, and sent without anyone outside the room testing how it lands. ## Why is this worth testing before it ships? Direct churn from a deprecation is the visible cost. The larger one is quieter: accounts that stay but now treat the vendor as one that pulls things out from under its customers. That label follows every renewal conversation afterward, felt first by support, then sales, then product marketing (LaunchNotes: How to Successfully Introduce and Announce Product Changes). A controlled experiment on the affected segment gives a team a read on that reaction before the announcement is public, while the draft can still be revised. ## How does the test work? The setup mirrors the audience at risk, not the whole customer base. If a report is being retired, the panel is built from the customers who use that report; if a configuration option is going away, it is the admins who touch it. The draft copy goes to that segment as written, with the questions an internal team is too close to ask honestly: What is the first reaction? What is missing? What would change that reaction from irritated to reassured? - Define the segment. Build the audience from job function, product usage depth, and tenure: the people the change touches, not a general population. - Share the draft as written. Use the real copy, not a cleaned-up version. - Ask for the reaction. First impression in one sentence, what feels missing, and whether the message changes how they see the product. - Test the migration path, if there is one. Confirm the segment understands what to do next and whether the timeline reads as reasonable or rushed. - Revise and re-test. Adjust the draft based on what the panel flagged, then run the revised version past a fresh sample to confirm the fix worked. None of this replaces a legal or compliance reviewer's decision about the announcement's language, or a support team's direct conversations with affected accounts. ## Three checks a draft should pass - Does it read as dismissive? A single line that minimizes why customers used the feature can undercut the message. - Is the next step clear? If a reader outside the drafting team cannot summarize what is happening and what they need to do in one sentence, the copy needs another pass. - Which direction does trust move? Ask directly whether the message increases, decreases, or leaves trust unchanged. Any announcement that moves it down needs rework before it ships. ## When the fix is not the wording Sometimes the result is not "revise the copy." It is "the feature this segment relies on is more load-bearing than the team assumed, and the deprecation needs to be reconsidered." That is a more valuable finding on a Monday than the same discovery from three escalating customers on a Friday (Product Teacher: How to Deprecate a Feature). [figure: Three-item checklist for a deprecation draft: does it read as dismissive, can a reader state the next step in one sentence, does the message move trust up, flat, or down. A downward move sends it back for revision.] ## Where this leaves the team A clear reaction test shortens arguments about which edits to keep. When an executive wants to keep a line that a broad, independent read of the affected segment flags as tone-deaf, that stops being a matter of opinion in the room. If your team has a deprecation planned, the next step is to see how the review runs, or read more about how a study like this gets structured before the draft goes out. [figure: Five-step horizontal path: define the affected segment, share the draft as written, ask for the reaction, test the migration path, revise and re-test with a fresh sample.] --- # Get VC-Level Feedback Without Knowing a Single VC URL: https://subconscious.ai/blog/use-cases/get-vc-level-feedback-without-knowing-single-vc Published: 2026-08-20 You have a startup idea, maybe traction, and no venture capitalist in your contacts. Cold emails to name-brand funds don't get replies. Founders with investor networks get feedback on decks, positioning, and ask before the partner meeting. Everyone else walks in blind, and a bad pitch burns a relationship that doesn't come back. The gap isn't access to opinions. It's access to evidence. A founder can guess which slide is weakest, or test the specific claim behind it against real audience data before the meeting, not during it. ## The claim that actually gets challenged Every pitch deck rests on a handful of claims: how big the market is, what customers will pay, which feature matters most, how the company should be positioned against competitors. Partners don't push back on the deck's design. They push back on these claims, because a check depends on them. An unproven claim in a partner meeting has two bad outcomes. Either it gets challenged on the spot and stalls momentum, or it goes unchallenged and the founder spends the next round of funding building toward a market that was never real. Testing the claim first avoids both. ## How do you test a claim instead of the room? Run a randomized experiment against the claim itself, instead of assembling opinions about how it will land. Subconscious runs controlled studies against a person-level audience graph covering 800 million real people. Point it at the specific claim in the deck, such as willingness to pay a given price or which feature matters most to a target buyer, and get back a causal estimate with a confidence interval instead of a founder's assertion. That's a different kind of evidence than a simulated reaction. A confidence interval tells you how much the estimate could be wrong. A guess doesn't. [figure: A five-step path: draft a market claim from the pitch deck, design a causal test for it, run the test against real audience data, validate the promising result with real people, then cite the result in the deck.] ## How do you move from a simulated test to real-human validation? A causal estimate from an audience graph is often enough to walk into a meeting with a defensible number instead of a hunch. When the stakes justify it, the same question moves to real-human validation without changing what was tested. Subconscious can test or validate studies with real human participants: start simulated to see whether a claim is worth pursuing, then validate the promising ones with real people before it goes in the deck. ## What doesn't this process replace? Publishing what a test cannot do is what lets a buyer check the number against the room. This process tests a market claim. It does not simulate an individual investor, predict how a specific partner will react to your pitch, replace a warm introduction, or negotiate a term sheet. Venture decisions involve judgment about team, timing, and fit that a causal test of a market claim was never built to answer. Due diligence checklists cover categories like team, market, and terms. Treat causal testing as evidence for one of those categories, market, not a stand-in for the whole process. A limit stated plainly is what makes the estimate usable. It also won't tell you why a specific fund passed. Founders who've collected the real reasons behind VC rejections report that vague feedback is common and the actual reason is often narrower than "not a fit." A tested market claim narrows what you have to guess about; it doesn't eliminate the judgment calls investors make about the rest of the deal. [figure: A branching diagram: an unproven claim splits into two failures, challenged live (stalls the meeting) or unchallenged (builds toward a false market). Both converge on one fix: test the claim before the meeting.] ## Which claim to test first Before the next pitch, pick the single claim most likely to get challenged and put a number behind it: - Demand size: how many target buyers actually have the problem you're solving - Price tolerance: what the target buyer will actually pay, not what you assumed - Feature priority: which capability the buyer values most when forced to trade off - Positioning: how the buyer ranks you against the alternative they'd otherwise choose Each of these can be run as a controlled causal test rather than argued from a founder's instinct. Test the specific claim that's weakest in the current deck, then bring the estimate and its confidence interval into the room. Learn how the tests work or see a walkthrough. --- # Persona Chat, Survey Research, or Causal Experiment? URL: https://subconscious.ai/blog/comparisons/persona-chat-survey-research-causal-experiment Published: 2026-08-20 A persona chat, a survey program, and a causal experiment answer different questions. Use a persona chat to explore what to ask, survey research to collect structured responses, and a causal experiment to measure the effect of a defined action. For a pricing, messaging, positioning, or launch decision, the right choice depends on whether the buyer needs ideas, reported preferences, or evidence that one option changes behavior. [figure: Three blocks, one per method, each labeled with the question it answers: persona chat explores themes, survey research collects structured responses, causal experiment measures which alternative changes the outcome.] ## What is at risk if you choose the wrong research method? The cost of choosing poorly is not an imperfect research artifact. It is the budget, positioning, or product decision made from evidence that cannot answer the underlying question. An open-ended persona conversation surfaces hypotheses, objections, and language worth investigating. It does not measure an effect against a defined population. A structured survey describes what respondents report under a specified questionnaire. It does not establish that the proposed action caused the outcome. Choosing one price, message, or launch option over another requires a design that compares those alternatives directly. ## Match the method to the buyer's question The three approaches are not substitutes. Each has a distinct unit of evidence. A discrete-choice experiment presents defined alternatives and analyzes observed choices to estimate how changes in their attributes affect preference. Statistical guidance treats experimental design, model selection, and interpretation as connected parts of that analysis. ISPOR's good research practices for discrete-choice experiments describes those method choices. ## Keep the causal question intact Subconscious is built for the third buyer question. The research approach uses controlled discrete-choice experiments to compare defined alternatives across a specified population and return a measured effect with a confidence interval. The practical advantage is decision alignment. A team testing a price does not receive an unstructured conversation about price; it receives evidence about the alternatives in the decision. The same applies to a message, positioning choice, or launch option. For a consequential call, Subconscious can test or validate a study with real human participants. That lets the team move from a simulated experiment to real-human validation without changing the causal question. The simulated result and the human result remain distinct evidence, and case studies show how that evidence is applied. ## What is Subconscious not designed to do? Stating where a method does not apply lets a buyer check it against their own question before committing budget. Subconscious is not an open-ended, ask-anything persona chat interface. It is also not a continuous managed survey-research subscription. It requires a defined decision and alternatives that can be compared. A validation result stated without its limits functions as marketing. Real-human validation does not turn a causal action test into an observed usability session, a clinical trial, or automatic proof of market performance. High-stakes decisions still require judgment about the population, study design, uncertainty, and external evidence. The validation leaderboard gives context for evaluating simulated research against human behavior. ## Use the procurement conversation to expose the difference Ask each provider to state what its output can support: - Can it name the population against which the result applies? - Does it compare the actual alternatives under consideration? - Does it return an effect estimate and confidence interval? - Can the causal question remain unchanged during real-human validation? - Which decisions fall outside the method? If the team cannot yet define the action or alternatives, begin with exploratory research. If it needs structured reported responses, design a survey. If it can name the decision and needs to estimate which option changes the outcome, a controlled causal experiment is the matching method. The experiment process shows what must be specified before that test begins. --- # Hierarchical Bayesian Models for Customer Lifetime Value Across Cohorts URL: https://subconscious.ai/blog/methods-and-validation/hierarchical-bayesian-models-customer-lifetime-value-across-cohorts Published: 2026-08-20 A marketing analytics team allocating next quarter's acquisition and retention budget across customer cohorts faces a modeling choice first: fit one customer lifetime value (CLV) model per cohort, fit a single global model, or let cohorts partially share information with each other. The choice matters most for small or newly acquired cohorts, where the wrong model turns sampling noise into a budget decision. ## Why cohort-by-cohort models break down Probabilistic transaction models such as BG/NBD (Beta-Geometric / Negative Binomial Distribution) estimate purchase frequency and customer dropout from transaction history; BG/NBD alone has no monetary component, so a full CLV figure requires pairing it with a spend model such as gamma-gamma and a discount rate. A common workaround for seasonal or cohort-level differences is to fit one BG/NBD model per acquisition month. This unpooled approach carries three costs: - Model proliferation. A company running in 10 markets with two years of monthly cohorts needs 240 separate models. - Cold start. A newly acquired cohort has too little transaction history to produce a stable estimate on its own. - Arbitrary boundaries. Treating a customer acquired May 31 as fundamentally different from one acquired June 1 is rarely justified by the underlying behavior. A single global model avoids all three problems, but it erases real differences between cohorts: a new, high-intent cohort gets the same parameters as an old, lapsed one. ## What is partial pooling in a hierarchical Bayesian model? A hierarchical Bayesian model treats each cohort's BG/NBD parameters as draws from a shared population-level distribution, rather than fitting each cohort in isolation or forcing every cohort to share one set of parameters. This partial pooling lets small cohorts borrow statistical strength from the population while keeping their own signal. The worked comparison behind this technique comes from Fader, Hardie, and Lee's original BG/NBD formulation, estimated by maximum likelihood over gamma and beta heterogeneity distributions (Marketing Science), and Abe's hierarchical Bayes extension of the related Pareto/NBD model, with customer-specific parameters drawn from a multivariate lognormal population distribution (Marketing Science). Both use the CDNOW dataset, transaction records from an online CD retailer, as the standard benchmark for this class of model. ## What the CDNOW example shows The CDNOW customers were split into four acquisition-cohort groups of uneven size: 1065, 815, 353, and 124 customers. The fourth group is a realistic stand-in for a small or newly acquired cohort. Fitting an independent BG/NBD model to each group produced four latent parameters, r, α (alpha), a, and b, governing purchase rate and dropout probability. For the two smallest groups, the a and b estimates showed high volatility and wide credible intervals: exactly the instability that makes an unpooled model risky to act on. Refitting the same four groups under a hierarchical structure, where group-level parameters are drawn from a shared global prior, narrowed the credible intervals for the small groups substantially, while their point estimates shrank partway toward the population mean, with the smallest groups shrinking the most and the largest groups barely moving. This "shrinkage" toward the population estimate is what a hierarchical model buys. ## Comparing the three approaches [figure: Three CLV modeling approaches compared: shrinkage pulls a small cohort's noisy estimate toward the global mean, unlike a fully pooled or fully unpooled model.] ## Where does this reasoning apply beyond CLV? The same logic applies whenever a team has to trust a per-segment estimate built on a small sample. Subconscious's causal experiments already report effects by segment; when a segment's sample is small, the defensible move is to pool the segment-level estimate toward the population-level effect rather than treat a noisy per-segment number as ground truth for a budget decision. See how segment-level results are structured in current research. ## What are the limitations of this approach? This is a modeling technique for observed transaction histories, not a description of a Subconscious product feature. No Subconscious benchmark or customer case exists for hierarchical BG/NBD specifically. The technique also assumes cohort membership is a meaningful grouping variable; if cohorts are defined arbitrarily, partial pooling narrows the noise but cannot fix a poorly chosen grouping. Extending this approach with time-varying global parameters for seasonality, covariates for acquisition channel or demographics, or additional levels of hierarchy for multi-market data is straightforward, but it adds inference complexity that should be weighed against the size of the decision it is informing. ## Next step Teams evaluating how segment-level estimates hold up under partial pooling can review how Subconscious structures segment-level causal results in case studies or read more on how we work. --- # How Research Leaders Govern Self-Serve Research URL: https://subconscious.ai/blog/implementation-and-procurement/how-research-leaders-govern-self-serve-research Published: 2026-08-20 Before stakeholders use AI to produce research output on their own, define what that output is allowed to decide. Exploratory answers can frame a question. Decisions about spend, public claims, pricing, positioning, or the roadmap require a causal experiment and, when the risk warrants it, a separate test with real human participants. ## Put the Decision Before the Tool Self-serve access changes who can produce an answer. It does not change what makes the answer fit for a decision. A stakeholder can summarize interviews, draft a customer narrative, or generate objections to a concept. They do not establish which action will change buyer behavior. The cost of a weak boundary appears when a plausible answer enters a decision memo as evidence. The business may commit budget, make a public claim, or change a roadmap against a hypothesis that was never tested. The research function then owns the consequences without having owned the method. The answer is a written governance rule. Name the decisions stakeholders may make from exploration alone. Name the decisions that require research review, a controlled experiment, and real-human validation. The research leader owns that boundary. This is not a case for slowing self-serve access. Demand for research judgment is not falling: the U.S. Bureau of Labor Statistics' 2024-to-2034 outlook for market research analysts and marketing specialists is still one of growth, even as AI takes over first-pass drafting and summarization. The role is shifting toward governing which answers are fit to act on, not producing more of them. ## What Is the Four-Layer Evidence System? A four-layer system separates question formation from evidence that can support action. Each layer has its own owner, decision right, and failure condition. The layers are cumulative. Review does not convert generated text into evidence. A simulated causal experiment does not become real-human validation because both address the same question. Each completed layer should be named in the decision record. [figure: A four-step path from stakeholder exploration to research review to a simulated causal experiment to real-human validation, with an owner and a failure condition attached to each step.] ## Match the Evidence Bar to the Cost of Error The right governance tier depends on what happens if the answer is wrong. ## What Decision Rights Should Researchers Have? The research team should design the self-serve system, not operate as a request queue. Its governance document should define: - which questions stakeholders may explore on their own; - which decisions require a named research reviewer; - what must be specified before an experiment begins; - when a simulated result is sufficient for a reversible internal choice; - when a separate real-human test is required; - how every output must state its method and limitation. The most important review question is not "Does this answer sound credible?" It is "Could this method identify which action caused the outcome we care about?" If the answer is no, the output remains a hypothesis. ## Roll Out One Decision Workflow Start with one live decision that crosses teams. Use the same sequence each time: - Write the business decision in one sentence. - Name the action, target population, alternatives, and outcome. - Classify the cost of error as reversible and internal or consequential and external. - Use self-serve output only to frame hypotheses and objections. - Have the research owner approve the evidence path. - Record the result with its method, limitation, and next validation step. As a planning example, repeat this review weekly for a month. The cadence is not a delivery promise. Its purpose is to expose unclear decision rights and turn the rule into normal operating practice. ## Where Does Subconscious Fit In? Subconscious is the causal AI company. Randomized experiments on a simulation of your market, validated against real human behavior, tell you why people choose and which action drives the outcome. Subconscious can test or validate studies with real human participants. The practical advantage is continuity: a team can move from a simulated experiment to real-human validation without changing the causal question. The validation remains a distinct study step. Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people. That reach supports a defined population for a controlled study. It is not a recruitable participant pool of 800 million people. See the research method and the study workflow for the distinction between a generated answer, a simulated causal experiment, and human validation. ## Limits of This Governance Model This framework is an organizational practice. Subconscious does not automatically enforce it inside a stakeholder's workflow. The research leader must write the rules, assign decision rights, and stop unsupported output from crossing the evidence boundary. Naming where simulation ends and human confirmation begins is what lets a buyer check the claim. Real-human validation is separate from simulated experimentation. A simulated result is not human-validated unless that step was explicitly run. The misses on this method sit next to its hits so a buyer can weigh both before acting. Neither a simulated experiment nor real-human validation automatically proves market performance. High-stakes financial, health, or policy decisions still require named expert review beyond any research tool. ## Buyer Questions Does self-serve access replace the research team? No. It changes the team's role from controlling access to governing questions, methods, and decision rights. Can exploratory output support a consequential decision? It can identify hypotheses. It cannot establish which action caused a behavioral outcome. Who owns the governance document? The research leader accountable for the evidence the organization acts on. Business owners remain accountable for the final decision. What is the next step? Choose one pending decision, assign its evidence tier, and document the boundary before the study begins. Discuss a validation workflow. --- # Bayesian A/B Testing at Scale: Why Millions of Observations Slow MCMC Down URL: https://subconscious.ai/blog/methods-and-validation/bayesian-b-testing-scale-why-millions-observations-slow-mcmc-down Published: 2026-08-20 A head of experimentation running an A/B test with millions of observations faces a real trade-off: full Bayesian inference swaps a single point estimate for three richer outputs, namely a Bayes factor, a Region of Practical Equivalence (ROPE) analysis, and a 95% credible interval, but a naive Markov chain Monte Carlo (MCMC) model becomes impractically slow once a test passes roughly a million rows. The team can re-architect the sampling pipeline to keep compute flat, or it can get the causal answer before the test ever needs a million live observations. ## Why does observation count break a standard MCMC model? At every MCMC step, the model evaluates the likelihood of each observation given the current parameters: 100 observations means 100 evaluations per step, 1,000,000 observations means 1,000,000, with sampling time scaling roughly linearly with observation count when the number of steps is held constant. Removing group indexing (splitting the data into separate vectors per group instead of indexing into a shared parameter array) makes the model run more than 3x faster at 1 million observations, yet the underlying scaling problem remains: both the indexed and non-indexed models keep slowing down as the dataset grows. ## How does the histogram approximation speed up MCMC? The fix is a planning-example technique from Bayesian statistics, not a Subconscious product feature: group all observations into a set number of bins (say, 500), then evaluate the likelihood only at each bin's center, weighted by how many observations fall in that bin. Rather than run 1 million likelihood evaluations at every MCMC step, the model now needs just 500 bin-center evaluations, a roughly 2,000x reduction in per-step likelihood evaluations at that observation count. That ratio describes the likelihood-evaluation count, not overall runtime: fixed per-step costs such as gradient evaluation, NUTS tree-building, and compilation dominate once the likelihood term is cheap, so measured runtime speedups are much smaller (75 seconds to 13 seconds, about 5.8x, at 500,000 observations). Because compute scales with bin count rather than observation count, a dataset of 1 million or 100 million observations requires the same number of evaluations per step. Accuracy improves with more bins as the approximation converges toward the continuous case, so bin count becomes the accuracy-versus-compute dial. In one documented run, a binned model built this way sampled 100 million observations in 22 seconds on a standard iMac, with total runtime including model compilation at roughly 30 seconds, versus a linear increase in runtime for the default (non-binned) model at that scale. Posterior uplift estimates from the binned model tracked closely with the default model across simulated true uplifts ranging from 0.1 to 1.0 at 500,000 total observations: the default model averaged 75 seconds per run at that volume, the binned model averaged 13 seconds, with no meaningful loss of inferential precision reported. A technique's reach and its edge belong in the same sentence, since that is what lets an engineer check it before relying on it. The technique extends to multi-group A/B/C/D tests but does not generalize to regression models with continuous predictor variables. ## What this means for the underlying decision A speed fix and a live-exposure problem are separate claims, and keeping them separate is what lets a team judge each on its own terms. Binning solves the "MCMC compute grows with observation count" problem for a team willing to build and maintain that engineering. It does not solve a different, earlier problem: even a binned sampler still means the test may need to run against millions of real users to resolve small effects, and until posterior probability or ROPE monitoring supports early stopping, a bad variant remains live, though ramped rollouts limit how much of that exposure is at full scale. Subconscious addresses that earlier problem instead of the sampling-speed problem. It runs controlled experiments on a simulated population and estimates causal effects with confidence intervals describing sampling variability within that simulation, before a change reaches a single real customer; external validity to the real-customer population is a separate, unquantified assumption. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market, per the causal fidelity paper. When a decision depends on it, a team can extend the same causal question into real-human validation on the same test without re-deriving the question from scratch. ## Limitations A capability claim stated without its boundary is marketing, so the boundary goes here. Subconscious does not implement or endorse the histogram-binning MCMC technique described above; that is a general Bayesian-statistics engineering pattern for scaling a live pipeline, not a Subconscious capability. And running a decision-quality simulated test first does not remove the need to monitor a live rollout at scale once the change ships. The two are complementary steps in the same pipeline, not substitutes for each other. [figure: Two-column comparison: discrete A/B/C/D groups map onto histogram bins and the approximation applies; continuous regression predictors have no fixed set of bin centers, so it does not generalize.] ## Further reading For teams that do need to scale live Bayesian MCMC pipelines, the binning approach traces back to established large-N approximation methods, including "Light and Widely Applicable MCMC: Approximate Bayesian Inference for Large Datasets" and "Informed Sub-Sampling MCMC: Approximate Bayesian Inference for Large Datasets". The open-source PyMC library remains a common tool for building these models. [figure: Naive MCMC slows as observations grow. Binning keeps compute flat but stays live at full scale. A simulated test branches off earlier, no real-user exposure. Both paths end in live monitoring.] See how this same class of causal question runs on a simulated population, before it ever needs live-test scale, on Subconscious's research page or the leaderboard of tested effects. To scope a specific test, book time with the team. --- # GPU Sampling vs. CPU Sampling for Bayesian MCMC: When Is the Switch Worth It? URL: https://subconscious.ai/blog/comparisons/gpu-sampling-vs-cpu-sampling-bayesian-mcmc-when-switch-worth Published: 2026-08-20 A data science team running MCMC-based causal inference in PyMC or Stan eventually hits the same question: move sampling to GPU, or stay on CPU? Getting it wrong either way costs. Accelerating a model that never needed it burns engineering hours nobody gets back. Staying on a CPU sampler past the point where GPU sampling would help stalls model iteration and delays whatever decision depends on the result. ## What did the benchmark actually measure? One published benchmark tested this tradeoff directly, fitting a hierarchical Bradley-Terry model, the standard pairwise-comparison model used for ranking competitors from win/loss records, to 160,420 professional tennis matches. It compared standard PyMC, PyMC with a JAX backend on CPU, PyMC with JAX on GPU, and Stan, at dataset sizes ranging from a single recent year up to the full match history. These are the original benchmark's figures, kept here as a planning example from that specific setup rather than a claim about current PyMC, Stan, or JAX performance: The fastest GPU configuration was at least 4x faster in wall time than the fastest CPU-only method. The benchmark also reported effective-sample-size-per-second gains up to roughly 11x for the fastest GPU configuration versus CPU-only PyMC and Stan, meaning the GPU run produced more usable independent samples per second, not just a shorter runtime. Even without a GPU, running PyMC with a JAX backend on CPU alone delivered a roughly 2 to 3x effective-sample-size-per-second improvement over standard PyMC and Stan. [figure: Decision path: below roughly 50,000 observations, CPU sampling wins in this benchmark; above that, GPU sampling wins. Ends in reproducing the comparison with the reader's own model and hardware.] ## Where does the CPU-to-GPU crossover point sit? The benchmark's runtime curves were close to flat for GPU configurations across dataset sizes, while CPU runtime grew with data volume. That produced a crossover point: below roughly 50,000 observations, GPU sampling ran behind CPU because of fixed per-run overhead; above it, GPU sampling won. For a team deciding where to spend engineering effort, that crossover point matters more than the headline 2.7-minute runtime, because it tells you whether your own dataset is even in the range where the investment pays off. ## What doesn't generalize from this benchmark? The setup that produced these numbers was narrow, which matters for anyone using the crossover point as a planning guide: - One model class. The benchmark used a single relatively simple hierarchical model. More complex models, including ones with additional covariates or dense covariance structures, may see a different crossover point or a different relative ordering of methods. - One consumer GPU. Testing ran on a single RTX 2070, not a data-center GPU or a multi-GPU cluster. Results on different hardware, or at larger scale, were not measured here. - Dated software versions. The benchmark used JAX v0.2.13 and CUDA 10.1. Current JAX and CUDA releases can produce materially different timings for the same model and hardware. - Unvalidated precision tradeoff. The benchmark noted that switching from double to single precision could theoretically yield up to a further 32x speedup on GPU, based on the consumer GPU's rated double- versus single-precision throughput, but flagged that numerical stability for that switch had not been validated. Treat that figure as an open question, not a confirmed gain. ## Before committing to GPU sampling Check where a candidate dataset falls relative to the roughly 50,000-observation range where this benchmark found GPU sampling starts to win, and treat that boundary as a reference for a similar model class, not a universal threshold. Then reproduce the comparison with the intended model, data, precision, chain configuration, and hardware before choosing a backend. For teams building causal research into decision infrastructure, the relevant question is whether the sampling design supports the decision without changing it. This historical benchmark does not establish current platform performance or imply that Subconscious uses the same stack. --- # How to Test a Pricing Tier Structure Before You Launch URL: https://subconscious.ai/blog/buyers/how-test-pricing-tier-structure-before-launch Published: 2026-08-20 A pre-launch founder must choose a pricing tier structure before committing engineering and sales motion to it, usually without enough buyer conversations to triangulate the choice. A structure that buries the sticky feature in the wrong tier or confuses buyers with too many tiers suppresses conversion and expansion for months. Re-packaging a pricing page after launch is far harder than getting it right the first time, since existing customers resist any change to a price they have already accepted. [figure: Diagram shows tier count, feature allocation, and packaging converging into one path to a structure comparison against a defined segment, while price points follow a separate path to a price-sensitivity method.] ## Separate the decisions before testing any of them A pricing page bundles decisions that call for different evidence: - Tier count. Two tiers, three tiers, or four tiers. - Feature allocation. Which features sit in which tier. - Packaging language. What each tier is called and how its value is described. - Price points. The actual dollar or euro amounts. Tier count, feature allocation, and packaging are relative-preference questions: which candidate structure a defined buyer segment prefers. Price point is a separate, absolute-value question. Mixing all four into one vague pricing question produces confused signals and a structure shipped on guesswork. ## Test structure and feature allocation against a defined segment Define the buyer segment first. Pricing research is sensitive to who answers it: the same structure can win with one segment and lose with another. Be explicit about job title and seniority, company size and stage, geography, current tooling and budget context, and buying authority. A vague segment returns consensus answers that do not predict anything. Draft candidate structures with the same feature surface. Hold the total feature surface constant across candidates so the comparison tests structure, not scope: Compare the structures against the segment. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, showing the candidate structures side by side with their feature lists and price points, to see which the defined segment prefers. Look for a structure that wins clearly across the segment, or a split where one sub-segment prefers a different structure than another. A split is itself useful: it can mean the launch needs two pricing pages, or a decision about which segment to prioritize first. Probe the winning structure. Test the feature allocation inside the leading structure directly: what happens if a feature is removed from a tier, moved up a tier, or added to the tier below it. This surfaces upgrade triggers and deal-breakers an internal pricing discussion tends to miss. When warranted, the same comparison can move from a simulated experiment to real-human validation without changing the underlying question. ## What does this pricing comparison not decide? Two questions sit outside it. The first is absolute willingness to pay. A structure comparison can show one price point beats another for a given tier, but not whether a given number is profit-optimal. For the absolute price point, run a dedicated price-sensitivity method such as the Van Westendorp Price Sensitivity Meter, which asks buyers directly about the price at which a product feels too inexpensive, too expensive, or appropriately priced. The second is enterprise contract-stage negotiation. Once a deal reaches enterprise B2B territory, procurement workflows, multi-year discount schedules, and negotiation specific to that account take over, none of which a structure-and-packaging comparison can simulate. For enterprise-tier pricing, use the comparison to settle structure and packaging, then handle deal mechanics with the actual buyer. ## What should happen before the pricing page ships? Testing tier count, feature allocation, and packaging against a defined buyer segment before the page goes live, and reserving a dedicated price-sensitivity method for the dollar amounts, gives a founder a structure they can defend on launch day rather than one shipped on guesswork. See the current case studies for how teams have used repeated comparisons to shape pricing, messaging, and positioning ahead of a launch, or start a study directly on a pricing decision. --- # How to Build a Customer Persona Worth Testing Against URL: https://subconscious.ai/blog/implementation-and-procurement/how-build-customer-persona-worth-testing-against Published: 2026-08-20 A customer persona worth testing against encodes role and context, behavioral history, core beliefs, decision patterns, and constraints, the internal logic that actually drives a buyer's decision. Most customer personas are useless for a real decision. "Marie, 34, marketing manager, enjoys yoga" describes a demographic, not a buyer. It cannot tell you whether Marie would pick your price over a competitor's, or which message changes her mind. A persona built to describe someone is a narrative. A persona built to predict a decision needs five inputs: role and context, behavioral history, core beliefs, decision patterns, and constraints. Publishing what a persona can't do is what lets a buyer check it before relying on it. Building those well is necessary groundwork, not a measured answer to a message, price, or feature question. ## What separates a usable persona from a narrative one A low-fidelity persona lists traits: - Five years into a marketing manager role - Employer is a mid-sized business - ROI is the main concern - Leans on data before deciding anything A usable one captures how the person decides: - Runs demand gen for a 150-employee B2B SaaS company, with three people reporting to her - Inherited a marketing stack where the ABM tool was never set up correctly - A prior vendor promised "AI-driven insights," then shipped dashboards nobody bothered to open - Evaluated on pipeline contribution, not MQL volume, since a change six months ago she is still adapting to - Gets tool recommendations from a peer community, then runs a two-week trial before purchasing gets involved Only the second version produces a distinct answer to a research question, because it encodes the internal logic that drives a decision instead of surface demographics. ## The five inputs that make a persona specific Beliefs are the strongest predictor of reaction to a new idea. Someone who believes formal research is theater needs a different pitch than someone who values methodological rigor. Constraints matter just as much: a persona that ignores a hard budget ceiling will hand back unrealistically positive feedback, while a persona with real constraints will tell you where the offer doesn't fit. ## Where do the inputs for a persona come from? The strongest personas are built from data a team already has, not imagination: - Sales call recordings. Hear the precise words customers reach for, what pushes back, and what they still want answered. Listen to five to ten recordings per target segment. - Support tickets. The problems customers hit, in their own words. This feeds behavioral history and constraints. - CRM notes. Decision-making dynamics, stakeholder involvement, and the objections that killed past deals. This feeds decision patterns. - Customer interviews. Verbatim quotes are the most reliable source for beliefs and communication style. - Product analytics. Usage patterns separate power users, occasional users, and churned customers into distinct behavioral profiles. ## The mistakes that flatten a persona back into a stereotype - Describing demographics instead of psychology. How old someone is, their gender, or their job title won't predict what they do. Beliefs, constraints, and decision patterns do. - Making the persona too agreeable. Real buyers carry skepticism, budget limits, and bad prior experiences. Leave the friction in. - Using one persona for a market with distinct segments. Different buying behavior means separate persona definitions, not one blended average. - Letting the persona go stale. Markets and roles change; revisit definitions on a regular cadence rather than writing them once. - Dropping the "burned before" factor. Most B2B buyers have been let down by a previous vendor, and a persona that ignores that experience reads as naive. ## Where does a well-built persona still fall short? A description of buyer thinking means little on its own without its limits stated next to it. A rich persona is a well-informed narrative: a description of how a segment of buyers is likely to think, not a measurement of how that segment responds to a specific message, price, or feature alternative. Mining sales calls, support tickets, and CRM notes answers "who are we talking to and how do they think." It does not answer "which of these two prices costs us fewer buyers." That second question needs a controlled experiment: define the alternatives, hold everything else constant, and measure which one a precisely defined segment chooses. Discrete choice experiments are the standard method for that kind of comparison, and research on how they're built treats qualitative persona-style input as the hypothesis-generation stage that precedes the experiment, not a substitute for it. Subconscious runs that controlled comparison across the same segment definition a team already built, and returns a causal effect with a confidence interval rather than a single plausible-sounding read. The cost of skipping that step shows up late: weeks spent mining calls and notes can produce a persona that reads convincingly and still turns out wrong once the decision ships and behavior contradicts it. ## Method boundaries worth keeping straight Naming where a method stops is what lets a buyer check the claim against reality. First, audience reach in a simulated experiment is not the same as the number of real people recruited for validation. Second, Subconscious can test or validate studies with real human participants, which lets a team move from a simulated experiment to real-human testing without changing the underlying causal question. That step doesn't turn a causal choice test into an open-ended chat with a persona, a usability session, or a clinical trial. It stays a controlled comparison of defined alternatives. ## A practical next step Build the five inputs from real customer data before testing anything. Then decide what the decision actually requires: if it only needs a plausible read on how a segment thinks, the persona is the deliverable. If it needs to know which price, message, or feature a segment will actually choose, take the same segment definition into a controlled experiment and check the result against comparable case work before committing. [figure: A chain: a demographic-only persona feeds five inputs (role, history, beliefs, patterns, constraints), forming a usable persona. That persona is still a narrative, so the chain ends at a discrete choice experiment.] --- # Target Group Simulation: Testing Campaign Concepts Before Production Spend URL: https://subconscious.ai/blog/use-cases/target-group-simulation-testing-campaign-concepts-before-production-spend Published: 2026-08-20 A brand or marketing leader deciding whether to run a campaign concept through a structured pre-launch test, or go straight to production and media spend, is weighing testing first against finding out only after launch that the creative never landed with the intended audience. ## The decision: test the concept, or commit the budget Before a campaign goes into production, someone has to choose among headline, offer, and positioning variants. That choice often gets made on instinct, a stakeholder's preference, or a quick internal read, then the team commits media and production budget to whichever variant won the room. Untested creative that misses its audience is a recognized source of wasted ad spend, which is why pre-launch testing exists: to catch a weak concept before it reaches media (Segwise: Creative Effectiveness: How to Score Ads in 2026). The alternative is to compare variants in a controlled test before committing spend, then carry only the surviving variant into production (Swayable: Creative Pre-Testing: Perfect Campaigns Before Launch). ## Where do informal testing methods break down? Two workflows commonly stand in for a real pre-launch test, and each has a distinct failure mode: The internal opinion round. A handful of colleagues, or the loudest voice in the room, pick a favorite headline. This is fast, but it tells the team what people inside the building think, not how the target audience will respond. The unmoderated focus group. A room of strangers discusses the concept, a moderator steers the conversation, and someone writes up "themes" afterward. The result is one person's interpretation of what a handful of participants said, shaped by group dynamics and whoever spoke first and loudest. Neither method holds a variant to the same question twice, compares variants against each other under identical conditions, or shows where the audience's response splits by segment. A structured comparison does all three. ## A structured way to compare campaign variants The underlying design is a controlled comparison, not a group discussion: - Define the audience. Name the segments the campaign needs to reach, described by role, context, and situation rather than only demographics. Three to five distinct segments is often enough to surface meaningful divergence without diluting the test. - Define the variants. Headline, offer, or positioning alternatives the team is genuinely choosing between. Testing fifteen campaign variants against a defined audience is not out of reach when the constraint is no longer recruiting a new room of people for each one. - Run the comparison. Expose each segment to each variant under matched conditions and record which one changes intent or preference. As a check on reliability, ask the same underlying question in twenty different ways and see whether the answer holds. - Read where segments diverge. A variant that works for one segment can fall flat with another; that split is often the most useful finding, and it is the one an internal opinion round, or a single unmoderated focus group of around eight participants, cannot reliably surface. - Carry the surviving variant into production, with the losing variants set aside rather than built. ## Where does Subconscious fit, and where does it stop? Subconscious runs this comparison as a randomized experiment against a simulation of the target audience, validated against real human behavior, so the team sees which tested variant changes intent or preference, and for which segment. Where the decision is consequential enough to justify it, the same study can move from a simulated read to a real-human validation of the same question without changing what is being tested. Stating what a tool cannot do is what lets a buyer check its scope before they rely on it. Subconscious does not replace an executive's or creative director's judgment call on which concept fits the brand, and it does not replace real human research entirely. Its audience graph is a testable population built for comparing actions, not a recruited panel standing in for a market-research firm's field study. A simulated read is the first pass on a campaign decision that carries real budget risk, not the last word. ## Limitations A method that only reports its wins is marketing. Here is what this one misses, published next to what it catches. A simulated comparison shows which tested variant is more likely to move the stated outcome for the defined audience. It does not forecast in-market sales, media performance, or channel-specific delivery. Treat the result as a filter that narrows the field before the team commits production and media budget, not as a guarantee of the surviving variant's real-world performance. [figure: Five-step horizontal path: define audience segments, define variants, run the comparison, read where segments diverge, carry the surviving variant into production; losing variants set aside.] [figure: Two-column comparison. Left, "decides this": which tested variant changes intent or preference, and for which segment. Right, "not this": in-market sales, media/channel performance, and the brand leader's judgment call.] ## Next step Name the audience segments and the variants under real consideration, then bring the comparison to a Subconscious working session before committing production and media spend. The case studies and research program describe how Subconscious validates simulated results against real human behavior. --- # First Hire Decisions: How Founders Should Test Role, Framing, and Pay URL: https://subconscious.ai/blog/buyers/first-hire-decisions-how-founders-should-test-role-framing-pay Published: 2026-08-20 A founder's first hire is a six-figure decision made with no HR team, no internal precedent, and no prior hiring experience. Get the role wrong and the company loses months of runway it cannot recover, along with the culture that first hire sets before anyone else joins. [figure: List of three variables a founder can test with candidates before hiring: role-priority ordering, job-description framing, and compensation structure, plus what testing does not replace.] ## Why the first hire is a decision problem, not a staffing problem Founders frame this as "who do I hire," when the real question is "which role, framed which way, at what pay, changes the outcome I need." Four patterns repeat across early hiring failures: - Role confusion. Instead of building the job around this quarter's actual mix of responsibilities, the founder hires against a title that only matches their own assumptions. - Culture projection. A solo founder hires people who think like them instead of people who cover a real gap. - Compensation guesswork. Pay is set too high, burning runway, or too low, losing the candidates worth hiring. - Job-description mismatch. A post written as an aspirational role description attracts the wrong applicant pool and repels the right one. Replacing an early hire costs more than that hire's salary once lost productivity, the search, and onboarding are counted; a bad-hire cost analysis built on U.S. Department of Labor data puts the real cost meaningfully above the base pay itself (Apollo Technical, checked 2026-07-27). A company with one employee cannot absorb that mistake and try again next quarter. ## What does a controlled comparison add that judgment alone can't? Reasoning does not reveal how a defined candidate population responds to a specific framing, priority list, or pay structure. A controlled experiment holds a candidate population constant, varies one thing at a time, and measures which version changes stated interest or choice. That is different from a handful of people reacting in an open-ended conversation. It answers "which version works better with this population," not "what does this one respondent think." ## Where does Subconscious fit in testing the framing before it goes live? Subconscious runs this kind of comparison as a randomized experiment on a simulation of the relevant population, then can test or validate the study with real human participants when the decision warrants it. For a first-hire decision, that means comparing: - Two or three role-priority orderings, to see which produces the strongest signal that the right candidates would apply. - Two job-description framings for the same role, to see which one changes stated interest most among a defined candidate profile. - Two compensation-and-equity structures, to see which one a target candidate profile finds credible enough to engage with. The output is not a persuasive-sounding critique. It is a measured comparison: which tested option produced the stronger response under the study conditions, for the population the founder needs to reach. The Subconscious research program explains how human baselines and replication support that method, and how Subconscious structures a working engagement covers what a founder needs to define before running one. ## What does this testing not replace? A controlled comparison does not conduct reference checks, run live interviews, execute onboarding, or substitute for the founder's own read on culture fit. A simulated comparison and a study run with recruited real human participants answer different questions; a founder should not treat one as proof of the other. The comparison narrows which framing, priority, or pay structure deserves the founder's limited time. The founder still owns the interview, the reference call, and the final decision. [figure: Two columns: simulated test on a modeled population shows which option scores stronger; human study shows how real candidates respond. Shared row: neither replaces the interview.] ## A founder's next step If the decision is which role to hire first, how to frame it, or what to pay, that is a narrow enough question to test before an offer goes out. Review how similar decisions have been tested, then bring the specific role, candidates, and pay structure to a working session. --- # AI Survey Tools Compared: Real Respondents, Live Sessions, or a Causal Experiment URL: https://subconscious.ai/blog/comparisons/ai-survey-tools-compared-real-respondents-live-sessions-causal-experiment Published: 2026-08-20 An insights lead choosing a research method has three real paths: a real-respondent survey, an AI-moderated live session, or a causal experiment. The wrong pick wastes budget on an answer to the wrong question. A survey reports what a sampled population says it prefers; only a controlled experiment shows which action moved the outcome. ## The three paths, side by side ## Real-respondent and live-session tools: historical landscape examples The figures below reflect vendor materials at the time this comparison was assembled. Treat them as historical planning examples, not verified current pricing or capability claims. - Attest: real respondents, self-serve fielding. - Qualtrics: enterprise survey platform, real respondents, days-long fielding, self-serve within an enterprise contract. Qualtrics documents its own survey software capabilities directly. - Quantilope: automated MaxDiff, conjoint, and TURF methodology on real respondents, days-long fielding, self-serve. - Listen Labs: real respondents interviewed by an AI moderator at scale, mixed self-serve, days-long fielding. - Remesh: live, large-group real-respondent sessions with AI clustering, hours-long sessions, self-serve. - Perspective AI: survey-shaped synthetic-respondent output, hours-long turnaround, self-serve, not benchmarked against real-human data in the materials reviewed. - OpinioAI: synthetic surveys and focus groups starting at $99 per month, hours-long turnaround, self-serve, not benchmarked against real-human data in the materials reviewed. - Evidenza: managed synthetic B2B buyer simulation, enterprise-only, days-long engagement, methodology-led validation story. - Aaru: agent-based behavioral simulation reporting roughly 90 percent correlation against EY-validated benchmarks, enterprise-only, days-long engagement. None of these figures are Subconscious claims. They describe tools a buyer might otherwise be evaluating against a causal experiment. ## What does a causal experiment answer? A causal experiment answers a different question than any row above: not what a population says, but which action moves the outcome. Subconscious can run controlled studies against a person-level audience graph of 800 million real people, then validate a study with real human participants on that same causal question. The design carries over; only the respondent source changes. Real-respondent surveys and live sessions remain the right choice when a report must show real-human provenance as the primary deliverable, or when the question is descriptive rather than causal. ## What doesn't a causal experiment replace? A causal experiment does not replace a fielded, real-human panel, and Subconscious does not publish a public price or accuracy leaderboard ranking every vendor above; the landscape figures come from vendor-published material, not an independent benchmark Subconscious runs. ## Where to start Buyers weighing this decision can review the research methodology behind the causal-experiment path, or check the leaderboard for how replication and validation results are reported. [figure: Three paths: survey settles what a population reports today, live session settles nuance behind a stated preference, causal experiment settles which action drives the outcome, validated against humans on request.] --- # Simulated AI Studies vs. Quantilope: Choosing a Research Path Before a Pricing or Claims Decision URL: https://subconscious.ai/blog/comparisons/simulated-ai-studies-vs-quantilope-choosing-research-path-before-pricing Published: 2026-08-20 An insights or growth leader choosing a pricing, claims, or segmentation study faces two paths: field a study to real respondents with an automated quant platform, or run a fast simulated study first and validate the result with real people before the decision ships. The right path depends on how much the decision costs to get wrong and how much time the team has before the window closes. ## The decision and the cost of choosing wrong Shipping a pricing or claims decision off a directional AI-simulated study alone risks an unvalidated say-do gap: what a simulated study suggests and what real people do can diverge, and a same-day chat output does not prove otherwise. Over-relying on a slower, fully fielded quantitative study risks missing the window when a launch date, budget cycle, or competitive response won't wait. ## What does an automated real-respondent platform deliver? Quantilope runs automated quantitative research against real respondents, using pre-built method templates such as MaxDiff, conjoint, TURF, segmentation, and brand tracking, with AI applied to the analyst layer to compress setup and reporting (quantilope). That combination is methodologically defensible: the sample is real, the methods are established survey-econometrics techniques, and the output is bound by how long it takes to field the respondents. This path fits an enterprise insights team that needs statistically defensible quantitative output, has budget for sample, and has time in the calendar for a study to field. ## Where does a fast simulated pass help, and where does it stop? A same-day simulated study run on AI-modeled respondents can compress the front end of research: teams can test a pricing question, a claim, or a concept against a modeled population before committing budget to a fielded study. The output is directional. It has not been checked against how real people respond to that question, and a chat-style summary of simulated conversations is not evidence a real market will behave the same way. A simulated pass is useful for narrowing options fast. It is not proof that the option it favors is the one real customers will choose. ## Where does Subconscious fit? Subconscious runs randomized, controlled experiments on a simulation of the buyer's market, comparing specific pricing, claims, or positioning actions and estimating which one is more likely to move the outcome, with confidence intervals where the study design supports them. Rather than stopping at a directional simulated result, a team can take the same causal question to real-human validation without redesigning the study. Subconscious's audience graph covers a person-level population of 800 million real people, which is a graph the platform can draw a study population from, not a recruitable panel of respondents waiting to answer a survey. That two-step path answers what an insights leader is actually asking: not whether the simulated read looks plausible, but whether the causal comparison holds once real people answer it. ## How the two research paths compare ## Limitations and failure conditions Named support for MaxDiff, conjoint, TURF, or brand-tracking study designs on the Subconscious platform needs confirmation before a buyer assumes parity with a dedicated quant platform. Subconscious does not currently publish an automated recommendation engine, a stated turnaround-time guarantee, or a per-respondent cost figure, and none of those should be assumed. That audience graph is a population Subconscious can draw a study from, not a standing panel of recruited respondents. ## What to do next If the deliverable has to stand on its own as statistically defensible real-respondent output and the calendar allows for fielding, an automated quant platform such as Quantilope is the direct fit. If the decision needs a fast first comparison of pricing, claims, or positioning options with the ability to check the result against real people before it ships, see how Subconscious approaches a study or book time to talk through the specific decision. Teams that want to see the method before committing can also review current replication results or the underlying research. [figure: Two paths to a decision. Left: an automated real-respondent study runs directly to the decision. Right: a fast simulated pass runs first, then real-human validation, then the same decision.] --- # AI Concept Testing Tools and Platforms in 2026 URL: https://subconscious.ai/blog/use-cases/ai-concept-testing-tools-platforms-2026 Published: 2026-08-20 Concept testing asks whether a target audience will care before a team ships a product, campaign, package, feature, price, or position. A recruited study can take 3 to 4 weeks, cost $15k to $80k, and compare six concepts at most. AI-assisted testing can shorten early screening, but the final decision still needs a validation plan. The 2026 market includes tools for synthetic panels, multi-agent simulation, qualitative interviews, and networked stakeholder research. Compare them on audience depth, supported formats, experimental control, validation, and pricing. [figure: Five-step path: define the decision and format; branch to interview or controlled comparison; screen fast with AI; check validation evidence rather than one accuracy number; route high-stakes launches to human research.] ## Neutral alternatives ### What does Electric Twin do? Electric Twin builds synthetic crowds for large consumer studies at enterprise scale and has reported $14M in funding. ### What does Aaru do? Aaru uses multi-agent simulation to model behavioral dynamics such as social proof and peer effects. Published positioning has cited around 90 percent correlation to real research and EY validation, with Fortune 500 teams as the stated fit. Treat those as vendor claims that require direct review. ### What does Evidenza do? Evidenza focuses on B2B audiences and cites customers including BlackRock, Microsoft, and JP Morgan. It is relevant when professional audiences are difficult to recruit. ### Synthetic Users Synthetic Users focuses on qualitative product and user-experience research for early concepts and prototypes. ### OpinioAI OpinioAI offers synthetic focus groups for first-pass reactions, with plans described as starting at $99 per month. ### Lakmoos Lakmoos uses a neuro-symbolic approach and emphasizes an audit trail. It is aimed at regulated categories such as automotive, finance, and energy. ### Societies.io Societies.io models how concepts land across connected stakeholder groups. It fits policy, public affairs, and B2B decisions with several constituencies. ### Sanctum Sanctum centers on feature-level testing before a product team exposes a change to real users. ### Experial Experial offers digital twins with real-time data integration and is positioned for German teams seeking a local provider. ## Choose the method before the vendor Start with the behavior and decision. Text, images, decks, landing pages, video, and prototypes require different evaluation methods. A qualitative interview can explain confusion. A controlled comparison can estimate which concept changes an outcome. Audience definition matters as much as the interface. The model should incorporate approved information about motivations, constraints, objections, and decision criteria. It should not be tuned to produce a preferred result. Validation claims need context. A range such as 80 to 95 percent accuracy against historical research benchmarks is not meaningful without the studies, outcomes, and comparison method. Independent research on LLM-simulated survey responses has found they can diverge from real survey data in effect magnitude even when they reproduce the direction of an effect (Political Analysis, Cambridge University Press). Do not treat that range as a universal property of AI concept testing. ## AI-assisted testing versus recruited research AI-assisted screening runs same-day, and synthetic panels support repeated comparisons on demand. That speed does not make the methods interchangeable: human research remains the reference for a high-stakes go/no-go launch. AI is strongest for early screening and iterative message, positioning, and creative tests where recruitment time blocks learning. [figure: Two columns: AI-assisted uses (early screening, iterative message tests, repeated comparisons) versus what stays with recruited human research (the final high-stakes launch decision).] ## A reliable evaluation checklist Check whether the platform: - Supports the actual concept format. - Defines and calibrates the target audience. - Separates qualitative explanation from causal comparison. - Records the alternatives, outcome, and assumptions. - Exposes validation evidence rather than a single accuracy number. - Lets a team repeat the study consistently. - Protects approved customer context. - Makes clear where human validation is still required. Subconscious is relevant when the concept decision can be framed as a controlled experiment on a defined audience. It should not be presented as automatically supporting every concept-test format or as replacing the final human-grounded decision. --- # Winning RFPs: Pre-Test Your Angle Before You Submit URL: https://subconscious.ai/blog/use-cases/winning-rfps-pre-test-angle-before-submit Published: 2026-08-20 Agencies lose RFPs they should have won for one reason more often than any other: they aimed the pitch at the wrong person inside the client organization. Procurement officers, brand directors, CMOs, and sometimes a CFO each take a pass through a typical RFP response before it wins or dies, and each can reject it on their own terms. Most teams write toward a single favorite reader and leave the remaining three to chance, only learning which gate mattered once the outcome is final. ## The decision that actually costs money The pitch team debates two or three strategic angles, drafts a narrative, prices the engagement, and locks all three before the client committee ever sees the document. That lock-in happens under deadline, on instinct, after the loaded hours are spent. If the angle was wrong for the procurement reader, or the pricing framing spooked the CFO, the team finds out only when the rejection email arrives, and by then the cost is sunk. The fix is not writing a better deck. It is testing the angle, the narrative, and the price against the specific readers who will kill or advance the response, before the team commits to one version of each. ## Four readers, four different kill conditions A response tuned for the CMO's point of view can still lose because the procurement lead couldn't defend the pricing internally. That's a reader-modeling problem, not a creative one, and it's testable in advance once the team defines these readers as distinct buyer segments. [figure: Four-step sequence: an RFP response tested against reader segments before submission, in order: strategic angle, drafted narrative, likely objections, pricing framing.] ## What's the test sequence before an RFP response locks? Test each decision against the defined reader segments before it becomes final, rather than debating internally and picking one version. - Test the angle first. Compare the two or three contenders the team is debating against the procurement, brand, CMO, and CFO segments. Read which one shifts stated preference and for which reader, not which one the internal team likes best. - Test the narrative once it's drafted. Run the draft response against the same segments and ask a pointed question of each: what's the top concern from a procurement read, what's compelling from a brand-director read, what earns a meeting from a CMO read. - Test the objections before submission. Ask each segment what would make them reject the response, and what a competing agency is likely to be stronger on. This produces a prioritized list of rebuttals the team can address in the document instead of finding out about them after the loss. - Test the pricing framing separately. A fixed fee, a retainer, and an outcome-based structure read differently to a procurement lead than to a CFO. Compare the framings against those two segments specifically, rather than defaulting to whatever the account lead drafted in isolation. The point of running these as four separate comparisons, rather than one holistic gut check, is that each removes a different untested assumption. ## How does Subconscious run this as a controlled comparison? This reframes the RFP problem as a controlled experiment rather than a review meeting: define the buyer segments (procurement-type economic buyer, marketing decision-maker, and so on), hold everything else fixed, and change one variable at a time (the angle, the narrative, or the pricing framing) to see which version shifts stated preference for which segment. Subconscious runs these comparisons against a person-level audience graph covering 800 million real people, which is a controlled-study audience, not a recruitable panel of people who agreed to participate. When a result needs stronger evidence than the initial comparison, the same causal question can be tested or validated with real human participants without changing what's being compared. For grounding on how directional results relate to human-validated replication: our best configuration reaches 87% of the measured human ceiling on one study, 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. Naming exactly where a fidelity score stops applying is what lets a buyer weigh it correctly for their own market. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. Independent research on the broader RFP funnel, not this comparison method, also points at the same upstream problem: agencies that are more disciplined about which RFPs they chase, and how they position within them, tend to see better returns than agencies that simply write more responses (Loopio, 2026). ## What doesn't this method prove? Publishing where a method stops applying is what lets a buyer check it against their own case. This method does not simulate a specific named procurement officer's judgment or identity. It compares stated preference across defined buyer segments instead, a more limited claim. It does not produce a public win-rate benchmark or a named case study showing measured lift for a specific agency; treat any efficiency or win-rate improvement as something to measure in your own pipeline, not a number this method guarantees. And the audience graph used for the comparison is not a recruited panel: real-human validation means running the study with actual participants, not just querying the graph again. ## Before the next RFP goes out If an RFP is already on the calendar, the sequence matters more than the tooling: define the four buyer segments for the client's category, test the angle before the team commits to one, and run the objection test before the document is final. Review how this fits alongside other buyer-decision testing on how Subconscious works and research, or see the method applied to other conditions in case studies. To scope a comparison for a specific RFP, book time with the team. --- # Simulating Data with PyMC URL: https://subconscious.ai/blog/methods-and-validation/simulating-data-pymc Published: 2026-08-20 A simulated output is only as trustworthy as the process that generated it. Before a data science or analytics leader commits budget to a decision built on a simulated experiment, the underlying data-generating process needs to be specified explicitly, fit to real data, and checked against that data. PyMC is a probabilistic programming library built for exactly that workflow, and its simulation tools show the discipline in miniature. ## Draws from a specified distribution PyMC models a highly structured data-generating process, useful even outside formal Bayesian inference: for simulation in optimization routines, risk analysis, and research design. A distribution is defined with explicit parameters, then sampled with draw: ### The same distribution, reparametrized The same Gamma distribution can be defined with a mean/standard-deviation parametrization instead of shape/rate: PyMC converts between equivalent parametrizations automatically, so the choice is about what's easiest to reason about, not a modeling compromise. ## Are PyMC distributions vectorized by default? Not every scientific-computing distribution allows array-style broadcasting across its parameters. Every distribution in the PyMC distributions API does. A two-row Dirichlet call in SciPy raises an error: SciPy raises: parameter vector a must be one dimensional, but its shape is (2, 3). ### Passing the array directly PyMC distributions are vectorized, so the same call runs directly: Each row draws its own three-way split that sums to 1: the first row (weights skewed toward 100) lands near [0.0017, 0.0396, 0.9586], and the second (weights mirrored) lands near [0.9500, 0.0418, 0.0082]. ## Meta-distributions: truncation and mixtures Real-world quantities are often bounded, or drawn from more than one underlying process. PyMC's Truncated class can bound any univariate distribution: ### How do mixture distributions combine components? Mixture distributions combine multiple components with weights. Combining two Normals with weights [0.3, 0.7] draws roughly 30% of samples from the first component and 70% from the second: ### Mixtures as a starting state The same mixture machinery composes with random walks, letting a mixture serve as the initial state for a multi-step process: ## Dependent variables in one draw Simulated variables are often not independent of each other. This example samples a categorical index, then uses it to select from a vector of three Normals with means [-100, 0, 100]: The five category draws come back as [2, 0, 1, 1, 0], and the matching selected values land near each drawn category's mean: around 99.6, -97.9, -1.9, -0.5, and -101.1. ### One draw sizing another A Poisson draw can also determine the shape of a downstream draw: how many Gamma-distributed events to sum. Across the three draws, the event counts come back [4, 10, 5], and the matching event-size sums come back roughly [36.4, 100.0, 57.8]: more Poisson events push the summed Gamma total higher. ## When the process itself is unknown, infer it The examples above start from a distribution the analyst already chose. A harder and more common case: a rough sense of what the data looks like, without a clear specification for how to simulate it. That is the setup behind fitting a model to real data, where the goal is realistic covariates whose marginals match an observed dataset. Because PyMC also performs inference, the same framework can recover the parameters of an assumed structure from real data. Given price data across five diamond cuts (53,940 rows), a reasonable guess is a mixture of three LogNormal distributions per cut: 5 × 3 = 15 means, 15 standard deviations, and 15 mixture weights. ### A MAP point estimate Fitting with find_MAP returns one local-optimum point estimate of weights, means, and standard deviations for each cut, with no uncertainty and no guarantee that component 1 means the same thing across cuts or runs: The dictionary returns one row of three means, three mixture weights, and three standard deviations per diamond cut. For the first cut alone, fit["mix_means"][0] is [6.66074039, 7.75610166, 9.05369536], fit["mix_weights"][0] is [0.35500674, 0.43518759, 0.20980567], and fit["mix_stds"][0] is [0.29090793, 0.58885309, 0.42603866]. The remaining four cuts return their own three-component fits in the same structure: five cuts times three components, each with its own recovered mean, weight, and standard deviation. One local-optimum MAP point estimate for the first diamond cut, from fit["mix_means"][0], fit["mix_weights"][0], and fit["mix_stds"][0] above. ### Validating the fit against real data The next step is drawing from the model conditioned on the fitted MAP values and checking whether the simulated output resembles the real data it was fit to: Two checks make that comparison concrete: overlaying simulated and observed histograms per category, and overlaying their empirical CDFs. The marginal simulated histograms resembled the original dataset's, and the two ECDF lines tracked closely enough to trust the fit. When they don't line up, the answer is to revisit the model: more mixture components, or a different distribution family. Don't ship the simulated output anyway. [figure: Four stages of a generative process: one named distribution, the same call vectorized across rows, meta-distributions that truncate or mix components, and a draw where one variable sets another's shape or value.] ## Performance notes for draw-heavy workflows draw compiles a function, seeds it, and calls it in a Python loop. That's convenient for exploration, but costly in a loop, because it recompiles the same function on every call. For performance-critical code, compiling the underlying random function once with compile_pymc and reusing it avoids that cost: (array(-0.10568235), array(-0.6541993)) ### Vectorizing the compiled call Defining the distribution with its final shape and calling the compiled function once, instead of looping, lets the random number generation vectorize: array([-0.10568235, -0.6541993 ]) ## Why does the fit-then-check step matter for a decision? Everything above is general-purpose PyMC, not a benchmark or case study. But the discipline it demonstrates generalizes past any one tool: specify a structured process, fit it to real marginal data, then compare the simulated draws back against the observed distribution before treating the result as trustworthy. Matching marginals like this establishes distributional realism for simulation inputs; it does not by itself validate the joint dependence structure or any causal effect estimated on the simulated population. The same standard causal experimentation applies to a simulated market's causal estimates, but those require their own separate validation before informing a launch, price, or message decision. A simulated experiment that was never checked against a real distribution is a model artifact, not evidence. Ship the wrong price or message on it, and the mistake surfaces only after the spend is committed. The full PyMC notebook and API documentation cover the mechanics in more depth. The discipline that carries over is simpler than the code: fit the parameters, then validate against the observed data before the output drives a decision. --- # AI Panel Research vs. Surveys: Use Each for the Right Question URL: https://subconscious.ai/blog/comparisons/ai-panel-research-vs-surveys-use-each-right-question Published: 2026-08-20 Surveys measure responses from people in a defined sample. Simulated panels explore how modeled audiences may respond to a controlled choice or open-ended prompt. [figure: A four-step horizontal path: simulate to map the question and generate hypotheses, refine the survey with those findings, field the survey with real respondents, then quantify only what the sample supports.] ## When should you use a survey? Use a survey when the decision requires quantitative breadth, a known answer space, and evidence from real respondents. It can estimate preference, awareness, or stated intent when the sample and instrument support the inference. Surveys answer only the questions written in advance, and low response rates can threaten the sample. Most B2B survey programs land under the 10% mark (Survicate, 2025 Survey Response Rate Benchmarks). ## When should you use a simulated panel? Use a simulated panel during discovery, when the team needs to compare early alternatives, surface objections, or improve the next human study. How we work covers where that discovery step fits before a study fields. An example panel compares 5-8 personas in a session that can take an hour rather than weeks. Simulated panels do not create statistical significance. Fidelity depends on elicitation, calibration, audience definition, and the task. They cannot replace observed behavior or primary research required by an external stakeholder. ## A three-step hybrid First, run a simulated experiment to map the question and generate hypotheses. A planning session can take a few hours. Second, use the findings to refine the survey. Remove ambiguous options, add missing answers, and define the behavior the survey should measure. Third, field the survey with real respondents and quantify only what the sample and method support. ## Planning examples for cost and time These planning ranges are examples, not current quotes or guarantees. One illustrative comparison puts a simulated panel's payoff at roughly 80% of a full study's insight for about 5% of its time and budget. Use simulation to decide what to test, drawing on Subconscious.ai's research methodology for the simulated stage. Use real respondents and behavioral data when the decision requires human proof. --- # AI Customer Conversations vs. Controlled Experiments: What Each One Can Prove URL: https://subconscious.ai/blog/comparisons/ai-customer-conversations-vs-controlled-experiments-what-each-one-prove Published: 2026-08-20 Before a launch, a pricing change, or a new message ships, most teams face the same choice: talk to a chatbot that stays in character as a customer, or run a controlled test that measures what a defined population would actually choose. The two produce different kinds of evidence, and only one of them gives you a number you can defend in a launch review. ## The appeal of an open-ended AI conversation An AI-driven conversation lets you type questions to a chat interface configured as a customer and get answers back in real time. You can ask follow-ups, probe an unexpected answer, and get a transcript that reads like a real interview. That immediacy is why teams reach for it: two weeks out from a launch with no time to recruit interviewees, testing several positioning directions before committing research budget to one, rehearsing an interview guide before running it with real customers, or getting oriented on a buyer segment the team has never sold to. None of that is wrong as a way to move faster. For example, a team that might realistically fit 5 to 10 real interviews into a research cycle could run 20 or 50 such conversations, across as many as 10 different customer types, in the same window. That ratio describes conversation throughput, not a validated or current Subconscious benchmark. The problem shows up when the output of any single one of those conversations gets treated as measured signal rather than a fluent guess. ## What a fluent transcript cannot tell you A chat-style AI conversation that stays in character produces answers that sound plausible and internally consistent. Consistency is not the same as accuracy. Independent research on using large language models for choice modeling has documented variance collapse, where the model's answers cluster more tightly than real populations do; demographic flattening, where distinct customer segments produce suspiciously similar answers; and high sensitivity to how a question is phrased (Can large language models assist choice modelling?, arXiv). A separate study on eliciting purchase intent from language models finds it can approximate human-level responses only under specific elicitation methods, not by default (LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings, arXiv). "Our findings reveal that proprietary LLMs can generate valid and behaviourally sound utility specifications, particularly when guided by structured prompts (CoT)." Sfeir and colleagues, Journal of Choice Modelling (source) None of that makes the transcript useless. It means a single open-ended conversation, run once, with no comparison group and no confidence interval, is a hypothesis, not a validated finding. A team that ships a launch, price, or message decision on that hypothesis alone is trading the cost of waiting for real signal against the cost of being wrong in a way nothing in the conversation flags. ## A different question: comparison, not conversation The underlying need in most of the scenarios above is not "have a conversation" but "find out which of these options a defined population would actually choose." That is a comparison question, and it calls for a controlled, randomized experiment across a defined population rather than a single scripted character. Subconscious.ai tests product, pricing, and messaging actions through causal experimentation and discrete-choice-style modeling: instead of asking one configured character what it thinks, the study randomizes an action across many simulated respondents and measures the resulting choice. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That figure describes the audience graph's reach, not a pool of people available to be recruited and interviewed; a study still needs a defined, relevant population drawn from that graph rather than an undifferentiated sample of it. ## Where a conversation with a real person is still the right tool Neither a scripted AI conversation nor a randomized experiment replaces talking to an individual customer. Real interviews remain the right method when the goal is specific, idiosyncratic detail from one person, a genuinely novel reaction that a model has no basis to predict, or the final validation step before a high-stakes decision ships. The instinct to run a fast, exploratory step before committing real interview time is sound; the difference is what that step should measure. A conversation that stays in character surfaces questions worth asking. A controlled experiment gives you a comparison to act on. ## Moving from a measured comparison to human validation When the decision is high enough stakes to warrant it, a study built as a controlled comparison can be validated against real human participants without redesigning the study. That matters because it preserves the one thing an open-ended conversation cannot offer on its own: a way to check the answer. A team that starts with a randomized comparison and, when the stakes justify it, confirms the result with real respondents gets both speed and a defensible number, rather than having to choose between them. [figure: Two columns compare a single AI conversation, judged by plausibility and used for fast orientation, against a randomized experiment across a defined population, used before committing launch budget.] For a launch, pricing, or messaging decision, start by defining the comparison you actually need answered, then decide whether a quick conversation or a controlled study is the right way to get it. Teams that want to see how a study moves from a simulated comparison to real-human validation can review how the process works or look at published studies. --- # What Is a Synthetic Persona? URL: https://subconscious.ai/blog/use-cases/what-synthetic-persona Published: 2026-08-20 A synthetic persona is an AI chatbot configured to respond as a specific type of person: a customer segment, a buyer role, an expert, or a stakeholder. You describe the demographics, context, and attitudes; the model stays in that voice across the conversation and gives one plausible answer to your question. It is not a controlled experiment, and it does not carry a confidence interval. ## What you're actually talking to A persona configuration bundles five inputs: demographics (age, location, income, profession), psychographics (values, attitudes, personality), role context (job, industry, decision authority), behavioral traits (buying patterns, frustrations, goals), and a communication style. Once set, the model answers in character: reacting to a price increase, objecting to a pitch, or reading a headline. That differs from a static research persona document, written once and referenced in a slide deck. A persona chat is interactive: ask it a question, get a response, then follow up. ## Why do teams use a synthetic persona before a decision? Marketing, product, and sales teams use persona chat the same way: to generate hypotheses before a launch, a price change, or a message goes live. Typical uses: probing early product concepts, drafting and testing campaign copy, rehearsing objection handling before a sales call, and sharpening a positioning idea before it reaches a deck. The value of the exercise is exploration. A single conversation surfaces objections and framing a team hadn't considered, without recruiting a focus group. ## Where does a single persona answer break down? The failure mode is treating that one conversation, from a single model run against a single configuration, as validated evidence. Research on large-language-model role-play finds models flatten demographic and psychographic variation into a narrower, average-sounding voice than the population represented, a pattern called persona collapse or homogenization (Investigating Persona Collapse and Homogenization in Large Language Models). The same work flags variance collapse: repeated runs of the same configuration converge on similar answers instead of reproducing the spread of opinion a real population would show (Population-Aligned Persona Generation for LLM-based Social Simulation). A persona also inherits whatever assumptions its creator built in, one reason researchers push for more transparency in how personas get specified (Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency). None of that makes the exercise useless. It means one chat session answers "what might this buyer say," not "what will this segment do." [figure: A four-step decision path: configure a persona, ask it a question, treat the single answer as a hypothesis, then run a controlled experiment to test it.] ## Turning an answer into a hypothesis worth testing The practical fix is sequencing, not avoidance. Use persona chat for what it's good at: exploring angles, objections, and framing before committing to a direction. Route anything a launch, price, or message decision depends on into a controlled discrete-choice experiment run against a defined population, so the answer carries quantified uncertainty instead of one model's best guess. That is where a causal behavioral platform fits: not a replacement for the exploratory chat, but the step that turns a hypothesis into a defensible decision. Subconscious runs controlled studies against a person-level audience graph covering 800 million real people, and can test or validate studies with human participants to move a finding from simulation to human confirmation without changing the underlying causal question. [figure: Two-row comparison: repeated runs of one persona configuration cluster near a single point, labeled variance collapse; a real population answering the same question spreads across a wider range of positions.] ## What doesn't real-human validation fix? Real-human validation doesn't turn a causal choice experiment into a usability session or a guarantee of market performance, and it isn't a step every question needs; reserve it for costly wrong answers. A persona chat can gesture at what a segment might say, but only a designed experiment with a defined population can put a number and an uncertainty range on what that segment will choose. Compare methods before you commit budget to one on the leaderboard. --- # AI Mind Clone Platforms in 2026: An Evaluation Framework URL: https://subconscious.ai/blog/use-cases/ai-mind-clone-platforms-2026-evaluation-framework Published: 2026-08-20 AI mind clone platforms cover named-expert replicas, customer personas, synthetic respondents, audience twins, and consumer characters, and choosing among them depends on what grounds the representation, whether state persists, and what kind of test the platform actually supports. “AI mind clone” is used in 2026 for several different products: named-expert replicas, customer personas, synthetic respondents, audience twins, and consumer characters. The label hides important differences in evidence, persistence, interaction, and validation. Subconscious should not be described as a persistent clone of a real person. It is a causal behavioral platform for decision-specific experiments on simulated buyers and target audiences. [figure: A four-step decision path: ask what data grounds the representation, ask whether state persists or resets, identify chat vs panel vs controlled experiment, then choose the matching method.] ## Three questions to ask ### What grounds the representation? A demographic prompt is not enough. Ask which data defines the audience, how assumptions are documented, and whether the representation is calibrated against relevant human evidence. Some vendors claim their profiles draw on roughly 100 times as much public-web material as a generic LLM would use, or cite 80 to 95 percent accuracy against a benchmark they do not name. Numbers like that need a named method and validation scope before they can guide procurement. ### Does state persist? Some tools preserve conversation history and update a representation over time. Others generate a fresh response for each prompt. Persistence may matter for coaching or entertainment, but it can also introduce drift and make controlled comparison harder. ### What can the team test? Direct conversation supports exploratory interviews. Panels support comparison across representations. Decision-specific experiments require controlled alternatives, an outcome, and a method for estimating how an action changes response, the discipline documented in the causal fidelity paper and behind Subconscious's research. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. ## Common platform types Named-expert tools such as Delphi focus on querying a replica built from a person’s public work. Character.ai focuses on entertainment and roleplay. Synthetic Users focuses on product research. BuyerTwin focuses on B2B sales practice. Electric Twin uses continuously refreshed audience-twin language. Aaru emphasizes population-scale simulation. A self-serve chat interface is not equivalent to statistical population simulation, and neither is automatically a causal experiment. [figure: A list of four failure modes in AI representation tools: demographic flattening, over-rationality, variance collapse, and prompt sensitivity. Each can produce a plausible answer that fails under a new decision.] ## Evaluate proof before features A market hypothesis for the next 12 to 18 months: continuous refresh, validation transparency, and native integration will become more important. It is not a product guarantee. Ask every vendor to define the unit being simulated, the human baseline, the task used for validation, and the known failure modes. Aggregate agreement is easier than individual fidelity. Demographic flattening, over-rationality, variance collapse, and prompt sensitivity can all produce plausible answers that fail under a new decision. Choose the method that matches the decision. For Subconscious, the relevant question is not whether an AI “clones a mind.” It is whether a controlled experiment can compare actions for a defined audience without overstating what the model knows, the standard behind Subconscious's case studies. --- # When to Test a Social Listening Signal Before You Act on It URL: https://subconscious.ai/blog/implementation-and-procurement/when-test-social-listening-signal-before-act Published: 2026-08-20 A spike in negative sentiment, a competitor campaign, or an emerging complaint theme tells a brand or insights lead that something changed. Before a team spends budget on a campaign, a product change, or a public statement built on a listening signal, the proposed response needs to be tested against a defined audience first. Detection tools show what people are already saying in public; they cannot show how people will react to something that has not shipped yet, because the people posting never agreed to be asked. ## The cost of guessing after detection The buyer who owns this decision, usually a brand or insights lead or a CMO, has to choose between two paths once a listening tool flags a signal: act directly on the finding, or test the proposed response before committing resources. Acting directly is faster but carries a specific risk: the people who post publicly about a brand are self-selected and vocal, not a validated sample of the broader customer base. A response built on their reaction alone can miss how the actual target audience responds. The cost of being wrong is the budget spent shipping a response that was never checked against the people it needs to convince. ## Why isn't a public comment a validated response? What a person says in a public post is a stated reaction, not a measured choice. Research on consumer behavior distinguishes stated preference from revealed preference, the pattern of choices people actually make when a decision has real stakes, and treats the two as different data types that do not reliably predict each other (Wikipedia: Revealed preference). A sentiment spike is a starting point for a decision, not proof of how a defined audience will respond to a specific fix. ## What monitoring tools and controlled experiments each do A monitoring tool tells a team a conversation exists. A controlled test tells the team which of the responses under consideration is more likely to move the outcome, for the audience that matters, before the team commits. ## A four-step process for testing before you act - Define the decision. Before opening a monitoring tool, state which business decision the signal needs to inform: a messaging change, a product fix, a positioning shift, or a public response. A listening effort without a named decision produces alert volume, not direction. - Set up detection queries. Configure monitoring across three areas: brand and competitor mentions, category and industry conversation, and specific customer-experience complaints. - Cluster and prioritize themes. Group raw mentions into distinct themes and rank them by how directly they touch the decision from step one. A theme tied to a core segment or a real product gap outranks isolated complaints. - Test the proposed response before it ships. Once a theme points to a specific fix, message, or concept, that proposed response, not the raw signal, is what needs testing against a defined audience before the team spends. ## How does Subconscious test a proposed response? Detection tools stop at step 3. Subconscious is built for step 4: running a controlled test that compares a proposed response, message, or concept against alternatives for a defined audience, and reporting the comparison as a directional estimate rather than a guaranteed outcome. This is testing a specific action a team is considering, not an automated system that decides what to do on its own. When the decision is high-stakes enough that the answer needs to hold up under real human scrutiny, a team can also validate the same study with real human participants. That step matters when the proposed response involves a regulated claim, a final price, or a public statement the brand cannot walk back. ## What does this not replace? A controlled test of a proposed response is not a substitute for the ongoing detection and monitoring work that surfaces the signal in the first place. Nor does it replace recruited human research for regulated claims or final pricing. And it cannot predict an entirely novel real-world event that has no analog in existing data. It can, however, tell a team which of several proposed responses to a known signal is more likely to work before that team spends the budget to find out the hard way. [figure: Four-step path: define the decision the signal informs, configure monitoring, cluster mentions into ranked themes, then test the proposed response against a defined audience before it ships.] ## Next step A social listening program that stops at detection leaves the highest-cost decision, what to actually do about the signal, untested. Pair the research methodology behind controlled response testing with a demo to see how a specific proposed message or concept gets tested against a defined audience before it ships. --- # 6 Aaru Alternatives for Synthetic Research and Causal Testing URL: https://subconscious.ai/blog/comparisons/aaru-alternatives-synthetic-research-causal-testing Published: 2026-08-20 Aaru sits at the enterprise end of behavioral simulation: large population models, implementations that run weeks to months, and contracts sized for Fortune 500 buyers (SiliconANGLE, 2026 covers the same wave of enterprise synthetic-data tooling entering research stacks). That profile fits population-scale simulation, not every product, pricing, or marketing decision a team needs answered this quarter. The question is which category of tool matches the decision, not which tool is biggest: a lightweight directional read, a research team's existing methodology carried over, an enterprise population simulation, or a causal experiment that returns a quantified, defensible effect. Committing to a heavy enterprise engagement for a question a lighter tool could answer wastes budget. Trusting an informal directional read for a pricing or launch decision with real financial exposure wastes the decision. ## What actually distinguishes these tools [figure: Branching path of five approaches ordered by rigor: directional read, sped-up methodology, population simulation, causal experiment, optional real-human validation, routed by stakes and turnaround.] Four questions separate the tools more reliably than their marketing pages do. Decision and method. A tool for simulated interviews answers a different question than a platform that compares product, pricing, messaging, or GTM actions against each other. Level of analysis. Some platforms model an entire population. Most commercial questions concern a defined buyer segment and a finite set of actions. Self-serve or specialist-run. Some products are operated by a vendor's research team; others are self-serve. A regulated or population-scale program may need specialists; a routine pricing or messaging test may not need to start with an implementation project. Procurement posture. GDPR, data residency, and SOC 2 status matter once a purchase moves past a pilot. Verify each requirement against a vendor's current, published evidence rather than inferring compliance from category or headquarters. ## The six alternatives ### 1. Subconscious Subconscious is built for teams that need to test actions before committing capital, not to talk to a persona. It runs controlled discrete-choice experiments on a simulated population and returns causal effects with confidence intervals, backed by a person-level audience graph covering 800 million real people. When a decision's stakes justify it, the same causal question can move to real-human validation without changing what is being measured. Subconscious is not a substitute for full population-scale simulation, an automated price optimizer, or a generic persona-chat product. Use uncertainty language only when the configured study supports it. ### 2. What does Highlight do? Highlight focuses on consumer research, with particular relevance to CPG workflows. It may fit teams that already run quantitative studies and want to add a synthetic-respondent layer. ### 3. What does Synthetic Users do? Synthetic Users focuses on simulated participants for product and UX interviews. It may fit teams that need a narrow interview workflow rather than population-level modeling or causal comparison (Synthetic Users describes itself as an AI user-research platform). ### 4. What does SYMAR do? SYMAR follows familiar market-research formats, such as surveys, focus groups, and structured interviews, with generated respondents standing in for participants. It may fit professional researchers who want to preserve an existing methodology while speeding up fieldwork. ### 5. Ditto Ditto offers a structured workflow for simulated consumer research. It may fit a smaller insights team that wants guided studies without standing up a large enterprise program. ### 6. Qualtrics Edge Qualtrics has added synthetic-data and AI-assisted research capabilities to its broader experience-management system (SiliconANGLE, 2026). Existing Qualtrics customers may value keeping surveys, feedback, and synthetic work inside one environment rather than adding a separate vendor. ## At-a-glance comparison ## A practical selection test Ask three questions before evaluating a specific vendor: - Does the decision need an answer this week or this quarter? - Is the required evidence persona-level, segment-level, or population-level? - Who will operate the work: a specialist research team, or the functional team making the decision? Run the same bounded question through the finalists and compare the supported method, the level of analysis, and who has to operate it. Pick the fit, not the vendor with the broadest category claim. For a decision that needs a defensible, quantified answer rather than a directional read, see how Subconscious structures a causal experiment or compare it against a specific alternative. Teams that want the option to move from a simulated result to a real-human check can review that path before starting a test. --- # What Are Synthetic Consumers? Knowing When to Trust the Answer URL: https://subconscious.ai/blog/methods-and-validation/what-synthetic-consumers-knowing-when-trust-answer Published: 2026-08-20 Synthetic consumers are AI personas built from a language model with persona conditioning, and trust in their answers hinges on question type: reasoning and preference questions hold up, sensory or emotional and novel-category questions need a human baseline. A synthetic consumer is an AI persona: a large language model given demographic, psychographic, and behavioral inputs as conditioning, then queried as a member of a target audience. Ask it whether a headline lands, which of three concepts it prefers, or how it would describe a brand to a friend, and it answers in character. The harder question isn't what the persona is. It's whether the answer it just gave is solid enough to move money against: a media buy, a packaging change, a launch date. ## What decides whether a synthetic consumer's answer can be trusted? Not every question a synthetic consumer answers deserves the same confidence: the question type decides it, not the platform or the persona's polish. A synthetic consumer stacks three components: a frontier language model for general reasoning, persona conditioning on demographics and psychographics, and calibration against real prior data from the same audience, such as panel data or prior survey waves. That third layer separates a research-grade persona from a model improvising a character. Even a well-calibrated persona reasons from patterns in its training data and conditioning, not from a lived nervous system or a real purchase history. That gap shows up predictably by question type: Where the reasoning travels. Stated-preference questions ("which of these three framings do you prefer, and why?"), brand-perception attitude, and message resonance hold up well, because they reward the multi-option reasoning a calibrated language model is built to do. Segment comparisons are the exception: persona conditioning tends to compress within-group variance and distort between-group contrasts, which matters for methods like mixed logit that model that heterogeneity directly, so route segment comparisons to a real-human baseline. Where it doesn't. Sensory and emotional response (the feel of a packaging design or the music cue in a video ad), genuinely novel product categories with no analog in the model's training distribution, and invented autobiographical detail ("tell me about the moment you switched providers last year") all produce fluent, confident answers with no real signal behind them. The practical rule: route reasoning and preference questions to a synthetic comparison, and route sensory or novel-category questions to a real-human baseline before committing budget. ## Why isn't a vendor's own accuracy figure the number to anchor on? Vendors in this category frequently cite a high correlation between synthetic and real-consumer answers on directional questions. That figure comes from each vendor's own validation work: its own personas, calibration data, and definition of a directional match. It measures the vendor's model of the world, not any specific decision a buyer is about to make. The open research question isn't whether language models can approximate human survey responses in aggregate. It's under which conditions that approximation holds, and how a buyer would know before spending money whether their question falls inside or outside the calibrated range. Argyle et al.'s foundational work simulating human samples with language models frames this directly: the technique can reproduce population-level patterns on certain questions, but the calibration and validation boundary is still being mapped, not settled.[^1] "We show that the "algorithmic bias" within one such tool -- the GPT-3 language model -- is instead both fine-grained and demographically correlated, meaning that proper conditioning will cause it to accurately emulate response distributions from a wide variety of human subgroups." Argyle, Busby, Fulda, Gubler, Rytting, and Wingate, "Out of One, Many," arXiv:2209.06899 (source) [^1]: Argyle, L. et al., "Out of One, Many," the paper that first generated simulated survey samples from a language model, appeared in the Cambridge journal Political Analysis; see arXiv. ## Where does Subconscious fit in this decision? Subconscious's fit here is causal experimentation, not another persona chat interface: it runs controlled comparisons between actions on a simulated population, so the output is which action moves the outcome in that simulated population, not what one character says it would do; whether that effect transports to real human behavior is a separate, unresolved question. Subconscious can test or validate studies with real human participants, which matters exactly when a question crosses from stated preference into sensory, emotional, or genuinely novel territory. A team can move from a simulated comparison to a real-human check without changing the causal question it's asking; the decision doesn't restart from zero, it extends the same test. ## What this doesn't fix Real-human validation doesn't turn a causal comparison into a usability session, a clinical trial, or automatic proof of market performance. It answers the same causal question with a human sample, not a broader one. No amount of calibration substitutes for real-human research on questions that depend on sensory or emotional response, or on a category the model has never encountered. [figure: Four steps: synthetic comparison runs; question involves sensory, emotional, or novel content; test extends to a real-human check; the causal question stays fixed while the sample changes to human.] ## The next question to ask Before treating a synthetic answer as grounds for spend, name the question type first. If it's reasoning or preference, a synthetic comparison is a reasonable basis for the next step. If it's sensory, emotional, or a category nobody has data on yet, that comparison is a starting point for a real-human validation, not a substitute. [figure: Two columns. Left, "Synthetic holds up": stated preference, brand perception. Right, "Needs a human baseline": sensory/emotional response, novel category, invented autobiography.] --- # Cross-Cultural Market Research Before an International Launch URL: https://subconscious.ai/blog/use-cases/cross-cultural-market-research-before-international-launch Published: 2026-08-20 The decision that matters before an international launch is not whether to research each market. It is which pricing, packaging, or positioning variant to run in each target country, decided before per-market launch spend, inventory, and campaign budget are committed. Get it wrong and the money is spent on a message or price point a local segment does not respond to, and the miss surfaces only once the commitment can't be recalled. [figure: Three columns for US, Japan, Brazil, each naming its buyer segment and the one factor that segment weighs most when deciding, showing one positioning statement landing differently in each.] ## Why One Market's Playbook Rarely Travels Traditional cross-cultural research means a separate local agency per country, translation and logistics across time zones, and a wait for reports before a launch date can move. Most companies cannot justify that sequence for every market they enter, so they default to exporting whichever positioning worked at home. That default causes the underperformance: a message tuned to one culture's purchasing psychology is not automatically legible in another. Cross-cultural research is harder than single-market research for reasons beyond logistics and translation: - Cultural context is implicit. What matters most about a culture is often invisible to an outsider. A US company entering Japan may never think to ask how much purchasing decisions there defer to hierarchy. A company based in Europe entering Brazil may not expect personal relationships to carry as much weight in B2B sales. - Identical scores carry different meaning market to market. A satisfaction or trust score that counts as strong in one industry or country can look mediocre in another. What counts as good quality, an acceptable price, or a trustworthy brand shifts by culture, so one scoring scale cannot be read the same way everywhere. - Language introduces compounding error. Even careful translation moves concepts that are normal in one language into something awkward or meaningless in another. ## How Do You Test Positioning in Each Market? The alternative to exporting one playbook is running a controlled discrete-choice experiment: the same pricing, packaging, or message alternatives, tested against defined buyer segments in each target country, measuring which option changes stated choice with a causal effect and confidence interval per market. That is a different claim than one open-ended conversation with a simulated persona per country: it produces a comparable, market-specific answer instead of an impression. A practical shape of this test, adapted from a company evaluating expansion into the US, Japan, and Brazil: Running the same positioning statement through these segments surfaces where the message needs cultural adaptation before a dollar is spent locally: the US segment responds to ROI evidence, the Japan segment needs integration and stability documentation, the Brazil segment responds to adoption and relationship signals. Without that test, a company exporting a single US-optimized message cannot know in advance which markets it will underperform in. ## What Cultural Framework Grounds the Comparison? A cross-cultural test works better when segments are built against dimensions known to predict where cultures diverge, not intuition about what "feels different." Hofstede's cultural dimensions theory is one well-established framework for this (Hofstede's cultural dimensions theory): ## What This Replaces, and What It Does Not Naming what an experiment does not replace is what lets a buyer weigh it against real launch risk. A controlled experiment answers which variant a defined segment prefers, with a causal effect and confidence interval. It does not replace in-market cultural expertise, local legal and regulatory review, distribution and channel relationships, or observed in-market sales behavior once the product is live. Treat it as the input that narrows which variant is worth local launch budget, not a substitute for that on-the-ground work. Subconscious runs these comparisons against a person-level audience graph covering 800 million real people, which makes precisely defined segments per country practical, not a handful of convenience-sample interviews. The path back to real participants is published here so a buyer can see exactly how a simulated finding gets checked. Audience reach is distinct from participant recruitment: when a finding needs confirmation with people who are not part of a simulated experiment, the same causal question can move to real human participants without changing what is being measured. ## A Practical Path to Building This Out Ongoing cross-cultural coverage comes from maintaining segment definitions per target market rather than rebuilding them for every campaign: - Cultural input per market, sourced from local employees, partners, or consultants who know how business gets done there. - Local market data: consumer research, prior campaign results, and market-specific signals that keep segment definitions grounded rather than assumed. - Segment drift gets stated here because only a rechecked segment counts as evidence. Periodic refresh, since consumer sentiment and cultural dynamics shift and a segment built a year ago can drift from the market it represents. ## Where to Verify This /research documents how Subconscious structures and validates causal experiments, and /case-studies shows the method applied to real launch decisions. /how-we-work walks through the process end to end. To see whether this fits a specific market-entry decision, book a demo. --- # A Research Operating Model for Matching Evidence to Decision Risk URL: https://subconscious.ai/blog/methods-and-validation/research-operating-model-matching-evidence-decision-risk Published: 2026-08-20 A research leader should route each question by the cost of being wrong. AI-assisted exploration can surface hypotheses. A controlled experiment on a simulated audience can compare actions directionally. An expensive or public decision may require the same causal question to be validated with real people. Human review governs all three. ## Route the question before choosing the method A plausible answer is not always strong enough to move a pricing, message, or launch decision. The research leader has to decide what evidence would change the action, what a wrong choice would cost, and whether the result will stay inside the team or support an external claim. The U.S. Bureau of Labor Statistics projects employment of market research analysts and marketing specialists to grow from 2024 to 2034. The Occupational Outlook Handbook describes the role and outlook. A projection published without its limits reads as marketing copy. That projection does not settle how a research team should use new tools. It reinforces the need for judgment about methods, caveats, and business decisions. Before selecting a method, write the decision in one sentence and state what evidence would be sufficient to act. ## What are the three evidence tiers and the review gate? A practical planning model has four layers: exploration, directional testing, human review, and validation. For decision routing, human review is a gate across three evidence tiers rather than a separate grade of proof. The gate is constant. A researcher checks the question, audience definition, alternatives, business context, and caveats before any output reaches a stakeholder. [figure: Three evidence tiers side by side: AI-assisted exploration, directional causal testing on a simulated audience, and real-human validation. One review gate crosses under all three.] ## How does Subconscious keep the causal question intact? Subconscious belongs between idea generation and external proof. It runs controlled causal experiments on simulated audiences to compare actions directionally. The team defines the audience, alternatives, and outcome before the comparison begins. Naming what a directional result cannot do is what lets a buyer check it. The result narrows the action set. It is not a verdict. When the decision requires stronger proof, Subconscious can test or validate the study with real human participants without changing the causal question: the simulated experiment and human validation ask the same question about the same alternatives. The study process should still record what changed, what stayed constant, and what the result does not prove. ## Put the review gate where claims change hands The human review step does not disappear when the method changes. Research judgment owns question framing, audience definition, neutral alternatives, interpretation, and the threshold for escalation. A fluent answer can become misleading when a deck removes the label that distinguished a hypothesis from a causal comparison or a validated result. Carry the evidence label into the decision artifact: - "Hypothesis from AI-assisted exploration: untested." - "Directional causal comparison: narrows actions, not validated with people." - "Validated with real-human participants: reviewed for this causal question." These labels keep the escalation path visible and prevent confident writing from turning a directional result into proof. ## What the operating model cannot delegate A method's limits belong on the record right next to where it gets used. Subconscious is one tier inside an evidence system. Its real-human validation extends a causal action test. It does not turn that test into a clinical trial, an observed usability session, or an automatic guarantee of market performance. The research team still decides which questions need exploratory work, which can use a directional comparison, and which require validation with people. That decision cannot be delegated to the tooling. ## Run a one-month routing exercise Choose one recurring research decision. Write the business decision in one sentence. Record its evidence tier, review gate, and validation trigger. Run that routing review once a week for a month, then revise the rule where the team found ambiguity. Use the research record to frame the method and its boundaries. If a current decision does not fit cleanly into one tier, discuss the study design before treating an output as evidence. --- # AI Audience Research for Market Decisions URL: https://subconscious.ai/blog/use-cases/ai-audience-research-market-decisions Published: 2026-08-20 A marketing lead deciding how to describe a target segment, before media or creative money moves, usually has two bad options: a persona document that is a year or two stale, or a survey that takes weeks to field and comes back confirming what the team already assumed. Neither answers the question before a campaign brief is due. ## Exploring a segment is not the same as proving a claim Simulated conversation with a modeled buyer segment skips the field-recruitment step. A team can describe a role, a set of pressures, and a category, then ask open questions and follow up on the answers, like an interview. That surfaces language, objections, and framing worth testing. A stated preference reported without its limits reads as marketing. Naming the limit here lets a buyer weigh it correctly. It does not settle a fact: stated answers, human or simulated, are a starting point, not a measurement of what a buyer will actually do. A recent study on elicitation methods and language models found that how a preference question is asked changes the gap between what a model states and what it would reveal under an incentive-compatible task (Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models). The same say-do gap that makes an open-ended survey answer unreliable applies to simulated dialogue. ## Why does the standard research toolkit stall the decision? A written survey requires the team to know which questions matter before the field period opens. That makes it strong for confirming a hypothesis and weak for finding one. A moderated group session solves the discovery problem but adds new ones: one dominant voice can steer the room, participants respond to each other instead of the moderator, and a single session is the whole dataset. Planning budgets for that kind of session commonly run into five figures. ## What simulated dialogue is good for Used as an exploration step, this supports a few concrete jobs: - Testing a message against several roles in the same buying committee before spending on creative. - Surfacing language and content angles a keyword list will not turn up. - Re-checking whether a persona built two years ago still matches how a role, its tools, or its constraints look today. - Comparing framing options for a positioning statement to see which one confuses buyers and which one lands. Research on whether language models can support choice modeling finds real promise in this generative role, alongside clear limits on treating model output as a substitute for fielded preference data (Can large language models assist choice modelling? Insights into prompting strategies and current models' capabilities). ## When does a decision need a causal test? The buyer question that carries budget risk is not "what does this segment say it wants," it is "which message, price, or positioning choice moves their choice, and by how much." Answering that requires a controlled experiment: present alternatives, hold everything else constant, and measure which option changes the outcome. Subconscious runs that kind of controlled, discrete-choice experiment across segments and reports which action moves an outcome, with a confidence interval attached. Studies can run against a person-level audience graph covering 800 million real people, a scale distinct from any recruited discussion panel. The most consequential findings can then move to real-human participants for validation, without redesigning the underlying causal question. That progression, from exploration to a measured result, is what stated-preference dialogue alone cannot close. [figure: Five steps: explore a segment via simulated dialogue, generate hypotheses, run a causal test measuring which option moves the outcome with a confidence interval, validate with real humans, then ship the decision.] ## What are the limits of this approach? A capability claim published without its limits is marketing. This one is stated plainly so a buyer can see where the tool stops and a designed study is needed. This approach does not produce a market-share number like "most of a target segment prefers monthly billing"; that kind of estimate needs a study built to support it. It does not replace transaction data, product usage, or direct observation when the decision hinges on what buyers actually do. And a simulated conversation, however fluent, is still a hypothesis-generation tool until it is checked against a controlled test or real participants. [figure: Four boxes: test a message against roles in a buying committee, surface language a keyword list won't find, re-check whether an old persona still holds, compare framing options for a positioning statement.] ## Where to start Pick one segment the team understands well and use it to calibrate: run the exploration, compare the output against what the team already knows is true, then decide whether the gap is signal or noise. From there, route the strongest hypotheses into a designed causal test and reserve real-human validation for the assumptions that carry the most budget risk. Review prior case studies for what a completed study looks like, or read how a study gets built before committing a segment decision to a media plan. --- # AI Can Summarize Consumers. It Still Needs Human Judgment. URL: https://subconscious.ai/blog/use-cases/ai-summarize-consumers-still-needs-human-judgment Published: 2026-08-20 AI can summarize consumers fast, but it still needs a human to review sources and judgment for high-stakes launch, pricing, or messaging decisions. A Head of Consumer Insights gets a stakeholder request: let an AI-generated consumer summary stand in as decision-grade evidence for a launch, pricing, or messaging call. The question underneath it does not change: is this directional exploration, or is it proof strong enough to commit a budget against? Get that call wrong and a team ships a launch, pricing, or positioning decision off a narrative that never held up with real customers: after the budget is spent and the research function's credibility with stakeholders is already burned. ## Why the pressure is showing up now AI has moved from a novelty layer into daily research workflows: analysis, reporting, data preparation, self-service insight. That has not removed demand for research judgment. Bureau of Labor Statistics data puts market research analyst and marketing specialist employment on a growth path running from 2024 through 2034. So the danger isn't "AI replaces researchers." It is a team accepting a confident-sounding AI or synthetic-panel narrative as fact before checking whether it is grounded in evidence. ## The decision that actually needs a system The old bargain in consumer research put expertise partly in access: knowing how to field a study, clean the responses, interpret the chart. AI weakens that advantage, but not the harder judgment call: which answer deserves trust, and what the decision in front of you requires. That call needs a structure, not a habit of reaching for whichever tool is fastest. A workable version has four layers: - Exploration: use AI to generate hypotheses, objections, and alternative explanations. - Directional testing: use a synthetic panel or AI-assisted read to compare options quickly. - Human review: check the audience definition, prompt neutrality, source grounding, and business context; look for contradictions across segments. - Validation: for decisions carrying real cost or public exposure, confirm the read against actual respondents, behavioral signals, expert judgment, or a fielded study. The output of step 2 is not the answer. It is an input to steps 3 and 4. ## Where does a structured evidence path hold up? Subconscious keeps the same separation this workflow argues for: directional exploration is distinct from causal proof. A simulated study read can move to a controlled experiment with real human participants for validation, without changing the underlying causal question, and results carry confidence intervals rather than a single fluent narrative. Naming what the audience graph is not lets a buyer check the claim against the right thing. Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people: that is audience reach, not a recruitable panel, and it does not substitute for the human-review and validation steps above. Calling out which layer a tool sits in is what a buyer needs before relying on it. This is the evidence layer, not the judgment layer. A human still has to define the decision, write the audience brief, spot contradictions, and decide what needs real-human validation before a claim goes external, regardless of which tool produced the directional read. ## What does honest labeling look like? The step that keeps a directional read from becoming a false decision-grade claim is naming it accurately before it leaves the research team: label a synthetic read as "directional synthetic panel read" or "AI-assisted hypothesis, still needs confirmation before it goes external." Stakeholders can then see exactly how much weight the finding can carry. The failure mode: a polished narrative ships before anyone confirms it rests on real evidence, usually because a deliverable is due and a fluent AI answer arrived before the source got checked. The fix: spell out the boundary in the deliverable itself, what the AI-assisted work covered, where it stopped, and what still needs validation. ## A first workflow to run this week - Pick a real project with a live decision attached to it. - Write the business decision in one sentence. - Define the audience and the risk level of getting it wrong. - Limit AI and synthetic-panel tools to the exploratory stage only. - Go through the output by hand and flag what holds up, what's shaky, and what's unsafe to use. - Present the answer with a clear caveat and a named next validation step. Repeat that loop across a handful of real decisions and the output is not a list of AI tools. It is a working evidence system that shows speed, judgment, and quality control together. [figure: Two-column comparison: real-respondent validation raises evidence strength (left); the causal question tested stays fixed, not a usability test or market-performance proof (right).] ## The limit that does not go away A validation claim without its limits reads like marketing, so the limit gets published next to the result. Real-human validation strengthens a directional finding; it does not turn a causal action test into an observed usability session or an automatic proof of market performance. What changes from simulation through validation is the strength of evidence behind the answer, not the question being tested. Saying plainly what AI does not do is what keeps the claim honest for the person deciding whether to trust it. AI changes what a first-pass consumer synthesis looks like. It does not remove the need for a human to decide what is true, useful, and too risky to act on. That decision, and the evidence path behind it, is what makes a consumer insight safe to put in front of a stakeholder. [figure: Four steps in order: Exploration generates hypotheses, Directional testing compares options via AI, Human review checks sources, Validation confirms with real respondents. Step two feeds step three, not the final answer.] To see how a directional read moves to a controlled study with real participants, review how Subconscious runs studies or look at published case studies. --- # Understanding the margin of error in simulations URL: https://subconscious.ai/blog/methods-and-validation/understanding-margin-error-simulations Published: 2026-08-21 A VP of consumer insights deciding whether to trust a synthetic panel's read on a pricing decision needs one number before signing off: the real margin of error on the study about to run, not a resemblance score from someone else's study. True margin of error comes from the response variance inside the randomized experiment you design and field this quarter, and it shrinks predictably as you add sample, landing in a confidence interval the study itself can defend. A high correlation to a prior human survey describes how well a model reproduced old data. It carries no provable bound on the choice set, segment, or question you are about to test. - Margin of error is a property of a randomized experimental design. It comes from the variance in the study you are running now, not from how well a model matched a past one. - A 0.90 Spearman rank correlation to a prior human survey, the most-cited case being EY and Aaru's recreation of a 3,600-respondent wealth study, is backward-looking validation. It says nothing provable about the next question you ask. - Resemblance scores can hide bias instead of revealing it: the same EY/Aaru study posted a -0.38 rank correlation on inheritance planning even while the median across 53 questions hit 0.90. - Predicting a single person's answer carries higher divergence than predicting the whole population's response pattern, on identical underlying data. - Discrete choice models estimated on your own randomized design, McFadden multinomial logit, Mixed Logit, or ICLV, produce coefficient-level confidence intervals the study itself can defend, not ones borrowed from another team's data. ## What does margin of error actually mean in a simulation? It means the range around an estimated effect, at a stated confidence level, derived from response variance inside the specific randomized experiment you fielded. In a discrete choice study, that variance comes from how respondents (human or simulated) traded off attributes across the choice tasks you built. Feed more respondents into that same design and the interval narrows in a way you can compute and audit. That is what a margin of error is: a property of this study, not a compliment paid to a different one. A correlation coefficient against a past survey does not narrow with more respondents in your current study, because it is not measuring your current study at all. ## Why a 0.90 correlation to last year's survey is not a margin of error The most-cited validation case in the market right now is EY's partnership with Aaru, which recreated EY's 3,600-respondent Global Wealth Research Report and reported a median Spearman rank correlation of 0.90 across 53 questions, with an average RMSE of 7.1 percentage points (EY, "How AI simulation accelerates growth in wealth and asset management"). That number answers a real question: did the model's synthetic population resemble the 2024-vintage human sample closely enough to trust the reproduction. It does not answer the question a buyer actually has going into a new study: what is the error bound on the specific decision I am about to make. Resemblance to a fixed, published dataset does not move when you change the question, the population, or the choice set. It is fixed the moment the benchmark study is fixed. ## The thin-segment failure: what resemblance scores hide The misses go on the record next to the hits, because that is what lets a buyer check the number before relying on it. Inside that same 0.90 median sat a -0.38 rank correlation on inheritance planning, a single topic where the simulation and the human sample diverged sharply while the aggregate score stayed reassuring (EY/Aaru). A separate study aggregating 29 real-world design-preference tests across 2,073 human participants found consistent, systematic discrepancies between LLM-simulated and real preferences, including position and order bias; the paper reports these discrepancies directionally rather than as a single effect size, so they are not directly comparable in magnitude to the correlation and divergence figures cited above ("Distorted Perspectives of LLM-Simulated Preferences," arXiv:2605.18311). Neither failure is a sample-size problem. A larger synthetic panel does not fix an order-bias artifact or a segment the model represents poorly; it just reproduces the bias with more decimal places. That distinction, bias versus variance, is the one a bare correlation number cannot make for you. "Our results unveil significant and systematic discrepancies between peoples' real design preferences and LLM simulations that are consistent across manipulations." Kuric, Demcak, and Krajcovic, "Distorted Perspectives of LLM-Simulated Preferences," arXiv:2605.18311 (source) ## Individual-level prediction carries higher error than population-level prediction Naming a failure mode is what lets a buyer check a study against it, so here is the one this method carries. A cross-domain benchmark comparing synthetic and human survey responses found aggregate-level Jensen-Shannon divergence of 0.011 to 0.046, versus 0.056 to 0.090 for single-answer, individual-level prediction on the same underlying data ("When Can Digital Personas Reliably Approximate Human Survey Findings?", arXiv:2605.10659). The gap between the two ranges is not a single ratio. Comparing the lower bounds, individual-level divergence runs more than 5x higher than aggregate-level divergence. Comparing the upper bounds, it runs close to 2x higher (0.090 versus 0.046). Either way, asking a simulation "what does this market do on average" and asking it "what does this one person do" are different questions with different error profiles. [figure: Bar chart showing Jensen-Shannon divergence ranging from 0.011 to 0.046 for aggregate population predictions, versus 0.056 to 0.090 for individual-level predictions, on the same underlying survey data.] That gap is a reason to ask what your study needs: a market-level read, where error is smaller, or a segment-level or individual-level read, where it is not. Neither question is answered by a single resemblance score against a past survey. ## How does a randomized discrete choice experiment produce a real confidence interval? It produces one because the interval comes from the variance in how respondents traded off attributes across the randomized choice tasks in that study, estimated with a discrete choice model. McFadden's multinomial logit, Mixed Logit, and ICLV are estimators applied to that randomized design; the causal identification comes from the randomization itself, not from the estimator. A standard multinomial logit carries the independence of irrelevant alternatives assumption, meaning it can distort preference-share and substitution estimates when alternatives are not genuinely independent; Mixed Logit relaxes that assumption by allowing preferences to vary across respondents, which matters when you are asking a substitution question rather than a simple main-effects question. Whichever estimator fits the design, the resulting confidence interval covers the effect within the population you simulated, in the study you ran. It does not extend that guarantee unconditionally to the real market, and any responsible read of the result says so. ## Where do Subconscious's own validation numbers fit? A buyer checks a claim by seeing its limit named plainly, and this one gets named here. Subconscious's own validation number carries the same limitation as the others, but it is stated differently: as a ratio against a measured human ceiling, not a bare correlation to one external dataset. Subconscious's best configuration reaches 87% of the measured human ceiling on one study, a 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959 against each other; across all 43 studies that passed design filters, the mean replication is 0.73 (Subconscious, Causal Fidelity paper). That figure is a validation result about how well the simulation reproduces known human studies. It is not a guarantee for a market you have not yet tested. Published benchmark studies can sit inside a model's training data; the replication protocol tests against studies designed to check for that, rather than assuming the problem away. The current model comparisons behind that number are tracked on the leaderboard, which updates as configurations change. The ratio against a measured human ceiling still does not substitute for a margin of error: that has to come from your own randomized design. ## Resemblance score or margin of error: which one should a buyer trust for a decision? Trust the benchmark comparison to screen a vendor, and trust the margin of error to bound the decision. The two answer different questions and neither substitutes for the other. For more on how validation studies are built and where their limits sit, see the methods and validation hub. The concrete next step: before running a new study, ask any vendor two separate questions, not one. First, what is your resemblance score against a published human benchmark, and what is the denominator. Second, what confidence interval does my specific design produce, and what population does it cover. A vendor with a strong answer to the first question and no answer to the second is offering you a description of their last project, not a margin of error for yours. If you want to see how that second question gets answered on a live design, meet with the team. --- # Download free Excel template for the Van Westendorp PSM URL: https://subconscious.ai/blog/methods-and-validation/download-free-excel-template-van-westendorp-psm Published: 2026-08-21 --- The pricing lead or VP Product setting a new price can find a free Van Westendorp Excel template in minutes, from Conjointly, OpinionX, Eloquens, eFinancialModels, or the open source pricesensitivitymeter package on CRAN (CRAN README). The real decision is not which template to grab; it is whether the acceptable price range that template returns should set the number you charge. Every one of these tools computes the same four-curve intersection correctly, but none of them can turn four uncontrolled survey questions into a causal estimate of what price actually changes behavior. - Free Van Westendorp templates exist and are functionally interchangeable: Conjointly, OpinionX, Eloquens, eFinancialModels, and the CRAN pricesensitivitymeter package all compute the same intersection points from the same four questions. - The templates get the arithmetic right; the problem sits upstream, in survey data collected with no purchase consequence, no competitive context, and no randomization. - Stated prices in subscription pricing research tend to run higher than what customers actually pay, consistent with hypothetical bias in unincentivized willingness-to-pay questions; the direction is well documented, the magnitude varies by study (GetMonetizely). - The method's four intersection points are known to shift under resampling and depend on respondents having a real reference point for the product's value, which is rare for anything new (Relevant Insights). - Van Westendorp is scoped to a single, well-defined product; once a decision involves multiple attributes or competitors, a randomized discrete choice design is the tool that can test many configurations in a single study instead of one (Drive Research). ## What does a Van Westendorp Excel template actually calculate? It plots four self-reported price points, too cheap, cheap, expensive, and too expensive, as intersecting curves and reads off the crossing points as an acceptable price range. Peter van Westendorp developed the method in 1976 specifically to keep pricing research cheap: four questions, no experimental manipulation, no control group (Wikipedia). A template automates that plot and the interpolation between data points. What it cannot automate is the thing the plot depends on: respondents giving prices that predict what they would actually do at the register. Nothing in the four questions puts a real choice, or a real cost, in front of anyone. ## Where the free templates actually come from The commoditization of this math is itself informative. Conjointly, OpinionX, Eloquens, and eFinancialModels all publish free spreadsheet versions. Hotjar, LimeSurvey, SurveyKing, and XLSTAT build the four-question flow directly into their survey tools. The CRAN pricesensitivitymeter package standardizes the calculation as three lines of R code. When a method's entire analytical output can be replicated by an open source package with no proprietary logic, the remaining decision is not computational. It is whether the input data was collected in a way that supports a causal claim about price and behavior. Van Westendorp's inputs never were. ## Why the spreadsheet math is not the hard part The four questions are unprompted, unconstrained, and unconsequential. A respondent names "too expensive" for a product they have never used, with no competing product named, no attribute traded off, and no assignment of who sees which price. Practitioner critiques of the method point to exactly this: direct price questioning invites lowballing, and respondents lack any reference point to value a new or unfamiliar product (Relevant Insights). A template cannot fix that, because the fix is not arithmetic. It is design: what varies, what is held constant, and who is randomly assigned to see what. None of that happens inside four open price questions, however cleanly the spreadsheet plots the result. ## Is Van Westendorp's price range accurate? It is directionally useful but not reliable enough to set a number with revenue behind it. Subscription pricing research shows stated prices running higher than what customers actually pay (GetMonetizely), the standard pattern of hypothetical bias in unincentivized willingness-to-pay questions: respondents name a price when nothing is actually at stake, and that price tends to sit above what they later accept once a purchase is real. The exact size of that gap varies by study and by market; the direction does not. A Van Westendorp range built entirely from that kind of self-report inherits the same bias, and a template has no way to correct for it because it never sees a real choice to check the answer against. ## The four-point estimate is not stable The acceptable price range comes from four intersection points calculated on whatever sample answered the survey. Resample that population, or even resample the same population at a different time, and the four intersection points move. Practitioner critiques document this instability repeatedly, but none attach a number to how far the range shifts; the finding is qualitative, not a stated margin of error (Relevant Insights). A wide, unstable range is a reasonable input to a conversation. It is a weak basis for a single number on a price sheet. ## Van Westendorp template vs. a randomized pricing experiment ## What would make a price estimate causal? Randomization, not a better spreadsheet. A causal estimate of what price changes behavior requires assigning different prices, or different products, to different respondents by design, then measuring which assignment actually moves choice. DCE, Mixed Logit, and ICLV are estimators for analyzing that kind of data, not causal methods in themselves; the causal identification comes from the randomized manipulation in the experiment, not from the model fit afterward. A plain multinomial logit model also carries an assumption worth naming here: independence of irrelevant alternatives, which means it cannot represent two products a respondent sees as close substitutes. Mixed Logit and ICLV relax that assumption by letting preferences vary across the simulated population, which matters once a pricing decision involves more than one competing option. [figure: Bar chart showing three values: human baseline reliability at 0.959, the best single-study replication at 0.832 (87% of the human ceiling), and the mean rank correlation across 43 studies passing design filters at 0.73.] Subconscious tests this approach with a randomized simulation, then checks the result against real human responses on published pricing studies. Our best configuration reaches 87% of the measured human ceiling on one study. That is a rank correlation of 0.832 against the published human result. Two independent samples of real humans reach 0.959 against each other. Across all 43 studies that passed design filters, the mean rank correlation is 0.73, a lower and more honest number to anchor expectations on. This is a replication result against published studies, not a guarantee for a market that has never been tested. Published studies can sit in a model's training data. The replication protocol behind these numbers is built to address that, not to pretend the problem doesn't exist. Full methodology and per-study results are in the causal fidelity paper, and current results by study are tracked on the leaderboard. ## When does Van Westendorp actually work? It works when the decision is genuinely small: one product, one price, low stakes, and a rough range is enough to move a conversation forward. Industry guidance treats it that way too: a cheap first pass before conjoint analysis, reserved for a single or very few product formulations (Drive Research). Once a decision involves multiple products, competitors, or attributes, the method is out of scope by design. A randomized discrete choice design can test many configurations in a single study. More background on how randomized experiment design and discrete choice estimation fit together is in the methods and validation hub. Before setting a price from a Van Westendorp output, run the same product through a small randomized discrete choice test with a named competitor and a holdout group, and check whether the range survives. If it does, the spreadsheet range was close to right for the wrong reason. If it doesn't, that gap is the actual finding. When you're ready to design that test properly, meet the team. --- # When One Metric Secretly Drives Another: A Buyer's Guide to Vector Autoregression URL: https://subconscious.ai/blog/implementation-and-procurement/when-one-metric-secretly-drives-another-buyers-guide-vector-autoregression Published: 2026-08-24 A data science or analytics leader choosing a forecasting approach for two or more business metrics has one decision to make first: do these metrics only respond to their own past, or does each one also feed back on the other? Pick a single-equation, point-estimate model when the answer is "they feed back on each other," and the forecast hides which metric is driving which, and drops the uncertainty band that would have told you how much to trust it. ## What can't a single-equation forecast see? A standard autoregressive forecast predicts one metric from its own history. That works when the metric truly moves on its own. It breaks silently when a second metric is quietly driving it: the model still produces a confident point number, but the number encodes a relationship it never checked. Here the left-hand side is the metric's value at the current time step, the first term on the right is a baseline intercept, each following term weights how much an earlier value of the same metric still matters, and the last term is unexplained noise. Nothing in this equation lets a second metric contribute at all. ## How does vector autoregression test both directions at once? A vector autoregression (VAR) model replaces the single equation above with one equation per metric, and each equation includes the recent history of every other metric in the system: Setting every cross-term to zero collapses the system back into two independent, single-metric forecasts. Keeping them in is what lets the model answer the buyer's actual question: which metric moves which, and by how much. VAR is not an economics-only tool. Published applications span patient health records in medicine, activity recognition from sensor data, disease-spread interactions in epidemiology, gene-regulatory networks in biology, and short-term supply and demand shifts in energy markets, anywhere two or more tracked quantities plausibly influence each other over time (Bayesian vector autoregression, Wikipedia). ## Why does the Bayesian version keep the uncertainty instead of dropping it? Classical VAR still returns point estimates for every coefficient and every forecast step. A Bayesian VAR (BVAR) treats each coefficient as an unknown quantity with a full probability distribution, updated from the data, so the output is a range of plausible futures instead of one number. PyMC is the open-source probabilistic programming library used to build and sample this kind of model; writing the VAR equations directly in PyMC produces a Bayesian VAR by default, without a hand-derived sampling routine (Bayesian Vector Autoregressive Models, PyMC example gallery). ## A historical example: the feedback ran one way, not both A widely used illustration of this discipline fits a two-metric BVAR to a public quarterly economic dataset spanning three decades. As a planning example, not a live capability, the exercise is instructive because of what the posterior revealed rather than any number it produced: past values of one metric were strongly associated with the other metric's future path, while the reverse relationship sat close to zero. A single-equation model applied to either metric alone would never have surfaced that asymmetry. That is the general shape of the buyer decision: before trusting a forecast that treats two metrics as independent, check whether the feedback between them is actually one-directional, bidirectional, or absent. ## How Subconscious approaches the same discipline Subconscious's causal experimentation and discrete-choice-style modeling start from an explicit decision, not an ambient dataset, and return directional comparisons with honest uncertainty language where the study design supports it. Read more about the research approach or how a study is set up. ## Where the analogy stops Subconscious does not run vector autoregression, Bayesian estimation, or macroeconomic forecasting as a product feature, and no confidence interval, posterior range, or runtime figure attached to this forecasting example belongs to a Subconscious output. ## Next step A team evaluating a forecasting or causal-modeling vendor can ask the same question raised above: does the proposed method check whether two metrics feed back on each other, and does its output keep an honest range instead of a single confident number? Learn about Subconscious to see how that question gets answered for a specific business decision. [figure: Four steps: check if either metric's past predicts the other; if not, single AR suffices; if so, use VAR with cross-terms in both equations; if a range is needed, use Bayesian VAR.] --- # Validating a Multi-Step Onboarding Flow Before Engineering Builds It URL: https://subconscious.ai/blog/methods-and-validation/validating-multi-step-onboarding-flow-before-engineering-builds Published: 2026-08-24 A Head of Product with a redesigned sign-up-to-first-action sequence has two options: validate it before engineering builds it, or ship on instinct and let production A/B tests find the broken step later. Each step in a sequential flow inherits the drop-off from the step above it, so a weak step early in the sequence taxes every step after it, and the resulting activation damage compounds into CAC payback and LTV before a fix ever reaches production. ## Why onboarding rewards evaluation before it ships Onboarding has a specific shape that rewards pre-build evaluation: - It is sequential. A step cannot be judged alone: what a person believes after step 2 changes how they read step 3. - Drop-off compounds. A typical multi-step flow loses 10 to 40 percent of users at any given step. - It resists live A/B testing. Production experiments need enough signup volume per arm to reach significance, limiting most teams to a few variants per quarter. - Most of the friction is in the words. Permission explanations, empty-state copy, and value-prop framing decide whether someone continues, more often than visual design does. A small structured evaluation run before a product ships has long been shown to surface most of a product's usability problems, at a fraction of the cost of finding them after launch (Nielsen Norman Group). The same logic applies here: evaluate the flow's decision moments before it is built, not after. ## How do you storyboard an onboarding flow before testing it? Before running any evaluation, break the flow into cards, one per step. For each card, capture three things: what the person sees, what they are asked to do, and why it belongs in the sequence. A typical B2B SaaS onboarding sequence (sign-up, account setup, connect a data source, invite teammates, pick a use case, a short tour, a first-action prompt, an upgrade nudge) turns into six to ten cards. Stating what a test leaves out is what makes its result trustworthy enough to act on. Each card becomes one question for the evaluation, testing that step's wording on its own; it does not capture carryover from earlier steps or the selection effect of who reaches that step in a real funnel. ## What should you test at each onboarding step? Different step types fail for different reasons, so the question asked has to match the step: Every method carries a blind spot, and naming it is what lets a buyer check the result against their own case. The questions above are direct, self-report elicitation, useful for a first pass but exposed to hypothetical bias, especially the upgrade prompt, where stated willingness to pay runs high. Subconscious's causal action testing uses a different comparison: a randomized discrete choice design (McFadden DCE, Mixed Logit, ICLV) puts a defined buyer segment in front of step-level alternatives (a permission explanation worded one way versus another, an empty state with guidance versus without) and reads which one a person continues past, before either version reaches a codebase. See how the method works and the underlying research. ## Where this method stops, and what has to pick up A miss recorded next to a hit is what makes a method usable for a real decision. A simulated evaluation reads decisions, not devices or timing. It does not observe page-load performance, rendering differences across phone and desktop, activation driven by push or email notifications, or retention behavior that only shows up weeks after signup. Those require production telemetry, not a pre-build test. When a finding needs that kind of confirmation, the team can move from a simulated comparison to real-human validation without changing the underlying question: the question of which step wording drives continuation stays the same, but whether the estimate transfers from simulated to recruited participants is a calibration question that needs holdout evidence, not an assumption. That step matters most for the one or two choices closest to launch, not for every step in the flow. ## When should you evaluate onboarding before it ships? The useful rhythm: run a full evaluation before any onboarding flow ships, revisit flows already in production on a regular cadence, and evaluate any new step or copy change before it merges, not after. Treat production A/B tests as refining a flow that already tested well, not discovering which steps were broken in the first place. For teams weighing whether a redesigned sequence is ready to build, browsing recent studies or booking a walkthrough beats another round of guessing. [figure: A row of onboarding step types in sequence, each with a small test checkpoint above it before an arrow leads to the next step, showing that a step only proceeds once its wording has been compared and chosen.] --- # Does a Synthetic Panel Replicate Wind-Developer Policy Preferences? The Luthi Replication URL: https://subconscious.ai/blog/case-studies/synthetic-panel-replicate-wind-developer-policy-preferences-luthi Published: 2026-08-24 [figure: Four-step path: confirm the fielded study's policy levers, run a matched synthetic study, compute rank correlation (r_s = .7884, p = .0004), then use it as directional support, not a replacement, for a new fielded study.] A synthetic replication of a 2011 fielded discrete-choice study on wind-energy policy preferences produced a rank correlation of r_s = .7884 (p = .0004) against the original developer survey. That is the evidence available to a research or policy lead deciding whether to trust directional signal from a synthetic panel before committing budget to a full fielded study of wind-project stakeholders. ## The population problem this replication addresses Wind-project developers are a narrow, hard-to-reach professional population. Fielding a new survey against them for every policy question (permit timelines, grid-access guarantees, incentive structures) is slow and expensive. Lüthi and Prässler used a discrete choice experiment to assess how these regulatory and incentive levers weigh on developer decisions, publishing their fielded results in Energy Policy in 2011. A synthetic panel re-ran the same comparison to check whether it recovers the same preference ordering. This case study does not document the operator, run date, or sample size behind this figure. ## What was compared ## What does the correlation support, and what does it not support? A rank correlation of r_s = .7884 (p = .0004) is a statistically significant, directional agreement between two preference orderings. The misses sit on this record next to the hits. It doesn't claim exact prediction accuracy or measure how closely any individual developer's stated preference matched. Lüthi and Prässler surveyed a narrow population under a specific set of policy attributes; the aggregate rank correlation reported here says nothing about agreement at the level of individual attributes, so it cannot be extended to attributes or populations the original study didn't cover. ## How do you decide whether to trust the synthetic read? A policy-design or advocacy team facing this choice has the same three options available for any synthetic-versus-fielded question: act on the synthetic panel alone, commission a new fielded survey of developers, or run a matched replication first. The third path is the only one that produces a testable statistic before spending the fielded-research budget. - Confirm the fielded study covers the same policy levers under consideration. - Run a matched synthetic study against the same attributes and population framing. - Compute the rank correlation and report its significance alongside it, as above. - Use a significant, positive correlation as directional support for scoping the next fielded study, not as a substitute for one, and not as proof for policy attributes the replication didn't test. ## Limitations and failure conditions Naming a failure mode here is what lets a buyer check it before relying on the result. This is one replication of one study against one narrow professional population. It says nothing about how a synthetic panel performs against regulators, utilities, or the general public; each needs its own matched replication. The 2011 benchmark also predates the shift in most major wind markets from feed-in tariffs to auctions, CfDs, and merchant/PPA structures, so this correlation is a method check on the original attributes, not current guidance on today's incentive structures. A team moving from this directional read to a live campaign or incentive-design decision should validate the specific attributes in play, ideally with real human participants, before treating the synthetic ordering as final. Teams evaluating policy-preference research more broadly can see how this replication compares to others on the leaderboard, and review the underlying method on about. --- # Which Draft Should a Founder Actually Publish? URL: https://subconscious.ai/blog/buyers/which-draft-should-founder-actually-publish Published: 2026-08-24 A founder who posts in public every week is not really choosing between "post" and "don't post." The decision happens earlier: which hook, which angle, which framing of the same idea goes out. Get that choice wrong often enough and the damage isn't one flat post. It's a target audience that quietly stops reading, a candidate who only ever saw the weakest version of the pitch, or a buyer who formed an opinion from a post that never should have shipped. ## Why does guessing get more expensive with volume? A single bad post is a rounding error. A pattern of bad posts is a reputation problem: slow to notice and slower to reverse. The founder rarely learns which post caused the miss. By the time a deal stalls or a candidate goes quiet, the post that misjudged them is long gone from the feed. LinkedIn's feed algorithm rewards posts that generate strong early engagement and suppresses accounts that repeatedly post low-performing content, which is exactly why a string of misreads compounds instead of averaging out. ## What "testing a post" should actually mean Reading a draft one more time before publishing doesn't tell a founder how a CFO, a product lead, or a head of marketing will read it. Asking a single AI assistant for an opinion returns one point of view dressed up as feedback. Neither approach separates the audience a founder is trying to reach from whoever happens to be watching. A useful test of a message variant needs three inputs, not a vibe check: Without a defined audience, a test of "does this post work" returns a generic score that flatters everything and clarifies nothing. The narrower the audience definition, the more the comparison means something. ## Testing the message, not the metric Subconscious tests messaging and claims variants against a defined target audience through causal experiments before a team commits to publishing, pricing, or positioning a message, the same class of decision as choosing which post to ship, framed around confidence and uncertainty rather than a promise about likes or impressions. That distinction matters because a founder's actual question is rarely "how many people will see this." It's "which of these messages lands with the people I need to reach." Only one of those questions is answerable before a post goes out. Stating what a method can't reach is what lets a buyer check the claim against what actually happens. Raw engagement (likes, impressions, how a specific platform's ranking system routes a post) depends on network size, timing, and platform mechanics that no message test, causal or otherwise, is built to predict. ## An illustrative comparison, not a promised outcome Take a founder deciding between a numbers-led hook, a contrarian claim, and a customer story, all making the same underlying point. A team might draft several variants, define the specific reader they're writing for (an early-stage operator wrestling with the transition the founder's product addresses, say), tag the intent, and compare relevance and resonance across that audience before choosing. The value is in forcing the comparison and the audience definition before publishing, not in a number promising how well any one post will perform. ## Where does this approach stop being useful? This approach answers one question: which message resonates better with a defined audience. It does not tell a founder how many people will see a post, how a platform's specific ranking algorithm will treat it, or whether a post will go viral. A team using message testing for public content should scope it to that comparison, not to a forecast of reach or engagement volume. When a founder wants more than a simulated read on a message, before a high-stakes launch post or a recruiting campaign, for instance, the same causal question can move from a simulated test to a study with recruited real-human participants, without changing what's being asked. That's a deliberate step up in stakes, not a requirement for every weekly post. ## The next post The founders who compound their reach are the ones who stop letting instinct decide which message goes out, especially on posts aimed at a specific buyer, investor, or hire rather than a general audience. Defining the audience and comparing the real candidates before publishing is the discipline; picking the winner is the easy part once that comparison exists. Read more on how Subconscious tests actions before a team commits to them, see the method behind the experiments, or talk to the team about a specific message decision. [figure: Left-to-right path: three draft variants (numbers-led, contrarian, customer story) feed into a defined audience and stated intent, which combine in a resonance comparison, ending in one publish decision.] --- # Bayesian Computation in Finance: Modeling Risk as a Distribution, Not a Guess URL: https://subconscious.ai/blog/industries/bayesian-computation-finance-modeling-risk-distribution-not-guess Published: 2026-08-24 A single number for expected return, volatility, or option value hides how much a model actually knows: a decision-maker cannot tell whether that figure is a confident estimate or a guess dressed up as one. Bayesian computation asks a narrower question: given the data and the assumptions stated up front, what is the full range of plausible outcomes, and how much weight does each carry? [figure: Two-row comparison: top row shows one fixed number each for market beta, value at risk, and option price. Bottom row shows the same three quantities as full posterior distributions instead.] ## Why point estimates understate risk Classical financial econometrics, including ordinary least squares regression and standard time-series methods, assumes model errors are normally distributed. Financial data routinely violates that assumption: asset returns, risk premia, and volatility exhibit skewness, heavy tails, and volatility clustering that a normal distribution does not capture. Treating parameters as fixed, single-point values then understates tail risk and produces forecasts that look precise but are not reliable. Bayesian methods treat model parameters as random variables with their own probability distributions rather than as fixed unknowns. Fitting a model to data returns a posterior distribution over parameters: a complete description of every plausible model configuration consistent with the data and the stated prior assumptions, so an analyst can see how much uncertainty a model actually carries into a decision. ## Quantifying uncertainty instead of picking one model A linear regression estimating market beta illustrates the difference. A frequentist regression returns one beta value and a confidence interval, often described but not literally interpreted as a distribution. A Bayesian regression on the same data returns a full distribution over beta, so an analyst can draw a whole set of plausible models that each fit the observed data, instead of settling on a single one. This matters most in asset pricing and factor models, where a fixed parameter can lead to underestimated tail risk, overconfident forecasts, and strategies that fail once real markets deviate from the fitted line. A distribution over parameters lets an analyst check for features a single estimate would hide, such as multiple plausible regimes or dependencies between parameters. ## How do you model errors that are not normally distributed? Bayesian modeling also relaxes the normality assumption on errors directly: an analyst can specify an alternative distribution, such as a skew-normal or Student's t-distribution, that better matches the empirical shape of the data. In an illustrative simulation with a known skewed, heavy-tailed error term, an ordinary least squares fit still recovers an unbiased slope estimate, but its standard errors and confidence intervals are unreliable because OLS assumes normal errors. A Bayesian fit using a Skewed Student's t likelihood recovers calibrated uncertainty intervals because it learns the error distribution's shape along with the intercept and slope. The same flexibility extends to the return series itself. Equity returns often show left-skewness, a higher chance of large negative shocks, which a skew-normal likelihood can represent directly. Compound returns across multiple periods, the cumulative product of period returns, tend toward a lognormal shape, which a Bayesian model can encode through a lognormal prior or likelihood rather than approximating with a normal one. ## Value at risk and option pricing under full uncertainty Parametric variance-covariance Value-at-Risk approaches assume normally distributed returns, and even methods that model volatility with GARCH or EWMA still carry model uncertainty in the tail parameters that a single point estimate hides. A Bayesian Value-at-Risk model, built by specifying a heavy-tailed likelihood such as a Student's t-distribution for portfolio returns, estimates the location, scale, and tail-thickness parameters jointly and propagates their uncertainty into the resulting loss distribution. The result is a probabilistic VaR estimate reflecting both future-return risk and model uncertainty, and it extends naturally to conditional VaR, scenario analysis, and stress testing built from posterior samples. Option pricing raises the same problem in a different form. A model like Black-Scholes takes fixed inputs, such as volatility, and returns a single price, but in practice volatility is uncertain and shifts over time. Treating volatility as a random variable, then simulating an option's payoff across thousands of paths drawn from its posterior distribution, produces a posterior distribution over the option's price, with credible intervals, instead of a single number. Despite these advantages, Bayesian methods remain less common in finance than in fields that adopted them earlier, a gap worth tracking as open-source tooling such as PyMC matures. ## Where this connects to testing a market decision before it ships Bayesian computation answers a modeling question: given historical data and stated assumptions, what is the plausible range of a financial parameter or price? Subconscious answers a different but related question: given a set of product, pricing, or messaging actions a team is choosing between, what would a target market actually do, and with what confidence? Both share the discipline of reporting a distribution instead of a single guess and stating the assumptions behind it. The research program applies that discipline to commercial decisions rather than portfolio and pricing models. [figure: Two fitted lines through the same skewed, heavy-tailed simulated data: the OLS line matches the true slope but its interval is miscalibrated; the Bayesian skewed-t fit's credible interval tracks the true uncertainty.] ## What are the limits of either approach? A posterior distribution is only as trustworthy as the prior and likelihood chosen to build it, and a poorly specified prior can bias the posterior just as a bad frequentist assumption biases a point estimate. Reporting a distribution instead of one number is progress only when the assumptions behind it are stated and checked, not treated as given. --- # AI Research for Enterprise Teams: Choosing the Right Tier of Rigor URL: https://subconscious.ai/blog/use-cases/ai-research-enterprise-teams-choosing-right-tier-rigor Published: 2026-08-24 An enterprise research function cannot staff a researcher for every product, marketing, sales, and strategy request that needs customer insight. The right response routes each request to the rigor level its stakes require: an open-ended self-serve AI session for a low-stakes gut check, a controlled causal experiment for a decision with real budget behind it, or a full fielded human study when the decision needs statistical proof across a defined population. [figure: Three-step path: no committed budget routes to a self-serve AI session; real budget with defined alternatives routes to a causal experiment; statistical or regulatory stakes route to a fielded study.] ## Why the routing decision matters Research demand inside a large organization grows faster than research headcount. Teams that cannot get a researcher's time within their decision window either wait until the decision is already made, skip research and guess, or run their own ad hoc questioning without a defined method. Self-serve AI panel tools remove the wait. A team asks an open-ended question, gets a fast synthetic-panel read, and moves on. That speed is the appeal and the risk. An open-ended AI session has no defined alternatives to compare and no defined population behind its answer. Treating that fast read as decision-grade evidence for a roadmap call, a pricing call, or a sales-strategy call is where the damage happens: the team spends budget and capacity acting on the decision, it fails, and the research function's credibility takes the hit across the organization. ## What causes the outcome? The failure is not that AI-assisted research is unreliable. It is a mismatch between the method and the stakes. An open-ended synthetic-panel conversation answers "what does this feel like to a plausible customer." It does not answer "which of these two options will more people actually choose, and by how much." Only the second supports a decision with real cost attached. A controlled causal experiment closes that gap. Instead of an open-ended conversation, it defines the specific alternatives under test and the specific population being asked, then measures which alternative drives the outcome and how confident that measurement is. ## Evidence Subconscious runs randomized experiments on a simulation of a defined market, validated against real human behavior, to show why people choose and which action drives the outcome. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (see the causal fidelity paper). /research documents how those experiments are structured and validated, and /case-studies shows the outcomes, context, and limitations of specific runs. Where scale matters, Subconscious can run controlled studies against a person-level audience graph covering 800 million real people. That audience graph is a modeling resource, not a recruitable panel of 800 million people available for real-human interviews. ## Comparing the three methods ## Recommended decision process Before a team commits budget or reputation to a decision, the research function should ask three questions in order: - Is this a low-stakes internal question with no committed budget behind it? An open-ended self-serve AI session is enough. A session like this typically turns around in 1-2 hours as a planning reference. - Does the decision commit real budget, headcount, or a public-facing commitment, and does it come down to choosing between defined alternatives for a defined audience? Run a controlled causal experiment. This is the governed middle tier: faster and cheaper than a full study, but structured enough to support the decision, unlike an open-ended session. - Does the decision require statistical proof across a large population, or does it carry regulatory, safety, or public-welfare stakes? Field a full human study. No simulation substitutes for that when the requirement is a fielded, recruited sample at scale. ## Where does Subconscious fit? Subconscious is built for the middle tier. A controlled discrete-choice experiment gives the research team a governed, repeatable answer (which alternative wins, by how much, and with what confidence) without asking every internal team to become research-literate first. When a decision depends on validation beyond simulation, a team can move from a simulated experiment to real-human testing or validation without changing the causal question it started with. That is useful when a finding from the governed middle tier needs to clear a higher bar before it reaches a board or a regulator. ## What are the limitations and failure conditions? A controlled causal experiment is not a substitute for a full fielded study when the decision legally or statistically requires one, for example when a claim needs to hold up across a large, recruited, real-world sample rather than a modeled population. It also does not itself provide the audit-trail or role-based access controls a regulated team may need around who requested and approved a study. And it does not replace the research team's judgment about which requests deserve which tier. Keep audience reach, simulated experiments, and recruited real-human participants distinct in every conversation about scale. A large audience graph is not evidence that a team can casually field real humans at that same scale on short notice. ## Putting a request on the right tier Two planning patterns are worth naming for teams sizing their own workflow: running a small number of real interviews (in the range of ten) and then extending that same line of questioning to a much larger simulated set (on the order of fifty) to check consistency, and keeping a standing panel of a handful of customer types (in the range of five to eight) that teams can query as an ongoing reference. A team with a live routing decision can see how Subconscious structures a controlled experiment at /how-we-work, or bring a specific request to a working session at /demo to size which tier it actually needs. --- # AI Customer Service Training with Simulated Customers URL: https://subconscious.ai/blog/use-cases/ai-customer-service-training-simulated-customers Published: 2026-08-24 Training customer service reps forces an uncomfortable trade-off. Classroom cases are safe but tidy. Live calls are realistic, but the customer pays for the agent's learning curve. A common compromise is two weeks of classroom work followed by a slow ramp under a senior agent's headset. The first ninety days carry the hardest learning. Simulation gives teams a controlled place to test training scenarios before those scenarios reach a real customer. [figure: Five-step chain: an agent's choice causes the scenario to branch, producing a scored transcript; the score is trusted only after trained agents are compared against a skipped cohort on live quality scores.] ## Why service training breaks down Three problems recur. Roleplays do not feel real. When two trainees play an angry customer and an agent, both know the script. Real callers interrupt, ramble, and change the subject. Live coaching is scarce. Senior agents split time between escalations and coaching, leaving little for repeated drills. Edge cases arrive late. A new hire may not see a rare regulated dispute, a bilingual call, or an unusually confused customer during the first month. When the case finally appears, the agent has not practiced it. Month one brings slow handle times and weak quality scores. Turnover tends to spike around that same ninety-day point, and understaffed onboarding is a documented driver of that early attrition. ## What should a useful simulation test? A useful simulation responds to the agent's choices. If the agent acknowledges the problem in the first thirty seconds, the scenario should change. If the agent moves straight to policy, the scenario should expose the consequence. The exercise should produce a transcript that a coach can inspect line by line. The same case should support repeated runs with different openings, questions, and escalation points, plus meaningful variation across customer contexts. The point is not to prove that a persona is a real person; it is to test whether a training action changes the behavior being measured. ## Scenarios worth practicing ### What does an angry caller scenario look like? The caller has been transferred twice, believes the bill is wrong, and wants to cancel. The agent has thirty seconds to acknowledge the frustration before diagnosing the problem. ### A billing dispute The customer believes they were overcharged while the account record shows the charge as correct. A workable script moves through four beats: verify what was billed, walk through why, name the surprise the customer is feeling, then lay out what happens next. ### Technical confusion The customer's description and the underlying issue differ. The agent must ask one or two well-placed questions without making the customer feel dismissed. ### A compliance edge case The exercise can test whether required language appears in the transcript. The scenario and scoring rules need review by the organization's legal and compliance owners before use. ### A non-native speaker The agent must slow down, reduce jargon, and confirm understanding. The exercise should measure the agent's adaptation, not judge the customer's language. ## What measures support coaching? Useful measures include de-escalation speed, acknowledgment before problem-solving, diagnostic accuracy, required-language adherence, and resolution-path quality. Each measure needs an explicit scoring rule. A transcript can show what happened; it does not establish that the score predicts live performance without validation against real calls. Simulation should complement the CRM, knowledge base, quality process, and human coaches. Text reduces the load while an agent learns the structure. Voice adds pacing, interruption, and tone. Teams often look for three patterns: faster progress toward acceptable quality, better preparation for unfamiliar cases, and lower early attrition. Those are hypotheses to measure, not guaranteed outcomes. Even hundreds of practice repetitions only matter if the practice transfers. [figure: Four steps: pick the worst-handled call types, run agents through repeated simulated drills, hold out a cohort that skips training, then compare live quality scores between the two groups.] ## A small evaluation design A first set of six to eight scenarios is buildable within a few days, though actual timelines shift with case complexity and who has to sign off on review. See how the design process runs for the underlying method. Start with the three calls the team handles worst. Put ten agents through five simulated runs of each one across a single week, then compare their live quality scores the following month against a cohort that skipped the drill. Set the outcome measure and the comparison group before training starts. Book a walkthrough to design an evaluation like this one around a specific team's worst calls. The causal question is simple: did the training action improve customer-service behavior for this group under these conditions? Simulation makes the practice repeatable. Validation against real work determines whether it helped. --- # Persona Documents, Persona Chat Tools, and Causal Buyer Tests URL: https://subconscious.ai/blog/comparisons/persona-documents-persona-chat-tools-causal-buyer-tests Published: 2026-08-24 Persona documents and persona chat tools both describe a buyer, while a causal buyer test measures whether a specific price, message, or feature change actually alters what that buyer does. A one-page persona document and a chat tool that answers questions "in character" both describe a buyer. Neither one tells a team whether a specific price, message, or feature will change what that buyer actually does. Before a launch, pricing, or messaging decision gets funded, the real question is not which persona format to use. It is whether the evidence behind the decision is a description or a test. ## The static persona document For two decades, the standard B2B and B2C planning artifact has been a one-page persona: a name, a role, a stock photo, a short list of pain points, and a set of preferred channels. A team builds it once in a workshop, stores it in a shared drive, and references it in planning meetings. In the persona-tooling industry, that workshop-to-finalization cycle has commonly run several weeks per persona, and teams have described the resulting document as stale again within about six months, at which point refreshing it means repeating the workshop. The document earns its keep as a shared shorthand: it gives a cross-functional team one name and one story to organize around. It was never designed to answer "how would this specific segment react to this headline," and using it that way asks it to do a job it cannot do. ## What is a persona chat tool? A newer category of tool replaces the static document with a conversational agent built from a demographic, behavioral, and psychographic profile. Instead of reading a page, a team asks it questions, follows up, and treats the responses as a stand-in for what a target buyer might say. Industry vendors in this category describe fast setup, incremental profile updates between sessions, and support for running several simulated respondents together to see a spread of answers rather than one. Academic work on this kind of language-model-simulated sampling has found that model-generated response distributions can track human survey and experimental responses on some stated-preference and economic-decision tasks, though the fit varies by task and population and is not universal (Argyle et al., 2023; Horton, 2023). That is evidence about how closely a language model's simulated answers can track a human sample on certain questions. It is not evidence that any specific chat persona has been validated for the launch, pricing, or messaging decision a team is about to make with it. ## What neither approach proves A persona document was validated by whether the team agreed it felt right. A persona chat tool is typically validated by comparing its aggregate answers to a published research benchmark, not to the specific segment, decision, and outcome the team cares about this week. Both are more description than proof: one tells you who the team believes it is building for, the other tells you what a simulated version of that buyer says when asked. ## The cost of treating either one as evidence When a team greenlights a headline, a price change, or a feature bet because a persona document "would agree" or a persona chat tool "said" a certain thing, it is extrapolating from a description to a decision the description was never built to answer. If the guess is wrong, that shows up after the campaign has run, the launch has shipped, or the roadmap quarter is already spent, and the budget along with it. ## Testing the actual decision Subconscious's fit for this problem is narrower than either persona format: run a randomized, controlled experiment on a simulated population to estimate which action, a price, a message, a feature, causes which outcome, for a defined segment, with uncertainty reported where the study design supports it. That reframes the buyer's question from "what would this persona say" to "does this specific action move this specific outcome, and how confident can the team be in that estimate." Subconscious can test or validate studies with real human participants. When a decision needs that stronger form of evidence, a team can move from a simulated study to real-human validation without changing the underlying causal question being asked. ## When is a persona description still the right call? A persona document, or a lightweight persona chat tool, is still the cheaper and faster choice for a kickoff deck, a stakeholder-alignment poster, or any deliverable whose job is "this is who we believe we are building for" rather than "will this specific change work." If the team is not going to act on a measured result, or procurement cannot support a testing vendor, a document requires none of that infrastructure. ## When is a controlled test the right call? The controlled test becomes worth the switch once a team is making a repeated, specific, budget-bearing decision against a defined segment, not orienting around one. A launch, a pricing change, or a message rewrite carries a real cost if the team is wrong. Consensus in a workshop, or a plausible answer from a persona chat tool, is not the same evidence as a designed experiment against that segment. ## Limits of a causal experiment here A controlled experiment does not replace the work of defining the segment, and it does not produce a browsable, always-on chat agent for a stakeholder wall. It answers the specific question it was designed to test, not every open-ended question a persona description might be asked. Real-human validation strengthens a specific causal estimate; it does not turn the study into an observed usability session, a clinical trial, or a guarantee of market performance. [figure: Four stages: a persona document and a chat tool describe a buyer without testing change; a controlled test estimates a change's effect with uncertainty; real-human validation checks it against real people.] Most organizations will keep some form of persona description around for planning and onboarding. The open question for any specific launch, price, or message decision is whether that description is being asked to do a test's job. When it is, see how Subconscious designs a causal test for that action, review worked decision examples, or start with a working session. --- # What Is Customer Simulation? 4 Use Cases and When Each Needs Human Validation URL: https://subconscious.ai/blog/use-cases/what-customer-simulation-4-use-cases-when-each-needs-human-validation Published: 2026-08-24 Customer simulation uses AI to model how a group of buyers thinks, reacts, and decides, without recruiting real people first. A calibrated persona, or a panel of them, responds to a concept, script, or pricing scenario the way that segment tends to, giving a team a directional read before committing budget to a full study. The open question is which jobs simulation can do alone, and which still need a real-human check. ## What Customer Simulation Actually Is A customer simulation is a model of a customer segment built from behavioral data, psychographic detail, and domain knowledge, then queried the way you'd query a person. Three things separate a real simulation from a chatbot wearing a persona prompt: - Segment specificity. The model is calibrated to a defined customer type, not "an average person." - Behavioral consistency. The same persona, asked a similar question in a different session, holds the same priorities, beliefs, and objections. - Validation against real outcomes. A trustworthy platform shows simulated answers reproducing a real human study, not just sounding fluent. Reliability varies by question type (Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents, arXiv). "We find that persona prompting does not yield a clear aggregate improvement in survey alignment and, in many cases, significantly degrades performance." Taday Morocho and colleagues, arXiv preprint 2602.18462 (source) ## 4 Use Cases Buyers Bring to Customer Simulation, and Which Ones Need a Validation Check [figure: A decision path showing four customer-simulation use cases: market research, sales coaching, and service training flow directly to use, while hiring and assessment routes through a required human validation step first.] Buyers searching for customer simulation usually mean one of four jobs, each carrying different risk if the team acts on a wrong read. ### 1. Market Research and Customer Insight The largest use case by spend: concept testing, message testing, pricing sensitivity, brand tracking, segmentation checks, and early buyer-journey mapping. The output is directional, useful for narrowing a wide set of options to the ones worth a closer, statistically grounded look. ### 2. Sales Coaching and Roleplay Reps practice against simulated buyer types: skeptical procurement, technical evaluators, security-focused stakeholders, price-sensitive owners. The simulation raises realistic objections and produces a rubric a manager can review. This is rehearsal, not prediction: lower-risk, because a bad rep answer surfaces a gap before a live call does. ### 3. Customer Service and Support Training Contact centers use simulated customers to train agents on de-escalation, empathy, and script adherence. Difficulty tunes from a calm billing question to an unresolved issue three contacts deep. Like sales roleplay, this is training: agent behavior improving over time, verifiable against call outcomes. ### 4. Hiring, Assessment, and Behavior Modeling Some vendors put candidates through a simulated discovery call or escalation and score the resulting behavior. A related use runs in academic and policy research, modeling how a population might respond to a price change or messaging shift at a scale recruitment can't reach. A number without its limits is marketing, so the risk on this use case gets named plainly here. This is the highest-risk use case in this list. A hiring decision made from a simulated interaction affects a real person, and hiring-related AI assessment carries added regulatory scrutiny. Treat a simulated score as one input, not the deciding one. ## What Separates a Trustworthy Platform From a Plausible One Fluent output alone isn't a signal. Five things to check first: - Calibration. Built from your audience's data, or a general-purpose model with a persona prompt on top? - Validation. Does the platform publish how it measures accuracy, and against what ground truth? - Panel structure. Can personas disagree in a group setting, or is every interaction one-on-one? - Auditability. Can you trace why a persona answered the way it did? - Workflow fit. Does the output move into the tools your team already uses, or stay locked in one interface? ## How a Customer Simulation Is Built A simulation has three working layers. The data layer combines public segment data, private customer data such as CRM records and interview transcripts, and structured psychographic detail. The modeling layer pairs a language model with rules that keep responses consistent with documented buyer behavior, producing stable objections and a traceable reason for each answer. The interaction layer is what a user sees: chat, a panel room, a survey, a voice call, or a roleplay scorecard. It's the most visible layer, though the modeling layer determines trust. ## Customer Simulation Compared to Traditional Research The pattern buyers converge on is sequencing, not replacement: run simulation first to sort which questions need a human study, then commission the smaller study where statistical confidence matters. ## Where Simulation Still Needs a Human Check Customer simulation doesn't replace talking to real people. It produces a directional signal built from a model of behavior, not the behavior itself. Decisions that require statistical certainty, a major repositioning, or a regulatory submission still need human research. This gap belongs on the record because a buyer checking sensory use cases needs to see it before relying on the tool there: it also can't observe anything sensory, including taste, scent, physical fit, or in-person reaction. In hiring and assessment specifically, a simulated score should never be the sole input to a decision about a real candidate. ## Proving the Result Before You Act On It The useful question is whether a result would reproduce what an actual human study finds, not whether it sounds plausible. Subconscious runs controlled causal experiments on simulated populations. The misses sit on the same public record as the hits, and that is what lets a buyer check the claim: our best configuration reaches 87% of the measured human ceiling on one study, at 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, drawn from roughly 300 replicated human studies across 9 domains (When Can Digital Personas Reliably Approximate Human Survey Findings?, arXiv). It is a validation result, not a guarantee for a new market. A question close enough to matter can move from a simulated population to real-human participant validation without changing the underlying causal question. See how Subconscious runs a study. Subconscious's person-level audience graph covers 800 million real people: the pool a simulated study draws its calibration from, not a claim that 800 million people answered the study. See current research. ## Frequently Asked Questions ### How does customer simulation differ from a chatbot? A chatbot is a language model steered by a system prompt. Customer simulation is a model of one segment's behavior, calibrated against real data. ### How accurate are customer simulations? Accuracy depends on question type and how rigorously the platform validates against real human outcomes. Treat any simulated answer as directional until checked against a validated benchmark or a real-human study. ### Can customer simulation replace traditional market research? For directional decisions, simulation is usually enough alone. Decisions requiring statistical certainty, a regulatory submission, or a large, hard-to-reverse spend still call for human research. ### Is customer simulation regulated? Most use cases carry little to no regulatory burden today. Hiring and pre-employment assessment are treated as higher-risk in some jurisdictions, meaning added scrutiny on audit trails and bias documentation. ### Who is customer simulation for? Marketing and insights teams, product managers, sales-enablement and service-training leads, and anyone who wants a directional read before a full study. ## Where to Start Pick one decision your team is debating, run it through a simulated panel that matches the relevant audience, and compare the output against what you already have. For hiring or any regulated decision, plan for a real-human validation step before the result changes what you do. See worked examples in case studies or book a walkthrough via a demo. --- # A Same-Day Triage Layer for Consumer Insights: When to Test Fast, and When to Wait for Humans URL: https://subconscious.ai/blog/implementation-and-procurement/same-day-triage-layer-consumer-insights-when-test-fast-when-wait-humans Published: 2026-08-24 A stakeholder wants a positioning decision by end of day. The insights team's fieldwork process runs on weeks, not hours. That gap is what pushes teams toward a guess, an internal opinion, or a shallow AI prompt with no audience grounding behind it. The way out is not to skip rigor. It is to sequence it: run a fast, controlled experiment first to eliminate the weak options, then send only the strongest candidate to recruited human participants for the validation that actually carries weight in a board deck. ## Why does the fieldwork clock keep colliding with the decision clock? Traditional consumer research is built around recruitment, scripting, fielding, and manual coding, and each stage adds real calendar time. Independent estimates of end-to-end market research timelines put full studies at multiple weeks once questionnaire design, sample recruitment, data collection, and analysis are sequenced (Drive Research; Nexus Expert Research). Product, brand, and marketing decisions do not wait for that timeline. When the research arrives after the choice has already shipped, insights stops being a decision input and becomes a postmortem. The failure mode that follows is predictable: the team routes around research entirely, using internal assumptions or a single unstructured AI query with no defined audience or controlled comparison behind it. Neither replaces a study. Neither tells anyone which action is more likely to change behavior. ## Is a triage layer a replacement for fieldwork? The fix is neither forcing every research question through the slow path nor abandoning recruited human research. It is splitting the work into two tiers with different jobs. The left column is where speed pays off because a wrong answer costs little: you iterate again. The right column is where the cost of being wrong is a launch decision, a regulatory filing, or a number that goes in front of the board, and only recruited human research carries that weight. ## Structuring the fast pass so it survives scrutiny A same-day-style triage pass only holds up under review if it stays disciplined: - Turn the stakeholder ask into a testable brief. Name the target audience, the competing hypotheses (for example, does the audience respond more to a functional-benefit claim or a transparency claim), and the exact stimuli to compare. Vague requests produce vague experiments. - Run the comparison as a controlled experiment, not a survey of opinions. Expose the same population to each alternative and estimate which one moves the outcome that matters, along with the objections and preferences that come with it. - Read past the top-line preference score. The reasons behind a rejected concept, and the specific language the audience used, are usually more useful for a rewrite than the win/loss number itself. - Iterate on the losing concept before moving on. A hybrid or revised stimulus that directly answers the objections raised in the first pass often outperforms both originals. Because the pass is fast, this loop can run more than once before the deadline. - Write the decision memo with confidence labels attached. State plainly what the simulated pass supports directionally, and what still needs recruited human validation before a launch, pricing, or regulatory decision is finalized on top of it. ## What evidence does a triage layer need before anyone trusts it? Speed is worthless if the fast pass does not track what real people would actually do. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. A live simulation can be stood up in about eight hours, with experiments running in under five minutes once it is live. Those figures are what make a same-day triage pass defensible rather than a shortcut dressed up as research. Subconscious can test or validate studies with real human participants using the same causal question the simulated pass was built around, so escalating a shortlisted concept to human fieldwork does not mean starting the study design over. It means holding the question constant and swapping in a higher-stakes population once the field has been narrowed. See the replication methodology and how a triage pass fits into a broader research operation. ## What this changes for the budget A triage layer does not shrink the human research budget. It reallocates it. Instead of spending recruitment and fielding costs testing twenty rough ideas, a team can spend that budget confirming the two or three ideas that already survived a controlled comparison: fewer wasted study cycles on options that were never strong candidates, and a decision memo that states in advance exactly what still requires a human study before it is safe to act on. Teams weighing where this fits against an existing research stack can review prior decision evidence or walk through how the triage-to-validation handoff works. [figure: Path: a same-day ask becomes a testable brief, runs through a controlled fast experiment, narrows to one candidate, goes to human validation, ends as a labeled decision memo.] --- # Prediction Solves the Wrong Problem for Most Business Decisions URL: https://subconscious.ai/blog/methods-and-validation/prediction-solves-wrong-problem-most-business-decisions Published: 2026-08-24 A pricing change, a new message, or a product decision rarely fails because a model predicted the wrong number. It fails because the team had no way to state what the model didn't know, and no way to diagnose the miss after the fact. ## Prediction and inference answer different questions Data science spent much of the last decade over-emphasizing prediction-focused machine learning. Algorithms like deep nets and random forests are strong at extracting non-linear patterns from large data sets. That is not the problem. The problem is that most business questions are not prediction problems. They are causal inference problems: what would happen if we changed the price, the message, or the product, an effect identified by randomized experimental design, not by model structure or priors alone. A model can be excellent at pattern-matching a training set and still have nothing to say about an action the business hasn't taken yet. ## Why does domain knowledge count as model input? A team deciding on price, positioning, or a product change usually already knows something: which variables plausibly relate to which outcomes, what range a parameter should fall into, which segments behave differently. Prediction-focused ML has no slot for that knowledge. It has to relearn structure from data alone, which is part of why it needs so much data to begin with. An inference-based approach takes the opposite path. It encodes what the team already knows into the structure of the model, then estimates the parts that are uncertain, rather than treating every relationship as equally unknown. [figure: A two-column comparison table contrasting prediction-only machine learning with inference built for a decision across four rows: output type, how domain knowledge is used, data requirement, and what ships to leadership.] ## Where has this approach already worked? A widely cited example is epidemiological forecasting during COVID-19, where researchers modeled the effect of interventions on disease spread and produced explicit uncertainty bounds rather than a single predicted curve (Flaxman et al., Nature). The value wasn't a sharper point prediction. It was a model that encoded epidemiological structure and reported a posterior interval conditional on that structure, in a situation where the data alone were noisy and incomplete; those NPI-effect estimates later proved highly sensitive to model structure and priors, with weak separate identification of individual interventions. The same shape appears in any setting where the team has strong priors about structure and comparatively sparse or noisy data: polling, planetary detection from telescope signal, and Subconscious's own domain of testing a marketing or product action before it ships. ## How does Subconscious apply this to go-to-market decisions? Subconscious's causal action testing builds a structured, discrete-choice-style comparison across pricing, messaging, and product actions, rather than asking a generic model to pattern-match past behavior. The team states the actions under consideration, and the experiment is built to estimate the causal effect of choosing one action over another. That output is explicit uncertainty framed qualitatively around the estimate. It is not a claim that every study design produces a formal confidence interval; the strength of the uncertainty statement depends on the specific study configuration. ## What this does and doesn't replace Two boundaries keep this framing honest. First, a structured causal experiment is not the same thing as recruited real-human research. Subconscious can test or validate studies with real human participants, and a team can move from a simulated experiment to real-human validation without changing the underlying causal question. Even so, the simulated result and the human-validated result are distinct claims, not interchangeable ones. Second, audience-graph reach, a person-level graph of the market, is not the same as a recruitable research panel. Scale in the graph describes how broadly a market can be represented; it does not by itself validate any single study's estimate. ## The practical difference for a leadership review When a pricing or positioning decision goes to leadership, the question that actually gets asked is rarely "what does the model predict." It's closer to "how sure are we, and what would change our mind." A model built for inference is built to answer that question directly. A model built only for prediction has to be reverse-engineered into an answer it was never structured to give, usually after the decision has shipped and something has gone wrong. Before committing budget to an irreversible go-to-market call, it's worth checking whether the model behind it was built to state its own uncertainty, or only to produce a number. See how Subconscious tests actions before they ship. --- # AI Brand Tracking: Brand Health Without Surveys URL: https://subconscious.ai/blog/methods-and-validation/ai-brand-tracking-brand-health-without-surveys Published: 2026-08-24 A brand or marketing leader who commissions a quarterly tracking wave knows its worst failure mode: a competitor moves, a message lands badly, or a public event shifts perception, and the next scheduled report arrives weeks after it mattered. The real choice is not whether to keep the quarterly wave, but what to do with the gap a wave-based program cannot answer when a specific event lands. [figure: Timeline of two scheduled quarterly tracking waves with a gap between them. A market event lands in the gap and routes to a scoped causal test, which can escalate to real-human validation before informing the decision.] ## Why the wave still leaves a gap A standard tracking program surveys a representative sample, commonly 500 to 2,000 respondents, once a quarter, reporting awareness, consideration, preference, NPS, and brand-attribute associations against the prior wave. That design is built for a defensible board-level number, not for a question that changes shape week to week. A shift that starts in week two of a quarter may not surface until the week-six report, by which point the response window has closed. A sample sized for the total market gets thin once it is cut into a segment of interest: a subgroup of urban professionals aged 25 to 34, for instance, can leave a base of roughly 80 respondents with wide confidence intervals. And a locked questionnaire cannot add a question mid-wave when a competitor does something the design did not anticipate. None of this argues for dropping the wave; the sample-based report is still the number leadership can defend. It argues for a second instrument covering the interval the wave misses. ## What goes in the gap The interval between waves is not empty; it is unmeasured. Filling it with an unvalidated always-on feed trades one risk for another: acting on noise costs budget on a reaction that never happened, and a genuine shift missed because nobody looked still ships a response months late. The better fit is a bounded, decision-specific test: pick the one action, such as a campaign concept, a repositioning message, or a response to a competitor's move, and run it as a controlled experiment rather than a running feed. Subconscious runs that kind of test as a causal behavioral experiment against a simulated audience built to reflect the buyer segment. The output is a measured effect within the simulated population, with an interval scoped to that simulated population, not a claim to have read the market's mind. When the decision is big enough, the same design can move to real human participants for validation without changing the question. See how we work for that escalation path. ## What should you measure once the gap is filled? A useful check in that gap covers a short list of outcomes, whichever action triggered it: Unaided recall. Which brands a respondent names first without a prompt, via a separate open-ended elicitation; useful as a signal of prominence in the simulated population, not a population awareness estimate. Attribute association. Which qualities attach to the brand, and whether they still match the intended position. Relative standing. Where the brand sits against a named alternative once a buyer chooses. Language in the room. What words or reactions a respondent reaches for; useful as a signal, not a precise population estimate. Message effect. Whether an exposed group differs from an unexposed one on recall, association, or stated choice, the exact shape of question a controlled test answers. ## When should you run the check? A scoped test is usually triggered by one of three things: a competitor launches something, a message or creative concept needs a read before it ships, or a segment-specific question comes up that the total-market wave can't answer alone, such as how Gen Z buyers read a positioning differently from an older cohort. Running the check before and after a launch, not just once, turns the result into a measured before-and-after instead of a single snapshot. [figure: A list of five outcomes a scoped causal test measures: unaided recall, attribute association, relative standing against a named alternative, language in the room, and message effect between exposed and unexposed groups.] ## What does this test not replace? A scoped causal test between waves is not a substitute for the quarterly program's sampling and its defensible report to leadership, nor a persona-based conversational panel standing in for it; a conversation with a simulated respondent answers a specific question, not the population-level number a board expects. Subconscious does not package an always-on brand-monitoring product. It answers one action-specific question where the wave has nothing to say yet. Two instruments, two different jobs: quarterly research for the defensible baseline, a scoped causal test for the decision that cannot wait. External sources make a similar case for reach without the survey wait: see Pulsar's overview of live brand signals for one industry framing of that gap, and Kadence's explainer on discrete choice modeling for the method behind testing a specific action rather than an open-ended preference. When the decision is worth the rigor, a case study or demo shows the design applied to a specific brand action. --- # Synthetic Dialogue or Real Interviews: What Can Justify a Market Decision? URL: https://subconscious.ai/blog/comparisons/synthetic-dialogue-real-interviews-what-justify-market-decision Published: 2026-08-24 A synthetic panel can help a team explore how a simulated customer might respond. An automated interviewer can capture what a real respondent says. Neither transcript alone establishes which concept, price, or message will cause a better market outcome. A launch decision needs evidence matched to the action and the cost of choosing poorly. ## Match the method to the decision The useful distinction is not synthetic versus human. It is the question each method can answer. Generated dialogue is not a record of what a recruited customer said, a limitation also made explicit in this first-party case for real customer research. ## Discovery and launch approval need different evidence An automated interviewer questions real respondents live. Its transcript records genuine testimony rather than generated dialogue. That is the right shape of evidence when the team needs customer language, unexpected objections, or open-ended discovery. Publishing the failure mode lets a buyer check the method against it before trusting it. The limit is the say-do gap. A respondent can explain a preference clearly without revealing what would happen when the team changes one element and holds the others constant. Interviews help define the hypothesis. They do not automatically estimate the causal effect of a price, concept, or message. The cost of confusing those jobs is material. Generated conversation can sound authentic even when no customer said it. Real testimony can sound decisive even when the study did not isolate the action. Either mistake can put launch budget and the go-to-market window behind the wrong choice. ## What question does a controlled comparison change? Subconscious is a causal behavioral platform, not a synthetic conversation panel or an interview tool. It runs controlled experiments on a simulation of the market and compares specific alternatives head to head. The question changes from “What does this respondent say?” to “Which action changes the outcome when the alternatives are tested under controlled conditions?” [figure: Four-step path: synthetic dialogue and real interviews both feed a hypothesis and language stage, which feeds a causal experiment comparing defined alternatives, which feeds an optional real-human validation step.] This method is strongest after discovery has produced a concrete choice. The team supplies the decision, target audience, alternatives, and outcome; the platform does not generate the hypothesis. ## What can the evidence carry and not carry? Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. A number without its limits is marketing, so the miss sits next to the hit. It is a validation result, not a guarantee for a new market. The method and results are available in the causal fidelity paper and on the research page. Subconscious can also test or validate studies with real human participants. A team can move from simulation to real-human validation without changing the causal question: the alternatives and outcome stay the same, only the source of behavioral evidence changes. Controlled studies can use a person-level audience graph covering 800 million real people. The reach figure is published with what it does not cover so a buyer can check it against the claim. That number describes audience reach. It is not a recruitable panel or a promise that every study includes that full population. Naming what a tool cannot do is what lets a buyer match it to the right job. Subconscious does not produce open-ended interview transcripts and is not the right tool for capturing verbatim customer voice. Confidence intervals, segment heterogeneity, and willingness-to-pay are study-specific outputs, not defaults for every study. ## What is the right sequence for each method? Use synthetic dialogue to widen the hypothesis space. Use real interviews to hear customer language and sharpen the alternatives. Use a causal behavioral experiment when the team must choose between defined actions. For a consequential decision, add real-human validation while preserving the same comparison. The practical next move is to write the decision as alternatives, audience, outcome, and cost of being wrong. The study workflow shows how that comparison is designed. When the choice is concrete enough to test, scope the decision. --- # Recollective vs a Causal Testing Platform: Matching the Tool to the Decision URL: https://subconscious.ai/blog/comparisons/recollective-vs-causal-testing-platform-matching-tool-decision Published: 2026-08-24 A team choosing a customer-research tool is usually choosing between three different jobs: a moderated online community, a fast AI conversation tool, and a controlled experiment that tests which action actually changes behavior. Picking the wrong one costs a launch cycle, a mispriced product, or a feature investment that never moves the outcome it was built to test. ## What does a moderated community platform buy? Recollective is a qualitative research and insight-community platform built for teams running structured, multi-session research programs: bulletin-board discussions, video diaries, and ongoing member engagement managed by a research team over an extended cycle. That format produces depth: a moderator can prompt participants repeatedly and steer the conversation toward emerging themes. Third-party reviews of Recollective on G2 place it in the same category as other community-research tools: something a dedicated research function operates as part of an established workflow, not a self-serve tool for a single fast question. The tradeoff is timeline and staffing. A community program needs a defined research question, a moderator, and a cycle measured in days to weeks before there's an answer to act on. That cost is justified when the goal is qualitative depth, ongoing member relationships, or a research program with its own team and budget. It fits poorly when a team needs one specific action tested before a decision this week. ## What does a fast AI conversation tool buy, and what does it miss? The other end of the category is a self-serve tool where a team member talks with an AI-generated stand-in for a customer type and gets a same-day response. That speed doesn't solve the evidence problem: a conversation, however fast, still returns stated opinion: what the AI-generated respondent says it would do, not a measured behavioral outcome. Aggregating several such conversations into a summary does not turn stated opinion into a test of whether a specific price, message, or feature actually changes what a buyer does. ## The question underneath: opinion or a measured cause Neither a moderated community nor a fast AI conversation tool is built to isolate cause. Subconscious runs randomized, controlled experiments on a simulated market and reports which action moves a defined behavioral outcome, with uncertainty, rather than aggregating stated opinion. That distinction matters most at the moment of the decision itself: shipping a price, message, or feature change built on community sentiment or AI-conversation output that reads as consensus, but doesn't predict what buyers actually do. Subconscious studies can run against a person-level audience graph covering 800 million real people, kept distinct from separately recruited real-human validation studies, which Subconscious can also run when a team needs to confirm a result against real participants without changing the underlying causal question. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. ## Where does a causal experiment not fit? Subconscious is not a moderated community platform or a chat interface for open-ended conversation: it doesn't run multi-week bulletin-board discussions, video diaries, ongoing member engagement, or general-purpose qualitative exploration. A team that needs a standing community, or an unstructured exploratory conversation with no defined action to test, should look at a platform built for that job. ## A practical test before choosing Before picking a tool, name three things: the specific action being tested (a price, a message, a feature), the outcome that action is supposed to move, and how fast the decision needs an answer. If the honest answer is "we need ongoing qualitative engagement with a defined member group," a community platform like Recollective is the right tool. If the honest answer is "we need to know which action moves behavior, and we need it this week," that calls for a controlled experiment. See how the method works in how Subconscious runs a study, review published research and the leaderboard of validated results, or book time to scope one specific decision. [figure: Three columns: a community platform returns stated opinion over days to weeks; an AI tool returns stated opinion same-day; a causal experiment measures an action's effect on an outcome in under five minutes.] --- # The Consumer Analyst's Decision: When Directional Reads Aren't Enough URL: https://subconscious.ai/blog/buyers/consumer-analysts-decision-when-directional-reads-arent-enough Published: 2026-08-24 A consumer analyst loses ground to AI not by using it, but by treating a fast, plausible-sounding answer as proof and sending it forward as if validated. The decision that protects the role is narrower than "use AI" or "don't": know which consumer-behavior questions can stay at quick directional exploration, and which ones must clear a controlled experiment with a measurable causal effect before the business acts on them. [figure: Four steps: generate hypotheses with AI, compare rough options and label them directional, have a human check audience and sourcing, then run a controlled experiment for high-stakes decisions.] ## Why Is This Question Showing Up Now? The mechanical parts of research analysis (drafting a survey, summarizing a transcript, formatting a chart, producing a first-pass read) are getting faster and cheaper to produce. That does not eliminate demand for the analyst's judgment; it concentrates risk in work that stops at production. AI adoption is reshaping which skills employers reward, with a widening premium for workers who pair AI fluency with judgment-heavy work over routine output (PwC, 2026 Global AI Jobs Barometer). For a consumer analyst, the exposed work is dashboard maintenance and recurring summaries. The protected work is choosing which claim deserves trust, attaching the right caveat, and knowing when a directional read is not enough evidence for the decision at hand. ## What Changes in the Role The old version of analyst expertise lived partly in access: knowing how to field a study, clean the data, and package a finding. AI erodes that advantage; more people can now draft a survey or generate a first-pass customer read. That judgment gets easier to test, because the bar shifts from "can you produce an answer" to "can you say which answer deserves trust, and why." The analyst who can name the audience assumption behind a claim, spot a thin or ungrounded result, and explain what would change the recommendation does work generation alone cannot replace. ## What Is the Four-Layer Evidence System? A durable analyst practice runs on a clear system for what each tool is allowed to do: - Exploration. Use AI to generate hypotheses, objections, and alternative explanations quickly. - Directional testing. Compare rough options to see which ideas are worth pursuing. Label this output as directional, not decision-grade. - Human review. Check the audience definition, the framing of the question, the source grounding, and whether the result fits the business context. - Validation. When the decision is expensive, public, or hard to reverse, move to real evidence: fielded research, behavioral data, expert review, or a controlled experiment that returns a measurable causal effect. The output of steps 1 and 2 is not evidence itself: it is a set of candidates step 4 confirms or rules out. ## Where a Controlled Experiment Closes the Gap Stating what a quick read cannot prove keeps it from passing as decision-grade evidence. A quick exploratory pass can surface which concept, message, or price story looks more promising. It cannot tell the business, with a stated confidence level, that changing the price or the message changes the outcome, because it was never designed to isolate cause from correlation. Subconscious runs controlled discrete choice experiments and returns causal effects with confidence intervals, using named methods (McFadden DCE and Mixed Logit) rather than a fluency claim about any AI model. An analyst can run that experiment against a person-level audience graph covering 800 million real people, a defined population rather than a recruited group of human respondents. When the decision is expensive enough to need it, the same team can validate the study with real human participants, without changing the underlying causal question. That answers the labeling problem: "directional read, not yet validated" becomes "tested causal effect, with a confidence interval and known limitations." ## What Mistake Erodes Analyst Trust? The costly error is not using AI. It is sending a chart or a recommendation without explaining the behavior behind it, letting a fluent, fast answer stand in for evidence because the deck needed a conclusion by end of day. The fix is to make the limits part of the deliverable, every time: name the job the AI-assisted work did, flag the questions it cannot answer, and spell out what still needs validation before the business acts on it. This does not make an analyst sound less confident; it makes them more credible, because they can name exactly where their confidence stops. ## One Habit to Start This Week Do not redesign the whole workflow at once. Start with one live decision: - Pick a real project tied to an actual decision. - Write the business decision in one sentence. - Define the audience and how much is riding on being wrong. - Use AI only for the exploratory and directional stages. - Review the output manually and mark it useful, weak, or unsafe to act on. - Present the answer with a clear caveat and a named next validation step, such as a controlled experiment where warranted. Repeat that on one decision a week for a month. The output is not a longer list of tools. It is a working system that shows speed, judgment, and where the evidence stops. ## What This Does Not Replace A causal effect number needs its limits published to count as evidence. A controlled experiment tests the action once the analyst has framed the right question and defined the right audience. It does not choose the question, decide which caveat matters most to a stakeholder, or substitute for real-human validation when a decision is legally exposed or highly public. The analyst still owns the framing. Naming where a method stops is what lets a buyer choose the right tier for a decision. When the decision is small, reversible, and cheap to test in market, a well-labeled directional read may be all a business needs. Deciding which tier a given decision requires stays the analyst's call. [figure: Five-step chain: deadline pressure leads to a fluent AI answer standing in for evidence, sent forward with no caveat, acted on by the business, then trust erodes when the answer proves wrong.] ## Where to Go Next Review the Subconscious research program for how the method and human-baseline validation work, or see the current case studies for examples of a directional question becoming a tested pricing or messaging decision. To scope a specific decision, bring it to a working session or read how Subconscious works. --- # What Is Simulated Market Research? A Buyer's Guide to When to Use It URL: https://subconscious.ai/blog/use-cases/what-simulated-market-research-buyers-guide-when-use Published: 2026-08-24 Simulated market research runs a defined audience through research stimuli, such as a survey, a concept test, an ad, or a messaging variant, using models conditioned to respond as members of that audience would. You describe the audience, the platform generates the responses, and the output looks structurally like real-respondent data: quant scores, segment cuts, and open-ended responses with the texture of an interview transcript. The category also goes by synthetic market research and AI-driven market research. ## The decision this framing is meant to help you make The question a research leader actually faces is not whether simulation "works." It is which stage of a study to run as simulation and which stage still needs real human participants. Route the wrong stage to the wrong method and you either present directional signal as a defensible population estimate, or you burn weeks of fielding time on a question that only needed a directional comparison. ## How the workflow runs - Define the audience. Set the boundaries that will condition the responses: age range, geography, income, occupation, attitudes, prior brand exposure, and any other demographic or psychographic trait that matters for the study. - Generate the simulated panel. Build individual respondent profiles into a panel, typically somewhere between 50 and 500 simulated respondents, split across whichever traits the study needs to vary. This is a common vendor-side panel size, kept here as a planning example rather than a Subconscious specification. - Design the research instrument. Survey, concept brief, ad pretest, or open-ended discovery script. Build whatever you'd normally use to field the study with real participants. - Run the session. Push the stimulus out to the panel and let each simulated respondent reply; the output pairs numeric scores with qualitative color. - Synthesize, then decide what still needs a human. Read the themes, compare segments, and identify which concept or message earns a real-respondent check before it ships. ## Where simulation earns its keep - Concept screening. Narrow a long list of concepts to a short list worth fielding for real. - Message and ad iteration. Compare many wording or creative variants before committing budget to one. - Cross-market comparison. Run the same study against multiple national or regional audiences side by side. - Hard-to-reach audiences. Senior B2B buyers, regulated professionals, and other groups where real recruitment is expensive or slow. - Continuous discovery. Recurring pulses on brand perception or message resonance that keep a team looking at data between formal studies. ## Where simulation stops A number without its limits is marketing. These three limits hold regardless of which platform runs the simulation: - This miss sits on the public leaderboard next to the hits. It does not produce a defensible population estimate on its own. A simulated study returns directional signal, not a confidence-interval-backed claim about what a real population thinks. - It cannot reason past its training distribution. A genuinely novel product, service, or scenario with no precedent produces a plausible-sounding response with no real signal behind it. - It cannot register sensory or emotional response. A model can reason about a packaging design or a TV ad. It cannot see, hear, or feel one. Independent validation work on persona-conditioned model responses backs this pattern: they approximate real survey response on directional questions but are not a substitute, especially where the question depends on lived sensory experience or falls outside the training data's coverage (Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents). "We find that persona prompting does not yield a clear aggregate improvement in survey alignment and, in many cases, significantly degrades performance." Taday Morocho and colleagues, arXiv preprint "Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents" (source) ## Routing a study ## Where Subconscious changes the shape of this hybrid Subconscious runs controlled experiments, not open persona interviews. Teams define the audience, the alternatives, and the outcome, then run a randomized experiment against a person-level audience graph covering 800 million real people. The research page documents the underlying method. Because the intervention and the outcome are fixed from the start, a team can move from the simulated experiment to real-human validation without changing the causal question. The shortlist a team narrows through simulation gets checked against real response before a launch decision, not after. The case studies show this pattern applied to specific pricing and positioning decisions. This audience graph is not a recruitable panel; it is the population a controlled experiment draws its comparison from. ## A glossary of adjacent terms - Synthetic market research. Same methodology, framed around the generated-respondent angle rather than the simulation angle. - AI-driven market research. Same methodology, framed around the model dependency. - Silicon sampling. The academic line of work behind using model-generated responses as a stand-in for survey samples. - Multi-step simulated research. A newer extension where simulated respondents act and react across a sequence of scenarios rather than answering single prompts. ## Next step If the decision in front of you is choosing among concrete alternatives, price points, messages, or launch options, that is a controlled-experiment question. Read how Subconscious runs a study or see who's behind the method before deciding which stage of your next study to simulate and which stage to validate with real people. [figure: A path sorting study stages into two lanes: simulation-fit (concept screening, message and ad iteration, cross-market comparison) and real-human-required (regulatory or hero claims, novel categories, sensory response).] --- # Automotive Research: Test EV Buyer Decisions Before Launch URL: https://subconscious.ai/blog/industries/automotive-research-test-ev-buyer-decisions-before-launch Published: 2026-08-24 Automotive teams can use controlled behavioral experiments to compare EV propositions, pricing, launch messages, and dealership interventions before committing a model or campaign. The method works best when the team names one action, one alternative, a defined buyer group, and the response that would change the decision. A number published without its limits is marketing copy. This does not replace vehicle clinics, observed driving behavior, engineering and safety review, dealer evidence, or in-market validation. It is an earlier layer that rejects weak options before the expensive evidence begins. [figure: Two-track diagram: traditional cadence has four sequential stages totaling 11-13 weeks; the fast track shows one 24-72 hour pass repeating about twenty times in that span.] ## Which automotive decisions are suitable for an experiment? The strongest candidates are decisions where the team can vary a commercial action while holding the rest of the choice context stable. Subconscious research centers on that intervention, not a persona conversation. ## Why does the research cadence break? Major automotive studies remain necessary, but their cadence leaves teams without evidence between them. The figures below are an illustrative planning example, not Subconscious delivery times. Separately, industry design cycles that once ran seven years are giving way to 18-month timelines (Arm Newsroom, 2025-07-10). A three-month research round forces teams to decide on older evidence. A high-frequency testing lane can run alongside the foundational studies: one illustrative capacity plan budgets 24 to 72 hours per pass, twenty passes in the span of a single clinic. These are planning inputs, not a Subconscious service level. The question is which decisions need a controlled first pass before the team spends more. ## How should an EV buyer experiment be designed? Start with the action, not the persona. Define the buyer group tightly, then expose each group to the same choice context with only the intervention changed. Planning examples, not universal segments: pragmatic switchers, premium loyalists, reluctant switchers, and conquest buyers for a proposition study; first-time EV buyers weighing a 35,000 to 50,000 euro purchase for pricing; drivers of premium German combustion vehicles for a dealership scenario; a 200 to 1000 vehicle corporate fleet mid-transition for a fleet study. Substitute your own market data and decision threshold. The experiment then needs five explicit parts: - The action under consideration. - The alternative or control. - The buyer population and relevant segments. - The choice or behavioral response that matters. - The evidence that would confirm or overturn the result. Pricing needs extra care: scenario testing compares defined price and package alternatives; it is not price optimization, and commercial modeling still determines whether a price ships. ## What can teams test across the automotive journey? ### EV propositions and model-year communication Compare buyer response to range, driving experience, premium ownership, charging support, or the framing of a model-year update: battery package, infotainment, exterior refresh, price adjustment. The output should change a concrete decision: which proposition advances to a clinic, which claim needs evidence, which message dies before creative production. ### Dealership interventions The misses sit on the public leaderboard next to the hits, so a buyer can check the fit before committing budget. Text scenarios can compare how sales scripts explain charging, financing, service, or the test-drive step, but cannot reproduce cabin materials, acceleration, haptics, salesperson behavior, or showroom pressure. Use the result to select interventions for a dealer pilot, not to claim observed performance. ### Fleet and B2B offers An early experiment can compare residual-value communication, transition support, pricing structures, or B2B sales messages across defined fleet profiles. Naming the failure mode here lets a buyer check the claim against what the method can actually do. The result is a pressure test, not a replacement for direct fleet-customer or dealer-network relationships. ### Competitive positioning A controlled comparison can place the proposed vehicle beside a named Chinese competitor and an established European competitor, testing which proposition earns consideration and how that changes by buyer group. Current product facts and live market evidence still need separate verification. ## What does a 9-week launch plan look like? The scenario below is illustrative: not a delivery estimate, customer result, or current Subconscious package. A premium European OEM brings a mid-size EV to market, priced between 45,000 and 55,000 euros, with 9 weeks from brief to first asset live and controlled comparisons at four points in the cycle. Each test should answer a decision question; a fast result that changes nothing is not useful. ## Where does this fit in the research stack? Controlled experiments fill the gap between major studies; they do not replace them. When validation is warranted, Subconscious can test with real human participants, using the same intervention, population, alternative, and outcome, not a different study. For evidence that has passed the claim gate, review the case studies. ## What will the experiment not tell you? Publishing the limit alongside the result lets a buyer see exactly what the method cannot yet answer. It cannot reproduce a ride-and-drive, cabin feel, acceleration, engineering performance, safety, dealer execution, or actual market demand. It is weaker when a category is truly novel and buyers have no stable reference, and results are sensitive to elicitation, calibration, and study design. The right stop point is a prioritized action with explicit limits. High-stakes vehicle, pricing, and dealer decisions still belong in clinics, real-human research, field pilots, or in-market tests. ## What is the useful first step? Choose one upcoming decision with real cost: an EV proposition, a defined pricing scenario, a model-year message, or a dealership intervention. Write down the alternative, buyer group, target response, and evidence that would change the decision. Then scope the experiment and its validation path before building the full launch plan. --- # Choosing a Pre-Launch Validation Method for Messaging, Pricing, and Positioning URL: https://subconscious.ai/blog/use-cases/choosing-pre-launch-validation-method-messaging-pricing-positioning Published: 2026-08-24 A launch is a few weeks out and the team still has to decide whose messaging wins, what price the market will accept, and whether the target buyer actually cares about the positioning. Those calls lock campaign spend, sales enablement, and packaging into production. Get the underlying behavior wrong and the fix arrives after launch, slow and expensive to unwind. ## What is pre-launch validation for? Pre-launch validation exists to answer one question before production work hardens: will this specific buyer segment behave differently under alternative A versus alternative B? That is a causal question about a defined audience and a defined set of alternatives, not an open-ended reaction to a single concept. Historically, teams answered it two ways. Traditional qualitative and quantitative research recruits real respondents, runs structured interviews or surveys, and takes several weeks to reach a defensible read. Faster tools generate simulated personas and let a team interview them conversationally. Neither, on its own, isolates which specific change in positioning, price, or copy caused the buyer's decision to shift. ## How to evaluate a pre-launch method Before choosing a method, check it against the actual decision it needs to support: - Does it name explicit alternatives? A single stimulus with a free-form reaction cannot tell you why a buyer preferred one option over another; the method needs to test defined alternatives against each other. - Does it isolate cause from preference? A comment thread of persona reactions or open interview transcripts reflects sentiment. A controlled experiment that varies one attribute at a time and measures the resulting choice returns a causal effect, not just a preference signal. - Does it report uncertainty? A single number with no confidence interval cannot support a launch decision as well as a measured effect with a stated range. - Can the team run it without a research team as a bottleneck? Iteration speed depends on whether a product or GTM lead can set up and read a test without waiting on a specialist. - Does it distinguish audience reach from who actually answers? A large audience definition is not the same as a large number of recruited respondents; confusing the two overstates what a single test proves. ## What to test before launch Four assets are worth testing before they harden into production work: - Positioning. Put category framing, problem language, proof points, and the opening line's value proposition head-to-head across defined buyer segments, then measure which version actually shifts the buyer's decision rather than which one simply reads better. - Pricing and packaging. Frame willingness to pay, how buyers perceive plan boundaries, and discount sensitivity as a discrete choice among named packaging options, rather than an open question about a single price point. - Launch assets. Put landing-page hero copy, subject lines, ad concepts, and sales-deck opening slides in front of the same defined segments so results are comparable across touchpoints. - Segment divergence. Run the same comparison across buyer roles, regions, company sizes, and category maturity. One universal message is rarely enough; a shared core promise backed by segment-specific proof works better. ## How does Subconscious approach this decision? Subconscious runs a controlled discrete-choice experiment: buyers or precisely defined buyer segments choose between the specific positioning, pricing, or launch-asset alternatives under consideration, and the result is a measured causal effect with a confidence interval. Discrete choice experiments are an established survey-research method for isolating which product attributes actually drive a choice, rather than which ones simply get mentioned in a conversation (Drive Research). See /research for how these experiments are structured and validated, and /case-studies for worked examples. [figure: Five-step decision path: each step is a pass/fail gate a method must clear. Failing any one gate rejects the method for the launch decision.] When the decision warrants it, a team can move from a simulated experiment to real-human validation without changing the underlying causal question. Subconscious can test or validate studies with real human participants, and it can run controlled studies against a person-level audience graph covering 800 million real people. That figure describes the scale of the graph a study can draw against, not a count of people recruited into any single test. ## Comparing method types ## What are the limitations of this approach? A pre-launch causal experiment does not replace recruited, vehicle-specific market research for very low-incidence audiences, regulated decision-making, or genuinely emerging behaviors with no historical data to calibrate against. It is a simulated experiment, not real-human validation. Where the decision is high-stakes enough to need recruited human confirmation, run the same causal question through real-human validation before committing production spend. ## Next step Start with the alternatives already on the table: two positioning statements, two price points, two hero lines. Structure them as a discrete choice rather than a single concept for reaction. Book a demo to see how a controlled experiment is set up, or read how Subconscious works for the underlying method. --- # AI Simulation vs. Causal Experiments: Structuring Research for a Consulting Engagement URL: https://subconscious.ai/blog/use-cases/ai-simulation-vs-causal-experiments-structuring-research-consulting Published: 2026-08-24 A consulting engagement has two different research jobs, and they need two different methods. Early in a project, the team is forming hypotheses, prepping for a workshop, or anticipating stakeholder objections: speed matters more than statistical rigor, and a directional AI simulation of likely reactions is a reasonable tool. Later, when the team has to defend a recommendation in front of a client's board, the standard changes. A role-played reaction from an AI stand-in is not evidence a partner can put in front of a skeptical CFO. That stage needs a controlled experiment with a defined population, defined alternatives, and a measurable effect. Confusing the two is the actual risk. Presenting a directional, uncontrolled AI read as if it were a defensible causal finding collapses under client or board scrutiny, and the damage lands on the firm's credibility, not just the deliverable. [figure: Four-step path. Steps 1-3, stakeholder prep, market-reaction hypotheses, and stress testing, use AI simulation. Step 4, the client-facing recommendation, requires a controlled experiment.] ## Where consulting research time actually goes A typical engagement spends its time on: stakeholder interviews (identifying the right people, scheduling across time zones, running roughly 45-minute conversations, transcribing, and synthesizing; a strategy project typically runs 15 to 25 of these), expert interviews sourced through paid networks, primary research when the client's question requires data that doesn't exist in secondary sources, workshop preparation, and the final synthesis into a recommendation. Not all of this compresses well with simulation. Some of it does. ## Where directional simulation earns its speed Three points in a consulting engagement are well suited to a fast, directional AI read rather than a full primary study. ### Pre-workshop stakeholder prep Before a strategy workshop, consultants need a working sense of each participant's priorities and likely objections. That means pre-interviews with 8 to 15 stakeholders; scheduling alone eats into the timeline. Building simulated stand-ins for each stakeholder type (a CFO skeptical of unproven technology spend, a CTO frustrated with legacy systems, an operations lead wary of disruption) and running the proposed strategy past them surfaces likely objections before the room does. This does not replace real conversations with the 3 to 4 most senior stakeholders; it reduces how many pre-interviews are strictly necessary and sharpens the ones that still happen. ### What are early market-reaction hypotheses? When a client asks how target customers will react to a market entry move or a pricing change and wants an answer fast, a directional simulation across modeled buyer segments (the satisfied incumbent, the actively-shopping switcher, the price-sensitive buyer, the early adopter) gives the team a structured first read, including which segments look more or less receptive. It is a hypothesis-generation tool, not a market-sizing study. ### Pre-presentation stress testing Before recommendations reach the client's leadership, teams traditionally rehearse internally, with partners playing devil's advocate. Running the deck past simulated stand-ins for the actual decision-makers in the room, built on real stakeholder characteristics rather than a colleague's guess at what the CFO would say, surfaces weak points in the argument before they surface in the boardroom. ## Where a controlled experiment is required The pattern above only holds for internal preparation. The moment a finding leaves the building and becomes the basis for a client-facing recommendation, the bar changes. A role-played reaction from a simulated stand-in answers "what might this type of person say," not "what would this population actually choose, and by how much." Subconscious runs a controlled discrete-choice experiment: a defined population evaluates defined alternatives, and the result is a causal effect with a confidence interval, not a scripted reaction (see the causal fidelity paper). When the client's decision is large enough to need defensible evidence (a pricing move, a positioning claim, a market-entry bet), that is the stage that needs it. Where audience scale matters, Subconscious can also run studies against a person-level audience graph covering 800 million real people, which is a reach claim about the graph, not a claim about how many people are recruited into any one study. ## How simulation and controlled experiments fit together ## Practical considerations The quality of the simulated stand-ins matters more here than in consumer research. Consulting applications need more depth than a generic buyer profile: organizational role and internal politics, decision-making authority, known positions on similar past issues, and communication style. That input has to come from public material (LinkedIn profiles, published interviews, company reports) and the project team's own prior knowledge of the client, not invention. Treat the shift from simulation to experiment as a pilot, not a wholesale swap. Run the directional read in parallel with the traditional method on one project, compare the two, and only fold simulation into standard methodology once it's clear where it usefully narrows scope rather than replacing judgment. Moving from a simulated read to a defensible one does not mean changing the question. When a directional hypothesis needs to become client-facing evidence, a team can move from a simulated experiment to real-human validation without changing the underlying causal question being tested. ## What are the limitations of a controlled experiment? A controlled causal experiment does not replace stakeholder interviews, expert calls, workshop facilitation, or a consultant's own synthesis and narrative judgment; those remain human work. It also does not run as an open-ended conversation with a persona. See /research for how Subconscious structures a controlled experiment, and /case-studies for how the method has been applied. For engagements where a recommendation has to survive board-level scrutiny, /how-we-work covers the process end to end, and a demo is the fastest way to see where a client-facing question in a live engagement fits. --- # Customer Simulations for Hiring Assessments URL: https://subconscious.ai/blog/use-cases/customer-simulations-hiring-assessments Published: 2026-08-24 An interview shows how well a candidate interviews. It does not show how the candidate handles a frustrated enterprise customer at 4pm on a Friday. Customer simulation adds a controlled work sample to a hiring process, the same discipline behind how Subconscious runs a controlled comparison for a marketing or product decision. Every candidate receives the same scenario, customer context, and scoring rubric. The exercise does not replace structured interviews, references, accommodations, legal review, or a human hiring decision. [figure: Five steps: define role and scenario, give every candidate the same prep and time, score the transcript on preset behaviors, a human decides, then check ratings against job outcomes after ten hires.] ## Why do hiring assessments vary? A 2023 planning estimate put the total damage from one bad customer-facing hire, including lost business, customer churn, and team strain, at roughly 1.5 times that employee's annual salary (Apollo Technical, checked 2026-07-27). A single number without its limits is marketing copy. Treat that figure as an example, not a universal cost model. Three sources of variation make customer-facing assessment difficult: Interviewer variation. Multiple studies comparing structured and unstructured interviews report 30 to 40 percent scoring variance across interviewers, even when they use the same rubric. Scenario inconsistency. A roleplay changes as the evaluator warms up, tires, or adjusts after the first three candidates. Candidate eight and candidate ten may face different levels of difficulty. Self-presentation. A candidate can rehearse stories about difficult customers. Recalling one moment is not the same as managing that moment in real time. Naming a failure mode in public is what lets a buyer check it. More interviews do not remove these problems. A standardized work sample reduces scenario variation but introduces its own measurement and fairness obligations. ## What does a controlled simulation workflow involve? Define the role and one representative scenario. A sales candidate might run discovery with a skeptical buyer. A customer-success candidate might handle a renewal conversation after two promised features were delayed. Give each candidate the same preparation and time. A conversation may run 20 to 40 minutes. Capture a transcript, then score only the behaviors defined before the first candidate begins. The candidate should know that the customer is simulated and that the exercise is assessed. Human reviewers should inspect the transcript and make the decision. Technical failures need a retry rule. Accommodations and working-language needs must be part of the design. ## Four roles where the method can be useful ### Sales A typical exercise gives the candidate a 5-minute product brief and 30 minutes with a skeptical buyer. Observe whether the candidate discovers the problem before pitching, handles an early price question, and confirms a next step. ### Customer success A renewal or escalation exercise can reveal whether the candidate listens before defending the company. This write-up puts the misses next to the hits. A 30-minute simulation may expose more job behavior than five hours of discussion about past behavior, but the team must validate that claim against its own outcomes. ### Customer service A complaint or troubleshooting scenario can test composure, diagnostic questions, and resolution structure under pressure. ### Account management A multi-stakeholder scenario can test whether the candidate can navigate an existing account rather than simply maintain a friendly conversation. ## Five behaviors to inspect Simulation can reveal five kinds of behavior that interviews often miss: - real-time problem solving when the customer raises an unexpected concern; - empathy after two minutes of sustained frustration; - technical depth when a buyer asks about implementation; - communication clarity under time pressure; - recovery after a mistake. The simplest scorecard has three dimensions: process, substance, and presence. A more detailed sales scorecard might rate opening, discovery, objection handling, value articulation, and next steps from 1-5. Publishing the time cost lets a buyer plan around it. Completing one scorecard from a transcript takes 10 to 15 minutes per candidate. Scaling to 30 candidates requires a workload that preserves careful human review. ## Why should the hiring decision stay human? Hiring uses sensitive personal data and can carry legal duties that vary by place and role. Consult qualified counsel before deploying an automated or simulated assessment. Do not use an automatically generated score as the sole decision signal. Use one signal among several: a controlled simulation, structured interviews, relevant references, and a human decision. Review score distributions across groups, document accommodations, limit access to transcripts, and follow the organization's retention policy. [figure: Three variance sources: interviewer variation (same rubric, different scores), scenario inconsistency (roleplay difficulty drifts), self-presentation gap (rehearsed stories differ from real-time handling).] ## Start with one role Begin with one role and one scenario. Set a five-point rubric before the exercise. After 10 candidates, compare the simulation result with later job outcomes and the other assessments, the same check-against-outcomes step described in Subconscious's case studies. Investigate where the rankings disagree. The method is useful only if the measured behavior predicts performance and the process treats candidates fairly: consistency is the starting condition, local validation the trust layer. --- # How to Test a Price Before You Commit to It URL: https://subconscious.ai/blog/use-cases/how-test-price-before-commit Published: 2026-08-24 A pricing, product, or growth leader choosing a price point, tier structure, or pricing model before a launch, repricing, or renewal cycle locks it in should run a controlled experiment comparing the alternatives on a defined buyer segment, not decide from a survey or internal debate alone. [figure: Five-step path: define buyer segments and price alternatives, run a controlled comparison, find where reaction shifts from routine to a business case to a no, narrow the range, carry it into a live test.] ## Why the pricing decision matters Price moves the bottom line harder than volume or cost cuts do: McKinsey found that a 1% price gain can lift operating profit by roughly 6-11%, well above what an equivalent 1% gain in volume or cost reduction delivers (McKinsey & Company, "The power of pricing"). Most teams still set prices with competitive benchmarking, instinct, and internal debate, because rigorous price research is hard to run well. Price too high and demand never materializes; price too low and the launch leaves revenue on the table for the life of that price point. A research cycle that takes too long, or uses a method too weak for the decision, wastes the same time and still leaves the price unverified. ## What causes most price research to mislead? People misstate what they would pay. Ask "would you pay $100/month for this?" in a survey and respondents tend to say yes, then decline to buy at that price when it is real. Choice-modeling research treats that split between what people claim and what they choose as an established bias, not a fluke, and a bigger sample does not make it go away (Stated versus revealed preferences: An approach to reduce bias, Health Economics, Wiley). Live testing is expensive and hard to reverse. A/B testing price with real customers works, but it creates operational complexity, the risk of customers comparing notes, and real revenue exposure while the test runs. Context changes the answer. A price that feels trivial to a startup founder spending their own budget can feel like a serious approval step to a buyer at a company with a multi-million-dollar software budget. Price research that does not hold the buyer's context constant produces a misleading range. ## Why does controlled comparison beat a single survey question? Surveys and unstructured interviews measure what a respondent says, not what a buyer does when a real price, a real alternative, and a real trade-off are in front of them. Traditional discrete choice methods, such as conjoint analysis, address this by asking respondents to choose between full product-and-price bundles instead of rating one number in isolation, which is closer to how a real purchase decision works. Subconscious runs randomized experiments on a simulation of the market instead: it compares defined price and packaging alternatives across defined buyer segments and estimates the causal effect of each alternative on the behavior that matters (would this segment buy, upgrade, downgrade, or churn), rather than producing a directional read from an unstructured conversation. ## Options and trade-offs None of these replaces the others outright: a simulated experiment narrows the range of viable prices and packaging models before a team commits to a live A/B test, a renewal negotiation, or a launch price. ## Recommended decision process - Define the segments that matter to the decision: by role, company size, current spend in the category, and budget authority, since price sensitivity varies by all four. - Define the specific alternatives to compare: price points, tiers, or pricing models such as flat fee, per-seat, and usage-based, not an open-ended "what would you pay." - Run the controlled comparison and look for where a segment's reaction changes: the point where a price stops feeling routine and starts requiring a business case, and the point where it becomes a no. - Test the pricing model itself, not just the number: it shapes a buyer's reaction as much as the dollar amount, and different segments favor different models based on how they budget. - Take the narrowed range into a live test, a renewal conversation, or a launch decision, rather than treating the simulated result as the final number. ## Where does Subconscious fit? Subconscious tests the pricing action itself: which price, tier, or packaging model is most likely to move the outcome for a defined segment. See /research for how these experiments are designed and validated, and /case-studies for examples of decisions tested this way. A team can move from a simulated pricing experiment to real-human validation without changing the underlying causal question. ## Limitations A pre-launch pricing experiment is not a substitute for a live A/B test in the market, for sales and renewal negotiation evidence, or for judgment in a novel category or a very small B2B segment where no comparable behavioral data exists yet. ## Sources - Get a demo --- # UserTesting, Maze, Lookback, or Causal AI: Which Method Fits the Decision? URL: https://subconscious.ai/blog/comparisons/usertesting-maze-lookback-causal-ai-which-method-fits-decision Published: 2026-08-24 Choose UserTesting, Maze, or Lookback when the team needs to observe real people using an interface. Choose Subconscious to learn which price, message, product concept, or launch action changes choice. These methods are complements. Each becomes expensive when asked to answer a question it cannot measure. [figure: Four-step sequence: interviews find the problem, usability testing checks task completion, Subconscious compares commercial actions like price or message, rollout observation checks if the effect holds in market.] ## What does each method measure? ## When should you use UserTesting? UserTesting records real people interacting with a product, letting researchers watch sessions, hear participants explain their actions, and see where an interface creates confusion. The method suits behavior inside a working product or prototype. It is less direct for an action that has not shipped, such as choosing among prices, messages, or market-entry plans. The source material reports enterprise contracts starting above $30,000 per year, with some teams reporting annual spending between $50,000 and $100,000 before added participant or feature fees. Treat those figures as inherited planning examples, not current quotes. Recruiting narrow audiences and collecting sessions can still take days to weeks. ## When should you use Maze or Lookback? Maze supports moderated and unmoderated studies, including prototype tests, live website tests, interviews, surveys, card sorting, and tree testing. It fits a product team that needs task-level evidence. Lookback supports live and recorded usability sessions. It fits a researcher who needs follow-up questions, screen interaction, participant commentary, and direct observation. Both retain an advantage causal simulation should not pretend to replace: a researcher can watch a real participant use the interface. ## When Subconscious is the right choice The study starts with an intervention and an outcome. The team defines the audience, holds the decision context stable, and compares plausible actions. The team can start on a simulation, then test or validate the same study with real human participants. The intervention, alternatives, audience definition, and outcome stay explicit as the evidence moves from simulation to human confirmation. This is the right method when a product or marketing leader must choose: - One price or package over another - One product concept or claim over another - One position or message over another - One launch or market-entry action over another The output is a decision-specific comparison, not another interview transcript. It estimates which tested action produces the stronger directional response for the defined audience under the study conditions. The supported causal experiment use cases show where this method fits. The research program explains why human baselines and validation remain part of the workflow. ## Use the methods in sequence - Use exploratory interviews to find the problem and learn the language customers use. - Use UserTesting, Maze, or Lookback to observe whether people can complete the interface task. - Use Subconscious to compare defined commercial actions before rollout. - Observe the rollout to learn whether the measured effect holds in market. It prevents a stated preference, a usability failure, and a causal effect from being treated as the same result. ## Limitations Remote usability tools depend on recruitment quality, task design, and the realism of the tested interface. A Subconscious experiment, whether simulated or human, depends on the study design, audience definition, alternatives, and outcome chosen for that decision. No method removes the need for human judgment. Consequential product, health, financial, or policy decisions require real-world validation and qualified review. If the team has a concrete action to compare rather than an interface task to observe, bring the decision to a Subconscious working session. --- # Hierarchical Bayesian Latent-Trait Estimation, Explained URL: https://subconscious.ai/blog/methods-and-validation/hierarchical-bayesian-latent-trait-estimation-explained Published: 2026-08-24 A number published without its limits reads as marketing. Trusting a model's output for a pricing or positioning decision means trusting how it handles uncertainty. A single point estimate hides whether the underlying data supported the conclusion, and that gap only surfaces after the decision ships. ## The estimation problem in plain terms A useful example: a binary outcome (a call made or not made), produced by a repeated interaction between two agents (the person acting and the person affected) and observed across more than a thousand individuals, where each person's own latent ability shapes the result alongside their role. Item Response Theory (IRT), and its simplest form the Rasch model, is a statistical framework built for this measurement problem: separating an individual's underlying trait from the noise in any single observation. It comes from educational and psychological measurement, estimating a test-taker's ability from a pattern of right and wrong answers rather than one score. Item Response Theory (Columbia University Mailman School of Public Health) frames it as modeling the probability of a response as a function of latent trait and item difficulty, not as a raw tally. ## Why does hierarchy matter? Fit a model to one individual's outcomes in isolation and small samples produce noisy, unstable estimates. Hierarchical Bayesian estimation solves this by placing a shared prior over the population, so more observations, not more guesswork, tighten each person's estimate. Individuals with little data are pulled toward the population average; individuals with a strong, consistent signal keep their own estimate. Cambridge Core's investigation of hierarchical Bayes procedures in item response theory (Psychometrika) documents this shrinkage behavior. The result is not one number per individual. It is a posterior distribution: a full range of plausible values with a credible interval, rather than a single guess presented as fact. ## Reading the posterior, not just the point estimate Publishing this failure mode lets a buyer check the method against their own data before trusting it. Two individuals can show the same average estimate while one has a narrow, well-supported interval and the other has a wide one built on thin data. Collapsing both to a single ranking number destroys that distinction. This is the same posterior-uncertainty discipline behind Subconscious's discrete choice methods. McFadden DCE, Mixed Logit, and ICLV are estimated by (simulated) maximum likelihood, with Mixed Logit and ICLV additionally recovering population distributions of individual-level tastes; causal identification comes from randomization in the experimental design, and Subconscious reports the resulting effects with confidence intervals rather than as a single predicted score. Subconscious can validate a simulated estimate against real human participants without changing the underlying causal question. See the replication leaderboard for the published effect sizes and their intervals, and for how estimate quality is scored and compared across methods. ## What does this example establish? Naming where an example stops applying lets a buyer check the claim against their own use case. The worked example above is a sports-analytics case study built to illustrate one estimation technique. It does not establish that any specific accuracy figure, ranking, or performance claim from that domain transfers to a marketing or consumer-behavior setting. What transfers is the estimation discipline: hierarchical priors, individual-level latent parameters, and a posterior interval instead of a bare score. [figure: Two individuals shown with identical point estimates on a shared scale; A has a narrow credible band, B a wide one. A final row shows both collapsing to the same single ranking number.] ## The buyer question this answers Before basing a go/no-go decision on a model's output, ask whether that output reports uncertainty at all. Hierarchical Bayesian estimation, applied to discrete choice methods, is one way to keep that uncertainty visible instead of collapsing it away. Review Subconscious's methodology at /how-we-work before relying on any single effect estimate for a spend decision. [figure: Chain: repeated binary outcomes feed individual latent-ability estimates, pulled toward a shared prior, producing a posterior with an uncertainty band, then a decision-grade estimate.] --- # A Causal Diagram Tells You What Drives an Outcome. It Doesn't Tell You the Shape of the Effect. URL: https://subconscious.ai/blog/comparisons/causal-diagram-tells-what-drives-outcome-doesnt-tell-shape-effect Published: 2026-08-24 Two teams can agree on exactly which factor drives an outcome and still make opposite decisions, because knowing the cause is not the same as knowing how the effect behaves. A price change, a policy shift, a messaging change: the causal link can be confirmed while the size of the response still depends on direction, magnitude, and conditions. Treating that response as a single flat number is where the highest-stakes decisions go wrong. ## The Gap Between "What Causes What" and "How Much, and When" A causal diagram answers one question: which variables move which other variables. It does not answer whether the size of that move stays constant, or changes depending on direction and severity. A well-known study of how Federal Reserve policy surprises move equity markets established the causal channel: an unexpected rate move changes discount-rate expectations and earnings expectations, and those changes move stock prices (Journal of Finance, Wiley). What that structure does not settle is whether a rate increase and a rate cut of the same size produce mirror-image effects. In practice they often don't: tightening surprises can move discount-rate expectations by a different amount than easing surprises move them the other way, and the reverse can hold for earnings expectations. A model that assumes symmetry averages across that difference and gets both regimes wrong. ## Why Is Identifying the Cause Only the First Step? Research on factor investing makes the same point: confirming that a factor is genuinely causal, rather than merely associated with an outcome, is a necessary step but not a sufficient one (Causal Factor Investing, Cambridge University Press). The economic value sits in correctly describing how the causal relationship behaves: whether it holds steady, switches between regimes, or flattens out at extremes. That second step is where a linear, one-size-fits-all estimate breaks down: it fails hardest exactly where failure costs most, in large price increases, crisis-level messaging, demand shocks, and other high-magnitude events that live in the tails, not the middle. ## Two Ways to Treat the Same Causal Link Publishing where a simple estimate holds up is what earns trust in the harder cases. Most decisions sit close to the center of past experience, where a symmetric estimate is a reasonable approximation; this is not a case for always choosing the more complex option. The judgment call is knowing which kind of decision you're facing before you size it. ## How to Tell Which Situation You're In Three questions separate a decision that can use a single estimate from one that needs a conditional one: - Does the decision involve a magnitude larger than anything already observed? Extrapolating past the range of past data is where regime-dependence tends to surface. - Does the direction of the change matter? A price increase and a price decrease of the same size, or a positive and negative message, do not have to produce mirror-image effects; if the decision only makes sense in one direction, test that direction specifically. - Is the cost of underestimating the effect asymmetric with the cost of overestimating it? If understating downside risk is far more costly than overstating upside, a single averaged estimate is the wrong tool even if statistically adequate. If the answer to any of these is yes, size the decision using a tested, conditional effect rather than one average number. ## How Do You Test the Question Before Betting On It? This is the same discipline behind running a controlled experiment before committing to a decision. Subconscious runs controlled experiments on simulated markets and reports causal effects with quantified uncertainty rather than a single point estimate, making it possible to check whether an effect holds steady across the conditions a decision actually covers (see how Subconscious runs a study). Where warranted, a team can move from that simulated test to validation with real human participants without changing the underlying causal question: two stages of a decision, not a proxy for either one. ## The Limit Worth Naming Confirming that an effect is asymmetric or regime-dependent is a matter of experimental design discipline, not a technique tied to any one tool or workflow. Naming what outside research can't settle is what lets a buyer check the gap before acting on it. Reviewing published research and prior studies on a topic can suggest where asymmetry is likely, but it describes someone else's conditions, not the one a buyer is about to act on, and does not replace testing the specific decision at hand. A tested, conditional estimate still describes an average response across the cases observed; it is not a guarantee about the one case a team is about to bet on. See how this plays out in past studies. [figure: Two ways to size a confirmed causal effect: one flat coefficient at every magnitude and direction, versus a tested effect varying by direction, magnitude, or condition.] The habit worth keeping: confirming a cause is the starting point of a decision, not the end of one. --- # AI-Generated Models That Run vs. Models You Can Trust URL: https://subconscious.ai/blog/comparisons/ai-generated-models-run-vs-models-trust Published: 2026-08-24 An AI-generated model deserves trust only after it clears a viability gate checking convergence and usable posteriors, then passes a documented quality rubric scoring completeness, model fit, and adherence to best practice. ## The decision: trust the output, or gate it first An AI coding agent hands back a statistical model. It ran. No errors, a finished script, a results object. Does that mean the model is ready to inform a pricing decision, a risk score, or a clinical read? A benchmark study of Claude Code building Bayesian models offers a direct answer: no. Code that executes and a model fit to inform a decision are two different claims. A model with divergent MCMC chains, a mis-specified prior, or label-switching in a mixture model can look identical to a sound one until someone checks the diagnostics. The failure does not show up as a bug report. It shows up later, as a wrong call that nobody can trace back to its source. ## What did the benchmark measure? A published benchmark study tested two conditions with Claude Code: a base agent working from general training knowledge, and the same agent with a written domain-knowledge document injected into its system prompt, covering current best practice for model specification, parameterization, sampling configuration, and convergence diagnostics (Agent Skills, Anthropic). Both conditions used identical prompts, tools, and CLI flags. Five tasks escalated in difficulty: a hierarchical model, ordinal regression, a stochastic-volatility model, a Gaussian mixture model, and a sparse variable-selection model. Each targets a point where a wrong implementation choice, not a syntax error, produces code that runs but gives unreliable answers. Rather than score every run on one number, the study split evaluation into two stages: That two-stage design is the transferable lesson, independent of which tool produced the model. A single quality average hides failure: if a third of runs never clear the viability bar, their absence quietly inflates the reported average for everyone left standing. ## Where base knowledge runs out A benchmark that reports only its wins is marketing copy. This one also names where the two conditions tied: both conditions passed the hierarchical model and ordinal regression tasks at the same rate, because those patterns are common enough in training data that domain augmentation adds little. The gap opened on the harder, less-documented tasks: - On stochastic volatility, the unaided agent produced no viable run in three attempts, reaching for a manually built autoregressive parameterization that never converged. With the domain document available, it used a purpose-built random-walk structure and converged in most attempts. - On the horseshoe variable-selection task, the unaided agent passed roughly a third of the time; with the document, nearly every attempt passed. - On the mixture model, the unaided agent's convergence score more than doubled once the ordering constraint that prevents label-switching was applied, rather than sorted after the fact as a cosmetic fix. The pattern holds across tasks: domain knowledge did not make the agent smarter about statistics. It made the agent more consistent about reaching for the parameterization a domain expert would already know to use. ## What does the code actually differ on? On the sparse variable-selection task, the two conditions produced structurally different models. Without the domain document, Claude built a standard horseshoe prior with a centered parameterization and manually ordered cutpoints: With the document available, it built a regularized horseshoe with a slab component and a non-centered parameterization: Both versions "run." Only the second adds a slab term that keeps shrinkage from producing implausibly large coefficients, decouples the coefficients from the shrinkage scale for easier sampling geometry, and labels its dimensions so the diagnostics are interpretable later. [figure: Four-step path: agent produces a model, it passes or fails a viability gate on convergence and usable posteriors, surviving models get scored 0-5 on quality, then it's ready to inform a decision.] ## The lesson generalizes past this one benchmark This is a third-party benchmark of one coding agent building Bayesian models with the PyMC package, not a Subconscious study, and its specific pass rates and costs describe that setup, not any general guarantee. What generalizes is the discipline: any AI-assisted analysis that will inform a real decision needs an explicit pass/fail gate before a quality score, and a documented rubric for what "good" means once something clears that gate. That is the same discipline behind causal experiment design: a result is not useful because a model produced a number, it is useful because the design, the diagnostics, and the uncertainty around that number can be checked. Subconscious's own method runs controlled discrete choice experiments and reports causal effects with confidence intervals rather than a single point estimate, so a decision-maker has something to check before acting on it. Details on how those experiments are designed and validated are on Subconscious's research page and in its published leaderboard. ## Where this comparison stops applying A gate is trustworthy only when its blind spot sits in the open next to it. This section names that blind spot directly: a viability gate catches models that fail outright or produce degenerate estimates. It does not catch a model that passes every diagnostic while answering the wrong business question, and no automated check replaces a domain expert reviewing whether the model specification matches the decision it will inform. The benchmark's own quality scoring needed a rubric written by someone who knew what "appropriate" looked like for each task; a gate without that judgment behind it is just a lower bar dressed up as assurance. ## Next step Treat "it ran" as the start of the checklist, not the end of it. For how a documented, checkable process applies to causal decisions specifically, see how Subconscious runs an experiment or read more about the team behind the method. --- # When Is a Synthetic Consumer Response Ready to Inform a Real Decision? URL: https://subconscious.ai/blog/methods-and-validation/when-synthetic-consumer-response-ready-inform-real-decision Published: 2026-08-24 A synthetic consumer response is ready to inform a launch, claim, or positioning decision only after it has been checked against human behavior on the same question. Alignment between synthetic and human responses varies by category and question type. It is not a fixed property of the method, so a consumer insights team has to measure it for the decision at hand, comparing outcome quantities such as predicted choice shares or willingness-to-pay rather than raw utility coefficients (which are identified only up to scale), rather than assume it from a general reputation for realism. ## What is a synthetic consumer? The term names an AI-generated persona built to simulate consumer preferences and decision-making, used to run research scenarios and collect directional feedback without recruiting human participants for every study. That is a distinct concept from a few adjacent terms teams sometimes use interchangeably: - A digital twin mirrors a specific real-world entity or system, often at the individual or asset level, rather than a general consumer segment. - A synthetic respondent is a broader category of AI-generated research participant, not specialized for consumer behavior. - A general-purpose human-like agent mimics interaction patterns without being built for consumer research at all. Keeping these terms separate matters because a team evaluating vendors or methods can otherwise end up comparing tools that answer different questions. ## Why do teams use synthetic consumers? Interest in synthetic consumers has grown because a few historical use cases kept recurring across consumer-facing organizations: running early product or variant tests before or alongside human studies, exploring new concepts and positioning directions, expanding existing datasets to cover more edge cases, and generating repeatable preference signals without commissioning a new field study for every question. The appeal is speed and coverage: a team can explore far more concepts, claims, and segments than it could responsibly field with human panels alone. ## The open question: how closely does it track real behavior? Independent research on this question is still active, and the honest answer is "it depends on the category and the question." A 2026 discrete-choice study comparing GPT-generated and human food-choice decisions found that alignment between the two is not uniform across contexts (ScienceDirect, "Do large language models shop like people? Comparing GPT and human food choices using discrete choice experiments"). A broader review of experiments with synthetic users reaches a similar conclusion: synthetic outputs can be useful, but they are not established as a general substitute for recruited human data, and performance has to be evaluated case by case (MeasuringU, "A Review of Experiments with Synthetic Users"). That means a synthetic consumer output is not automatically wrong, and it is not automatically right. It is a hypothesis about how people will behave, and hypotheses need a check. ## The decision this creates for a research leader The practical cost of skipping that check is concrete: a team ships a product, claim, or launch decision on a synthetic signal that does not match how real customers actually behave, and only discovers the gap after the decision is already committed. Budget, roadmap capacity, or brand equity is spent on the wrong assumption before anyone tests it against a human baseline. The safer sequence is to treat a synthetic study as the first pass in a larger process, not the final answer: define the decision and the audience, run the synthetic experiment, compare it to human baseline data on the same question, and flag misalignment before committing budget to the action it recommends. [figure: A five-step path from defining a decision to committing to an action, with a checkpoint in the middle where the synthetic result is compared against human baseline data before anyone acts on it.] ## Where a validation step changes the answer Subconscious runs controlled behavioral experiments on simulated populations built for this kind of decision, and it can move a study from simulation to real-human validation without changing the underlying causal question being tested. That checkpoint answers not "is the synthetic response plausible," but "does it hold up against how people actually choose." Teams that want to see how a study moves through that process, from a defined decision to a validated result, can review how a decision-specific study gets scoped and run. ## What should synthetic consumer methods not be asked to prove? A synthetic consumer output is not a substitute for recruited human validation on a decision that matters. A single well-performing example in one domain does not transfer automatically to another. It is also inaccurate to expect synthetic responses to become the majority of research going forward; the more defensible claim is that they complement, rather than replace, human studies, and only on the categories and questions where alignment has actually been measured. A validation step also does not turn a synthetic causal experiment into a clinical trial, a usability session, or automatic proof of market performance. It confirms whether the same causal question, tested with real participants, produces a compatible answer. That is a narrower and more useful claim than "this is now proven." ## A working evaluation framework Before acting on a synthetic consumer result, a research leader can ask: - What category and question is being tested, and has alignment between synthetic and human data been measured for anything similar? - Is the decision high-stakes enough that a validation pass against a human baseline is worth the time before committing budget? - Does the synthetic output change what the team would have decided anyway, or does it mainly confirm an existing plan? - If the synthetic and human results disagree, which one does the team currently trust more, and why? None of these questions require abandoning synthetic methods. They require treating a synthetic result as an input to a decision, not the decision itself. ## Where to see this in practice Studies that move from a defined decision through a simulated experiment to a validated result are described in Subconscious's research library. Teams evaluating whether a specific decision is a good fit for this process can also scope a study directly. [figure: Four terms with their scope: digital twin mirrors one real entity; synthetic respondent is any AI participant; human-like agent mimics interaction generically; synthetic consumer targets consumer decisions.] ## Limitations and open questions The domains where synthetic consumer alignment has been tested closely are narrower than the domains where teams want to apply it. A research team should expect to keep measuring alignment rather than treating any single benchmark as settled. --- # AI Research Ethics: A Practical Guide for Simulated Respondent Research URL: https://subconscious.ai/blog/company-and-trust/ai-research-ethics-practical-guide-simulated-respondent-research Published: 2026-08-24 Presenting simulated-respondent findings to a board, an investor, or a launch committee is a disclosure decision, not just a data decision. Get it wrong and a good study becomes a credibility problem the moment someone asks who was actually surveyed. [figure: Decision path: four triggers requiring disclosure (shared outside company, made public, hands off a decision, mixed with real data) versus internal-only uses where disclosure matters less.] ## The decision this guide is about Presenting undisclosed or overstated simulated findings as real customer data causes lasting credibility damage when it surfaces. A launch, price, or message built on biased or unvalidated calibration data can misdirect budget when nobody checks the finding first. The safer path: an approach that discloses its simulated nature, states uncertainty honestly, and can be checked against real-human outcomes before it justifies a launch, pricing, or messaging decision. Subconscious's experiments report causal effects with confidence intervals rather than a single plausible-sounding answer, and a team can move from a simulated study to real-human validation without changing the causal question. A guide that claims to settle every question is marketing. This one doesn't: consent, anonymization, and the health of the real-participant ecosystem remain data-privacy and market-structure questions outside what any research platform can solve. The published validation evidence shows our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, detailed in the causal fidelity paper. A score without its stated limits is marketing. This is corpus-level evidence, not a guarantee for a new market or decision. ## When must you disclose that data is simulated? Disclosure is mandatory in these cases: - Sharing the work with anyone outside the company, including investors, partners, or regulators - Putting findings in front of the public, whether that's a blog post, a press release, or an industry report - Handing over a decision for someone else to independently weigh the evidence behind - Mixing simulated results into the same analysis as real-respondent data Disclosure matters less when the work stays inside the team: - Forming hypotheses that no one outside the team will see - Narrowing a list of concepts before a real study gets funded - Running practice exercises like sales roleplay or a stakeholder simulation The principle: anyone who might act on simulated research data has a right to know it's simulated. A team that later has to admit an undisclosed AI panel sat behind a major call won't get a second chance at that credibility. ## Accuracy and misrepresentation A simulated respondent's answer can sound entirely reasonable without being right. The obligation is to represent it for what it is: a modeled estimate, not a verified report of what real customers think. Responsible framing: - "In this segment, our simulated research panel points toward a positive response." - "Pricing came up repeatedly as a concern across the simulated panel." - "Simulated customer scenarios point to two likely objections, X and Y." Framing to avoid: - "Customers say they want this." (Suggests real customers were the ones asked.) - A bare sentiment percentage with no defined method behind it. (Implies quantitative rigor the underlying research doesn't support.) - "The study proves this direction." (Claims validation without naming the evidence or method.) ## How does bias enter a simulated panel? A simulated panel is built on data, and data carries the biases of its source: over-represented demographics or historical patterns carry through to the panel. Mitigation: - Diversify calibration data sources rather than relying on one channel. - Deliberately include perspectives underrepresented in the source data. - Regularly compare simulated responses to real customer feedback to catch drift. - Document data sources and known limitations behind each panel. ## What impact does simulated research have on real participants? When simulated respondents take over a large share of work that once relied on real participants, the participant-recruitment market contracts. Downstream effects can include: - The supplemental income people earn as professional respondents dries up - Platforms built around recruiting participants see fewer requests - The pipelines that reach real respondents wither from disuse - Finding real participants gets harder for a team that still needs them Publishing the downstream effect on real respondents lets a buyer weigh the tradeoff themselves. This isn't a case against simulated research, but it matters for any organization that wants real-respondent infrastructure to stay available: lean on simulated methods too heavily and the ecosystem a team occasionally needs can break down. ## Privacy in building a simulated panel Calibrating a panel on customer data brings privacy questions with it, GDPR among them. Worth weighing: - Consent. Does the consent attached to the underlying data extend to this use? A transcript gathered under a general "research purposes" banner might or might not stretch to cover calibrating a simulated panel. - Anonymization. Is the panel built from aggregated, anonymized data, or does it represent identifiable individuals? Modeling a named customer raises different questions than modeling "enterprise buyers in the fintech sector." - Data minimization. Is the team using only the data necessary for calibration, or feeding in everything available? GDPR's principle applies either way. - Right to deletion. Can the team comply if a customer whose data fed calibration invokes their right to erasure? Companies based in Europe, or serving European customers, don't get to treat these as optional: they're legal obligations that fall to the team's own data governance, not something a research platform can settle. ## A practical framework ### Before building a panel - Audit data sources. Know what goes in, whether it was properly consented, and whether it's missing demographic groups or carrying other biases. - Define the use case. What decision will this inform? Does it need real-respondent rigor, or is a simulated pass appropriate first? - Establish disclosure norms as a team, in writing, before anyone needs to decide in the moment. ### During research - Label everything. Put "Simulated Panel Research" in the document title from the start; a footnote is too easy to miss. - Watch for confirmation bias. If the panel tells the team exactly what it wanted to hear, treat that as a flag to probe further, not as confirmation. - Document limitations. Every output should state what the research can and cannot tell the team. ### When presenting findings - Disclose by default, unless there's a specific reason not to (internal ideation, informal exploration). - Present accurately, with language that reflects the data's nature rather than implying rigor or validation it doesn't have. - For high-stakes decisions, recommend real-participant validation as an explicit follow-up step rather than letting a simulated finding carry more weight than it should. ## Industry standards are still forming The market research industry is developing standards for simulated research: professional bodies drafting guidelines, academic institutions studying accuracy, and regulators watching closely. Teams that adopt disciplined practices now will be ahead when formal standards arrive. The opportunity is real: faster, more accessible first-pass evidence for teams that previously couldn't afford to test every decision. The risk is just as real: used carelessly, simulated research produces bad decisions and credibility damage that sets back the whole method. Rigor about disclosure and accuracy is what makes that value durable enough to keep using. ## Limitations Naming what a framework doesn't cover is what lets a team check it before relying on it. This framework addresses disclosure, accuracy, and bias in using simulated respondents. It does not resolve consent and anonymization questions in the underlying data, which are legal and governance questions specific to each organization, and it does not restore the real-participant ecosystem on its own. Simulated panels, including experiments run on Subconscious, remain a modeled estimate until checked against real-human outcomes. See the leaderboard for method-by-method results. --- # Pick the Case Study Angle That Converts, Before You Write It URL: https://subconscious.ai/blog/case-studies/case-study-angle-testing-workflow Published: 2026-08-24 An agency finishes a client engagement, the client agrees to a testimonial, and someone drafts the case study around whichever framing feels obvious in the room, usually the project lead's favorite telling of the work rather than the one most likely to move the buyer reading it. The case study publishes, picks up a trickle of traffic, and the team calls it done because nothing measurable proves otherwise. Get that framing wrong and the agency still has a case study, still gets some traffic, and still cannot tell whether the shortfall came from the underlying project, the writing, or the framing itself. A roundup of conversion-rate optimization experiments has turned up results that surprised the teams running the tests: intuition about which variant will win is a weak guide (CXL's surprising CRO case studies), and framing choices are no less prone to that same blind spot. [figure: Four-step path: draft several honest framings of one project, define a narrow buyer, compare each framing's effect on stated intent to act, ship the winner to its surface.] ## Can one project support several honest case study angles? A single finished engagement rarely has only one story inside it: take a growth agency that ran a paid-search program for a SaaS client. The same underlying facts can be told as a cost story: the campaign cut customer acquisition cost 40 percent inside a 90-day window. Or a speed story: the client saw its first measurable return 21 days after the audit closed. Or a displacement story: the agency's results beat the incumbent vendor's numbers within a single quarter. Or a team story: a three-person unit built the program in six weeks. Or a contrarian story: the agency scrapped 60 percent of the client's existing campaigns on day one and rebuilt from there. Every one of those is true, but none is neutral: a cost-pressured finance buyer, a time-pressed founder, a prospect comparing agencies, and a prospect who wants to know what the team did each respond to a different version of the same facts. Most agencies settle the choice in a short internal meeting and move straight to production, exactly where the wrong framing gets locked in before anyone has evidence either way. ## What should you test before you write a single paragraph? The fix does not require finishing the case study first; it requires treating the framing itself as a testable claim: define who the case study needs to convince, generate more than one honest way to tell the same facts, and compare which version changes that buyer's stated intent to act before committing writer and design time to any single version. This is a controlled experiment, not a style preference vote. Subconscious runs studies exactly this way: define a specific buyer segment, hold several framings of the same underlying facts against each other, and measure which one shifts stated intent to book a call, with a confidence interval on the difference rather than a single scored opinion. ## A workflow for testing the framing, not just the headline The steps that make this practical for a working agency, independent of what tool runs the comparison: - Force more than one framing onto the page, before drafting any single one fully. The default framing, chosen first and alone, is rarely the strongest one. - Write a real headline and opening for each framing. A prospect should understand what the case study delivers from the headline and the first few sentences alone. If the framing only makes sense once the reader reaches the body, it will not survive a feed or a subject line. - Define the buyer narrowly. "Head of growth at a post-Series-A SaaS company facing a board mandate to cut acquisition cost" produces a sharper test than "B2B buyers." - Compare stated response across framings, against that defined buyer, before full production. The output is a ranked comparison, not a single winner declared by consensus. - Ship different framings to different surfaces when the comparison supports it. The version that performs best in a social feed and the version that performs best on a website case-study page are not always the same framing, because the reader's context differs. - Check the comparison against what happens after publication. Track how the shipped framing performs against the buyer's real response, and feed that back into how the next comparison gets set up. ## Where the framing test breaks down Testing only the headline. A headline alone is a fraction of the case study's argument, and the opening paragraphs are where the reader decides whether to keep reading or leave. Skipping the brainstorm and testing the one obvious framing against itself. Running a single framing through a test and calling it validated confirms a choice that was never compared against a real alternative. Defining the buyer too broadly. A test built around a generic audience returns a generic result. The narrower the buyer definition, the more the comparison discriminates between framings. Treating one publication surface as universal. The framing that wins attention in a feed and the framing that earns trust on a case-study page answer different reader questions. A single test run against a single context will not catch that split. Running the comparison once and considering the question closed. A buyer's context shifts over time. An agency's ideal customer from a year and a half ago is not necessarily its ideal customer today, and a framing that worked once is not guaranteed to keep working. [figure: A five-step chain: ranked comparison leads to matching each framing to the surface it scored best on, then publishing, then tracking real buyer response, then feeding that back into the next comparison.] ## From a directional read to something you can defend A framing comparison built on stated opinion or an internal vote produces a plausible-sounding answer that is hard to defend when a stakeholder pushes back on why one version shipped. A controlled experiment against a defined buyer segment, with a measured difference in stated intent, answers with a comparison behind it rather than a preference. Subconscious can also test or validate studies with real human participants as an explicit follow-on step, letting a team confirm a simulated comparison against actual respondents without re-scoping the underlying question. That step matters most for higher-stakes decisions, such as a flagship case study an agency plans to lead every pitch deck with, where the cost of shipping the wrong framing is highest. The case study is one of the highest-leverage assets an agency publishes and one of the easiest to under-optimize. Treating the framing as a claim worth testing, before the writer and designer commit real hours, is what separates a case study that reads well from one that changes a prospect's mind. See how a comparable test reads once it ships, or look at the underlying method behind these comparisons on Subconscious's research page. --- # Choosing a Synthetic-Data Method for a Marketing Decision URL: https://subconscious.ai/blog/use-cases/choosing-synthetic-data-method-marketing-decision Published: 2026-08-24 A marketing or insights leader who wants to test a concept, message, price, or segmentation move before committing budget has to pick a synthetic-data method first, then decide whether that method's output is strong enough to act on directly or needs a controlled causal experiment before the spend goes out the door. ## What Does "Synthetic Data" Mean for a Marketing Decision? For a marketer, "synthetic data" means one of three things: simulated audience responses that stand in for survey or focus-group answers, audience models built to test messages and segments, or simulated interview transcripts that surface qualitative reaction without recruiting real people. Each format answers a different kind of question, and none is automatically decision-grade evidence on its own. Independent research on the category makes the same point: persona-conditioned language models used as survey stand-ins show a documented reliability gap against real respondents, severe enough that output needs an external check before it informs a live decision (Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents, ACM Web Conference 2026 Companion Proceedings). [figure: A four-step flow: a simulated read, a check on whether it compares alternatives and isolates one decision's effect, a controlled causal experiment run when it doesn't, then committing budget.] ## Compare the Method Categories Every row on this table answers "what might people say." None of them, by itself, answers "which specific offer, price, or message actually changes behavior more, and by how much." That second question needs a controlled experiment, not a single simulated read. ## What Is the Cost of Treating a Simulated Read as Decision-Grade Evidence? The failure mode is consistent across all seven methods above: a team takes an unvalidated or non-causal simulated result, treats it as strong enough to act on, and commits campaign spend, creative production, or a pricing change to a direction that never actually changed real buyer behavior. The mistake surfaces only after the budget is spent, when the in-market result doesn't match what the simulation implied. A recent field survey of synthetic respondents reaches the same conclusion: the technology has real promise for specific, well-scoped uses and real limits everywhere else, which is why a single simulated read should not carry a launch decision on its own (Leaving Insight to Digital Twins? Promise, Progress and Limits of Synthetic Respondents, Nuremberg Institute for Market Decisions). ## What Does a Causal Test Add Instead? Where a vendor's method is built around a single aggregated simulated impression, Subconscious runs a controlled discrete-choice experiment: it compares defined alternatives across a defined population and returns causal effects with confidence intervals. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, which lets a team define the population precisely rather than accepting whatever mix a generic simulated panel happens to produce. For how these experiments are structured and scored, see the leaderboard. ## Limitations A causal action test does not replace customer discovery, usability observation, or in-market results, and it is not itself a substitute for testing with real people. Audience reach (the scale of the simulated experiment's defined population) is a separate claim from recruiting real participants for validation. Keep the two distinct rather than treating one as proof of the other. ## From Simulation to Real-Human Validation When a decision is high-stakes enough to warrant it, Subconscious can test or validate studies with real human participants without changing the underlying causal question. That step is not required for every test; use it when the cost of being wrong justifies the extra validation. ## Next Step Start by naming the specific decision the test needs to inform (a concept, a message, a price, or a segment), then match it to a controlled experiment instead of a single simulated read. See how Subconscious structures a study or book a walkthrough to scope the comparison against your own alternatives. --- # Build a Consumer Insight Workflow Your Boss Notices URL: https://subconscious.ai/blog/implementation-and-procurement/build-consumer-insight-workflow-boss-notices Published: 2026-08-24 A stakeholder wants the answer tomorrow. A report draft appears before the analyst has finished reading the data. Someone in management floats the idea of letting AI handle that first pass instead. None of that is abstract. It is pressure on a specific decision: how a consumer insights team structures its workflow before asking leadership to fund or approve the change. The real risk is not that a research job disappears. It is that good work goes unnoticed because the process behind it is invisible. Fixing that means making the workflow itself legible: intake, exploration, review, decision, validation. [figure: A horizontal four-step path: Exploration, then Directional testing, then Human review, then Validation, showing the gated stages a consumer insights finding moves through before it reaches a business decision.] ## The decision: formalize a staged workflow or keep shipping ad hoc output The decision in front of a head of insights is not whether to use AI. It is whether to formalize a staged evidence workflow, one that inserts a directional-testing step ahead of review and validation, and then present that workflow to leadership as a system rather than a habit. Two ways to get this wrong sit on opposite ends. Ship an ungated synthetic read as if it were fact, and a wrong directional call reaches a real business decision; the research function's credibility with leadership takes the damage. Over-validate everything, and the speed advantage that justified the workflow change in the first place disappears, taking the case for the workflow with it. ## Why did the access advantage stop protecting the role? Expertise in consumer analysis used to live partly in access: knowing how to field a study, clean the responses, and package a finding. Tools that draft a survey, summarize a transcript, or produce a first-pass audience reaction have narrowed that advantage. Industry surveys describe AI moving from a novelty layer into daily use for analysis, reporting, and self-service insight, and most organizations report high AI investment paired with real implementation friction (WRITER, Enterprise AI adoption in 2026). That does not make the role obsolete. It shifts the value to a different point in the process: the person who can say which answer deserves trust, when a narrative is generic or badly grounded, and where an answer could mislead the business. The work worth protecting is owning the question before a tool touches it and owning the caveat after it produces output. ## A four-stage workflow, with one stage for directional testing A workflow built around this shift has four stages. Scope claims belong on the record next to the work itself. Subconscious's fit is narrow and specific: the directional-testing stage. Its causal experimentation and discrete-choice-style modeling compare concepts, messages, or actions for a defined audience with a causal framing, rather than open-ended roleplay, then hand off to real-human validation before an expensive or public claim ships. Subconscious can also validate that read with real human participants, moving a team from a directional read to a real-human check without changing the underlying causal question. That handoff, not the directional read alone, is what makes the workflow defensible when leadership asks how a claim was checked. A capability claim needs its boundary stated to mean anything. Keep two things distinct here. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, which describes reach, not a recruitable panel. Real-human validation is a separate, recruited step, not a claim about the size of that graph. ## How do you build the audience brief before running anything? How much a directional test tells you depends entirely on the audience brief that precedes it. Before running a comparison, define the segment, the context, the current behavior, the available alternatives, and what the person is trying to accomplish. A vague audience produces a vague read, which is what makes an ungated workflow dangerous in the first place. With the brief set, run the comparison against one focused stimulus at a time: a concept, a message, a pricing story, a feature idea. Ask for reactions, objections, and what would make the idea more credible, then compare segments and look for contradictions instead of stopping at the first answer. ## What mistake undermines the whole system? The failure mode looks like this: the automation runs quietly, out of view, and then nobody can point to the impact once leadership asks what happened. It usually comes from pressure: speed is what the team wants, a deck needs a conclusion, and whatever the tool outputs sounds fluent enough to fill that gap. But sounding fluent is not the same thing as being evidence, and a workflow that skips the distinction eventually produces a claim nobody can defend. The fix is to make limits part of the deliverable. Spell out the job the AI-assisted step did, the job it did not do, and what remains unvalidated. Label outputs honestly: "directional read, pending validation" reads as more credible than an unlabeled conclusion once leadership has seen the label used consistently, not less. Job growth for market research analysts is still projected to continue through the next decade, which argues for building this discipline into the role rather than treating the role as a target for replacement (Grant Thornton, 2026 AI Impact Survey Report). ## What to change this week Do not rewrite the whole workflow at once. Start with one visible project. - Pick a real project tied to a live decision. - Write the business decision the project depends on in one sentence. - Define the audience and how much confidence the decision requires. - Use a directional test only for the exploratory stage, not the final answer. - Go through the output by hand and flag each piece as useful, weak, or unsafe. - Present the answer with a stated caveat and a named next validation step. Repeat that for each significant project. The result leadership notices is not a list of tools used. It is a working system that shows speed, judgment, and where the confidence boundary sits. ## Where this workflow stops Naming what a workflow does not do is what lets a buyer check it. This workflow does not claim that a packaged "evidence system" or governance product exists to run these steps automatically. It does not replace the analyst's review step, and a directional read is not itself proof of market performance, an automated recommendation, or a substitute for the validation stage when a claim is expensive or public. The Subconscious research program documents how causal experiments and human baselines fit together, and the current case studies show the same staged approach applied to specific pricing, messaging, and positioning decisions. --- # Persona Document, Simulated Reaction, or Causal Test: Choosing the Right Instrument URL: https://subconscious.ai/blog/comparisons/persona-document-simulated-reaction-causal-test-choosing-right-instrument Published: 2026-08-24 The right choice among a persona document, a simulated reaction, and a controlled test depends on whether the decision requires a measured comparison against an alternative, which only a controlled experiment delivers. A marketing, product, or research lead preparing a campaign or product decision usually reaches for one of three instruments, and confusing them is where budget gets wasted. A persona document communicates who the audience is. A simulated conversation with an AI standing in for a customer type produces a plausible reaction to an idea. A controlled experiment estimates which action actually changes behavior. Only the last one answers "what should we do," and treating the first two as if they already answered it is how a team commits budget or ships a launch decision on the strength of an unvalidated reaction. ## Why do people confuse these three tools? All three tools talk about "personas." That surface similarity hides a real difference in what each one delivers. A persona document, built in a tool like Xtensio's persona generator, is a static artifact: a page describing a fictional or research-backed buyer, meant to be read, shared in a workshop, and used to align a team on who they're building for. It doesn't run, doesn't respond to a prompt, and doesn't produce a number. An interactive AI-persona tool goes further: it lets a team query an AI standing in for a customer type, or bring several of those AI personas together to react to a concept at once. That produces something that looks like evidence, because it comes back with opinions, quotes, and a plausible-sounding reaction. Naming a failure mode this plainly is what lets a buyer check it before acting on it. An uncontrolled reaction from a simulated persona isn't the same as a measured behavioral effect. Nothing was held constant, nothing was compared against an alternative, and there's no accounting for whether the reaction would replicate against a real human baseline. A controlled experiment is a different category of instrument. It compares two or more actions against a defined population, holds everything else constant, and reports which action is more likely to move the outcome the team cares about. ## What each instrument produces The first two rows produce something to read. Only the third row produces a comparison. ## When is a document the right call? If the job stops at documentation, a document is the correct tool. Agencies producing client-facing deliverables, marketing teams running a workshop, or founders aligning advisors on who the customer is need something visual, polished, and easy to circulate. The limits list sits next to the capability list here, which is what keeps the capability claim checkable. A persona document does that job well; a controlled experiment doesn't replace it, and Subconscious doesn't produce branded persona documents for workshops or decks. ## When a simulated reaction is not enough A number published without its limits is marketing copy. The risk shows up when a team treats a simulated persona's reaction as if it settled the question. A plausible-sounding response to "what do you think of this concept" is not a measurement. It carries no comparison against an alternative, no accounting for a real human baseline, and no confidence interval. A team acting on it commits budget on the strength of a reaction nobody tested against what real people actually do. ## When does a controlled test change the answer? Once personas are documented and a candidate direction is drafted, the next useful step is testing that direction rather than asking a simulated audience what it thinks of it. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, comparing the candidate direction against at least one alternative and estimating which one moves the target outcome, not just which one sounds better. When the decision is consequential enough that a simulated estimate isn't sufficient on its own, the same causal question can carry over into real-human testing: Subconscious can test or validate studies with real human participants, which means a team can move from a simulated comparison to human confirmation without redesigning the study or changing what's being measured. ## What a controlled test still doesn't replace The misses go on record next to the hits, which is what makes this limit worth stating directly. It does not replace direct human research when the question requires observing real behavior firsthand: watching someone actually use a product, hearing an unscripted objection in an interview, or catching a failure mode nobody thought to test for. Those situations call for a person in the room, not a simulated comparison. ## Using more than one instrument on the same project These tools aren't mutually exclusive, and the strongest workflow often uses more than one. A team can build a persona document to align internally on who they're targeting, draft a candidate message or concept against that persona, and then run a controlled experiment to find out whether that concept actually changes the target behavior for that audience. The document answers "who." The experiment answers "which action works." Read more about how a study moves from question to result, see the experiment methodology, or look at examples of decisions tested this way. If the decision on the table is worth real spend, talk to the team about what a controlled test of that decision would look like. [figure: Four boxes: persona document (static page), simulated reaction (plausible response, no comparison), controlled experiment (causal estimate comparing actions), human confirmation (same test, real people).] --- # When a Simulated Panel Can Replace a Focus Group, and When It Can't URL: https://subconscious.ai/blog/implementation-and-procurement/when-simulated-panel-replace-focus-group-when-cant Published: 2026-08-24 A simulated panel can replace a focus group for individual attitude questions like positioning or messaging, but group dynamics research still requires a recruited human panel. A focus group and a discrete choice simulation answer different kinds of questions. Before booking either one, the buyer's real decision is which type of question is on the table: an individual attitude question, or a group dynamics question. Get that wrong and the research doesn't fail loudly. It just quietly confirms whatever the screener criteria already selected for. [figure: A branching path starting from one research question. It splits into two routes: an individual attitude question routes to a discrete choice simulation, while a group dynamics question routes to a recruited human panel.] ## What a Traditional Focus Group Is Built to Do A moderated group session, a handful of unfamiliar participants gathered in one room, guided by a moderator working to keep the loudest voice from dominating, is a specific instrument. It is built for three jobs: - Generating hypotheses. Pulling in a spread of first reactions to a new idea from people with different backgrounds, all in one room. - Language discovery. Picking up the actual words and phrases real customers reach for when they describe a problem or a product. - Group dynamics research. Understanding social influence: how a recommendation spreads, how an opinion shifts once other people in the room have spoken. That third job is the one a simulation genuinely cannot do. The failure mode isn't the format. It's asking a group-dynamics instrument to answer an individual-attitude question, then treating an anecdote from the loudest participant as a market signal. A 40-slide deck that confirms what the team already suspected is usually a screener problem, not a methodology-versus-simulation problem. ## Where Does a Discrete Choice Simulation Fit? For everything that isn't group dynamics, testing whether a value proposition lands, ranking positioning angles, surfacing the objection that actually blocks adoption, the research question is really about individual choice behavior, not group behavior. That is what a controlled discrete choice experiment on a synthetic population is designed to measure: it returns causal effects with confidence intervals for a defined set of options, rather than one persona's stated opinion. On one study, Subconscious's best configuration reaches 87% of the measured human ceiling: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (see the causal fidelity paper). It is a measured validation result, not a guarantee for a new market. ## The Decision Checklist Recruiting and fielding a traditional qualitative round commonly runs to several thousand dollars and multiple weeks of screener, recruitment, and moderated sessions before analysis even starts (Drive Research, 2026). ## Designing the Study 1. Write the actual research question. "Let's hear what customers think about X" is an agenda item, not a research question. A usable question has a specific, falsifiable answer: "Of these three positioning angles, which one drives the strongest intent to purchase?" or "What's the objection that most blocks adoption, and how much does it matter relative to the others?" 2. Define personas as specifications, not labels. A useful persona names demographics, category behavior (current solutions, usage frequency, brand preferences), attitudes, and knowledge state, for example, someone might specify: 35 years old, works as a marketing manager, has cycled through three separate project management tools over the past two years, still uses one of them today, is frustrated with its reporting, and needs clear proof of improvement before switching again. "A marketing professional" is not a persona; it's a label. Traditional practice runs three to five personas per study, enough coverage without spending the whole study on edge cases. 3. Keep the discussion guide short. A traditional in-room guide runs 12-15 questions across roughly 90 minutes. A tighter set, 6-8 questions structured as warm-up, problem exploration, concept introduction, probing, and comparison, produces cleaner signal per question than a long guide does, whether the sessions are moderated or simulated. 4. Run each persona independently. One structural difference from a group session: running personas one at a time removes groupthink and the dominant-personality effect that a moderator otherwise has to manage. Each persona produces an unfiltered, individual response to the same question set, the input a discrete choice model needs. 5. Analyze for pattern, divergence, and language. Look for what every persona says, or what none of them say (both are signals), where segments diverge, what's unexpected, and the exact words used. The words are the messaging input; the pattern and divergence are the finding. ## Can You Iterate Without Re-Fielding a Panel? A traditional focus group gives one round of feedback per fielded session; testing a revised concept means re-recruiting and re-booking. A simulated study can be re-run against the same personas with a revised stimulus without re-recruiting a panel from scratch, which matters most when an early round surfaces an unanticipated objection. ## When Does the Answer Still Need a Human Room? Group-dynamics research, video or non-verbal footage for stakeholder buy-in, and compliance contexts that require documented respondents all stay with a recruited panel; a discrete choice simulation is not built to produce any of those. Treating simulation as a universal replacement, rather than the right tool for the individual-attitude half of the research plan, is the mistake to avoid. ## Next Step Start with the research question, not the format. If it's an individual-attitude question, which positioning angle wins, which objection blocks adoption, whether the value proposition lands without an explanation, a discrete choice simulation can resolve it before committing budget to a group session built for a different job. Where the shortlisted answer is consequential enough to need a documented human check, that validation step runs on the same causal question, not a different one. See how the method is validated in past studies, or book a walkthrough to scope a specific research question. --- # AI Focus Groups: Uses, Limits, and a Practical Workflow URL: https://subconscious.ai/blog/implementation-and-procurement/ai-focus-groups-uses-limits-practical-workflow Published: 2026-08-24 An AI focus group is a simulated research panel. Defined audience models respond to questions, stimuli, and scenarios. The output can expose agreement, objections, and questions worth testing with people. A number without its limits is marketing copy. The method is directional. A claim that the whole study runs in minutes rather than weeks, or reaches 80 to 95 percent accuracy, both common in vendor marketing, needs a defined task, human baseline, and validation design. [figure: A five-stage path: define the audience, hold the stimulus constant, probe divergence, synthesize where responses agree and differ, then validate the surviving options with a human study.] ## What problem the method addresses Traditional focus groups can suffer from groupthink, social desirability, small samples, recruitment bias, and long fieldwork. Nielsen Norman Group identifies groupthink and social-desirability bias as the most common failure modes in focus group research. Naming the sample size is what lets a buyer check the estimate against it. Eight people produce qualitative observations, not a statistically projectable estimate. One planning example places a traditional group at €5,000 to €15,000 and 3 to 4 weeks. A participant saying, "Yes, I would pay €50 for that," is still stated preference rather than observed behavior. These limits make method choice important, not focus groups worthless. "Focus groups don't accurately predict future behavior. However, they can help gauge attitudes and guide future exploration, thus avoiding wasted research time." Therese Fessenden, Nielsen Norman Group, "Focus Groups 101" (source) ## A four-step simulated workflow ### 1. Define the audience Specify the roles, context, attitudes, and constraints that matter to the decision. In one planning example, a team builds 5 personas in 20 minutes. Treat the timing as an example from that setup, not a delivery promise. ### 2. Hold the experiment constant Present the same stimulus and question to each audience definition. If the study tests group effects, state what information each participant can see and when. ### 3. Probe divergence Ask follow-up questions when one response differs. Run the same session 10 times with different framings only when the study records the changed variable. ### 4. Synthesize without hiding variation Record where responses agree and where they differ. Divergence may signal a segment difference, a weak audience definition, or model sensitivity. It does not establish the cause by itself. ## What questions can AI focus groups help screen? Early concept tests can ask whether a problem and solution make sense. Message tests can compare headlines and value propositions. Objection mapping can capture the first three reasons a buyer might reject an offer. Competitive-positioning exercises can compare reactions to alternatives. Publishing where a model's read of a market stops is what lets a buyer plan around it. Localization work needs special care. A comparison among Germany, the UK, and the US is a model-based hypothesis until people in those markets validate it. ## Use real people for the final evidence The misses belong on the record next to the hits. Human research remains necessary when the question depends on observed behavior, body language, emotion, physical stimuli, sensory experience, or external proof that real customers participated. High-stakes decisions also need evidence proportional to their consequences. The useful sequence is simulated exploration followed by focused human validation. Subconscious frames this as decision-specific experimentation, not generic roleplay. Define the action, audience, and outcome, then test which action changes behavior under the study conditions. ## Planning examples for study design See worked examples of simulated-to-human study sequences before scoping a new one. A concept-screening example starts with three positioning concepts for a skincare line. Traditional research would recruit 30 to 40 women across two markets, run four groups, allow four weeks, and cost roughly €18,000. The simulated first pass uses a 5-persona group defined as urban consumers ages 25 to 40, then takes one concept into a focused 20-person human study. In that example, the team reports €12,000 saved and a schedule compressed from 4 weeks to 8 days. Those figures describe one planning example, not a current Subconscious price, saving, or delivery guarantee. A message test might compare five statements for a €120,000 campaign using 6 defined buyer perspectives. If two statements remain strong across all six perspectives, an A/B test can validate them in market. The simulated result should eliminate weak options, not replace behavioral evidence. A public-affairs exercise might compare three frames in two markets where traditional recruitment would cost €18,000 per market. The example uses 8 simulated perspectives per market, then a 200-person tracker after launch. A reported two-to-one difference is still directional until the human tracker confirms it. ## What questions should you ask vendors during procurement? Ask vendors how the audience is constructed, what data grounds it, how prompts and versions are retained, how validation works, and whether humans can inspect the transcript. Review data processing, security, retention, and sub-processors with the appropriate internal owners. Do not infer compliance from a vendor's location or marketing language. Do not accept claims of zero setup cost, unlimited participants, or a one-hour complete study without checking the contract and workflow. For budget planning, one comparison places online qualitative work at €2,000 to €5,000 over 1 to 2 weeks with 10 to 30 participants. The same comparison places do-it-yourself interviews at €500 to €2,000 over 2 to 4 weeks with 5 to 15 participants. Confirm the scope and local rates before procurement. [figure: A decision path: concept, message, and objection questions route to simulated screening. Localization claims and any question needing observed behavior, emotion, or proof route to human validation instead.] ## A practical starting point Use 5 to 10 simulated perspectives for a focused research question as a planning example. Fewer than 5 may hide divergence. More than 10 may repeat the same patterns. For segment comparison, one example uses 5 to 8 perspectives per segment and multiple panels. Setup might take about 20 minutes, an asynchronous pass may take minutes, and interactive follow-up may take 30 to 60 minutes. Compare them with a traditional 3 to 4 week recruit-and-field process only after defining the same scope. --- # Customer-Intelligence Dashboards vs. Controlled Experiments Before Launch URL: https://subconscious.ai/blog/comparisons/customer-intelligence-dashboards-vs-controlled-experiments-before-launch Published: 2026-08-24 A pricing, messaging, or launch decision needs evidence before it ships, not after it. Two different tool categories both get marketed as an "AI customer panel," and they answer that need at opposite ends of the timeline. ## Two categories, one shared label A customer-intelligence dashboard connects to first-party data (CRM records, product analytics, marketing signals) and turns existing customers into a segmented, continuously updated view of who they are and what they do (Native AI, "AI Customer Panel"). A controlled-experiment platform runs a study against a simulated population built to represent a target segment, whether or not a company has customers in it yet. Only one of them has anything to say before first-party data exists to feed it. ## What a dashboard needs before it is useful A dashboard is only as good as the data behind it. It ingests first-party signals and surfaces patterns in observed behavior: strong for understanding who a team already has and what they are doing, and offering nothing for a segment or product that has no customers yet. ## What a controlled experiment tests before that data exists Subconscious runs controlled experiments on simulated populations to estimate which pricing, messaging, or launch action is likely to change behavior, before a team has the first-party data a dashboard-style tool requires. The studies run against a person-level audience graph covering 800 million real people. That graph is distinct from any recruited real-human panel: it defines who can be represented in a study, not who was interviewed for it. The output is a causal effect with a confidence interval, comparing one specific alternative against another, rather than a continuous view of an existing base. ## Where the two compare directly ## Where each one breaks down A controlled experiment does not replace a voice-of-customer program. Subconscious does not ingest a continuous CRM or CDP feed, and it is not built to monitor an existing customer base over time. That is the dashboard's job, and it does it well once the data exists. A dashboard, in turn, cannot answer a pre-launch question: there is no signal to surface about customers or a product that does not exist yet. The two are complementary. A team with an established base and a mandate to track it continuously needs the dashboard. A team deciding what to price, or what to ship next, needs the experiment first, and can still stand up the dashboard once the launch produces customers to track. ## From simulated result to real-human confirmation When a decision is expensive enough to double-check, the same causal question can be tested twice: once against the simulated population, and again with real human participants, without changing what is being asked. Subconscious can test or validate studies with real human participants, which lets a team move from a fast simulated read to a confirmed one on the studies that matter most. ## A practical next step If the question is which version of a decision is more likely to work before it ships, that is a controlled experiment, not a dashboard. See how a study gets built for a specific decision in past work, or book a walkthrough of a study against a target segment. [figure: A branch on whether first-party data exists: no data leads to a controlled experiment yielding a causal effect with a confidence interval; data leads to a dashboard yielding a segmented behavioral view.] --- # Why Per-Test Bayesian Model Loops Stop Scaling URL: https://subconscious.ai/blog/methods-and-validation/why-per-test-bayesian-model-loops-stop-scaling Published: 2026-08-24 A data science team running many concurrent Bayesian A/B, ABC, and ABCD tests eventually hits the same wall: compute cost and batch runtime scale with the number of tests, not with the amount of insight the tests produce. Left unaddressed, that scaling problem forces a choice between throttling how many experiments run at once or waiting long enough for results that they arrive too late to change a decision. A documented HelloFresh case study on the PyMC Discourse forum shows how this problem was diagnosed and fixed in production, and is a reference for teams choosing between one independent model per test and a structurally consistent, shared model across test types. ## Where the original pipeline broke down HelloFresh ran large numbers of A/B, ABC, and ABCD test campaigns at once, each fit as its own independent Bayesian model. Diagnosing the existing models turned up strong correlations between some posterior parameters and high autocorrelation in the MCMC chains, both of which slow convergence and cut the effective sample size a team gets for a given compute budget. That combination of scale and inefficient sampling meant overnight batch runs took long enough that teams either waited past the point where results were still actionable, or limited how many tests they could run concurrently. Neither option scales with test volume. ## Why did the team fix the model before optimizing speed? The team's first move was a correctness pass, not a performance optimization. They proposed a structurally different Bayesian model that removed one parameter compared to the original formulation, used revised priors tailored to the problem, and applied the same structure consistently across A/B, ABC, and ABCD tests. The new priors also matched domain knowledge: for an ABC test, most prior mass sat on scenarios where the variants' conversion rates were close together, not far apart, while still spreading roughly evenly across the 0–1 range. Under the new model, MCMC chains showed well-behaved mixing, and the correlation and autocorrelation problems that had been slowing convergence were resolved. Before trusting the new model's output, the team ran parameter recovery simulations: known conversion probabilities were used to generate simulated data, and the posterior distributions were checked against those true values. For an ABC test, the posteriors were correctly centered on the ground truth. That result confirms correct implementation and identifiability under the assumed model; it does not by itself establish calibration or that the model matches the real conversion process. At this stage, the corrected model alone was already faster: roughly 1.2x for A/B tests and roughly 2x for ABC and ABCD tests. Worthwhile, but the team judged it insufficient given the batch's scale. ## Why didn't tuning knobs solve the speed problem? Two conventional speed levers were tried next, and both fell short. Cutting MCMC tuning steps from the default 1000 down to 100 saved only about 0.1 seconds per A/B test, not enough at HelloFresh's batch scale. Avoiding model recompilation by defining one PyMC model with data swapped in through pm.Data containers, rather than rebuilding a model object per dataset, improved engineering hygiene but produced a negligible speed gain in practice. Both attempts optimized how a single test was fit. The bottleneck was somewhere else: how many separate fits the pipeline ran. ## What was the structural fix that solved the scaling problem? The breakthrough was reframing how the batch was fit, not tuning any individual fit further. Instead of looping over datasets and running MCMC separately for each test, the team constructed one large unpooled model that fit all datasets simultaneously. In this unpooled structure, each test keeps its own statistically independent parameters (nothing borrows strength across tests), but every test shares the same model structure, so the pipeline runs one warmup and adaptation phase and evaluates gradients across all tests in a single vectorized computation instead of restarting the sampler for each one. The pipeline stopped looping over small models entirely; PyMC compiled and sampled one model encoding every test at once. This is the step that most changed the pipeline's scaling behavior: one larger run replaced many small independent runs carrying the same total data and parameters, and it was measurably more efficient. HelloFresh's own reported runtime figures for this change are documented in the PyMC Discourse discussion; treat them as that team's engineering result, not a Subconscious benchmark. ## What generalizes beyond this one pipeline The unified model also supported two-arm, three-arm, and four-arm comparisons under one consistent prior structure, so A/B, ABC, and ABCD tests could run through the same system instead of separate bespoke pipelines. That consistency simplified both implementation and interpretation, and kept inference accuracy verified by the same parameter recovery approach across test types. The general lesson transfers past this specific stack: when Bayesian test volume grows, look for the structural bottleneck, how many independent models a pipeline is fitting, before assuming the answer is faster hardware or a lighter-weight per-test model. ## Where this fits a causal-testing decision This case study describes one vendor's engineering work on one customer's pipeline; it is not a Subconscious capability, customer result, or benchmark, and Subconscious does not claim to reproduce this exact pipeline, tooling, or result for any customer. What it illustrates is the argument behind treating causal experimentation as software infrastructure rather than a one-off analysis script: a testing system that has to run at scale needs structural design decisions, not just faster per-test loops. Teams evaluating a causal-testing platform can apply the same standard: see how Subconscious approaches experiment design and what its validated results look like, and where a decision needs grounding beyond a simulated run, pair it with real-human validation on the same causal question rather than treating scale and rigor as a tradeoff. [figure: Four-step chain: correlated posteriors and autocorrelation cause slow convergence, which makes runtime scale with test count, forcing a choice: throttle tests or ship late results.] ## Limitations The proof points here, model diagnosis, structural redesign, and parameter recovery validation, come from a single vendor's single-customer engineering case study on the PyMC Discourse forum, not from a controlled study or a Subconscious result. Method boundaries also apply in the other direction: even a well-validated Bayesian pipeline like this one estimates conversion probabilities from the data it's given; it does not substitute for real-human validation when the buyer's decision depends on confirming a causal question with actual people rather than historical conversion data alone. For general background on the underlying test design, see A/B testing. [figure: Chain: many per-test models cause slow MCMC mixing, fixed by a corrected model for a modest speedup. Two tuning attempts branch off with negligible gain. Chain ends at one joint model fitting every test at once.] --- # 10 UserInterviews Alternatives for Restructuring Your 2026 Research Stack URL: https://subconscious.ai/blog/comparisons/10-userinterviews-alternatives-restructuring-2026-research-stack Published: 2026-08-24 UserInterviews is still the default recruited-panel platform for qualitative research, and the first line item most teams reconsider when they replan their 2026 research stack. The real question isn't whether to drop it, but which category to add alongside it or substitute for it: a synthetic-first tool for fast iterative testing, a different recruited-human panel for the same job UserInterviews already does, or a hybrid stack that reserves recruited humans for high-stakes or hard-to-reach segments. Get this wrong in either direction and it costs you. Treat a plausible synthetic opinion as validated evidence for a launch, price, or positioning call, and you ship on an untested assumption. Keep paying recruited-panel rates for decisions that don't actually require an individual qualitative voice, and you waste research budget while slowing iteration cycles. ## Why Teams Look Beyond UserInterviews - Speed. Recruitment cycles for niche audiences still run 2 to 5 days. Synthetic-first tools respond in minutes. - Cost per participant. Recruited-panel fees are billed per session, and they add up fast at product-manager iteration cadence. - Coverage on niche segments. Hard-to-reach B2B audiences, such as CFOs, security leads, and procurement, are hard to recruit economically through a general consumer panel. - Iteration cost. Testing five concept variants means five separate recruitment runs on a traditional panel. - International coverage. UserInterviews' panel is strongest in the US; European and APAC coverage is thinner. ## The 10 Alternatives Teams Are Actually Evaluating ### 1. Subconscious, for testing which action actually moves behavior before you commit budget to it A tool that names what it doesn't do is one a buyer can check against their decision before they commit budget. Subconscious isn't a like-for-like UserInterviews replacement, and it doesn't run 1:1 qualitative interviews. It fits the slice of this decision where the question is which pricing, messaging, or product action is more likely to change behavior: controlled experiments run against a simulated market, backed by studies against a person-level audience graph covering 800 million real people. High-stakes decisions can validate the same study against real human participants without changing the underlying causal question. Best for: teams deciding a launch, price, or message before they commit spend to it, not teams that need an individual's unscripted qualitative voice. ### 2. What Is Respondent? The closest like-for-like alternative to UserInterviews: a recruited, real-human panel on a different platform. Source material describes stronger B2B coverage in some categories. Pricing is per participant, the same recruited-panel model Respondent itself runs on. Best for: teams that want recruited real humans, on a different panel. ### 3. Synthetic Users, a UX-focused synthetic-respondent tool Qualitative AI respondents in a self-serve workflow, positioned for product teams that want qualitative-style user research without recruitment overhead. Best for: product teams running qualitative research without a recruitment cycle. ### 4. Prolific, an academic-grade survey panel A recruited panel built for academic-grade audience targeting, positioned for survey-heavy research at volume. Best for: teams running survey-style research at higher volume. ### 5. dscout, diary and ethnographic research Mobile-first diary and ethnographic research, a different shape of study than UserInterviews' interview format, and often complementary to it rather than a substitute. Best for: longitudinal, in-context research. ### 6. UserTesting, video-led usability with a built-in panel A heavier, full-funnel platform: video-led usability testing paired with its own recruited panel. Best for: teams that want video, a recruited panel, and a platform in one contract. ### 7. Evidenza, synthetic B2B decision-maker panels Founded by the former LinkedIn B2B Institute team, Evidenza runs synthetic respondents modeled on B2B roles, such as CFOs, IT decision makers, and procurement, that are difficult to recruit economically through a general panel. Best for: B2B teams researching enterprise decision-maker audiences. ### 8. Maze, real-user prototype testing Live usability tests on prototypes with real users, often paired with a synthetic-first tool: synthetic exploration first, then real-user validation on the prototype that survives it. Best for: PMs running prototype tests with real humans. ### 9. Lyssna, lightweight real-human checks Self-service testing with real people, covering preference checks, first-click reads, five-second impressions, and short surveys. Best for: teams that want a low-cost way to check design or copy with actual people. ### 10. OpinioAI, a budget-tier synthetic focus group tool AI-moderated synthetic focus groups priced at the low end of the category, aimed at boutique agencies and early-stage teams. Best for: boutique agencies and early-stage startups with a tight research budget. ## Which Research Need Points to Which Tool [figure: Branching path: the question "what does this decision need" splits into four routes, each ending in a different category of replacement tool for UserInterviews.] ## What Stack Pattern Is Showing Up in 2026? Most teams don't fully replace UserInterviews on day one. The pattern we see most often: add a synthetic-first tool as the daily driver for iteration, and keep a recruited panel under contract for the moments that still need a person to stand behind the answer, among them final launch checks, pricing calls with real stakes, and compliance-sensitive work. That panel contract typically shrinks in volume over time as the daily-driver tool absorbs routine iteration, and gets renegotiated at renewal. ## What Doesn't Subconscious Do In This Stack? Published limits are what let a research lead map this tool against the rest of their stack before they buy it. Subconscious does not run 1:1 qualitative interviews, moderated video sessions, or longitudinal diary studies. It doesn't replace a recruited-panel platform for discovery-stage research, where hearing a real person's unscripted language is the point rather than a distraction from it. Real-human validation on Subconscious extends a causal question already under test; it doesn't turn the platform into a usability lab or a clinical trial. ## Making the Call for Your 2026 Budget If most of your research spend is going toward individual qualitative voice, such as discovery interviews, diary studies, and moderated usability sessions, a recruited panel like Respondent, dscout, UserTesting, Lyssna, or Maze stays the right tool. If a meaningful share of that spend is going toward deciding which pricing, message, or product action to ship, that's a different question, and it's worth testing it as one. See how the studies run before you decide where the next research dollar goes. --- # AI Purchase Intent Detection: How It Works URL: https://subconscious.ai/blog/use-cases/ai-purchase-intent-detection-how-works Published: 2026-08-24 Purchase intent detection estimates whether a person, account, or market segment is likely to buy. Most intent systems make that estimate from evidence created after buyers begin acting. A launch team has a harder question: which audience and message should receive budget before those actions exist? Synthetic pre-launch testing can supply an early directional signal. It belongs before behavioral scoring, then must be checked against real outcomes once the market responds. [figure: A five-step decision path: no buyer activity yet leads to a synthetic pre-launch test, then choosing a segment and message, then launching and observing buyers, then calibrating against real outcomes.] ## The missing input in a pre-launch intent stack Live intent models can draw on first-party activity, firmographic priors, and third-party research signals. Those inputs become useful only after there is a buyer or account to observe. The immediate problem is not how to score an active account. It is how to choose an initial segment, message, or position before committing GTM budget. Launching without evidence risks backing the wrong audience. Treating an untested synthetic result as observed buyer behavior creates the same exposure behind a more precise-looking number. ## How does a synthetic purchase-intent test work? A controlled experiment presents alternatives to a simulated market and measures how the intended outcome changes. For purchase intent, the alternatives might be audience definitions, messages, or product positions. The output is comparative: it indicates which tested action is more likely to move the stated outcome. Subconscious supports this early decision with causal action testing: controlled alternatives run against a person-level audience graph covering 800 million real people, not a recruited panel. Once real activity appears, teams should compare the early result with observed outcomes and let those observations govern ongoing scoring. See the workflow from question to experiment. ## What does credible evidence look like? An accuracy number is useful only when its benchmark and measurement are clear. Independent research on machine-learning approaches to purchase-intent and consumer-behavior prediction shows that model accuracy depends heavily on the features, training data, and evaluation method used, and that no single benchmark transfers across products or markets (Application of machine learning in predicting consumer behavior and precision marketing, PMC). "The results show that CatBoost and XGBoost have the best prediction results when dealing with complex features and large-scale data, F1 scores are 0.93 and 0.92 respectively, and CatBoost's ROC AUC reaches the highest value of 0.985." Jin Lin, PLoS One (source) Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. Replication accuracy means how closely simulated studies reproduce the direction and outcome of the original human study, drawn from roughly 300 replicated studies across 9 domains. The method and supporting evidence are available in the causal fidelity paper. This is the relevant proof standard for the simulated studies, not a claim that every buyer or segment prediction matches that ratio. Subconscious can also test or validate studies with real human participants, so a team can move from the simulated pre-launch result to real-human validation without changing the causal question being asked. ## Decisions the signal can inform The test is most useful when the team names an action before running it: - Choose a starting segment. Compare defined audiences before allocating campaign spend. - Select a message. Test competing propositions against the same purchase-intent outcome. - Set a calibration point. Record the pre-launch result, then compare it with real buyer behavior as it arrives. They do not turn a market-level experiment into an individual lead score. ## What guardrails apply when acting on the result? Calibration against real outcomes is required when those outcomes become available. Known failure modes include variance collapse, demographic flattening, prompt sensitivity, and weaker fidelity for individuals than for aggregates. Segment-level performance therefore cannot establish individual-level accuracy. The study design also determines what can be reported. Teams should not assume that confidence intervals, segment heterogeneity analysis, or a finished decision memo are standard outputs. They should ask which alternatives were tested, which outcome was measured, and what real-human evidence will be used to challenge the result. [figure: Diagram: a segment-level purchase-intent result branches to three valid uses (choose a segment, select a message, set a calibration point) and one invalid use, individual lead scoring, marked out of scope.] ## Scope the decision before the study Start with one launch choice: a defined set of segments, messages, or positions and a purchase-intent outcome that distinguishes them. Review the underlying research approach, then bring that exact decision to a demo. The useful deliverable is evidence for the next allocation decision, followed by a plan to calibrate it with real buyer behavior. --- # Persona Simulation Tools in 2026: What They Answer, and When You Need More URL: https://subconscious.ai/blog/use-cases/persona-simulation-tools-2026-what-they-answer-when-need-more Published: 2026-08-25 A marketing, product, or research leader picking a persona simulation tool in 2026 is deciding something narrower: is a directional impression from a queryable AI character enough evidence to greenlight a launch, feature, or price change, or does the decision need a controlled experiment with a measured outcome? The answer depends on how much budget and engineering time are riding on being right. [figure: Flowchart: four tool categories each give an impression or document, funneling into one question, then to a controlled experiment with a confidence interval.] ## What do persona simulation tools do? Persona simulation tools use AI and data to build queryable, virtual stand-ins for customers, users, or stakeholders. A team can chat with the stand-in, run it through a group setting alongside other stand-ins, or ask it to react to a message. That differs from a static persona document, a profile someone reads once for alignment; a simulation is something a team can interrogate repeatedly as questions change. The 2026 market splits into four categories. ## Choosing a category Choose an interactive AI persona platform for a fast directional read on messaging or a concept without a research specialist: narrowing ten headline options to three, or checking whether an objection is worth addressing before a bigger test. Choose a data-driven generator when the team already has analytics, CRM, or social-listening data and wants personas that update as that data changes. Pick a template builder when the deliverable is a deck-ready profile for a workshop or pitch, not something to query. Choose a specialized simulator when the use case is narrow (documentation testing, code-first scripted scenarios) and the team has the engineering capacity to integrate it. ## Four questions to push on with any vendor - Interactivity. Does the persona respond to queries the team runs directly, or does it just sit as a static document? - Validation. Does the platform publish how its simulated responses compare to real human responses, and against what benchmark? - Speed. How long from signup to a usable directional read? Traditional qualitative research runs three to four weeks per round, a baseline for comparison rather than a claim about any specific tool's turnaround. - Team access. Can marketing, product, and sales run the tool directly, or does every query route through a research specialist? ## When does a directional read stop being enough? An interactive persona platform answers "what might this character say," not "which specific option causes more of the outcome we care about, and by how much." A chat-style reaction to one message doesn't establish that the message caused a measurable lift over an alternative; it reports one simulated character's plausible response. Subconscious runs randomized, controlled experiments on a simulation of the market, comparing defined alternatives across a defined population, and reports the causal effect with confidence intervals. That approach follows the same discrete-choice experiment design used in peer-reviewed research practice for estimating how people trade off defined attributes between real alternatives (ISPOR Conjoint Analysis Good Research Practices Task Force report). The distinction matters most when the cost of being wrong is high: a positioning change, a price test, or a launch decision where budget and engineering time are already committed by the time results come in. Methodology and worked examples are at /research and /case-studies. When scale matters, those controlled studies can run against a person-level audience graph covering 800 million real people: the population available to a simulated study, separate from recruiting real human participants. ## What are the limitations of persona simulation tools? A controlled causal experiment does not replace customer discovery calls, moderated qualitative research, or watching how a change performs once it's actually in market. It is also not, by itself, real-human validation. Subconscious can test or validate the same study with real human participants, letting a team move from a simulated study to that validation without changing the underlying causal question, useful when a decision warrants a second, independent check before committing. None of this replaces open-ended exploratory conversation with a persona when the goal is narrowing options fast; it matters once the next step is a specific, resourced decision. Teams weighing that trade-off can see how Subconscious structures a study or book time to scope one. --- # 5 Questions That Decide Between Synthetic Personas, Panel Data, and Causal Testing URL: https://subconscious.ai/blog/comparisons/5-questions-decide-between-synthetic-personas-panel-data-causal-testing Published: 2026-08-25 A research lead comparing tools starts from the wrong question: "which platform is better?" The better question is what the decision needs: a directional hunch, a measured audience profile, or proof that an action changes an outcome. Picking the wrong tool wastes budget or time. ## Three ways to research a market decision Directional synthetic-persona tools generate AI-driven responses from modeled knowledge about a market or segment. Real-panel audience intelligence platforms, such as YouGov Profiles, collect responses directly from permissioned human panelists and refresh core variables on a regular cadence. A third category, causal behavioral experimentation, runs randomized studies designed to isolate which action changes a decision, not describe an audience or generate a directional response. A comparison that lists only strengths reads like marketing. This one states where Subconscious stops, so a buyer can check it against the decision in front of them. Real-panel platforms remain the stronger source when a decision needs direct human evidence, ongoing trend tracking, or activation into ad platforms. Subconscious does not replace panel recruitment, weekly-refreshed demographic tracking, or direct ad-platform activation (YouGov Profiles). ## Five questions that route the decision ### 1. Do you need a hunch or a measurement? If the goal is to surface objections, screen concepts, or generate hypotheses before committing budget, a directional persona tool fits. If the goal is to measure an existing audience's reported attitudes over time, a panel-based platform fits. Neither answers whether a specific action (a new price, a new message, a new feature) changes what someone chooses; that needs a designed experiment. ### 2. Does the answer need to hold up as causal proof, not just a directional read? A directional persona response can suggest an idea is worth testing. It cannot tell a team a message caused a change in intent; it was never designed to isolate cause. A randomized experiment is designed for that: it varies one thing at a time and measures the resulting change, which a pricing or launch decision requires before it ships. ### 3. Is statistical representativeness required for this decision? Panel-based platforms give direct, permissioned human responses, supporting representativeness claims that directional synthetic output cannot make. A causal platform closes that gap: Subconscious runs controlled studies against a person-level audience graph covering 800 million real people, and can validate a study with real human participants when needed. Naming exactly when validation is required, and when it isn't, is what makes a causal claim checkable rather than asserted. That step matters when the answer would depend on trusting simulated behavior alone; for most exploratory work it changes nothing. ### 4. Do you need to activate the audience in ad platforms, or decide what to do first? Panel platforms with recontact and API access hand audiences to campaign activation. A causal platform handles the step before that: deciding which price, message, or feature to activate. Treat these as sequential jobs, not competing ones: decide with a causal test, then activate with a platform built for reach. ### 5. What stage is the decision at: ideation, measurement, or pre-launch validation? Early ideation tolerates a directional signal; the stakes are low. Ongoing measurement and segmentation need panel-grade evidence collected over time. Pre-launch validation, the point where a wrong call is expensive, needs a causal answer: a randomized study on the specific action, checked against real human behavior when the stakes justify it. ## Where each method stops A method that only claims wins is a pitch. Listing where each one stops is what puts the misses next to the hits, in public. None of these three approaches replaces the others. Directional persona tools do not produce statistically representative audience data. Panel platforms do not isolate cause; a reported attitude is not the same as a demonstrated causal effect. Causal experimentation still depends on a well-specified study design; a poorly framed question produces a precise answer to the wrong problem. Matching the tool to the question, not defaulting to whichever is fastest to open, keeps a launch, pricing, or positioning call from resting on the wrong kind of evidence. Teams that need to test a specific action before a launch decision can see how a causal experiment is structured, review how a study moves from design to validation, or talk through a specific decision. [figure: A branching path with four research needs, each routing to a tool: hunch to persona tools, measured audience to panel platforms, proof of causal change to causal experimentation, activation back to panel platforms.] --- # Do Simulated Patient Preferences Match a Published DCE? The Adam Replication URL: https://subconscious.ai/blog/case-studies/simulated-patient-preferences-match-published-dce-adam-replication Published: 2026-08-25 A health-services researcher or HEOR/medical affairs team deciding whether to trust a synthetic discrete choice experiment on patient treatment preferences needs a check against real patient behavior before that ranking informs a patient-experience or communications program. Adam et al. ran a published discrete choice experiment (DCE) on treatment-process attributes with patients in Berlin and Munich, comparing complementary and conventional medicine. An internal synthetic run on the same design produced a rank ordering that can be checked directly against the original human ordering. A ranking without its run details is marketing, so population size, run date, and method are not published here. [figure: Two ranked columns, one from the human DCE and one from the synthetic DCE, showing active listening, time with provider, and cost in matching order across both columns.] ## The decision this replication informs Before committing budget to a full human fielding, a market-access or health-system team can run a synthetic version of the same choice design first. The question isn't whether the synthetic run is correct in isolation. It's whether its ranking of attributes agrees with how real patients in a published study actually chose. ## What did the original human study measure? Adam et al. fielded the DCE with patients in Berlin and Munich, asking them to trade off active listening, time with the provider, and cost when choosing between complementary and conventional medicine. The published result is a ranking of which attributes patients weighted more heavily. "DCE results showed that the treatment process attributes 'active listening' and 'time' were most relevant to all patients." Adam and colleagues, Patient (2018) (source) ## What did the synthetic replication compare? The misses matter as much as the hits, so a synthetic version of the same DCE design was run against a simulated population, with population size and run date not published here. The outcome of interest is the rank ordering agreement between the two orderings, not a re-derivation of the original study's individual respondent data. The synthetic ranking and the published human ranking agreed in direction. Naming what a comparison does not cover is what lets a buyer check it: it is not a claim that the synthetic run reproduces the original study's respondent-level data or extends automatically to a different patient population. ## Where this fits into a validation workflow A team doesn't have to choose between a synthetic study alone and a full human field study upfront. Subconscious can move a team from a simulated experiment to real-human testing or validation without changing the underlying causal question, using a synthetic DCE as a first-pass check against a published study like this one before deciding whether a new population needs its own fielded validation. Related synthetic-to-human comparisons are collected in case studies, and the underlying method is documented on the research page. ## What does this replication not establish? A published result is only useful to a buyer who can see its edges, so a rank correlation against one published study is not a live clinical validation. It does not certify accuracy for a specific new health-system decision, and it does not substitute for direct patient research on the buyer's own population. The comparison speaks to whether attribute ordering agreed in direction between the two studies. It does not speak to individual-level prediction, calibration for a different patient group, or performance across a new set of treatment-process attributes. --- # A Persona Document Names the Buyer. It Doesn't Test the Decision. URL: https://subconscious.ai/blog/comparisons/persona-document-names-buyer-doesnt-test-decision Published: 2026-08-25 A buyer persona document is not enough evidence to greenlight a campaign, message, or pricing decision. It organizes what a team believes about its buyer, not which action will change that decision. Use the document to align the team on whom it wants to reach, and a decision-specific experiment for evidence on what to do next. [figure: Four-step path: Population (buyers concerned), Action (choice under consideration), Comparison (credible alternative), Outcome (what would change the decision). Persona feeds only Population.] ## The two artifacts solve different jobs HubSpot presents its free offering as a buyer-persona template generator. Its output is an alignment artifact: a shareable profile that captures the target buyer's role, goals, challenges, and motivations. A causal behavioral experiment compares defined actions for a defined buyer population and estimates which action changes the decision-specific outcome. ## The costly category error A persona can say that a buyer values simplicity. It does not show that a simplicity claim will change preference, adoption, or purchase choice relative to another claim. The category error happens when a team treats the description as validation, selecting a campaign or price because it sounds consistent with the profile, then committing budget without testing the action. If the assumption is wrong, the team learns only after the market has seen the decision. Subconscious, a causal behavioral platform, helps teams test product, pricing, messaging, and go-to-market actions before committing capital, comparing controlled alternatives and estimating causal effects. ## How do you turn a persona into a testable decision? A persona can frame an experiment if the team converts its assumptions into four choices: - Population: Which buyers does the decision concern? - Action: Which message, price, or product choice is under consideration? - Comparison: What credible alternative will the team test against it? - Outcome: Which choice or behavior would change the decision? This framing keeps the persona in its proper role: defining the buyer population and generating hypotheses, not evidence for an action's effect. ## Demand proof in proportion to the decision A number without its assumptions is marketing copy. A simulated experiment remains evidence under a model, study design, and set of assumptions, not automatic proof of market performance. Teams should inspect whether the experiment was replicated or validated, where it failed, and how uncertainty was estimated. Confidence intervals belong in the result only when the current study design supports them. For decisions that need another layer of evidence, Subconscious can test or validate studies with real human participants. A team can move from a simulated experiment to real-human validation without changing the causal question, though the recruited participants and the simulated population remain distinct evidence sources. Naming what a method can't do is what lets a buyer check it against the decision at hand. Neither one turns the study into a usability session, clinical trial, or guarantee of the launch result. ## How do you choose based on the decision at hand? Choose a persona generator when the deliverable is a document for a workshop, pitch, or internal brief. Scoping what a tool does not do is what keeps a buyer from misapplying it. Subconscious does not produce that artifact. Add a causal experiment when choosing among actions and the cost of choosing poorly is material. Keep the buyer definition from the persona. Test the campaign, message, feature, or pricing scenario as a controlled comparison. Escalate to real-human validation when the evidence's stakes and limits warrant it. If you have a defined buyer, competing actions, and an outcome that matters, bring the decision to Subconscious. --- # AI Market Research for Startups: Faster Decisions Without False Certainty URL: https://subconscious.ai/blog/use-cases/ai-market-research-startups-faster-decisions-without-false-certainty Published: 2026-08-25 Startups need customer evidence before they can justify a large research budget. Causal behavioral experiments compare early product, pricing, and message decisions before a team commits capital. The result is a sharper hypothesis for the human research that follows, not a reason to skip real customers. [figure: Four boxes labeled problem/solution, pricing/packaging, segments/positioning, and message/landing-page copy, each with an arrow pointing into one box labeled human validation via interviews, tests, and product data.] ## Why startup research gets deferred A basic qualitative study runs $10,000 to $40,000 (Drive Research's 2026 market research cost guide), and a brief typically needs four to eight weeks to reach insight (Drive Research). That timing conflicts with two-week sprints, where even four weeks can span several product decisions. Founders often replace formal research with a handful of friendly conversations. Those conversations do not establish a market-wide pattern. The goal is to separate exploration from validation and use the right evidence for each. ## Decisions to test early ### Problem and solution hypotheses Define the audience, the behavior that matters, and the action under consideration. Ask how often the problem occurs, what the current workaround is, and what would cause a buyer to switch. An example problem-framing session takes two to three hours. ### Business model and pricing scenarios Compare specific alternatives rather than asking whether one price “seems fair.” Test monthly versus annual packaging, the evidence a buyer would need before committing, and the alternatives they already consider. Treat the result as a directional scenario comparison, not willingness-to-pay proof. ### Segments and positioning Compare the same product or message across distinct roles, industries, company sizes, or contexts. The point is to find where an action changes response, not to turn demographic labels into fixed truths. ### Message and landing-page choices Test concrete copy variants before paying for production or media. Ask what each version communicates, which audience it appears to address, and what action a buyer would take next. [figure: Five-step path: a five-question brief feeds a causal experiment; only questions with a real difference branch into human validation (interviews, tests, product data); that output loops back into a sharper next question.] ## Pair simulation with human evidence Use behavioral simulation to generate and prioritize hypotheses. Validate important findings with customer interviews, usability tests, experiments, or observed product data. Novel markets and unusual early adopters remain hard to represent. A startup’s first ten customers may differ sharply from the average buyer. A simple starting brief contains five questions about the most important unknowns. Run the comparison, note what changed your view, and use those findings to improve the next real customer conversation. An example end-to-end working session takes two to three hours. The aim is better decisions, not a synthetic stamp of approval. --- # How Agencies Decide Which RFPs to Pursue, Decline, or Partner URL: https://subconscious.ai/blog/buyers/how-agencies-triage-rfps-simulated-stakeholders Published: 2026-08-25 Agencies decide by testing a draft pitch angle against the prospect's actual buying group, then sorting the result on two signals, angle strength and comparative standing, into pursue, rewrite, partner, or decline. A borderline RFP lands in the inbox: pursuable, not obviously winnable. The question before anyone opens a slide template: does this one earn a full pitch team, get declined, or go to a partner? Answering that late, after the deck is half built, wastes the same margin either way. ## Pitching everything is a real cost Unbilled pitch work is not free labor. One analysis of agency pitch economics put the average cost of a single competitive pitch at roughly $44,000 in unbilled staff time, and full-year pitch investment at close to $475,000 for a typical agency, consuming about 17 percent of the revenue those pitches eventually win (Duval Partnership). Industry-norm win rates on competitive pitches run 25 to 33 percent, by the same analysis. Deal size compounds the problem: broader B2B win-rate research shows that as the value at stake rises, the odds of winning fall, with the largest opportunities converting at roughly half the rate of smaller ones (Landbase). Committing a full team to every RFP is a bet the math does not support. ## Sort the brief before anyone opens a slide deck Not every RFP deserves the same scrutiny. Four categories cover most of a pipeline: - Clear win. An existing client expanding scope, or a warm relationship with an obvious fit. Proceed without extra review. - Worth the fight. Strong category fit, a real budget, and no entrenched incumbent standing in the way. This is where sharpening the angle before the deck locks pays off most. - Genuinely uncertain. Mixed signals on fit, an opaque buying process, or an incumbent that may or may not be dislodgeable. This group benefits from testing before committing. - Pass. Wrong category, wrong size, no visible budget, or a brief that reads like the decision is already made. Decline quickly and move on. ## Why test the angle before finishing the deck? For the uncertain middle group, the goal is to find out whether a draft positioning angle survives contact with the prospect's decision-makers before spending real production budget on it. Subconscious can run a causal test of that draft angle against a modeled audience built from the roles who will actually weigh in on the award: the primary decision-maker, the people who will use the work day to day, and whoever signs off on spend (the causal fidelity paper). The test compares the draft angle against the objections and gaps those roles are likely to raise, with uncertainty language attached where the study design supports it. Build the audience from what the brief and public role information show. A useful version of that lineup, drawn from a typical mid-market brief, might include two or three people who own the final decision, one or two who will work with the output directly, one person evaluating technical or legal fit, and one or two who control the budget. Treat that lineup as a starting template; every brief signals its own buying group. Write the angle down before testing it: the reframe of the problem, the recommended approach, and roughly what the engagement looks like. Keep it to a page. Testing early finds the weak spot before it is buried in fifteen slides of production value. ## How do you read the test results? Two signals decide what happens next: whether the angle answers the prospect's real problem, and whether the agency compares favorably against the alternatives the prospect is considering. Combined, they point to one of four actions. [figure: 2x2 grid: angle strength vs comparative standing. Each cell names a next action, from pursue as drafted to decline or partner.] A capability gap here is worth a second look before it becomes an automatic decline. If the missing piece is a real hole in the agency's offering, that argues for a partner or a pass. If it is a capability the agency has but buried in the pitch, the fix is a rewrite, not a walkaway. ## What a structured test catches that an internal review misses Internal pitch reviews still matter for tactics, craft, and rehearsing the room, but they carry three built-in blind spots. The internal team wants the agency to win, which makes it reluctant to voice the harshest objection. The team has already seen earlier drafts of the angle and cannot read it cold the way a prospect will. And the team has no structured way to compare the pitch against what competing agencies are likely to offer. A modeled test of the prospect's buying group has its own limits, including sensitivity to how the brief is framed, but it adds a different kind of signal than a closed-door review can produce. For a higher-stakes RFP, a team that wants more confidence before the pitch is finalized can extend the same test to a recruited group of real people without changing the underlying question. Subconscious supports that move from a modeled test to human validation as one continuous causal experiment, not a separate study (how-we-work). ## What this does not do This is a test of a draft angle, not an automated recommendation engine. Subconscious does not package a ranked shortlist or an auto-generated decision memo from this workflow; someone on the new-business team reads the result and makes the call. There is no published case result or benchmark yet showing how this workflow performs against actual pitch outcomes. And the test cannot see everything that decides an award: existing relationship strength, internal politics at the prospect, or agency-of-record history. ## Where to start Pick the next uncertain RFP in the pipeline. Draft the one-page angle, test it against the buying group before the deck is built, and treat the result as one input alongside the team's own read of the account. Teams that want to see how the testing works before committing a real brief can review Subconscious's published study results or book a walkthrough. --- # Looking for a Simile Alternative? Ask What Proves the Simulation First URL: https://subconscious.ai/blog/comparisons/looking-simile-alternative-ask-what-proves-simulation-first Published: 2026-08-25 Teams searching for a Simile alternative aren't shopping for a cheaper clone. They're trying to make a pricing, messaging, or launch call this quarter, and the deep-research, enterprise-simulation category is built for a different buyer: a dedicated research function, a long procurement cycle, and a budget line to match. Simile positions itself as an AI simulation platform for testing human behavior, built around agents trained on real human interviews to model how a population responds. That grounding is a real strength, and it's also why the product is sold through a services engagement rather than self-serve, out of reach for a team that needs an answer before the next planning meeting. ## The real question isn't speed, it's proof Every synthetic-research vendor claims high fidelity to real human behavior. Almost none show their work in public. A buyer comparing tools rarely sees the actual gap between a simulated answer and what real people did, on the decision they're making, failure cases included. The question worth asking is not "how quickly can I get a persona running," but "what happens when I check this simulation's answer against a real human study, and does the method hold up when it's wrong." ## Where does a causal behavioral platform fit? Subconscious is a causal behavioral platform. Instead of asking an AI persona what it would do, it runs a controlled discrete-choice experiment against a simulated population, so the output is a causal estimate of which action moves an outcome, not a paragraph of simulated opinion. The practical advantage: a team can move from a simulated experiment to a real-human validation study of the same causal question without redesigning it. That closes the gap that makes persona-style tools hard to trust for a decision with real money behind it: the buyer never has to guess whether the simulated answer would have held up. ## What does a causal behavioral platform not solve? A causal behavioral platform isn't a replacement for population-scale generative-agent research trained on large interview corpora, and it doesn't publish a persona library for open-ended interview roleplay. If the research question is about simulating a large, heterogeneous population at that depth, an enterprise simulation vendor with that research grounding remains a reasonable fit, procurement timeline included. Subconscious also doesn't claim a specific setup time, self-serve price point, or accuracy benchmark not published on a current, sourced evidence page. Any number attached to a decision like this should come from a page you can check yourself, not from a vendor's back-of-comparison table. ## A framework, not a leaderboard [figure: Simulation platforms output a persona's self-description, checked by internal claims. A causal platform outputs a causal estimate, checked against real-human validation.] ## Start with the decision, not the vendor Before evaluating any synthetic-research tool, including Simile, write down the actual decision: the action considered, the cost of getting it wrong, and what would count as proof either way. Then ask each vendor, including Subconscious, to show, not claim, how its simulated answer holds up against real human behavior on that question. A platform that can walk you from the simulation to the validation without changing the question has already answered the harder half of "can I trust this." Related reading: research, leaderboard, case studies, or book time to walk through a specific decision. --- # Bayesian Spatial Modeling for Evaluating Hockey Goaltending Performance URL: https://subconscious.ai/blog/methods-and-validation/bayesian-spatial-modeling-evaluating-hockey-goaltending-performance Published: 2026-08-25 A goalie's save percentage answers a narrow question: what share of shots did they stop? It does not answer the question a general manager needs answered: did this goalie perform well given the shots they faced? A goalie behind a strong defense sees fewer dangerous chances and looks better than their true skill. A goalie facing a barrage of high-danger rebounds looks worse. The raw number conflates skill with the difficulty of the job, and it reports that conflation as a single confident figure with no sense of how much of it is noise. The same problem shows up anywhere a team judges performance from a raw outcome rate: a sales rep's close rate without adjusting for lead quality, a campaign's conversion rate without adjusting for audience, a support agent's resolution time without adjusting for ticket complexity. Trusting the raw number rewards position over skill. ## How do you separate shot difficulty from goalie skill? One worked analysis of a full NHL season addressed this by building a spatial model of shot danger, then measuring each goalie against that model instead of against a league-wide average. The dataset covered shots from the 2023-2024 season: 121,670 raw shots across 99 goalies and 921 shooters, reduced after filtering out empty-net, power-play, and non-offensive-zone shots to 91,247 shots, a 75.0% retention rate. The core idea: model the probability of a goal as a function of where the shot came from, using a Gaussian process. A Gaussian process learns a smooth surface over the ice rather than a single number: it estimates goal probability at every point in the offensive zone, plus how uncertain that estimate is, based on nearby observed shots. Points close together on the ice are assumed similarly dangerous; the model lets the data determine how fast that similarity decays with distance. Fitting an exact Gaussian process to tens of thousands of shots is computationally expensive, so the analysis used a Hilbert Space Gaussian Process approximation, which projects the surface onto a finite set of basis functions to make the computation tractable at that scale. ## What does the baseline model get wrong? A location-only model has an obvious flaw: it treats the ice as stationary, meaning it assumes danger changes at the same rate at every distance from the net. That is not true near the net, where danger rises sharply over a short distance, compared to farther out, where it changes far more gradually. The analysis corrected for this with coordinate warping: transforming the input coordinates with an arcsinh (inverse hyperbolic sine) function of distance from the goal, using a 5-foot offset and a 20-foot scale parameter, so the model could vary its effective sensitivity to location faster near the net and more slowly farther out. From there, the model added a rebound indicator, since a shot immediately following a rebound is a different kind of scoring chance than an unassisted shot from the same spot. Rather than adding a constant rebound bonus everywhere on the ice, the analysis modeled the rebound effect as its own spatial surface and combined it multiplicatively with the baseline danger surface, so a rebound in the slot, where baseline danger is already high, gets amplified more than a rebound from the corner. Finally, the model added a hierarchical shooter effect: a per-shooter adjustment, estimated jointly across all shooters so that shooters with few recorded shots borrow statistical strength from the overall distribution rather than producing a noisy individual estimate. That hierarchical structure (modeling many related units together instead of one at a time) is the same discipline behind treating any small-sample group (a new sales territory, a newly launched product SKU) as informed by, but not identical to, the population it belongs to. ## What is Goals Saved Above Expected? Once the model estimates expected goal probability for every shot a goalie faced, it can compute Goals Saved Above Expected: the sum of expected goals across all shots faced, minus goals allowed. A positive value means the goalie prevented more goals than the model expected given shot quality; a negative value means they allowed more. Goals Saved Above Expected is one of the standard advanced hockey metrics for separating goalie performance from shot quality. The distinguishing feature of the Bayesian version of this metric is not the point estimate. It is the interval around it. Because the underlying goal-probability model produces a full posterior distribution rather than a single number, the resulting Goals Saved Above Expected figure comes with a credible interval attached to every goalie's estimate, not just a ranked list. Reading that interval, not the point estimate alone, separates a meaningful performance signal from a plausible-looking coincidence. In the original season-long analysis, several goalies with a negative point estimate had credible intervals that included zero, meaning the model's estimate of the shot difficulty those goalies faced was too uncertain to say with confidence that the negative point estimate reflected true underperformance rather than uncertainty in how dangerous their shots against were. That is not a modeling failure. It is correct behavior for a metric meant to prevent overconfident conclusions from small, noisy samples, and it is a more honest answer than a leaderboard position implies. ## Why the uncertainty is the deliverable, not the ranking It is tempting to read a model like this as a ranked list of goalies, the least useful thing to take from it. The original analysis frames itself as one worked example on one season of public shot data: a demonstration of a modeling approach, not a validated production rating system. Its goaltender rankings belong to that one analysis, not to any standardized industry benchmark. What generalizes is the discipline: before trusting a raw outcome metric, ask what confounds it, model those confounds explicitly, and report the estimate as a range rather than a single number. That discipline is the same one Subconscious applies to causal action testing: a tested pricing change, message, or product variant is reported with a quantified effect and an uncertainty band, not a single confident number standing in for "this worked." A range that overlaps with "no effect" is treated as inconclusive rather than rounded up to a win. [figure: Five boxes left to right: isotropic baseline, coordinate warping near net, rebound surface multiplied in, hierarchical shooter effect, then corrected danger surface used to score goalies.] ## Where this method stops This kind of model estimates relative performance against expectation within one dataset and one modeling choice; it does not certify universal truth about a player's skill. The underlying danger surface's uncertainty is widest in ice regions with fewer, noisier shots; on the summed Goals Saved Above Expected metric, interval width instead grows with the number of shots a goalie faced, wider for high-volume starters than for rarely-used backups. That is a feature, not a limitation, of an approach designed to keep small samples from masquerading as strong signal. A production deployment would need cross-season validation, additional contextual variables such as team defensive quality, a goalie term so that each goalie's own shots do not inform the expected-goals baseline used to grade them, and sensitivity checks against alternative model specifications, none of which are attempted here. The broader takeaway for any team deciding whether to act on a performance number: ask whether that number has been adjusted for the difficulty of the situation it was measured in, and ask whether it comes with a range or a single point. If the answer to either is no, treat the number as a starting hypothesis, not a decision. Subconscious's research program and replication leaderboard apply that same standard, quantified effects with visible uncertainty, to the commercial decisions a go-to-market team works through. [figure: Four-stage chain: raw save percentage conflates skill and difficulty; a location-only danger model; corrections for angle, rebounds, shooter; result is Goals Saved Above Expected with a credible interval.] --- # Koji vs a Controlled Synthetic Experiment: Recruited Interviews or Fast Filtering First URL: https://subconscious.ai/blog/comparisons/koji-vs-controlled-synthetic-experiment-recruited-interviews-fast-filtering Published: 2026-08-25 A product marketing or consumer insights lead facing a launch, message, or pricing decision usually has to choose between two different research jobs, not two competing brands. One job is recruiting real people for interviews that produce quotable, transcript-level evidence. The other is running a controlled experiment against a defined synthetic population to filter a long list of candidate ideas down to a few worth testing with real people. Koji is built for the first job. A causal simulation approach is built for the second. [figure: A left-to-right path: many candidates enter a controlled synthetic experiment box, narrow to a shortlist of two or three, flow into recruited interviews, then end in a decision box backed by real-person evidence.] ## What does Koji actually do? Koji is an AI-native interview platform that runs and moderates customer discovery calls with real, recruited participants, then synthesizes the resulting transcripts into themes and recommendations. It is one of several automated interview platforms that a recent industry roundup of nine tools grouped together as software-run discovery research (Koji, "Best AI Interview Software in 2026: 9 Platforms Compared"). The job it does well is scaling discovery-interview volume: moving a team from roughly 10 manually moderated calls per quarter to roughly 50 per month, without adding headcount to recruiting, scheduling, or synthesis. That output has a property a simulation cannot substitute for: a transcript from a real person that a stakeholder can read and trust as evidence of what an actual customer said. ## What does a controlled synthetic experiment test instead? A causal simulation approach does not recruit anyone. It defines a synthetic population that stands in for a target audience, then runs a controlled experiment that compares candidate actions, messages, prices, or concepts against each other. Subconscious runs this kind of experiment to compare actions and estimate which one is more likely to move a defined behavioral outcome, with uncertainty reported where the study design supports it. The output is a causal comparison across the options tested, not a transcript. A transcript answers "what did this person say about the idea?" A controlled experiment answers "which of these alternatives is more likely to change behavior, so the team can stop testing the rest?" ## Where each method wins ## When recruited interviews are the right starting point Choose recruited interviews first when the output needs to be cited to stakeholders as real-customer evidence, when the decision carries high-stakes investment behind it, or when the team is exploring a genuinely new market segment where no population definition yet exists to simulate against. ## When a controlled experiment is the right starting point Choose a controlled synthetic experiment first when the team has more candidate messages, prices, or concepts than traditional exploratory market research can responsibly work through in the 3 to 4 weeks that kind of study usually takes, when the decision needs a same-week directional read before it can commit to a longer recruiting cycle, or when a marketing, product, or growth team without dedicated research operations needs to narrow a long list before asking for real-human research budget. ## How the two connect in one workflow A controlled experiment can run upstream, comparing many candidate options and surfacing the two or three worth carrying forward. Recruited interviews then run downstream, on that narrowed shortlist, to validate the finalists with real customers before the team commits budget behind one. ## How Subconscious fits into that workflow Subconscious can also test or validate studies with real human participants, and can run controlled studies against a person-level audience graph covering 800 million real people when scale of population definition matters to the decision. Neither claim substitutes for the interview transcripts a platform like Koji produces; both describe how a filtering pass and a validation pass can share the same decision framing. ## Limitations and failure conditions Subconscious does not moderate live interviews, does not produce human-respondent transcripts or quotes, and does not replace recruited real-human research when a decision requires that kind of provenance. Treat any specific sample size, panel configuration, or turnaround number described for a particular research vendor's own product as a feature of that vendor's setup, not a universal limit of every simulation-based method. ## Next step See how Subconscious runs these experiments, review prior case evidence, or read the current research before deciding which starting point fits the decision in front of you. Teams ready to scope a specific decision can book a session. --- # Segmentation, Synthetic Personas, or a Causal Experiment: Choosing the Right Research Method URL: https://subconscious.ai/blog/comparisons/segmentation-synthetic-personas-causal-experiment-choosing-right-research Published: 2026-08-25 Choose an audience-segmentation platform to define, reach, and measure a US audience. Choose synthetic-persona research to explore possible language, objections, and hypotheses. Choose a controlled causal experiment when a price, message, or feature must be compared with a defined alternative and measured before it ships. Treating audience classification or an open-ended conversation as proof that an action changes behavior can waste campaign or product budget. [figure: A four-step path: segmentation defines who to reach, persona exploration surfaces hypotheses, a causal experiment compares alternatives to find which action changes choice, and measurement observes results after launch.] ## Start with the decision, not the tool ## What can audience segmentation do? A national consumer-segmentation platform can combine household classification with marketing activation and measurement, according to its current first-party product description. Use that category when the decision is who to reach, how to activate the audience, and how to measure campaign delivery. One historical planning example described four established segmentation systems; treat that figure as historical scope context, not a current vendor claim, and verify present coverage during procurement. A group size only becomes decision evidence once the caused-choice result is tested, so this limit is stated here for that reason. Segmentation can identify a commercially relevant group. It does not establish that one proposed message, price, or feature will cause a different choice. The commercial action still needs its own test. ## What can persona-style research do? Persona-style research can support open-ended exploration of possible objections, language, and motivations. It is useful when the team is still shaping the question or deciding which alternatives deserve formal comparison. This boundary is stated here so a team can weigh a hypothesis correctly before spending against it. Its boundary is measurement. A plausible conversation is not a controlled comparison. It cannot show that a proposed action caused a change in buyer behavior. Use the output to form hypotheses, not to approve campaign or product spend. ## When is Subconscious the right fit? Subconscious is the fit when a team has a specific commercial action to test. It compares defined alternatives across a defined population using causal experimentation and discrete-choice-style modeling. The result is a causal comparison, with uncertainty language where the study design supports it. The question is narrow: which price, message, feature, or go-to-market action changes choice relative to a defined alternative? ## Keep each method in its lane The methods can work in sequence: - Use segmentation to define the audience. - Use persona-style exploration to surface language and hypotheses. - Use a controlled experiment to compare the proposed actions. - Use activation and market measurement to observe what happened after launch. The handoffs protect the decision. A controlled comparison estimates the effect of the action; market measurement then shows what happened in deployment. ## Evidence standards before budget moves A study should define the population, alternatives, intervention, outcome, and uncertainty before anyone interprets the result. /Research explains how Subconscious structures controlled experiments. The /leaderboard shows its validation approach. These exclusions are published here so a buyer can check fit before committing budget. Subconscious does not perform identity resolution, device or email linkage, multichannel media activation, or attribution. It does not replace an established segmentation system for audience definition at scale. This scope is stated plainly so a simulation result is never mistaken for market proof. A simulated experiment is decision-support evidence. Audience reach in a simulation is not recruited human participation, and a result does not automatically prove market performance. High-stakes decisions still need real-human validation. Subconscious can test or validate studies with real human participants, so a team can move from simulation to real-human validation without changing the causal question. If the unresolved decision is a defined action rather than an audience description, review /how we work or book a walkthrough. --- # Why a Frozen CAC Number Misleads Your Next Budget Reallocation URL: https://subconscious.ai/blog/methods-and-validation/why-frozen-cac-number-misleads-next-budget-reallocation Published: 2026-08-25 [figure: Two paths from the same spend and customer data: static CAC blends improving and declining periods into one misleading average; time-varying CAC drifts over time, revealing the channel's current regime.] ## The decision this affects If you own a media mix model and set next quarter's channel budget from it, the question is not whether your MMM fits the historical data. It is whether the model still assumes each channel's cost-per-acquisition (CAC) is constant across the whole window it was trained on. Most production MMMs make exactly that assumption, and it quietly reshapes every reallocation decision built on top of it. ## Why does a constant-CAC assumption cost money? A media mix model that treats a channel's effectiveness as fixed has to average across every period in its training window, whether the channel was improving, holding steady, or decaying. That average blends two different stories into one number: it overstates acquisition cost on a channel that is actually getting more efficient, and understates it on a channel whose performance is quietly sliding. Neither error is visible from the model's headline fit; both only show up in the reallocation decision itself, when budget gets pulled from a channel that was working and pushed toward one that was already weakening. Audience behavior, auction dynamics, creative fatigue, platform policy changes, and macro shocks all move channel performance week to week. A model trained on a full year of that movement, but built to output one CAC per channel, cannot tell you which regime you are currently in, and even the time-varying version tracks the low-spend CAC rather than the marginal CAC at current spend that reallocation actually depends on. ## How does a time-varying model represent the same channel? The fix does not require abandoning the interpretable model most teams already use. A standard saturation curve maps spend to acquired customers with two parameters: one sets the ceiling on how many customers a channel can produce at full saturation, and the other, the inverse of the curve's initial slope, represents the cost of acquiring a customer when spend is still low. Classically, that second parameter is treated as a fixed constant for the whole model. The documented alternative keeps the same saturation curve but lets that initial-slope parameter move with time instead of holding it fixed. Spend and acquired customers are still the only inputs; the parameter representing current channel efficiency is now an unobserved, time-indexed quantity that the model has to infer rather than assume, with the split from baseline trend, seasonality, and correlated cross-channel spend shaped by the model's kernel, lengthscale, and hierarchy priors. ## How does the model infer effectiveness it never observes directly? Because you only ever observe spend and customers acquired, the time-varying efficiency parameter has to be estimated, not read off. A Gaussian process is a standard way to do that: it puts a prior over smooth functions of time, encoding the assumption that channel effectiveness drifts gradually rather than jumping erratically day to day. Combined with the saturation curve, that lets the model produce a posterior estimate of gradual drift in efficiency; capturing seasonal patterns typically needs a periodic kernel, and abrupt shocks like a sudden shift in platform rules typically need a changepoint or shorter lengthscale, since a single smoothness prior trades off against representing both at once. A further refinement is documented directly in the open-source PyMC-Marketing time-varying media baseline example: a Hilbert space Gaussian process (HSGP) over time with a channel dimension, using shared basis functions across channels with channel-specific coefficients, rather than every channel moving independently.[^1] ## Where this connects to testing an action before you commit budget A technique explainer that never names its own boundary reads as a sales pitch, so this piece states it directly. This is a modeling-technique explainer, not a claim about what Subconscious builds. Subconscious does not construct Bayesian MMMs. But the underlying discipline is the same one Subconscious's causal experimentation approach is built on: don't treat "what happened last quarter" as a stable readout of "what will happen if you change your action." A time-varying MMM refuses to assume yesterday's channel effectiveness holds today. A causal experiment refuses to assume a correlational pattern in historical data tells you what will happen if you actually change price, packaging, or channel mix. Both reject a frozen average in favor of estimating how conditions are changing, with uncertainty attached to the estimate rather than hidden inside it; only the causal experiment estimates which action moves the outcome. Teams that want to see how that discipline runs end to end, including moving a tested action into real-world validation, can look at how that process is structured in practice. ## What this technique does not tell you Listing what a method cannot do is what lets a team check it before betting budget on it. A time-varying MMM changes what you can say about a channel's past performance. It does not, by itself, tell you what will happen if you shift budget in a way your historical data never covered; that is a forecasting extrapolation, not a causal test of a new action. It also does not replace validating a decision with real customers; it only makes sure the historical baseline you are extrapolating from reflects current conditions instead of an average of old and new ones. [figure: Five-stage flow: observed spend and customers feed a saturation curve, which splits into a shared market-wide trend and a channel-specific deviation, combining into the posterior CAC estimate for that channel.] ## The practical takeaway A CAC number earns its use as a planning input only when its limits are published alongside it. Before your next quarterly reallocation, ask whether your MMM's CAC estimates are allowed to move over time or are frozen for the whole training window. If they're frozen, the "underperforming" and "overperforming" channels in your dashboard may just be measuring which direction each channel's effectiveness has drifted, not which channel deserves more budget next quarter, and even a time-varying estimate describes the current regime, not next quarter's, since its posterior reverts toward the prior mean outside the training window. --- [^1]: MMM with time-varying media baseline: PyMC-Marketing documentation, PyMC-Marketing (open source documentation). --- # Simulated Personas vs. Causal Experiments: What to Trust Before a Launch Decision URL: https://subconscious.ai/blog/use-cases/simulated-personas-vs-causal-experiments-what-trust-before-launch-decision Published: 2026-08-25 A concept, message, or launch plan often has to move this week, not after a fielded study clears the calendar. The question that matters is not whether an AI persona sounds convincing, but whether the read comes from a controlled comparison of alternatives, with a causal question behind it, or from a chatbot improvising in character. Only the first kind should move a decision. [figure: Two columns. Left, persona chat: an assistant improvises in character, one opinion, no comparison. Right, causal experiment: a simulated population compares alternatives, producing an effect estimate.] ## What is a simulated persona, and where does the chat version stop counting as evidence? A simulated persona isn't a bio sheet with a stock photo pinned to it, a customer-database report, or a prompt telling a chatbot to "act like a 35-year-old marketing manager." Those produce plausible-sounding text, not a tested answer. Freeform AI persona chat sits one step further along: an assistant improvises in character and returns an opinion, a prediction from language patterns. It says nothing about which of two or more concepts would actually change a customer's choice, because nothing was compared under controlled conditions. Subconscious takes a different approach to the same buyer question. It runs a controlled experiment on a simulated population: two or more actions are compared, and the platform estimates which one is more likely to change the outcome a team cares about, with uncertainty reported where the study design supports it. The research page documents this method and its validation. ## Three moments where this decision shows up ### A concept needs a directional read before a fielded study is scheduled A classic fielded concept test, once an agency is engaged, a sample is recruited, and a focus group or survey runs, has historically taken three to four weeks. That timeline is a reference for budgeting a study, not a claim about how quickly any simulated alternative resolves the same question. A controlled experiment on a simulated population can compare multiple concepts against the same defined audience before that fielded study is committed to. The result is directional: which concept is more likely to win, and where the objections are likely to concentrate. A precise market-share forecast still requires a real, fielded sample. ### Why can't a team convene stakeholders like executives or investors for a real session? Executives, journalists, regulators, and investors are rarely available for a fielded focus group. A simulated stakeholder comparison is not competing with a real session here; there usually is none to compare against. The relevant question is whether the simulated read is useful at all, not whether it beats a fielded alternative. ### What decisions are too small to justify a study but costly to get wrong? Which subject line to test, which headline to run for a specific market, how a CMO buyer is likely to react to a pitch deck: none of these justifies commissioning a full study, and getting them wrong is not free. For questions at this scale, Subconscious can run the comparison against a person-level audience graph covering 800 million real people. That graph defines reach for the simulated population, not a claim about recruiting 800 million people into a live study. ## Where a simulated read still falls short, chat-based or causal - Genuinely new categories. When a product sits in an entirely new category with no comparable behavioral history, a simulated population has no experience to draw the comparison from. Real exploratory research stays the stronger tool here. - Precise, sample-level quantitative outputs. Directional comparisons are the strength of a simulated experiment. A board presentation needing statistically solid, sample-level numbers still requires a real, fielded sample. - Emotional reactions no one puts into words. Ethnographic observation captures reactions a participant would never say out loud. A simulated respondent, human or language-model-driven, only articulates what it is asked to articulate. A causal experiment on a simulated population is not a clinical trial, an observed usability session, or automatic proof of market performance. It answers a comparison question under a defined population and design. ## Three ways to test the same decision ## Validating a simulated read before it drives a decision Three checks before it feeds any real decision: - Historical backtest. Take a question whose real-world answer is already known, from a past study or a past launch reaction, and put the same question to the simulated population. A read that reproduces the known outcome has passed a meaningful test; one that does not should not be trusted on a new question either. Independent research on when digital personas reliably approximate human survey findings backs this discipline: agreement with real respondents varies by domain and population, so a backtest against a known outcome is the check, not an assumption (Northwestern Media and Cognition Group). - Internal cross-check. Ask the simulated population a question about a team's own product, then compare the answer against real signals already on hand, such as support tickets or NPS responses. - Moving to real-human validation on the same question. Subconscious can test or validate studies with real human participants, letting a team move from a simulated experiment to a real-human study on the same causal question without redesigning it. This step matters when the decision's cost of being wrong is high enough to warrant confirmation; it is not required for every routine comparison. ## The smallest next step Pick one decision a team is actually facing this week. Define the target population in two or three sentences: role, context, and what that population already knows. Then compare two or more concrete alternatives, a headline, a concept, a claim, against that defined population, rather than asking one open-ended question. Compare the result with what the team would have expected without it. Either the result is directly usable, or it is a useful surprise worth investigating before the fielded study runs. Teams evaluating this can see how a comparison like this runs or review past decisions built this way. For teams deciding whether a simulated read fits how they already work, how we work covers where this fits alongside existing research processes. --- # Where Should the Next Wegmans Open? 7 Checks for a Bayesian Site-Selection Forecast URL: https://subconscious.ai/blog/industries/where-should-next-wegmans-open-buyers-guide-bayesian-site-selection-its Published: 2026-08-25 A new-store forecast should separate observed evidence from assumptions before a retail real estate team commits build-out and lease capital or accepts the risk that a new location will divert demand from nearby stores. The historical grocery example behind this guide used the open-source PyMC package to combine road access, local demographics, analyst input, and probabilistic sales estimates. At the time described, the chain had operated for over 100 years, had 100+ stores across the East Coast, and opened at most 2-3 new locations per year. Those figures describe the original planning context, not current operating claims or delivery benchmarks. Use these seven checks to interrogate any comparable site forecast. Each check identifies an input, modeling choice, or judgment that a buyer should be able to inspect. ### 1. Define the trade area by travel time A simple radius treats every direction as equally accessible. A road-network catchment instead reflects the routes people can use, including street layout and physical barriers. Travel-time isochrones estimate how far a customer can travel within a chosen interval, which is why they matter for catchment analysis at retail sites (TravelTime). The team constructed a driving graph around each store from a roughly 20-mile buffer. Road classifications supplied edge speeds and travel-time values. A shortest-path calculation produced nested catchments from 2 minutes out to 24 minutes. When catchments overlapped, the location with the shorter trip received the contested area. A buyer should ask whether the network data, travel assumptions, and overlap rule match local shopping behavior. ### 2. Allocate demographics within partial boundaries Census geography and a store catchment rarely share the same edges. Weighting a partially included block group only by land area can misrepresent where households actually sit, especially when settlement is concentrated in one corner. The example addressed that mismatch with a quadkey tile grid and geocoded delivery-point density. The density inside each clipped block group determined how much of its population and household data contributed to the store profile. The buyer's check is straightforward: confirm that demographic allocation follows inhabited places rather than empty acreage. ### 3. How are store and ecommerce demand modeled separately? The historical model estimated in-store and ecommerce sales through two distinct channels while sampling them together, so total demand carried uncertainty from both outcomes. It also let the model constrain coefficient direction where domain knowledge justified it and incorporate an analyst's ordered visibility assessment without pretending that judgment was a precise physical measurement. Both sales outcomes used the trailing 52 weeks of data in the source setup. That was a historical modeling choice intended to reduce seasonal distortion, not a universal prescription. A buyer should inspect whether channel definitions, averaging windows, and judgment variables fit the decision now being made. ### 4. How is each store tested as a true holdout? With roughly 100 stores in the fitting data, the example used full-refit leave-one-out validation. One store was removed, the model was fitted again, and the omitted store became the prediction target. This checks whether the approach generalizes beyond the observations used for fitting. Each store contributed two observed outcomes, one per sales channel. The evaluation combined the held-out evidence from both. That prevents a strong result for one channel from concealing a weak result for the other. Buyers should ask to see performance across sites and channels, not only an aggregate score. ### 5. How is uncertainty preserved when combining models? The example compared several model variants and assigned weights according to predictive fit. It then combined posterior samples using those weights rather than blending only the central forecasts. Keeping the samples preserves distribution shape and supports probability questions, such as whether sales clear a planning threshold. This matters because a single expected value can hide asymmetric or multi-peaked risk. The useful buyer artifact is the range of plausible outcomes and the assumptions behind its shape, not merely the headline estimate. ### 6. Recalculate neighboring-store inputs Adding a proposed site changes the territory assigned to nearby stores. The historical workflow first partitioned the map without the candidate and then partitioned it again with the candidate included. Demographic covariates were aggregated for both versions. Applying paired posterior samples to those two input sets produced a distribution of estimated sales displacement for each affected store and channel. [figure: A left-to-right path of five stages a buyer walks a site forecast through: trade area by travel time, demographics in partial boundaries, separate demand models, leave-one-out validation, and preserved uncertainty.] The sequence is important: proposed site, revised trade areas, recomputed covariates, updated sales forecast, then estimated displacement. A buyer can trace that chain to identify where measured evidence ends and analyst judgment begins. ### 7. Keep the analyst in the decision loop An analyst supplied a candidate address, square footage, parking type, visibility score, and operating zone. The interface returned expected weekly sales with an uncertainty interval, affected-store estimates by channel, and a map of the revised trade areas. Those outputs informed a broader decision that still included market visits, competitive intelligence, and real estate judgment. The interface is useful when it exposes assumptions and uncertainty. It is dangerous when polished output encourages the team to treat the forecast as an automatic site decision. ## The question that remains after the shortlist These seven checks can improve confidence in how a site forecast was assembled. They do not establish which concept, assortment, or message will change behavior among shoppers in the selected trade area. That is a separate causal decision. Once spatial analysis has narrowed the candidates, Subconscious can run a controlled experiment comparing defined actions for the shopper population in question. The causal test should retain the same decision and population if the team later moves to real-human validation. This handoff adds behavioral evidence without recasting the forecast as proof of market performance. Read more about the research approach and its application to CPG decisions. [figure: Five-step chain: territory without the candidate, then with it added, feeding recomputed neighbor demographics, run through paired posterior samples, ending in a displacement distribution per store and channel.] ## Keep the boundary visible Subconscious does not replace spatial demographic analysis, travel-time network modeling, real estate judgment, market visits, or competitive research. It also does not produce the neighboring-store displacement estimates or automated location forecasts described here. Those methods estimate where a store may work and how it may affect the network. A controlled causal experiment tests which defined action changes a shopper decision. If the unresolved choice concerns the concept, assortment, or message rather than the physical site, scope that behavioral test against the target population. --- # How Brand Strategists Choose Which Campaign Route to Greenlight URL: https://subconscious.ai/blog/use-cases/how-brand-strategists-choose-which-campaign-route-greenlight Published: 2026-08-25 [figure: A five-step path from naming which route gets budget, through internal review and exploratory feedback, to a controlled causal experiment comparing routes, ending in a greenlight backed by a measured effect.] A brand strategist rarely fails because a campaign concept was bad on paper. The failure shows up after production and media are locked, when a route that won internal debate does not move its intended audience. What matters is not which concept the room likes, but which route causes the audience to respond, and whether the strategist can defend that answer when a stakeholder challenges it after launch. ## Why This Decision Gets Made on Taste Committing production and media budget behind a campaign route is expensive to reverse. When that commitment rests mostly on internal taste and late-stage stakeholder opinion, the strategist inherits two risks: the route never resonated with the target audience, and there is no defensible evidence trail when someone asks why it was chosen. Faster drafting and summarization do not remove this risk; they make it easier to reach a confident-sounding recommendation before anyone has tested whether the audience actually prefers the route. AI is taking over drafting and first-pass analysis, but research work is not disappearing: federal labor projections put market research analyst and marketing specialist employment on a growth path from 2024 to 2034, faster than the average occupation.[^bls] What changes is where judgment adds value: less in producing an answer, more in owning the question before a tool touches it and the caveat after it produces one. [^bls]: Market Research Analysts: Occupational Outlook Handbook, U.S. Bureau of Labor Statistics. ## What Does a Pre-Production Test Need to Answer? A useful pre-production test does not ask "does this concept feel right." It answers a narrower question: given a defined audience segment, which campaign route or message causes a measurably different response, and how confident can the team be in that difference. That requires four things before any test starts: - The decision the test is meant to inform, stated as a sentence: which route gets budget. - The audience segment, defined by who they are, their current behavior, and their alternatives, not a generic demographic label. - The candidate routes or messages, treated as an explicit comparison set rather than one idea tested in isolation. - A stated confidence bar for when the result is strong enough to act on. ## Comparing the Ways Teams Test a Route Before Production Different methods answer different parts of this question; none replaces the others. Subconscious fits the third row: a controlled discrete choice experiment that compares candidate campaign routes or messages across a precisely defined audience segment and returns a measured causal effect on response. Subconscious can define that audience segment against a person-level graph covering 800 million real people for the study-design step: that scale describes how the population is defined, not a pool of people recruited or contacted for every study. Research documents how these experiments are structured and validated, and case studies show the method applied to specific decisions. ## What Is the Practical Advantage of This Approach? The advantage over taste-based selection is not speed, but that the answer ties to a specific audience and comparison, letting the strategist state why a route was chosen and defend it later. The advantage over open-ended exploratory tools is that the result is a comparison built around the actual routes being decided between, not a single fluent reaction to a generic prompt. When the decision justifies it, a team can move from the simulated experiment to real-human validation without changing the underlying causal question. That step matters when a stakeholder needs the result to hold up against people who actually saw the campaign, not just the modeled result. ## Where Does This Method Stop? Publishing where a method stops is what separates a real result from marketing copy. A controlled pre-production test does not replace client or stakeholder discovery work, moderated qualitative research, or watching how a campaign actually performs once it is live. It does not tell a strategist whether the underlying brand positioning is right, whether the creative execution matches the tested message, or how the campaign performs against competitive activity in market. ## Putting It Into Practice Before the next campaign route decision reaches a room full of opinions, write the decision as one sentence, define the audience segment precisely, and list the candidate routes as a real comparison set. Run the controlled test against that comparison before committing production and media budget, not after. How Subconscious works covers how to set up that comparison, and a demo walks through applying it to a specific campaign decision. --- # Testing a Narrative Before the Board Sees It URL: https://subconscious.ai/blog/use-cases/testing-narrative-before-board-sees Published: 2026-08-25 The deck is finished. It goes to three colleagues first, each invested in the presenter's success, so each one gentle. Then the narrative meets the executive team or the board, and the first hard question exposes exactly what the friendly reviewers were too polite to name. Test the narrative against a defined set of buyer or stakeholder perspectives before the room that actually matters, instead of relying only on colleagues who want the presenter to succeed. ## Why does internal review miss the break? Internal review is not careless. It is structurally unable to find what it is looking for. Reviewers are invested in the presenter's confidence. A colleague who crushes morale before a high-stakes meeting has done more damage, socially, than one who stays quiet about a weak slide. The reviewer pool is a biased sample. People who already work with the presenter, or already buy into the idea, are not the audience in the room. Reviewers optimize for the relationship, not the deck. They will see the presenter again next week. Research on workplace feedback backs this pattern: colleagues systematically avoid the criticism a presenter needs most, because delivering it carries a real social cost to the reviewer (Buckingham and Goodall, Harvard Business Review). ## What does a structured test do differently? A randomized experiment run against defined buyer or stakeholder segments is not a replacement for the real meeting. It is the rehearsal a presenter cannot get from people who already want them to succeed. Subconscious runs controlled experiments that compare narrative or messaging variants against defined buyer segments and reports which version changes the outcome, rather than collecting the unstructured opinion an internal reviewer gives. That distinction matters because sycophantic agreement is not unique to human reviewers. Recent research on generative AI decision support finds the same bias appears when leaders lean on AI tools for ambiguous judgment calls: the tool tends to agree with the framing it is given rather than challenge it (arXiv:2603.03970). ## A workflow to run before the review Read the narrative once, flat. Ask one question: what does this say the company does, who is it for, and what is being asked for. Several claims will not land the way the presenter assumed. Identify the single biggest objection per segment. Different buyer or stakeholder segments will flag different concerns: market size, team, defensibility, go-to-market, competitive position. That is a prioritized list of risks to address before the real meeting. Test whether the story holds together. Does an early claim contradict a later one? Does the stated ask match the stage implied by the traction shown? Test the hardest questions in advance. The most useful part of the exercise is simulating the room's Q&A, not the read-through. That gives the presenter rehearsed, evidence-based answers instead of improvised ones. Test the "why now" framing directly. Most narratives hand-wave the moment. Running two or three versions of that framing against the same buyer segments shows which one actually lands. ## What this kind of test tends to surface Four patterns show up often enough to expect them: - The problem statement is not as universal as the presenter assumed. Segments disagree about whether it is real, big, or urgent, and that disagreement is the market risk, surfaced early instead of in the room. - The traction or proof point is calibrated wrong in one direction or the other: overclaiming for the stage, or underselling material evidence the deck buries. - The team or capability slide does not pattern-match to what the audience is looking for. - The ask is misaligned with the story the rest of the narrative tells. ## Limitations and failure conditions Naming where a test stops is what lets a buyer check the method against the claim. A structured test surfaces disagreement and structural weakness. It does not predict how one board or one buyer will react in the room, and it does not replace the live meeting. The misses get published next to the hits, so anyone can weigh both. A narrative can score well against every tested segment and still fail because of timing, internal politics, or a mismatch the test was never built to see. The test catches fixable weaknesses in the argument. It cannot manufacture conviction the material does not earn. Testing narrative variants this way is a simulated experiment. When the decision is high enough stakes to justify it, the same causal question can move to real-human validation without changing what is being tested. See how Subconscious runs studies for that path. Subconscious runs these controlled studies against a person-level audience graph covering 800 million real people, each defined in place, documented in the causal fidelity paper. That audience graph is not a recruitable pool of participants; it is the definition layer the experiment draws its segments from. ## Where to see this in practice Research and case studies show this pattern applied to product, pricing, and positioning narratives before they reach a board or executive review. For a narrative facing a real deadline, a scoped session is the fastest way to see what a defined-segment test would surface before the room does. [figure: Five-step path to test a narrative before a review: read it flat, rank the top objection per segment, test whether the story holds together, test the hardest likely questions, then test versions of the why-now framing.] --- # How much validation does a research decision need? URL: https://subconscious.ai/blog/company-and-trust/how-much-validation-research-decision-need Published: 2026-08-25 A buyer weighing causal experimentation against incumbent market research asks two questions: does this method answer "why," and how much validation does this decision need before the budget commits. Getting the second wrong is the expensive mistake: paying for human-scale validation on a low-stakes call, or shipping a high-stakes launch, pricing, or messaging decision on simulation alone when the cost of being wrong is large. ## The decision this method is built to test Subconscious runs randomized controlled experiments on synthetic respondents to estimate which action moves a target behavior, for which population, with quantified uncertainty where supported. That is a different unit of output than a dashboard (what happened) or a stated-preference survey (what people say they would do). The method uses discrete choice, McFadden-style choice modeling, Mixed Logit, and causal inference rather than generic AI persona roleplay. Earlier internal planning framed this ambition in stages: transcribing behavioral-science research, indexing which language models align with human responses in which domains, building an open API, and eventually producing a causal map spanning macro market trends and individual-level decisions. Those are historical planning examples, not a current product timeline or delivery promise. ## Matching validation depth to the cost of being wrong A low-stakes messaging test can often run and resolve entirely in simulation. A launch, pricing, or positioning decision with real capital behind it usually needs a second step: comparing the simulated result against a human baseline, then moving to real-participant validation on the same causal question if the decision justifies it. Subconscious can test or validate studies with real human participants, letting a team move from a simulated experiment to human validation without changing the underlying causal question. Independent research backs this: large language model responses can diverge from real human survey data and should not be treated as a guaranteed substitute for human-subject research (a Cambridge University Press study in Political Analysis examining how large language model outputs measure up against real human survey responses). That is why human validation is a distinct, optional step, not a guarantee baked into every result. [figure: Five-step path: define the causal question, run a synthetic experiment, compare to a human baseline, validate with real participants, decide with a confidence interval. Baseline and validation are optional steps.] ## Where scale and validation stay separate concepts Two different claims get confused in research conversations: how many people a platform's audience graph can reach, and how many are recruited and validated for a given study. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, but that figure describes the addressable graph, not a recruitable panel of validated participants. A buyer sizing a validation plan should ask which of the two, modeled reach or recruited participants, applies to their study. ## What is still a roadmap item, not a live claim Extending the causal-experiment method beyond text into image, video, or experience modalities, and building standardized benchmarks of how closely different language models align with human responses across domains, remain roadmap items rather than current, live capabilities. A buyer evaluating this method today should scope the decision to what can be tested now, discrete-choice and preference-style questions in text, and treat multi-modal or benchmarked alignment claims as direction, not delivery. [figure: Two-column comparison: an 800 million person modeled audience graph, addressable but not recruited, against recruited participants actually enrolled and validated for a specific study.] ## Next step When being wrong is expensive, the right first move is to define the causal question precisely, run the synthetic-respondent experiment, and decide up front whether the result needs a human-baseline comparison before it goes to the team making the call. Explore the research behind the method or see how the process runs end to end before scoping a study, or book time to walk through validation depth for a specific decision. --- # Conversational Personas, Structured Studies, or Causal Experiments: Choosing an AI-Simulated Research Method URL: https://subconscious.ai/blog/comparisons/conversational-personas-structured-studies-causal-experiments-choosing-ai Published: 2026-08-25 Teams evaluating AI-simulated-participant research tools often compare brand names instead of the underlying method: an ongoing conversational persona, a structured self-serve study, or a controlled causal experiment. Each answers a different question, and picking the wrong one for a high-stakes decision is where the risk sits. ## Three methods, three different questions Conversational persona tools let a team build a standing simulated customer and talk to it over time. Marketing, product, and sales can all draw on the same persona. The interaction is open-ended dialogue: follow up on an answer, challenge it, explore a tangent. That makes this approach good at surfacing questions a team didn't know to ask, not at proving what will happen if a specific action is taken. Structured self-serve study tools work differently: define a research question and participant parameters, run the study, and get back an aggregated qualitative output. The workflow mirrors traditional UX or product research, just faster than recruiting real participants. It answers a predefined question well; the output is a study result, not a causal comparison between one action and another. Causal experimentation is a third method: a controlled comparison of specific actions, such as a price, a message, a product concept, or a go-to-market move, against simulated buyers, reporting the effect of choosing one option over another. Subconscious.ai is built around this method. ## Comparing the three methods ## What's the cost of picking the wrong research method? The risk isn't using a conversational or study-based tool; it's treating a plausible-sounding synthetic conversation or study output as proof of what real buyers will do. Independent research on AI-generated survey and social-research responses has documented real limitations in how closely simulated respondents track actual human behavior (Verian Group, "Synthetic Sample in Social Research: significant limitations of AI generated responses"). A pricing change, a messaging shift, or a campaign built on narrative-level synthetic feedback can ship before the gap between what the simulation implied and what the market actually does shows up. "From a statistical perspective, if you simply generate a large enough synthetic sample size, every difference becomes statistically significant and thus loses its meaning." Verian Group, "Synthetic Sample in Social Research: significant limitations of AI generated responses" (source) Conversational and structured-study methods are well suited to open-ended qualitative discovery: figuring out what to ask, or getting fast directional feedback on a concept. Subconscious does not replace that kind of exploratory work, and it is not built as a study-management or persona-library product. Its fit is narrower: testing a specific action before a team commits real budget or reputation to it. ## When does a decision need a causal answer? If the decision is high-stakes, such as a pricing move, a positioning change, or a launch decision, a directional impression from a conversation or a single study isn't the same as evidence that one option causally outperforms another. That's the gap a controlled experiment closes: it compares defined actions against simulated buyers and reports the effect of the difference. When the stakes justify it, Subconscious can move a study from simulation to real-human validation without changing the underlying causal question. See how this works in practice. That step validates the same comparison with real people, not a usability session or a guarantee of market performance. ## How do you choose a starting point? Reach for a conversational or study-based tool when the goal is open-ended discovery or a fast directional read on a defined question. Reach for causal experimentation when a specific action needs to be compared against an alternative and a wrong call carries real consequences, with the option to confirm the result with real participants before it ships. Review examples of this method in the leaderboard, or talk through a specific decision. [figure: Three methods and what each answers: persona tools give open-ended dialogue, studies return an aggregated result, causal experiments compare actions and report an effect, optionally validated with real people.] --- # Qualtrics vs. Synthetic Panels: Which Stage of a Launch Decision Needs Which URL: https://subconscious.ai/blog/comparisons/qualtrics-vs-synthetic-panels-which-stage-launch-decision-needs-which Published: 2026-08-25 A research lead running every launch, pricing, or messaging question through a structured survey platform is asking one tool to do two different jobs: pruning options and proving a decision. Those jobs have different costs, timelines, and evidence requirements, and mixing them up is expensive in both directions. ## The two jobs a research question actually has Before a launch, pricing, or messaging decision ships, a team needs two separate things from research: a fast way to kill the weak options, and a defensible way to prove the survivor works. Running every question through a multi-week fielded survey burns budget and calendar time on ideas that would have died in a same-day test. Running a regulated or board-level claim through simulation alone, with no real-respondent check, risks shipping a decision nobody can defend when a stakeholder asks how many real people confirmed it. ## What are structured survey platforms built for? Qualtrics is the enterprise default for fielded research: real respondents, established survey methodology, and integration into existing research operations (Qualtrics Core XM). The category's strength is real-respondent provenance: when a stakeholder needs a study that a defined number of real people confirmed, this is where that comes from, using survey-building and reporting tools built for structured data collection and analysis at that standard (Qualtrics survey software). The trade-off is timeline and cost. Designing a study, recruiting respondents, fielding it, cleaning the data, and producing a report is a days-to-weeks process, and every added wave adds both. ## What a causal, simulation-first pass changes Subconscious runs controlled experiments on a simulated population and returns causal effects with confidence intervals, validated against real human outcomes. Subconscious's best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959; mean 0.73 across the 43 studies that pass design filters), validated against a corpus of roughly 300 replicated studies across 9 domains, per the causal fidelity paper. It is historical validation evidence, not a current per-project guarantee. That lets a team narrow which pricing tier, message, or feature framing is worth taking to a fielded survey. Simulation is not a substitute for real-respondent recruitment when a regulatory body, board, or stakeholder requires human-sourced evidence. Subconscious's real-human validation studies exist as a distinct, separately run service, not the same simulated output relabeled: a team can move from a simulated experiment to real-human testing without changing the underlying causal question, but that step is deliberate, not automatic. ## Where the two approaches actually diverge The table understates one thing: these are stages, not competing categories. A pricing or message test that skips straight to a fielded survey pays full price and full timeline to rule out ideas a same-day causal pass would have already killed. A launch claim that skips straight to simulation and ships without a validation step leaves the team with no answer when someone asks who confirmed it. ## How do you sequence Qualtrics and simulation testing for a real decision? For a launch, pricing, or messaging decision, the practical sequence is: run the causal experiment first to identify which options have a real effect and prune the rest, then decide whether the decision's stakes require a real-human validation pass before it goes to a board or regulator. Not every decision needs that second step: a marketing team iterating on message variants for an internal test rarely does. A pricing change going into a board deck, or a claim that will face regulatory scrutiny, usually does. Subconscious's research and how we work pages describe this sequencing in more detail, and the case studies page shows it applied to specific launch decisions. Teams evaluating whether a given decision needs the validation step can book time to walk through it against their own launch calendar. [figure: Flow diagram: options feed a same-day causal test that prunes weak ones. The survivor hits a decision point: board or regulatory scrutiny? One path goes to real-human validation, the other ships directly.] ## What is the limit of real-human validation testing? Real-human testing through Subconscious's validation service does not turn a causal experiment into an observed usability session, a clinical trial, or automatic proof of market performance. It confirms whether the causal question already tested holds up with real respondents, nothing more. Teams that need clinical-grade evidence or in-market performance proof need a different instrument than either of these. --- # When a Fixed Forecasting Rule Should Become a Distribution URL: https://subconscious.ai/blog/methods-and-validation/when-fixed-forecasting-rule-should-become-distribution Published: 2026-08-25 A forecasting pipeline that hard-codes its shrinkage constant either over-regresses every subject or under-regresses every subject, since a single mis-set constant pushes all of them in the same direction. A single point projection compounds that problem: it hides how much confidence a business should place in the number before allocating budget, roster spots, or pricing decisions against it. A case study rebuilding a well-known baseball projection system in a Bayesian framework shows what changes when fixed rules become estimated parameters with distributions attached. ## The Baseline: MARCEL MARCEL is a deliberately simple system for forecasting Major League Baseball player performance, developed by Tom Tango. The name is a nod to Marcel the Monkey, and the underlying idea is a bar: whatever a forecaster builds should beat this baseline of three years of historical data, with recent seasons weighted more heavily, regression toward the league mean, and an age adjustment. MARCEL rests on three hard-coded rules: - Fixed recency weights. Each season is weighted 5/4/3: the most recent season counts as 5, the one before as 4, the oldest as 3. - Fixed mean regression. A constant total of league-average plate appearances is added to a player's own data, regardless of how much individual data that player has produced. - Fixed age adjustment. A linear age curve applies a constant slope before and after an assumed peak age of 29. None of the three is estimated from the data it forecasts. ## Recasting the Same Structure as a Bayesian Model The original model is deterministic: one point projection per player, no attached uncertainty. A Bayesian version preserves MARCEL's three-component structure while replacing each fixed constant with a parameter the data estimates directly. Applied to hard hit rate (the share of batted balls hit at an exit velocity above 95 miles per hour, a leading indicator tracked because it correlates with later outcomes like batting average on balls in play). The original MARCEL specification supplies the deterministic baseline this rebuild departs from. The three substitutions: - A hierarchical beta-binomial model replaces the fixed regression constant. Instead of forcing every player's average toward the mean by the same fixed amount, the pooling strength scales itself to each player's own data volume. Players with few batted balls get pulled harder toward the league average; players with large samples keep more of their own signal. - A Dirichlet distribution replaces the fixed 5/4/3 weighting scheme. The model estimates season weights directly from the evidence rather than assuming Tango's ratio is correct for this metric. - A triangular aging function with an estimated peak age replaces the fixed age curve. Both the slope of the aging effect and the peak age itself become free parameters instead of assumed constants. ## What Did the Estimated Parameters Show? Historical planning example, not a current benchmark: for hard hit rate, the estimated season weights corresponded to roughly a 6/2/1 split, which leaned much harder on the most recent season than the original 5/4/3 scheme did. The estimated peak age for hard hit rate came out near 28 years, close to but not identical to Tango's fixed assumption of 29. ## Why Is Convergence Not Optional? An estimated distribution is only as trustworthy as the sampling process that produced it, but convergence alone does not confirm the model is correctly specified. Before treating any of these parameters as decision-ready, the case study checks two things: - Energy plots, which show whether the sampler explored the full parameter space or got stuck in part of it. - R-hat statistics, which compare variance across independent sampling chains; values at or near 1.0 indicate the chains agree on the same distribution. A model that reports a tight-looking credible interval without passing these checks is not more trustworthy than the fixed-rule baseline it replaced: it is just a distribution nobody has verified. ## Where Does This Generalize? The same three-part structure (weighted past performance, mean regression inversely proportional to sample size, and an age or lifecycle adjustment) applies to any repeated-measurement forecasting problem: customer lifetime value, renewal propensity, demand projection, or performance tracking outside baseball. The specific weights and peak age are properties of hard hit rate; the modeling discipline of replacing an assumed constant with an estimated, checked distribution is the transferable part. That discipline is the same modeling logic behind how Subconscious's causal behavioral platform reports uncertainty in decision-specific studies. Subconscious runs controlled experiments on simulated markets and returns causal effects with confidence intervals rather than a single deterministic estimate. This is a parallel in modeling philosophy, not a claim that Subconscious runs this baseball model or has replicated this specific analysis. Where a team's decision depends on validating a simulated result, Subconscious can also test or validate studies with real-human participants without changing the underlying causal question. See how we work. ## Limitations This rebuild is a starting point, not a finished system. A triangular aging curve is a simplification: most real aging effects are not perfectly linear before and after a peak, and older subjects in any population are typically survivors (the ones still performing well enough to keep being measured), which biases a naive aging curve unless the model accounts for that selection. The same caveat applies outside baseball: a fixed constant replaced by an estimated distribution is progress, but the distribution still needs domain-specific checks before a business treats it as ground truth. For teams evaluating this kind of shift on their own forecasting pipeline, research covers how Subconscious approaches uncertainty and validation in more depth. [figure: Four-step path: a fixed rule becomes an estimated parameter with a distribution attached; that distribution passes a convergence check; convergence is necessary but not sufficient, so posterior predictive and out-of-sample checks are still needed before it is decision-ready; otherwise it loops back as unverified.] --- # Panel Marketplace, Persona Chat, or Causal Experiment: Choose by the Decision URL: https://subconscious.ai/blog/comparisons/panel-marketplace-persona-chat-causal-experiment-choose-decision Published: 2026-08-25 A panel marketplace, a synthetic persona chat, and a controlled causal experiment answer different research questions. Choose a panel marketplace when the study requires recruited respondents. Use persona chat to develop hypotheses. Use a causal experiment when the business must determine which action changes buyer choice before committing budget. Naming where a method breaks lets a team catch the misuse before it reaches budget. The costly error is asking exploratory output to carry a pricing, messaging, or positioning decision. If the method never compared actions under controlled conditions, it cannot isolate what caused the result. [figure: Branching diagram: needing recruited respondents leads to a panel marketplace, needing hypotheses leads to persona chat, needing to choose which action changes buyer behavior leads to a causal experiment.] ## Match the tool to the evidence required A panel marketplace supplies recruited respondents to research applications through an API, as described in its respondent-sourcing documentation. That is a fieldwork capability. It does not turn a survey into a causal experiment. ## What is the substitution risk? Persona chat can surface language, reactions, and hypotheses worth testing. The boundary appears when a team treats those outputs as evidence that one proposed action will outperform another. A conversation is not a designed comparison, and a directional response is not an estimate of causal effect. The same boundary applies to ordinary fieldwork. Recruiting real people improves the relevance of the respondent source, but respondent identity does not determine study design. A survey can involve real people and still fail to isolate the effect of a price, claim, or launch message. When the evidence does not match the decision, the contradiction arrives after spend is committed. The larger cost is not the research spend but acting on a signal that was never designed to answer the business question. ## How do you build one causal question through the stack? Start with the action the business can take. For example: choose price A or price B, lead with claim A or claim B, or launch position A or position B. Then define the buyer choice that would distinguish the alternatives. Subconscious runs controlled discrete-choice experiments to isolate which action changes buyer decisions. Its studies draw on a person-level audience graph covering 800 million real people. A scale number stated without its limits reads as marketing. That is an audience-reach claim, not a claim that 800 million people are available for recruitment. When the stakes warrant human evidence, the same causal question can move to real-human testing without changing the study's decision logic. A capability holds up under scrutiny only when its description also states what it excludes. That does not convert the platform into a panel marketplace, guarantee market performance, or replace the judgment of research teams. ## Keep fieldwork in the design when it belongs there A research stack still needs a panel marketplace when: - A tracking study or syndicated research product depends on repeated fielding with recruited respondents. - An existing research application needs a respondent-supply API, following a workflow such as the provider's documented demand integration. - The methodology or stakeholder requirement calls for recruited human fieldwork. Stating what a method will not do is what lets a buyer check the claim against its actual scope. A causal experiment does not replace that layer: Subconscious does not operate a panel marketplace, sell fieldwork by the completed interview, or replace human research operations and agency relationships. ## How do you route the study by its decision? Need recruited respondent supply: choose the fieldwork layer. Need candidate hypotheses: use exploration. Need to choose which action changes buyer behavior: design a causal comparison and decide whether real-human validation is necessary. Review the research approach to assess the method. When the decision and alternatives are defined, scope the study. --- # Twelve Published Studies Subconscious Uses to Check Simulated Experiments Against Real Human Behavior URL: https://subconscious.ai/blog/case-studies/twelve-published-studies-subconscious-uses-check-simulated-experiments Published: 2026-08-25 A research leader deciding whether to trust a simulated experiment for a live product, pricing, or policy call needs one thing first: proof that the simulation reproduces what real people chose. Subconscious checks this by replaying published, peer-reviewed discrete-choice and conjoint studies inside its own simulated markets and comparing the result to the original human data. ## Why does the validation check matter before a decision ships? A synthetic method that quietly diverges from real human choice behavior only becomes visible after a launch, price, or policy decision has shipped. A validation check run beforehand, against studies whose outcomes are already known, catches that divergence before it shows up in market results. ## How the replication check works Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. A number without its limits attached is marketing. It is a validation result, not a guarantee for a new market. The full methodology and results are published in the causal fidelity paper, drawn from a validation corpus of roughly 300 replicated studies across 9 domains. Prior academic work shows conjoint designs track real-world behavior, including Hainmueller, Hangartner, and Yamamoto's PNAS study comparing stated preferences from survey experiments against real-world referendum voting behavior. Publishing the scope of a metric next to the metric is what lets a buyer check it instead of taking it on faith. This is an aggregate metric, not a guarantee for any single study or a claim of universal predictive accuracy for a decision that has never been tested. ## Which published studies does Subconscious use as checks? The examples below are a subset of the corpus: named, peer-reviewed discrete-choice and conjoint studies that Subconscious replicates as an independent check against real human choice data, not illustrative marketing examples. Each row is a distinct check, not a ranking. [figure: Five-stage chain: a published human study, run as a simulated replay, compared to the original result, aggregated across the 43 studies passing design filters, into 87% of the measured human ceiling (0.832 against a 0.959 human-to-human ceiling; mean 0.73 across the 43 studies).] ## What can a team do after seeing the validation results? A team that wants more than an aggregate accuracy figure can move from a simulated study to testing or validating with real human participants, on the same population and causal question, confirming a specific result without redesigning the underlying study. ## Limitations Naming where a method stops working is what makes its accuracy number usable for a buying decision. Replication accuracy is an aggregate validation metric, not a guarantee for any single study or decision. It does not establish universal predictive accuracy for a study outside the validation corpus. Audience or data-access claims tied to a specific third-party tool referenced in older materials are not carried forward here without current confirmation. ## Next step Read the validation methodology and full study list before treating any single simulated result as decision-grade. Teams comparing this approach against other case evidence or checking current standing on the replication leaderboard can also review how Subconscious's research is structured or book a walkthrough of a specific study design. --- # 8 AI Buyer Simulation Approaches for B2B Sales Teams in 2026 URL: https://subconscious.ai/blog/use-cases/ai-buyer-simulation-tools-b2b-sales-2026 Published: 2026-08-25 B2B sales teams in 2026 choose among eight rehearsal-software patterns, from call scoring and account-specific practice to ramp simulation, delivery coaching, and self-serve buyer cloning, differing mainly in how deep the buyer profile goes. A sales-enablement leader shopping for AI rehearsal software is answering two questions at once: can this tool make a rep more comfortable on a call, and does it know anything true about the buyer on the other end. Most 2026 platforms answer the first question well. Almost none answer the second, because the objections and pitch elements a rep drills against are usually a trainer's best guess, not measured against how a real buyer's stated preferences actually move. That gap has a cost: a rep who has rehearsed twenty plausible objections can still hear the one nobody scripted, because the tool never tested whether it moves the buyer's decision at all. ## Two different jobs wearing the same label Rehearsal tools give a rep repetition: a simulated conversation, a scripted or semi-scripted counterpart, a scorecard on delivery. They build confidence and pacing, and catch an unprepared rep before a real buyer does. Decision research answers a narrower, harder question: of the objections, offer terms, and framing choices a team could put to a buyer segment, which ones actually change what it chooses? Discrete choice experiments are the standard method economists use to isolate that causal effect: present a defined audience with structured tradeoffs and measure which factor moves the outcome. A rehearsal tool can make a script's delivery smoother. It cannot tell a team whether the script contains the right objections. That question sits upstream of any roleplay session, and it is the one most B2B sales teams skip. ## What is a rehearsal platform actually built from? Strip the branding and every 2026 sales-roleplay product is assembled from the same three parts: A buyer profile, built from a role, a company context, stated priorities, likely objections, and a decision style. The depth of that profile most determines whether practice transfers to a real call. A simulated conversation that walks through discovery, a demo, a pricing discussion, or a procurement exchange, with the simulated counterpart pushing back the way a real buyer might. A feedback layer that flags a missed signal, a weak question, or an objection the rep handled poorly, then rolls that up into a team-level view of where the gaps sit. ## 8 approaches to rehearsal software worth understanding These are not ranked. They are the recurring patterns across 2026's sales-roleplay category, described by what each approach optimizes for rather than by vendor name, so a buyer can match the pattern to the actual bottleneck. ### 1. Cold-call and discovery-call scoring Built for structured outbound, this pattern grades call mechanics: talk-listen ratio, pacing, and whether the rep covered the discovery questions the playbook calls for. It fits SDR and BDR teams where volume and consistency matter more than deep account context. ### 2. Named-account buyer rehearsal This pattern ties a simulated counterpart to a specific account already in the pipeline, so an account executive rehearses against something closer to the actual deal than a generic role. It suits reps preparing for one high-stakes call, not a broad practice cadence. ### 3. What is new-hire ramp simulation? Scenarios here are sequenced around a new rep's first ninety days, with scored simulations used as competency gates before a rep is trusted on a live call. This pattern treats readiness as something to measure and clear, not assume. ### 4. Broad enablement suites with roleplay attached Some platforms bundle rehearsal into a larger training system that also handles product certification and other competencies. Buyer simulation is one module among several, not the core product, which suits an organization consolidating multiple training tools into one system. ### 5. What is communication-mechanics coaching? This pattern treats buyer simulation as secondary to delivery: voice, pace, filler words, and structure. It is closer to a presentation coach than a buyer-modeling tool, and fits teams whose main gap is how a pitch is delivered, not what is in it. ### 6. Playbook and certification bundles Here rehearsal sits inside a wider enablement stack that also tracks playbook adherence, certification status, and ramp progress. The value is one system of record, not depth on any single piece, including the buyer model. ### 7. Research-plus-rehearsal bundles This pattern pairs public-source account research with a practice session immediately after, so a rep walks into a call having researched and rehearsed. It reduces the gap between researching an account and being ready to talk to it. ### 8. Self-serve buyer-cloning platforms The most flexible pattern lets a team build a persona-level buyer or account model and reuse it across objection mapping, discovery practice, demo rehearsal, and pricing conversations, often shared with marketing and product teams, not owned solely by sales. The tradeoff is that the depth of the underlying buyer model varies widely by vendor and is rarely independently verified. ## Where the practice ends and the decision research starts None of the eight patterns above answers whether a real buyer would actually have this conversation. That question needs a controlled comparison: define the action under consideration (an offer term, a pricing structure, a framing choice, a timing decision), define the buyer segment, and measure which version actually shifts stated intent. Subconscious runs controlled discrete choice experiments built for exactly that comparison, with a documented human-baseline replication check behind the method. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. A fidelity number without its scope is marketing, so the scope is stated here. That figure comes from the causal fidelity paper and is not a claim about sales-roleplay conversation quality specifically. Subconscious does not build persona-level rehearsal personas for live call practice, and does not score a rep on cold-call delivery mechanics. Its fit sits upstream of the roleplay tool: testing which offer terms, pricing structures, and framing choices actually move a target buyer segment's decision before those choices get written into a training scenario. See how that shows up in practice in the case studies and the current leaderboard of tested approaches. ## A sequence that avoids the blind spot A team evaluating a high-stakes pitch or objection set can run both tools without conflating their jobs: First, test the underlying decision: run the controlled comparison described above against the target buyer segment before any script gets written. Second, script the rehearsal tool with what the experiment found, instead of with a trainer's best guess at what a buyer might say. Third, run the rehearsal repetitions: five practice sessions before a six-figure demo, or fifty simulated conversations in a new rep's first two weeks against the priority buyer profiles. If three of five sessions surface an objection the rep had not prepared for, the rehearsal step earned its keep. One planning example: teams running that kind of ramp-week repetition have reported moving time-to-first-meaningful-deal from around twelve weeks to seven. Treat that as a range to test inside a specific team, not a guaranteed outcome, since ramp speed depends on pipeline quality, territory, and rep tenure as much as on practice volume. [figure: Three-part diagram of a rehearsal platform: buyer profile, simulated conversation, feedback layer. The buyer profile is marked as the part that most determines transfer to real calls.] ## Limitations to keep straight Naming this failure mode here is what lets a buyer check it against how their own team uses both tools. A rehearsal tool that scores delivery mechanics is not decision research, and decision research is not a substitute for repetition: conflating the two leads a team to over-trust a scorecard as proof a pitch works, or skip rehearsal because the offer was already tested. Real-human validation matters here too: Subconscious can test or validate studies with real human participants, which lets a team move from a simulated comparison to a human-confirmed one without changing the underlying causal question. That step answers "did the measured effect hold with real people," not "did the rep deliver the pitch smoothly." [figure: Two columns: rehearsal tools build repetition and score delivery, testing rep comfort. Decision research uses structured tradeoffs to test which objections actually change a buyer's choice.] --- # AI Customer Journey Mapping Through Touchpoint Experiments URL: https://subconscious.ai/blog/use-cases/ai-customer-journey-mapping-through-touchpoint-experiments Published: 2026-08-25 Customer journey maps are often created once in a workshop and left unchanged. Research-based maps can be slow and static. Workshop maps can be fast but dominated by internal assumptions. Simulation offers a third input: defined customer types reacting to specific touchpoints. The result can be updated as the experience changes, but it remains simulated evidence that should be compared with customer and behavioral data. [figure: A four-step loop diagram: build audience panel, map touchpoints, run monthly updates when a touchpoint changes, then a quarterly review, with an arrow looping back from quarterly review to monthly updates.] ## How long does a customer journey map take to build? A research-based journey study can take 6-10 weeks and cost €20,000-50,000. A workshop-based map can take a day and cost only the participants' time. Those are planning examples, not current Subconscious prices or delivery commitments. Both methods produce a snapshot. A map from six months ago may no longer reflect a changed product, competitor, or customer context. ## What audience stages should you simulate? Define audience states such as an unaware prospect, active researcher, first-time buyer, repeat customer, at-risk customer, and loyal advocate. Do not treat one generic persona as the whole journey. At each touchpoint, present the actual experience and probe the response. Examples include an ad, product page, first purchase, onboarding step, support interaction, or review request three months after purchase. Record confidence, confusion, excitement, frustration, and indifference. Ask what information is missing and which action the customer would take next. ## Treat simulation as a different kind of evidence Interview research asks customers to reconstruct an experience. Memory can omit touchpoints, compress time, or rationalize a choice, a pattern documented in research on recall bias in retrospective self-reports. "there was an under-reporting of 14 % in self-reported health service utilisation for GP visits compared to national insurer claims data over the same period" Brusco and Watts, BMC Health Services Research (source) Simulation presents the touchpoint in the current session. It is not automatically more accurate, and it does not reveal what an actual customer did. ## A repeatable workflow ### Week 1 Build a panel of 5-7 audience definitions across important stages. Calibrate them with approved interview transcripts, NPS comments, support tickets, and reviews. ### Week 2 Walk each audience through the major touchpoints. Record reactions, expected behavior, and friction. Build the first map with the assumptions visible. ### Monthly updates When onboarding, checkout, email, or another touchpoint changes, run the same comparison again. Keep unrelated variables stable. ### Quarterly review Repeat the full journey exercise. Compare what improved, degraded, or introduced new friction. ## Focus on high-value transitions The move from awareness to consideration depends on the information and proof that starts active evaluation. The move from first purchase to second purchase often determines whether a customer relationship develops. Study onboarding, first use, follow-up, and the next purchase decision. The move from risk to churn requires observed warning signals such as lower use or more support contact. Use simulation to form intervention hypotheses, then test the action. [figure: Two columns: Interview (recalling a past event, risks: omits touchpoints, compresses time, rationalizes) and Simulation (reacting live, limit: not real behavior). Both feed a box: Compare against real data.] ## How do you connect the map to existing evidence? NPS and CSAT show where satisfaction changes. Analytics shows where customers stop. Support tickets show recurring failures. Simulation can suggest why each pattern exists and how an alternative touchpoint may change it. Subconscious can support controlled comparisons of journey actions for a defined audience, alongside the case studies where those comparisons ran. The journey map becomes useful when each proposed change is tied to an outcome and checked against real behavior. --- # What Is AI Market Research? Definition, Methods, and Where It Still Needs a Human Check URL: https://subconscious.ai/blog/use-cases/what-ai-market-research-definition-methods-where-still-needs-human-check Published: 2026-08-25 AI market research uses artificial intelligence to conduct, accelerate, or interpret market research: AI-generated synthetic respondents, automated analysis of qualitative data, AI-assisted survey design, and predictive modeling of how a segment will respond to a change. The common thread: AI generates or synthesizes evidence, rather than storing and displaying data collected traditionally. The buyer question that matters is narrower than "does it work?" It's whether simulated evidence can stand on its own for the decision in front of you, or needs a real-human check before you commit budget or roadmap time. ## The methods that fall under the label - Synthetic respondents. AI personas configured against demographic and psychographic profiles answer research questions, standing in for unrecruited segments. - Automated qualitative analysis. Natural language processing scans transcripts, open-ended survey responses, and support tickets for themes and sentiment at scale. - AI-assisted research design. Language models help draft questionnaires, discussion guides, and flag bias in the instrument. - Predictive behavioral modeling. Models trained on behavioral and attitudinal data predict how a segment responds to a product change, price move, or message. ## What it replaces in the traditional research timeline Traditional research runs through defining the question, designing the instrument, recruiting participants, fielding it, analyzing results, and writing up findings. Each stage costs time: a typical qualitative study runs four to eight weeks from brief to report, with recruitment alone often consuming two to four weeks of that window. AI market research compresses specific stages, not the whole pipeline: - Recruitment becomes optional when synthetic personas stand in for real participants. - Fieldwork happens immediately: a synthetic session runs in hours instead of the days it takes to schedule and conduct one with real people. - Analysis speeds up: natural language processing can identify themes in 500 interview transcripts in minutes, work that takes human analysts weeks. - Report drafting gets a head start from generative summarization, freeing analyst time for interpretation. ## How reliable is a synthetic respondent? Naming where persona models fail lets a buyer check a synthetic respondent before trusting it for a decision. Persona-conditioned language models can sound plausible without being a reliable stand-in for what real people would say. Recent work testing persona-conditioned LLMs as synthetic survey respondents finds accuracy that depends heavily on the specificity of the persona and the type of question asked, and a separate study conditioning personas on socio-economic microdata finds the same pattern: closed-ended questions about established attitudes hold up better than open-ended questions about novel behavior (Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents; Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?). That means accuracy isn't answerable as a single number: it depends on what's being asked and what the answer is used for. "We find that persona prompting does not yield a clear aggregate improvement in survey alignment and, in many cases, significantly degrades performance." Taday Morocho and colleagues, arXiv preprint 2602.18462 (source) ## Reframing accuracy as a validation question The useful question isn't whether a persona sounds convincing, but whether the result reproduces what a real study would find, and whether a team can check that when needed. Subconscious runs controlled causal experiments on simulated populations. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, drawn from roughly 300 replicated studies across 9 domains. A fidelity number only helps a buyer when its limits are stated next to it. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. When a decision needs more certainty than that baseline provides, the same causal question can move to real-human participant validation without redesigning the study. Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people; that reach is distinct from a recruitable panel available to respond directly. ## What to trust simulated evidence for, and what still needs a human check The right column is where the cost of being wrong outweighs the time saved: a launch built on a synthetic signal that doesn't hold in market. [figure: Two columns: left "use simulated evidence for" lists hypothesis generation, instrument pre-testing, concept testing; right "get a human check first" lists market sizing, capital decisions, unprecedented events.] ## Why are teams adopting AI market research despite the gap? Cost pressure explains most of the growth: traditional research is expensive enough that many teams can't run it as often as they'd like, and product cycles increasingly outpace a multi-week research timeline. AI methods also put research access in front of non-specialists: a product or marketing manager can run a session without owning research methodology. The adoption reasons sit next to the accuracy limit on purpose, so a team can weigh both before deciding. None of that changes where the accuracy ceiling sits. It changes how often a team can afford to check a question, which is why knowing when a simulated answer is sufficient matters more than the existence of the tool. ## How do you run a first study? Start with the research question that matters most this quarter. Run it as a controlled experiment rather than an open-ended chat with a persona: define the decision, the action being tested, and what result would change it. A short exploratory session is enough to see whether the direction of the answer is useful, compared against secondary research or a single interview. If the decision has enough at stake to need a confidence interval, plan for a real-human validation pass on the same causal question before committing budget. --- # Synthetic Panel Tools vs. Fielded Real-Respondent Research: How to Sequence Testing URL: https://subconscious.ai/blog/comparisons/synthetic-panel-tools-vs-fielded-real-respondent-research-how-sequence Published: 2026-08-25 A fast synthetic panel tool and a fielded real-respondent research platform answer different questions. Treating either as a complete substitute for the other ships expensive concepts, prices, or messages on weak evidence. The panel tool tells you how a modeled audience reacted to a concept. The fielded platform tells you how a sample of real people reacted. Neither, by itself, tells you which action caused the reaction, and that gap is what an insights or brand team needs answered before capital moves. Fielded platforms such as Upsiide are built for fast, mobile-first concept and message testing with real recruited respondents; independent reviews describe that as its core strength. ## Why this decision matters The decision most teams are making is not "which vendor," it's "how much evidence is enough before we commit spend." A synthetic panel can produce an accuracy score against historical data in minutes. A fielded study can produce a statistically defensible sample in days. Both outputs look like decision-grade evidence. Neither one, on its own, is a causal test of the action under consideration. A score published without its limits attached is marketing copy. The failure mode worth naming directly: greenlighting a concept, price point, or message on an aggregate accuracy score from a synthetic tool. The category has documented weaknesses: variance collapse (every simulated respondent converges on the same answer), demographic flattening (subgroup differences get smoothed away), and over-rationality (simulated respondents behave more consistently than real people do). A team that treats a high accuracy percentage as proof the concept will work discovers the miss after the launch, not before it. ## What causes synthetic panels to diverge from real behavior? Synthetic panel tools work by conditioning a language model on demographic, survey, or behavioral data and asking it to answer as if it were a member of that audience. This is useful for fast iteration: teams can compare far more concept variants than a fielded study budget allows, and the cost per comparison is low. But the underlying mechanism is pattern completion against training data and prompt context, not a controlled experiment against real market response. Published research on large language models in stated-preference and choice tasks finds they often recover plausible aggregate tradeoffs but struggle with individual-level heterogeneity, cultural and demographic nuance, and sensitivity to question framing. Publishing this tradeoff lets a buyer check the method before trusting it. Fielded research with recruited human respondents avoids that failure mode because the respondents are real, but it introduces its own well-known ones: people don't always do what they say they'll do, respondents can satisfice or answer aspirationally, and the sample size and cost that make results defensible also make broad exploration slow and expensive. Neither limitation is fatal, but both are reasons to be precise about what each method is evidence for. ## What does an accuracy score actually measure? An accuracy percentage reported by a synthetic panel tool is a correlational measure: how often the tool's aggregate answer matched a historical human answer on a past study. That signals whether the tool is well-calibrated in general. It is not the same claim as "this specific concept, at this specific price, will produce this specific market outcome." Confusing the two is the substitution error, because a tool can be well-calibrated on average and still miss the one comparison that matters for a launch decision. Subconscious takes a different approach: rather than reporting how closely a modeled population's answers match historical survey data, it runs controlled discrete-choice-style experiments and evaluates whether the causal direction and effect of a tested action reproduce what real human studies find. That is a stricter bar than matching an aggregate answer, because it requires the comparison, not the response, to hold up. ## Comparing the three approaches ## A recommended decision process - Frame the decision, not the tool. Name the specific action being tested (a price, a claim, a concept), the population, and the outcome that matters. - Use fast, low-cost methods for triage when you have many options and low individual stakes. This is where synthetic panel speed is useful, provided the team treats the output as a narrowing signal, not a launch decision. - Use a causal test when the decision determines what gets built or how capital gets spent. An accuracy benchmark against historical data answers "is this tool generally calibrated." It does not answer "will this specific action move this specific outcome." A controlled experiment designed around the decision does. - Reserve recruited, fielded human research for the cases that require it. Regulated decisions, board-level launches, and any case requiring an audit trail of real human response should go to real, recruited respondents. ## The third option between "fast score" and "fielded sample" Subconscious is a causal behavioral platform: it runs controlled experiments on simulated markets to estimate which action most likely changes a specific behavioral outcome, and it can test or validate studies with real human participants. The distinction: Subconscious is not reporting a general accuracy score against a benchmark; it is testing the specific comparison a buyer is making, and it reports the result alongside its limitations. See research and validation methodology, how it fits an existing testing workflow at how-we-work, and worked examples at case studies. Teams can book time to walk through a specific comparison. ## What can't these three methods do alone? The misses sit on the record next to the hits in this comparison. Subconscious does not replace fielded, recruited-human research for final regulated or high-stakes validation. Audience reach (the breadth of who can be simulated), simulated experiments (a causal method for comparing actions), and recruited real-human validation (fielded survey or panel research with respondents) are three distinct concepts. A causal simulated experiment does not become a fielded study, and a fielded study does not become a real-time simulation, no matter how the workflow is described. [figure: Four-step path: wide field of concepts narrows via a synthetic panel, moves to fielded research for a defensible sample, then a causal experiment tests the surviving action, then budget commits.] ## Adjacent questions Can synthetic and fielded methods be combined? Yes. A common pattern narrows a wide set of options with fast synthetic methods, then moves it to a fielded or causal test before committing budget. The mistake is skipping that second step for a confident-looking number. Does a high accuracy percentage mean a synthetic panel tool is decision-grade? Not by itself. Accuracy against historical data measures general calibration, not whether the tool identified the causal effect of the specific action under consideration. Is a causal simulated experiment the same as a clinical trial or usability study? No. Real-human validation extends a causal question from a simulated population to recruited human participants without changing what is tested. It does not turn a causal action test into an observed usability session, a clinical trial, or an automatic proof of market performance. --- # How to Choose an Email Subject Line Before You Commit List Volume URL: https://subconscious.ai/blog/use-cases/how-choose-email-subject-line-before-commit-list-volume Published: 2026-08-25 The decision is which candidate subject line, and paired preheader, to send to a live list, without burning list volume or waiting a full send cycle to find out you guessed wrong. Most lifecycle and growth marketing teams still make that call in a Slack thread: someone proposes a line, and the team finds out whether it worked only after the open-rate report lands. ## Why the wrong subject line is expensive A typical B2B lifecycle program ships dozens of unique subject lines per quarter across nurture, product, and broadcast sends. Each underperformer burns list volume on a variant that never had a real chance, delays the campaign while the team waits for the next send window, and on a small list can mean the A/B test never reaches statistical significance at all, so the team never learns anything from the loss. ## Why does a live send-test alone struggle with this decision? A real send-test is the correct final check, but it is a poor first pass for exploring subject-line options: - It needs volume. Detecting a small lift in open rate at a reasonable confidence level requires enough opens per variant (HubSpot). Below a certain list size, that volume simply is not there. - It only tolerates a couple of variants. Splitting a list across more than two arms shrinks each cell until the result is noise. - Every loser costs something real. A variant that was going to lose still goes to real inboxes, and repeated losing sends can compound against sender reputation on lifecycle automations. ## What causes a better outcome? The outcome improves when the team can compare more candidate subject lines against a description of the actual audience before any of them reach a real inbox, and reserve the live send for confirming the strongest one or two. Subconscious runs controlled discrete-choice experiments against a modeled version of the target audience, drawn from a person-level audience graph covering 800 million real people, to compare subject-line and preheader variants before a send. It can then validate the study with real human participants, moving from the simulated comparison to a real-human read without changing the underlying causal question of which variant people prefer and why. ## Options for testing a subject line ## A practical decision process - Generate a wider set of candidate subject lines and preheaders than feels comfortable. - Describe the actual audience segment specifically. A named role, company stage, size band, and current tooling produces a sharper comparison than a generic label like "marketing leaders." - Compare the candidates against that modeled audience, and record not just which one is preferred but why: what a reader expected to find after opening, and which lines read as something to skip. - Narrow to a small number of strongest candidates, refine the winning pattern, and take one structurally different challenger forward as well. - Confirm with a live send. ## Does this apply to B2C as well as B2B? The same process applies to consumer sends, with one adjustment: purchase intent in B2C email is more emotion-driven, so the modeled audience should reflect the demographic and emotional range of the actual list rather than a single profile. A variant that wins in aggregate can still lose with a specific high-value segment, which an aggregate open-rate report alone would not surface. ## Limitations and failure conditions - Deliverability is a separate problem. No pre-send comparison can tell you whether a subject line lands in a Promotions tab instead of the inbox. That requires a dedicated inbox-placement check. - Brand voice is a human call. A variant that scores best on open intent can still be off-brand. Have a person check the top candidates against brand voice guidance before shipping. - Simulated comparison does not replace a live send. It informs which variants deserve real list volume; it does not confirm deliverability, sender reputation, or actual recipient behavior on its own. ## Adjacent questions What about subject lines with personalization or merge tags? Test the merge-tag text as it will actually render for the recipient, ideally several sample versions with realistic name, company, and trigger context, so a forced-feeling personalization shows up before the send. Should preheader text be tested separately from the subject line? No. Subject line and preheader are what an inbox actually displays together, so they should be compared as a unit. A strong subject line can still be undercut by a flat preheader, a pattern that open-rate data alone will not reveal. Is there a list size below which this doesn't matter? The smaller the list, the less a live A/B test alone can tell you, because it may never reach significance. That is exactly the situation where comparing candidates before the send has the most to offer. Teams that want to see how this fits alongside other testing programs can review how Subconscious structures a study, look at case evidence, read more on the underlying research, or book time to walk through a specific campaign. [figure: Five-step path: draft many candidate lines, describe the audience segment, compare candidates against a modeled version of that audience, narrow to the strongest few, confirm the winner with a live send.] --- # Why a Favorite Still Loses Most of the Time: A Bracket Forecast Under Uncertainty URL: https://subconscious.ai/blog/methods-and-validation/why-favorite-still-loses-most-time-bracket-forecast-under-uncertainty Published: 2026-08-25 A single-elimination bracket does not ask a team to be good once. It asks a team to be good repeatedly, against opponents decided by other matches it does not control. That is why a strong favorite in the opening round can still be a clear underdog to win the whole thing, and why a single point estimate is the wrong tool for a decision that depends on a sequence of outcomes rather than one. A 2026 World Cup knockout-round forecast makes this concrete. It models each team's attack and defense strength, updates that strength with group-stage form, then simulates the entire bracket match by match: regulation time, extra time, penalty shootouts, and the next round's opponent produced by another branch of the same draw. [figure: Five-stage chain: England's simulated survival probability falls each round: 88.1% to survive round one, 61.7% quarterfinal, 38.7% semifinal, 25.7% final, 14.8% title.] ## A favorite is not a forecast of the title The Round of 32 produced some lopsided matchups: England at 88.1% against DR Congo, and Argentina at 88.0% against Cape Verde. Those are real edges in a single match. They are not title predictions. Following England's simulated path through the bracket shows why. Round of 32 survival lands at 88.1%, then each later stage knocks the number down again: 61.7% for a quarterfinal appearance, 38.7% for the semifinals, 25.7% for the final, and just 14.8% for the title itself. None of those drops is a modeling error. It is the compounding cost of winning four or five matches in a row against opponents who are themselves survivors of hard matches. Canada's case makes the same point from the other direction. Against South Africa it opened as the 66.1% favorite, and it won. But its quarterfinal probability was only 16.1%, its semifinal probability 4.3%, its final probability 0.9%, and its title probability 0.2%. A team can win the match in front of it and still have a thin conditional path to the trophy. Morocco shows the reverse pattern is possible too. It was only a 41.8% favorite against the Netherlands, yet its quarterfinal probability was 30.4%, its semifinal probability 15.0%, its final probability 7.1%, and its title probability 3.0%. A team can be an underdog in its first match and still hold real upside deeper in the bracket, once measured round by round rather than judged off the opening game alone. ## Where the uncertainty comes from The forecast is not just a ranking of teams. It carries uncertainty forward from three distinct places: - Long-run team strength. A slow-moving estimate built from years of international results, not a single tournament. - Group-stage form. A state-space adjustment layered on top of that long-run baseline, drawn from each team's actual group matches and modeled with a Kalman filter through the pymc-extras statespace module. This adjustment is deliberately small and centered across the field, so a hot three-match stretch does not get mistaken for a new baseline. - Bracket path. Every simulated match produces a scoreline, which can go to extra time and then a penalty shootout. The winner of that draw becomes someone else's next opponent, and the simulation runs forward through the whole tree rather than treating each round as independent. Penalty shootouts are treated as a coin flip in this model, because the underlying data has no per-player shootout history. That is a real limitation, not a hidden precision claim: it is honest about what the model does not know, rather than inventing a penalty-taking skill rating it cannot support. ## The decision this maps to Swap "which team wins the tournament" for "which version of a launch, price, or message wins in market": the discipline of carrying uncertainty forward instead of trusting one point estimate applies, though only a randomized experiment on the manipulated attribute, not a bracket simulation, can identify which version wins. A single point estimate, one predicted winner or one predicted lift, hides how much of that number is signal and how much is noise. A calibrated, uncertainty-quantified estimate forces the harder and more useful question: how much of the outcome is one throw at replicating, and how much would survive a repeated trial? That is the same discipline Subconscious applies to a business decision: run a controlled causal experiment on a simulated population, and report the causal effect with a confidence interval, scoped to that simulated population, rather than a single favorite number. Where the decision is safety- or budget-critical enough to justify it, a team can take a simulated result to real-human validation without changing the underlying causal question being asked. Subconscious does not build sports forecasting models, and this is a comparison about probabilistic reasoning under uncertainty, not a claim that the two systems share tooling. [figure: Four inputs the forecast combines: long-run strength, a small Kalman-filter adjustment from group form, a full bracket-path simulation routing winners to next opponents, and a coin-flip assumption for shootouts.] ## What to demand before a decision Before committing budget to a launch, a pricing change, or a messaging test, the same three questions this bracket forecast answers are worth asking of any model producing a recommendation: - What is the full path of probabilities, not just the first-round number? - Where does the uncertainty come from, and is it being carried forward honestly? - What does the model admit it does not know, rather than papering over with a false precision? Reviewing a leaderboard of tested causal effects or a summary of how these experiments are structured in current research makes it easier to compare a single confident-sounding forecast against one that shows its work round by round. --- # Which Validation Check Should a Product Manager Run Before Engineering Starts? URL: https://subconscious.ai/blog/buyers/which-validation-check-should-product-manager-run-before-engineering-starts Published: 2026-08-25 Most teams engage with only 6% of the features that ship, according to product-usage benchmark data from Mind the Product. Before a feature reaches that fate, a product manager makes a validation decision: run a quick simulated read, put a prototype in front of real users, or run a controlled experiment comparing variants. Picking the wrong check is how engineering time lands on a feature nobody uses. ## Three checks, three different questions A simulated read on a concept, prototype, or feature description surfaces early objections cheaply, before a final design exists. A check earns trust when its limits are stated plainly. It tells a PM what stands out or confuses, not what a buyer will actually choose when the alternative is doing nothing. A moderated or unmoderated usability session with recruited users tells a PM whether people can complete a task and where they get stuck. This method's limits belong on the record next to its uses. It does not isolate which of two competing feature variants, price points, or messages caused a change in behavior. A controlled causal experiment puts two or more variants in front of the same target audience under the same conditions and estimates which one moves the outcome. It answers the question the other two leave open: which specific change caused the difference. ## Matching the check to the decision The first two checks are exploratory: fast, cheap, and good for narrowing options. The third is confirmatory: slower to set up, and built for the moment a PM has to commit engineering time to one variant over another. ## When does a controlled experiment change the launch decision? New Age Floral used a series of controlled experiments to test pricing before a launch decision. Across five iterative experiments with consumer behavior modeled across 125 participants, the research identified $60 as the price that maintained market share, with sales expected to drop off above $100, and avoided an estimated $65,000 in traditional research cost in the process (case study). Finta ran a similar pricing test and moved to a $99-per-month or $950-per-year structure, closing five new customers at the new price point. Subconscious runs this kind of test as randomized experiments on a simulation of the target market, validated against real human behavior. It answers "which action" questions: which price, which feature variant, which message changes what a buyer chooses. It fits once exploratory or qualitative work has narrowed the field to a short list of variants, when the remaining question is which one to ship. ## What does a causal read not do? Naming a method's limits is what lets a buyer check it before relying on it. A simulated experiment does not replace watching a real person try to use the product, nor does it run a recruitable panel of human participants from an audience graph. When a launch decision is high-stakes, run the simulated comparison first, then validate with real human participants before committing, without changing the underlying causal question (approved methodology). ## Before the ticket goes to engineering A concept that has not been shaped yet needs a fast exploratory read. A flow that might confuse people needs real users watching it. A feature variant, price, or message that will only ship once needs a controlled comparison. Talk to Subconscious about setting up that comparison before the engineering ticket gets written. [figure: Three branches by feature stage: unshaped concept to simulated read, confusing flow to usability session, ship-once variant to controlled experiment. All converge on the ticket.] --- # From an Unexplained Survey Result to a Validated Why URL: https://subconscious.ai/blog/methods-and-validation/unexplained-survey-result-validated-why Published: 2026-08-25 A four-week brand tracking wave lands on your desk and a key metric has moved in a direction nobody can explain. Stakeholders want the reason by tomorrow morning, with no time or budget to refield a qualitative wave. The choice is between guessing at a narrative from a handful of open-ends and running a structured comparison that tests candidate explanations against the population that produced the result. Guessing is the expensive option. If the "why" you present to stakeholders is a plausible-sounding story rather than a tested one, the messaging, pricing, or packaging decision built on top of it can be wrong in a way nobody catches until the next wave. ## What can a fielded survey result tell you, and what can't it? A survey wave is a snapshot: it tells you a metric moved, and often by how much, but it does not let you ask the respondents who produced that number a follow-up question. If 40 percent of respondents in a hypothetical wave say they dislike a new packaging design, the topline gives you the size of the reaction, not the mechanism. Historically, closing that gap took weeks for a fresh qualitative round, or settling for a thin read of open-ended comments. The alternative is to treat the fielded data as the input to a controlled experiment rather than the end of the analysis. Instead of asking "what happened," design a comparison that narrows which candidate explanation is most consistent with the outcome. ## Why ground a simulated experiment in your own data? A simulated follow-up's credibility depends entirely on what it is grounded in: a model with no context about your specific respondents defaults to generic, average assumptions about the world. Argyle and colleagues addressed this gap in a Political Analysis paper, "Out of One, Many": conditioning a model on the detailed backstory of a real survey respondent produced response distributions that tracked human subgroup patterns in benchmark national surveys more closely than an uninformed model (Cambridge University Press, 2023). In practice, the fielded wave itself (segment definitions, response patterns, the open-ended language your respondents actually used) becomes the grounding layer for a simulated population built to represent that same audience, which means a candidate explanation the population surfaces may reflect language already present in that wave rather than an independent read on causation. The simulation does not stand in for the survey; it is a way to keep interrogating the survey after the field window has closed. [figure: Five steps: import the fielded wave and segments, state testable candidate explanations, query a population grounded in that data, treat the result as a hypothesis, then confirm the winner with recruited respondents.] ## Running the pressure-test Start from the fielded baseline. Import the wave that produced the confusing result, along with the segment definitions that matter for the question at hand, instead of starting from a generic category description. State the candidate explanations as testable alternatives. Instead of asking a simulated panel an open-ended "why," frame two or three specific hypotheses (a competitive price move, a messaging shift, a packaging change) and compare how a grounded population responds to each. Query the grounded population, not a single persona. Naming a failure mode here is what lets a buyer check it before trusting the output. Segment-level divergence is the most heavily caveated output of a simulated pass, not its most trustworthy one, given the variance-collapse and flattening failure modes below. Treat the result as a hypothesis, not a conclusion. The output narrows the field of candidate explanations and tells you which one is worth testing further. It does not prove which mechanism is real in the market. ## When a simulated pass is enough, and when it isn't Some parts of a confusing-result investigation suit an iterative simulated pass; others require recruiting real respondents before anyone acts on the answer. ## What this method does not do Publishing what a method cannot do is what lets a buyer check it before they rely on it. A simulated pass narrows explanations; it does not manufacture certainty. A number without its limits is marketing, so this limit is stated directly. It is not built to produce a population estimate with a defined confidence interval. A claim that an exact share of a population holds a view requires a study designed and fielded with real respondents, not a follow-up query against a grounded population. The misses go on the record next to the hits, and this is one of them. It is also bounded by the data used to ground it. A population grounded in one fielded wave reflects the patterns present in that wave and will not anticipate a genuinely novel shift, such as a sudden competitive move or macroeconomic shock the original data never captured. The broader literature on this kind of simulation documents recurring failure modes worth taking seriously: response distributions can collapse toward the average (variance collapse), demographic differences can flatten out, and generated respondents can behave more consistently "rational" than real people do (arXiv, 2026). These are the reason a candidate explanation from a simulated pass should be treated as a hypothesis worth validating, not a finding worth shipping on its own. Keep three things distinct: a simulated experiment grounded in your data is a lab bench for testing explanations; a large audience-reach figure describes real people who could in principle be reached, not a panel available for recruitment; and a recruited real-human validation study confirms a specific finding. Collapsing any two of those into one claim overstates what the evidence supports. ## Moving from a candidate explanation to a validated one When a candidate explanation is going to drive a real decision (a repositioning, a price change, a packaging rollback), the causal question does not change moving from the simulated pass to a human check. Subconscious can test or validate studies with real human participants on the same comparison a simulated pass already narrowed, testing the same comparison with a second source of evidence rather than restarting with a different question. The practical value of running the simulated pass first: it can be rerun against a new hypothesis without a new field period, and the recruited human study tests an already-narrowed set of explanations rather than starting from a blank page, though narrowing carries the risk of pruning the true driver before the human study ever sees it. Read more on how this fits into a broader research workflow, or see how we work through a decision end to end. ## Frequently asked questions Does a simulated pass replace the survey wave that produced the confusing result? No. It uses that wave as its grounding, then lets an analyst keep asking questions of the population the wave already described, without a new field period. Can this method process the open-ended comments from the same wave? Yes. Clustering themes and objections across the full set of open-ends covers the whole set rather than a manually coded sample. When does a finding need a recruited human study before anyone acts on it? Whenever being wrong is costly, or the claim needs to hold up to external audit, compliance review, or a regulatory body. See a worked example in our case studies. What should an analyst distrust about the output of a simulated pass? Any single, confident-sounding narrative offered without comparison against alternatives. The method is strongest when it is used to rank or eliminate candidate explanations, not to generate one plausible story and stop there. --- # Testing a Welcome Sequence Before It Reaches Your List URL: https://subconscious.ai/blog/use-cases/testing-welcome-sequence-before-reaches-list Published: 2026-08-25 Email is the one channel a marketing team fully owns. No algorithm decides who sees it, no platform takes a cut. Getting the welcome sequence wrong, or landing a nurture flow with the wrong tone, carries a real cost: the team already paid to acquire every subscriber on the list, and a weak sequence pushes them back out the door. The decision a lifecycle or CRM marketing lead faces before any sequence ships is whether to test message order, tone, CTA escalation, subject lines, and send timing against the subscriber base first, or find out after the send whether the sequence worked. ## What does it cost to guess wrong on a welcome sequence? The math is unforgiving. A welcome email that draws a 12% unsubscribe rate instead of 2% is burning 10 percentage points of the list every week it runs. That is not a problem a content refresh fixes next quarter. It is a compounding loss on a channel that has no way to recover reach once someone opts out. Unsubscribe rate is typically calculated as unsubscribes divided by emails delivered for a given send (Omnisend, 2026), which means the leak compounds with every later email in the sequence, not just the one that triggered it. Re-engagement flows fail the same way, just louder: send the wrong message to a subscriber already drifting toward churn and the sequence accelerates the decision to leave. Nurture flows fail more quietly, people simply stop opening, but the lost pipeline is the same. Live A/B testing cannot catch this before it happens. It takes weeks per variant, needs list volume many teams do not have, and only shows what went wrong after real subscribers already saw it. ## How do you test a sequence before the list sees it? Subconscious runs a controlled experiment against a simulated subscriber population defined from the traits that describe the real list, including job titles, firmographics, and behavioral signals already sitting in the CRM, before the sequence ships. The team sets the sequence variants up as controlled alternatives (a different message order, a softer or more direct tone, a faster or slower CTA escalation), and the experiment estimates which variant most changes the outcome that matters: continued engagement, click-through, or unsubscribe likelihood. The population is defined against Subconscious's person-level audience graph, covering 800 million real people, used to match a subscriber base by role, industry, and other defining traits. That audience graph is a basis for defining who the simulated population represents; it is not a pool of people who receive the test emails. ## Where each sequence type tends to break Different sequence types fail in predictable ways, and knowing the failure mode in advance is what makes an experiment worth designing. Subject lines and send timing are worth testing the same way: which subject line sets the right expectation for the email body it opens, and which day and time the segment would plausibly want to hear from the brand, rather than defaulting to whatever slot is open on the calendar. ## What does this not replace? This is a pre-filter, not a substitute for live testing, built to screen out the sequence variants that would clearly underperform before any reach a subscriber. The team still ships the strongest two options to the real list and measures what actually happens there. Subconscious does not guarantee a specific unsubscribe rate, and it does not claim persistent memory of a subscriber's full inbox history across every future send; each email in a sequence is evaluated with the context the sequence itself provides. When the direction of a result needs confirming with people instead of a simulation, the same causal question (which variant most changes the outcome) can carry over into a real-human validation study without being redefined. That step matters when the stakes are high enough to warrant it; for a routine subject-line test it usually is not. ## Where to start The clearest place to begin is the sequence most likely already losing subscribers: the welcome flow. Define a population that matches the real subscriber base, run the first email in the sequence against it, and look at where engagement would plausibly break down before the second email goes out. From there, case studies and how Subconscious runs an experiment cover what a full sequence test looks like end to end, and a live walkthrough covers the workflow directly. For more on the underlying method, see Subconscious's research. [figure: Five steps left to right: define a matching population, set up sequence variants, compare engagement and unsubscribe risk, ship the two strongest, confirm with a live test if warranted.] --- # Diagnose Feature Adoption Drop-Off Before You Commit the Next Sprint URL: https://subconscious.ai/blog/use-cases/diagnose-feature-adoption-drop-off-before-commit-next-sprint Published: 2026-08-25 A feature ships, the rollout plan executes, and adoption still stalls. The dashboard shows what happened: opens, clicks, drop-off, but not why. The team that owns the metric has to decide the next sprint's action: iterate on a specific fix, reposition the feature, or kill it. Waiting on a full recruited-user research cycle costs weeks the team doesn't have, and guessing wrong costs more: rebuilding the UI when the real problem was discoverability burns a sprint on the wrong fix, and killing a feature that only had a naming or placement problem throws away otherwise-valid product work. ## Why post-launch diagnosis is harder than pre-launch research Pre-launch research is comparatively easy: mockups, fake-door tests, directional signal before a line of code ships. Post-launch diagnosis has to explain why real behavior diverged from expected behavior, something analytics alone doesn't show. Users opened the feature, clicked around, and left, but did they not know it existed, find the interface confusing, or understand it perfectly and decide it wasn't useful? Each of those has a different fix, and a formal interview cycle to distinguish them typically takes weeks to recruit, run, and synthesize, matching the lead time market research analysts report for structured studies (U.S. Bureau of Labor Statistics). ## What's a faster way to narrow the hypothesis before interviews start? Subconscious can run a controlled simulated session against a defined audience segment to generate and narrow adoption-failure hypotheses before committing engineering time to a fix. A four-part diagnostic sequence covers the most common failure modes: - Discovery check. Describe the product without mentioning the new feature, then have the segment guess what's tucked into the settings or feature menu. If nobody comes close to naming what actually shipped, the problem is discovery, not value. - Value proposition stress test. Walk the segment through the feature and the benefit it's meant to deliver, then ask whether it would change how they work, and why or why not. Hesitation, confusion, or a flat "that's nice, but..." response signals the feature doesn't solve a problem the segment actually has. - Workflow friction audit. Walk the segment through the actual user flow and note where confusion or resistance appears. This maps onto the drop-off points already visible in analytics, but attaches reasoning to each one. - Competitive context check. Ask how the segment currently solves the problem the feature addresses. An existing workaround means the feature is competing with an established habit, not with nothing, and that is a harder bar to clear. ## Failure patterns worth checking for first Four patterns recur often enough to check before running a full diagnostic cycle: ## Deciding to kill or iterate A simulated session can also inform the kill-or-iterate decision. A segment that consistently says "I don't need this" or "I already have something better" is a clear signal to kill the feature and reallocate the engineering time. A segment that says "this is exactly what I need" and then struggles to use it is describing a fixable UX problem, not a value problem. ## Where does this fit next to real user interviews? Subconscious can test or validate studies with real human participants on the same causal question the simulated session narrowed, without re-litigating what's being asked. The sequence looks like: run a session on day one post-launch instead of waiting weeks for interviews, narrow five real interviews to the specific issue the session surfaced instead of scheduling fifteen open-ended ones, and test a proposed fix in simulation before committing engineering time to build it. The audience behind these sessions is a controlled, person-level audience graph covering 800 million real people, not a recruitable panel for spontaneous, open-ended interviews; recruiting real participants for validation is a separate step. ## How do teams get started? If a feature is struggling with adoption, the sequence to run this week is: define the audience segment, run the four-part diagnostic framework, and use whatever it surfaces to decide the next sprint before the team defaults to a multi-week interview cycle. For teams evaluating whether this fits their research process, it's worth walking through one session live before committing a roadmap decision to it: book a session. ## Limitations A simulated session narrows which explanation is most plausible for the defined segment; it does not replace watching real users struggle with the interface, and it is not proof that a fix will work until it moves through real-human validation or ships and analytics confirm the change. [figure: A left-to-right sequence of four diagnostic checks (discovery, value proposition, workflow friction, competitive context), converging on a final decision point: kill the feature or iterate on a specific fix.] --- # Self-Serve Persona Chat vs. Managed Research Advisory: Which One Answers Your Decision? URL: https://subconscious.ai/blog/comparisons/self-serve-persona-chat-vs-managed-research-advisory-which-one-answers Published: 2026-08-25 A team evaluating AI-driven research tools in 2026 usually faces two shapes: a self-serve tool where anyone opens a chat window and talks to a synthetic persona, or a managed advisory service where a vendor's team builds executive-level personas and delivers a strategy readout. Neither shape, by itself, tells you whether a real customer's choice would change. ## Two synthetic-research models, one missing question ### What is the self-serve conversational model? This model lets a marketing, product, sales, or research team create a synthetic persona, give it role and context, and hold a conversation with it. The workflow suits daily use: build a persona, ask it questions, compare answers across personas, and keep a shared library of the ones the team returns to. Self-serve conversational tools target growth-stage and mid-market organizations that want an answer inside a single session rather than a multi-week engagement. ### What is the managed executive-advisory model? The other model targets large enterprises and pairs an AI persona layer with a vendor's own analysts. Personas here are typically modeled on executive or leadership thinking rather than individual customers, used to evaluate a strategy, positioning statement, or go-to-market plan at a senior level. Delivery includes onboarding, scoping, and an interpreted readout rather than direct self-serve access; pricing is usually a custom annual contract, not a published rate. Both are legitimate for what they do: fast qualitative exploration for the first, structured executive-level strategy review for the second. Neither is designed to test whether an actual choice would change under a real alternative. ## What a persona conversation measures, and what it doesn't A persona conversation returns a plausible, articulate answer. It does not, on its own, establish that the answer predicts a real decision. Independent research on AI-generated respondents raises the same concern: composite or generated personas can produce fluent, confident output that displaces the friction and disagreement real user research is supposed to surface (ACM Interactions, "The Synthetic Persona Fallacy: How AI-Generated Research Undermines UX Research"). Work on how closely large language models can reproduce individual survey respondents from socio-economic microdata finds real promise at the aggregate level alongside persistent gaps at the individual level (arXiv, "Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?"). That is the say-do gap in a new form. A stated opinion, synthetic or human, is not the same thing as a measured change in behavior. Treating a fluent persona response as proof that a pricing move, a message, or a launch decision will work is where the cost of being wrong shows up: budget, positioning, or a go-to-market plan gets committed on an answer with no error bars and nothing to replicate. [figure: Three-row comparison: persona chat gives an opinion, not proof a choice changes; advisory gives a strategic readout, not an alternative test; experiment gives a causal effect against a human baseline.] ## Comparing what each approach actually answers ## When does each model fit the decision? A self-serve conversational tool is the right choice for early exploration: drafting message variants, stress-testing a rough concept, or generating a starting hypothesis before a bigger commitment. A managed executive-advisory service fits when the deliverable is a synthesized strategic point of view for a senior audience and the organization has budget for a scoped engagement. Neither fits the moment a real choice needs to be made and the cost of guessing wrong is high: a pricing change, a launch claim, a positioning bet, or a go-to-market plan that competes for scarce budget. ## How a controlled experiment changes the answer Subconscious runs randomized, controlled experiments on an audience graph to measure which action is more likely to change a real choice, then reports the result with uncertainty rather than a single confident sentence. Where the decision depends on it, a team can move from that simulated experiment to real-human validation on the same causal question rather than switching methods. That validation step is the practical difference from a persona conversation or an executive-level readout: an estimate of which action changes the outcome, checked against how Subconscious approaches experiment design, not generated once and trusted. ## What a controlled experiment does not do Naming what a method cannot do is what lets a buyer check it before betting on it. A controlled behavioral experiment does not replace executive judgment. It informs a decision; a person still has to make it. It is not a branded persona chat session, and it is not built for open-ended conversational exploration of a rough idea. It is also not a research consultancy offering professional-services interpretation of a strategy deck. Teams that need that kind of managed, executive-facing readout are better served by a service built for it. ## Where to start Before committing budget to a self-serve session or a managed engagement, identify whether the underlying question is exploratory or a real decision with a measurable cost of being wrong. For the second case, a demo walks through how to design a controlled experiment around that specific decision. --- # Agency Study, Synthetic Panel, or Causal Experiment: Choosing a Research Approach URL: https://subconscious.ai/blog/comparisons/agency-study-synthetic-panel-causal-experiment-choosing-research-approach Published: 2026-08-25 An insights or marketing leader choosing a research approach usually starts from two known models: commission a full-service research agency, or run an AI persona panel for a fast directional read. Neither is built to answer a narrower question: of the specific actions under consideration, which one actually changes the outcome the team cares about. Committing a large agency engagement to a question that only needed a controlled test of one action wastes budget and time. Treating a directional synthetic-panel read as adequate evidence for a high-stakes brand or pricing call risks a decision the read was never designed to support. [figure: Four blocks: agency study measures market belief; synthetic panel measures reaction to one concept; causal experiment measures which action changes an outcome; human validation confirms high-stakes results.] ## What Is the Agency Research Model? A global research agency runs the traditional playbook: bespoke quantitative and qualitative work fielded against real respondents, plus brand-tracking programs, segmentation, advertising-effectiveness benchmarks, and a consulting layer. Kantar, for example, maintains proprietary brand-equity frameworks such as BrandZ, calibrated against years of real-respondent data (Kantar BrandZ). That depth is why agency studies remain the default for board-level brand decisions. The tradeoff is scale. Real-respondent fieldwork, proprietary framework licensing, and a global consulting layer are built for a small number of high-stakes studies a year, not for a team that needs an answer on Tuesday. ## What Is the Synthetic-Panel Model? A newer category of tools generates AI personas from public and customer-provided data, then runs structured conversations or simulated focus-group panels against them. The output is conversational and directional: summaries, quotes, and a read on how a modeled audience might react to a concept, message, or pitch. That model is useful for teams fielding many low-stakes questions in a short span: pre-testing a campaign angle, stress-testing stakeholder messaging, or prepping for an internal pitch. The output is a plausible read against a modeled population, not a fresh, statistically fielded sample. ## What Both Models Leave Unanswered Neither model, by design, runs a controlled comparison between two or more actions and estimates which one changes a specific outcome. An agency study describes what a market currently believes, prefers, or recalls. A synthetic panel describes how a modeled audience is likely to react to one concept at a time. Neither answers: if we ran action A instead of action B, which one moves the number we're tracking? ## What Is a Controlled Causal Experiment? Subconscious is a causal behavioral platform. It runs controlled experiments on simulated populations to estimate which product, pricing, packaging, or messaging action is more likely to change a specific behavior, compared against the alternatives under consideration. Learn how the experiments are designed. When a decision depends on validation beyond simulation, a team can extend the same causal question to real human participants without changing what's being tested. That step matters most when a wrong call carries enough downside to justify confirming the simulated result against real behavior before committing. ## Comparing the Three Models ## Where a Causal Experiment Doesn't Replace the Other Two A causal experiment on a simulated population doesn't replace the global panel networks or decades-deep proprietary brand-equity frameworks an agency maintains, and it isn't a faster or cheaper alternative to any specific vendor without a current, sourced comparison to back that claim. Causal experimentation complements real-human validation rather than substituting for it: the simulated and human-validated results should answer the same causal question. If the question is "what does the market currently think," an agency study or a synthetic panel already answers it. If the question is "which of these actions will actually move the outcome," that's the gap a controlled experiment is built to close. See how a study moves from a defined decision to a tested action. --- # How to Sequence a Year of Research Into One Experiment Roadmap: 10 Steps URL: https://subconscious.ai/blog/implementation-and-procurement/how-sequence-year-research-into-one-experiment-roadmap-10-steps Published: 2026-08-25 A research leader who runs one study at a time re-answers the same question every quarter. A launch decision needs a segment read, so a study gets commissioned. A pricing decision needs another read three months later, and nobody connects it to what the first study already showed about the same buyers. Phase 2 gets designed without phase 1's evidence, and the program can never show which action moved which outcome over a year of decisions. The fix is not more studies. It is sequencing: turning one goal into a prioritized set of questions, each with its own hypothesis, ordered so each phase's evidence shapes the next. This kind of sequential and mixed-methods research design is a documented practice in the research-methodology literature, not a proprietary framework (Research design unlocked: roadmapping for integration of paradigms, strategies, and tools, ScienceDirect). ### 1. Define the target goal Start with the outcome the program exists to move, not the next study on the calendar. "Increase sales" is not a target; "identify which mobile app features drive the highest engagement among a defined segment" is. A vague goal produces disconnected studies because there is no shared question for them to answer against. ### 2. Break the goal into research questions Each experiment should answer one piece of the goal, and together they should cover it. For a customer loyalty goal, that could break down into three questions: which messaging drives repeat purchases, whether easier navigation improves retention, and which payment methods lift checkout completion. Rank the questions by impact and feasibility to set the run order. ### 3. Design each experiment around a specific hypothesis Give every experiment a testable statement tied to one research question, and keep each one focused on one or two variables so the result is unambiguous. Note dependencies explicitly: an experiment on navigation and retention is more useful once an earlier experiment has already identified which features buyers value. ### 4. How do you map the experimental path into phases? Group related experiments into phases and decide the run order so earlier findings inform later design. A typical path: phase 1 on product features and navigation to learn what drives usage, phase 2 on messaging and incentives to influence behavior around that usage, phase 3 on payment methods and pricing to optimize conversion once usage and behavior are understood. Each phase is a planning input to the next, not a standalone report. ### 5. Design for flexibility Not every experiment turns out as expected, and that is useful information, not noise. Build room to pivot: a result that surprises you should change the next phase's design, not just get filed. ### 6. Test product, pricing, and GTM actions before committing capital Subconscious fits inside the roadmap, not around it: each experiment in the sequence can run as a causal comparison across actions before a team commits budget to build, price, or launch. Subconscious frames each test as a discrete-choice-style comparison of specific actions, and reports which action moved the outcome and by how much, with the limitation stated alongside the number. It is a tool for one phase of the roadmap, not a replacement for the sequencing work in steps 1-5. ### 7. Standardize data collection and analysis across phases Every experiment in the program should produce comparable data. Standardizing collection and analysis lets a team line up results from different phases and see patterns a single study would never surface: which segment showed up in both the messaging and the pricing phase, which driver kept mattering across the program. ### 8. How do you review and adapt after each phase? After a batch of experiments closes, check whether the results actually answered the research questions for that phase, and whether the next phase's plan still makes sense in light of them. Assumptions sometimes turn out wrong, or a phase surfaces a question nobody planned for; refining the plan mid-program is the point of running it as a sequence rather than a single study. ### 9. How do you keep one central record of hypotheses, experiments, and results? Document every hypothesis, the experiment that tested it, and the result, in one place a team can reference when the next phase starts. A dashboard or shared file system that shows which hypotheses were supported, which weren't, and what new questions came out of each phase turns a series of studies into an institutional record instead of a stack of disconnected decks. ### 10. Synthesize the program back into the original goal Once several phases have run, bring the pieces back together against the goal from step 1. What did each phase add to the answer. What patterns showed up across phases. Where did the program change direction because of what an earlier phase found. Summarize the findings into recommendations a team can act on, and feed the open questions into the next round of the roadmap. [figure: Five stages left to right: target goal, ranked questions, phases of hypothesis-driven experiments, review after each phase, one central record of hypotheses and results.] ## Where this breaks down A roadmap only compounds evidence if the phases are actually sequenced by dependency, not convenience. Running phase 2 before phase 1's data exists just produces two disconnected studies with a shared file name. And simulated experiments answer "which action moves the outcome" inside the causal question you designed; when a decision needs a check against real human behavior, that is a separate, deliberate step, not something a simulated result implies automatically. Confidence intervals, segment-level heterogeneity, and automated recommendations are not standard outputs of every study; they depend on how a given phase is configured. Treat this roadmap as the discipline for sequencing decisions, and treat each phase's specific configuration as the place to confirm what that phase will and won't report. To design the causal experiments for a specific phase, or to see how the sequencing and synthesis work in practice, book time to walk through a roadmap. [figure: Four-step chain: a phase closes with an unexpected result, the team checks it against the original research question, the next phase's design changes in response, then the revised phase runs and the record is updated.] --- # Pre-Test Facebook and LinkedIn Ads with Simulated Buyers URL: https://subconscious.ai/blog/use-cases/pre-test-facebook-linkedin-ads-simulated-buyers Published: 2026-08-25 A performance marketing lead can build several ad variants, launch them all, and let Facebook or LinkedIn's algorithm spend real budget finding the winner during the platform's learning phase. Every losing impression or click in that phase still costs money, and a platform can reset the learning phase after a significant edit to budget, audience, or creative, restarting the exploration cost (Meta Business Help Center). A number without its limits is marketing. So the limit gets stated up front: a controlled pre-launch experiment against a defined buyer population narrows which variants deserve that live budget. It does not replace the live test. It decides which alternatives are worth exposing to real spend. ## What does a pre-launch experiment add? A discrete-choice experiment against a defined population gets a causal read on which creative or message variant drives the target outcome. That differs from asking a model to guess a preference: the experiment holds the audience definition fixed and varies only the thing being tested, so the result is a comparison, not an opinion. The misses sit on the record next to the hits, so this one gets said directly: real audience behavior under live auction conditions, and the platform's own delivery algorithm, still determine the final outcome (Meta Business Help Center on significant edits and the learning phase). ## What to test before launch ### Hooks Compare the headlines meant to stop the target buyer while scrolling. Ask which behavior each hook is meant to change before testing the alternatives. ### Message framing Test whether the buyer responds to time savings, cost reduction, risk mitigation, or competitive advantage. Free-form explanations can help generate hypotheses, but a decision-specific comparison is needed before treating one frame as the stronger action. ### Visual concepts Compare a person using the product against a product screenshot, or serious, playful, aspirational, and functional treatments. Keep the offer and copy fixed if the goal is to isolate the visual. ### Offer and call to action Compare a free trial, a money-back guarantee, and a limited-time discount only when those are real options. Do not ask a simulated audience to choose an offer the business cannot actually support. ## Define the buyer precisely "B2B SaaS buyers" is too broad to test against. A more useful definition is a VP of Marketing at a Series B SaaS company with 100-500 employees who is responsible for choosing and implementing marketing tools. Capture the variables that could plausibly affect the decision: - Role, seniority, company size, stage, industry, geography, and team size - Quarterly and annual goals - Current pain and workarounds - Evaluation criteria and common objections - Professional information sources - Buying process and trust signals These are audience-definition inputs, not proof that every individual in the segment behaves identically. A tighter definition produces a more useful comparison. [figure: A path from draft ad variants through a pre-launch experiment against a defined buyer, into a live platform A/B test, ending at a confirmed winner, with a loop back from performance drop to a new pre-launch round.] ## Connect pre-testing to the live campaign A workable sequence: - Draft several copy and visual variants. - Run a pre-launch experiment to identify the strongest candidates. - Launch those candidates in a real A/B test on the platform. - Let live behavior determine the winner. - Return to a pre-launch experiment when live performance drops and a new round of creative is needed. The pre-launch stage screens alternatives. The live stage measures real behavior under real auction and delivery conditions. ## How do you measure whether the screen works? Track the pre-launch ranking beside live click-through rate, conversion rate, cost per acquisition, and downstream lead quality. The goal is not to prove the model already knows the answer, but to learn where the screen removes an obvious loser before spend, and where real behavior changes the ranking once the campaign is live. [figure: A diagram with the pre-launch ranking on one side and four live platform metrics on the other, connected by comparison lines, showing the two are checked against each other rather than one replacing the other.] ## Limitations Naming a method's edge is what lets a buyer check it against real use. A pre-launch panel narrows the set of options worth testing live. It does not replace live platform optimization. Teams that want to see the underlying experiment methodology can review the case studies, read more at research, or talk through a specific campaign. --- # AI Research for Management Consultants: Choosing the Right Stage for Simulation vs. Causal Testing URL: https://subconscious.ai/blog/buyers/ai-research-management-consultants-choosing-right-stage-simulation-vs Published: 2026-08-25 The right stage depends on what the answer needs to survive: use open-ended AI simulation for early hypothesis generation, and reserve controlled causal testing for the number a client or investment committee will scrutinize. A management consulting engagement runs on a compressed clock, and the research plan has to match the decision it feeds. Early hypothesis generation and a client-facing recommendation are not the same job; treating them as interchangeable gets a recommendation dismantled in front of an investment committee or a client's leadership team. ## What is the engagement-timeline problem in consulting? A typical strategy engagement front-loads stakeholder interviews, then spends the middle weeks scoping and running research before the team has anything to synthesize into slides. A research vendor cycle that eats the middle of the calendar leaves too little runway for iteration, and the client is effectively paying the firm to project-manage that vendor. Open-ended AI-simulated discussion sessions have become a common way to fill that early gap. A team can stand up a synthetic panel matching a client's target audience and run a discussion guide through it before committing budget to a formal study. ## Where the two methods actually differ The failure mode is not using simulation. It is presenting an open-ended, uncontrolled panel session as if it were client-ready evidence. An engagement lead who can't tell the difference between "directional impressions from an exploratory session" and "a defensible, causal answer to a pricing or positioning question" is the one whose recommendation collapses under IC or client scrutiny. Subconscious sits on the right side of that table. Where an exploratory AI discussion produces useful early impressions, a controlled discrete-choice experiment tests defined alternatives against a defined population and returns causal effects with confidence intervals. ## Matching the method to the engagement stage Due diligence support. When a PE firm asks a team to assess a target's market position ahead of an acquisition, the early work (sentiment reads across existing customers, competitor customers, and category non-users) is exploratory. But the number that goes in front of the investment committee (would customers switch, at what price, under what positioning) needs a controlled comparison behind it, not a synthesized transcript. Case studies show how that comparison stage is structured for a specific claim. Pricing strategy. Directional reads on willingness-to-pay from an open discussion can point a team toward which pricing structures are worth testing. The recommendation itself (which tier structure customers actually prefer, and by how much) is a causal question that traditional market research has typically answered with a formal conjoint analysis, and it is the kind of question a controlled experiment is built to answer with a confidence interval attached. Market-entry assessment. Evaluating a new geographic or demographic segment benefits from broad population coverage. Subconscious runs controlled studies against a person-level audience graph covering 800 million real people, which lets a team compare defined market-entry alternatives at a scale that early exploratory conversations can't reach on their own. ## What doesn't a causal experiment replace? A causal experiment is not a substitute for stakeholder interviews, due-diligence fieldwork, expert-network calls, or the consultant's own synthesis and judgment. It answers one kind of question: which of these defined alternatives performs better, and by how much. And a simulated experiment's audience scale is not the same thing as a recruited panel of real participants: the two need to stay distinct in how a team describes them internally and to a client. When a recommendation is large enough to warrant it, a team can move from the simulated experiment to real-human testing or validation without changing the underlying causal question: the same comparison, tested against recruited participants instead of the audience graph, when the stakes call for it. ## Putting it into an engagement Before running a panel exploration on the next research-bound engagement, identify which output actually needs to survive committee or client scrutiny. If it's a directional read for internal hypothesis-building, an open-ended session is the right tool. If it's the number the recommendation rests on, see how a controlled study gets structured before the team commits to a method it can't defend in the room, or book a walkthrough of how a specific pricing or market-entry question would be set up as a causal test. [figure: Branching path: hypothesis generation leads to an open panel. It asks if output feeds a committee decision. No stays internal. Yes routes to a controlled experiment with confidence intervals.] --- # Pricing Research Methods: Choosing the Right Method Before You Field URL: https://subconscious.ai/blog/methods-and-validation/pricing-research-methods-choosing-right-method-before-field Published: 2026-08-25 [figure: Four pricing methods in a row, each labeled with what it measures: a single stated price, a range of acceptable prices, price weighed against attributes, a real purchase-intent response to an offer.] A pricing or product leader facing a launch, a tier change, or a repricing decision must choose a research method before fielding: willingness-to-pay, price sensitivity, conjoint or trade-off, or direct offer testing. Picking the wrong one produces a confident-looking number that does not measure the behavior the decision actually depends on. That is a wasted research spend and a mispriced launch, dressed up as evidence. ## The four methods answer different questions Each method answers a narrower question than "what should we charge." Willingness-to-pay asks how much a buyer would pay for a defined offer, usually through open-ended or scaled questions. It is fast to field but vulnerable to hypothetical bias: buyers tend to overstate what they would actually pay when no real transaction is at stake. Price sensitivity, most commonly run as a Van Westendorp Price Sensitivity Meter, asks buyers to name the price points at which a product feels too cheap, a bargain, expensive, and too expensive. The method produces a range rather than a point estimate, and it depends on buyers being able to reason clearly about a single price in isolation (Sawtooth Software, Van Westendorp Pricing Model). "Peter Van Westendorp introduced the Price Sensitivity Meter in 1976 and it has been widely used since then throughout the market research industry." Andy James, Penn State College of Earth and Mineral Sciences, BA 850 course notes (source) Conjoint and trade-off methods present buyers with bundles of attributes, including price, and ask them to choose or rank alternatives. This captures how price trades off against features, tiers, or positioning, closer to how a real purchase decision gets made (Penn State University, Van Westendorp Meter course notes). Direct offer testing puts an actual price or tier structure in front of buyers and measures a concrete response: intent to purchase, upgrade, or churn. It is closer to a real transaction than the other methods, but the response is still stated intent, not a real transaction, so hypothetical bias remains; it is also the most expensive and slowest to run at scale. ## Which method belongs before fieldwork The choice depends on what the team can already describe clearly, not which method is best in the abstract. A team that cannot yet describe its target buyer group and the specific pricing question clearly should close that gap before selecting a method. ## How can you pressure-test a pricing plan before fieldwork? Before committing budget to formal fielding, a team can run a controlled experiment that compares the pricing actions under consideration against a defined buyer segment and estimates the causal effect of each action on simulated choice within the experiment, with confidence intervals scoped to the simulated population where supported. This is a way to pressure-test which method and price points deserve real-buyer fielding, not a substitute for it. Used this way, a simulated pass can surface which price points are clearly dominated, which trade-offs buyers seem to weight most, and which assumptions in the research brief need to be checked with real respondents before the team spends on recruiting and programming. ## What still requires real buyers in pricing research? Simulated experiments do not replace real recruited buyer validation, representative statistics, or final price-elasticity estimation for decisions with financial or compliance stakes. A team can move from a simulated experiment to real-human testing without changing the underlying causal question, which is the practical value: the research design carries over, only the respondent population changes. The main failure mode across every pricing method, simulated or fielded, is false precision: a number that looks more certain than the underlying evidence supports. The discipline that prevents this is the same regardless of method: define the target buyer group, state the exact decision, list the assumptions the number depends on, and decide upfront which parts of the answer require real human data before anyone treats it as final. ## Next step Before fielding, write down the exact pricing decision, the buyer segment, and the alternatives under consideration. Match that description against the table above to identify the method. If the team wants to pressure-test the plan first, compare pricing actions in a controlled experiment, or read more on how Subconscious runs controlled experiments. --- # Audience-Data Activation vs. Open-Ended AI Exploration: Where Causal Testing Fits URL: https://subconscious.ai/blog/comparisons/audience-data-activation-vs-open-ended-ai-exploration-where-causal-testing Published: 2026-08-25 Teams comparing tools for understanding US buyers usually land on two very different categories, and neither one answers the question that actually determines spend: which specific action, a message, a price, a product change, moves the outcome. ## Two categories, one missing step The first category is predictive audience-data infrastructure: platforms built around a large identity graph and per-person attribute data, used to build, enrich, and activate marketing segments at scale. The second is open-ended AI exploration tooling, built for fast, conversational back-and-forth with simulated consumer voices to surface language, objections, and early reactions before a team commits to bigger research or spend. Both are useful for what they do. Neither runs a controlled test of the action a team is actually deciding on. Audience-data platforms tell a team who to target once a message is chosen. Exploration tooling tells a team what a simulated conversation partner says about a concept. Neither returns a causal estimate of which version moves a real outcome. Treating either output as proof of what a decision will do is where budget gets wasted: activating a large segment against an unvalidated message spends media money at scale on something never tested against a holdout, and treating open-ended exploratory chatter as validated direction risks shipping a positioning or product call never checked against a causal baseline. ## What are audience-data activation platforms built for? This category's strength is scale and reach: a very large identity graph connecting consumer profiles to granular per-person attributes, used to define, enrich, and activate marketing segments across digital channels, plus measurement tooling to track campaign performance afterward. It fits once a team already knows which message or offer it wants to put in front of which audience and needs to find and reach that audience at scale. It is not built to tell a team which message, price, or product change should win in the first place. Segmentation and activation presuppose the decision has already been made. ## What are open-ended AI exploration tools built for? This category runs fast, conversational sessions with simulated consumer voices, including one-to-one interviews, surveys, or multi-voice group formats, to surface language, objections, and directional reactions before a team commits to larger research. That speed is genuinely useful early: it can shape the questions a later study should ask, or surface an objection a team hadn't considered. What it does not do is estimate a causal effect. A simulated conversation can produce plausible-sounding opinion, but that is not the same as an estimate of which action moved an outcome, with uncertainty attached. Recent method work on language-model-based consumer response shows that even techniques built specifically to approximate human purchase intent depend on a validated elicitation method, not on open-ended conversational output alone (LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings, arXiv, 2026). Directional exploration and causal proof are different claims, and a buyer deciding where to spend needs to know which one they are getting. "SSR achieves 90% of human test-retest reliability while maintaining realistic response distributions (KS similarity > 0.85)" Maier and colleagues, arXiv preprint 2510.08338 (source) ## Where a controlled causal test fits instead Subconscious is a causal behavioral platform. It runs controlled experiments on a simulated population and estimates which specific action, a message, a price point, a product change, moves a defined outcome, with uncertainty where supported. That sits upstream of audience activation and is a different kind of tool than open-ended exploration: a controlled experiment that produces an estimate a team can act on. ## The one place these categories agree: none of them replace real people Audience-data activation, exploration tooling, and controlled causal testing all describe themselves as inputs to human research rather than a substitute for it. Subconscious can run a study against a person-level audience graph covering 800 million real people, and can move from a simulated study to validation with real human participants without changing the underlying causal question. That step is not needed for every routine question, but it matters whenever the decision is large or novel enough that the cost of being wrong is high. ## What does Subconscious not do? Subconscious does not run cross-channel media activation or campaign measurement, and it is not a persistent audience-segment or enrichment database. It is not an open-ended exploration workspace either. It answers a specific, bounded causal question, does this action move that outcome, rather than serving as an always-on audience system or a general conversational tool. ## Before the next dollar goes to activation or exploration If the question is "who do we target with the message we've already picked," audience-data activation is the right category. If the question is "what objections might come up before we invest in a bigger study," exploration tooling can move fast. If the question is "which message, price, or product version actually moves the outcome we care about," that is a causal test, not a segmentation or conversation problem. Run the controlled experiment before committing the budget, and see how the method has been used in practice or start with a specific decision. [figure: List of three tools mapped to the question each answers: activation targets a chosen message, exploration surfaces early language, causal testing finds which action moves the outcome; all three check against real people.] --- # How to Use Synthetic Consumers for Early Concept Testing URL: https://subconscious.ai/blog/implementation-and-procurement/how-use-synthetic-consumers-early-concept-testing Published: 2026-08-25 Synthetic consumers work best as a fast directional read during exploration and panel testing, sharpening a concept before it moves to human review and causal validation for any external claim. A brand, product, or insights leader planning concept work faces the same fork every cycle: which stage gets a fast synthetic-panel read for directional learning, and which stage needs real validation before a claim ships externally or a budget gets committed. Treating a directional AI read as proof, using it to greenlight a launch or predict how the market will respond, produces a decision built on unvalidated output: wasted spend and credibility damage when real customers later contradict it. ## Why the fork matters more than the tool AI-generated consumer feedback has moved from novelty into daily research workflow. That does not eliminate demand for research judgment; it relocates it. Between 2024 and 2034, the U.S. Bureau of Labor Statistics expects the market research analyst and marketing specialist workforce to keep expanding, even as AI absorbs more of the mechanical production work: drafting surveys, summarizing transcripts, generating first-pass personas. The risk is narrower: teams either trust a synthetic read too much, or dismiss it before it can do the job it's actually good for, sharpening a concept fast, before the expensive step. ## A four-layer path from question to claim A workable evidence system keeps four layers distinct instead of collapsing them into one AI habit: - Exploration: use AI to generate hypotheses, objections, and alternative framings for a concept. - Directional testing: run a synthetic panel against one focused stimulus (a concept, message, or pricing story) to see where it draws confusion or objections. - Human review: check the audience definition, prompt neutrality, and source grounding before treating any output as a finding. - Validation: test the sharpened version against real behavior before the claim goes external or a budget is committed. [figure: Four steps left to right: Exploration generates hypotheses, Directional panel testing surfaces objections, Human review checks the audience and evidence, Causal validation tests the concept before an external claim.] ## Where does a causal experiment fit? Steps one through three are exploratory by design: fast, cheap, and meant to be wrong sometimes. Step four is where Subconscious fits: a controlled discrete-choice experiment with confidence intervals tests whether the sharpened concept actually changes buyer choice, distinct from a synthetic panel's directional read. That distinction only matters when the decision is expensive or the claim is public. For routine internal iteration, the exploratory layers alone are often enough. ## What does the workflow look like from start to finish? Before running anything, name the decision itself and note what changes depending on which way the read points. Then define the audience brief behind the panel, covering who they are, their situation, what they do now, what else they'd consider, and the goal driving that choice. Run the panel against one focused stimulus and ask for reactions, objections, and what would make the idea more credible; don't stop at the first answer. Then do the human work: read the responses, strip generic themes, and separate an interesting hypothesis from actual evidence. Only then does a decision that matters warrant moving to causal validation. Label the output honestly at every stage. Phrases like "directional panel read" or "hypothesis from AI-assisted exploration" make the method more credible, not less, because they tell the next reader exactly how much weight the finding can carry. ## What are the limitations of this approach? A causal experiment does not replace human judgment about which decisions are worth testing, and it is not itself the fast, low-cost exploration step this workflow depends on for early iteration. Running validation on every concept variant defeats the purpose of having a cheap exploratory layer; save it for the decisions where being wrong is expensive or public. ## Next step See how the validation step plays out in practice in case studies that moved from a sharpened concept to a tested claim, or book a session to scope which of your current decisions actually needs it. --- # Use Causal Experiments to Improve B2B Marketing URL: https://subconscious.ai/blog/use-cases/use-causal-experiments-improve-b2b-marketing Published: 2026-08-25 Causal experiments improve B2B marketing by testing specific campaign alternatives, such as headlines or channels, against a defined buying-group decision and measuring which action actually changes behavior. B2B campaigns often reflect what an internal team thinks buyers care about. Real buyer evidence arrives late, or not at all. A 6-week research cycle does not fit every campaign decision. The answer is not to ask a generic model what a buyer would say: how a preference question is asked changes the gap between what a model states and what it would actually reveal under an incentive-compatible task (Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models). Define the decision, alternatives, audience, and outcome. Then use a controlled experiment to estimate which action changes behavior for which segment. [figure: Four checkpoints in sequence: planning sets buyer roles and outcome, copy review compares headlines, pre-launch check audits ad-to-CTA, and the report states what was tested, changed, and uncertain.] ## Start with the buying group B2B purchasing involves multiple stakeholders. Four roles often matter: The champion wants the product and builds the internal case. The economic buyer controls the budget and needs a defensible business reason. The technical evaluator tests fit with the requirements and existing systems. The skeptic looks for reasons to stop the purchase. These are study definitions, not universal truths. Adapt them to the actual buying process. ## Test marketing actions ### Message alternatives Write three versions of the value proposition. Hold the audience and outcome constant. Compare which version changes the intended response and which objections appear by segment. ### How do you test campaign concepts? Before production, compare the proposed concepts against the same decision criterion. Separate creative preference from the behavior the campaign must move. ### Channel and call-to-action choices Treat channel preference as a hypothesis. Compare channel or call-to-action alternatives when study design and available behavioral evidence support the question. Do not present a simulated opinion as proof that a buyer will click. ### Competitive positioning Test how different claims perform against the same alternatives. Avoid asking a model to invent private competitor plans or customer beliefs. ## A three-part sprint example Campaign planning (2 hours). Define target buyer roles, current behavior, the action being considered, and the outcome. Use the session to identify the assumptions that need an experiment. Copy review (30 minutes). Compare headlines and key messages before design. Keep the stimulus and scoring rule consistent. Pre-launch check (1 hour). Review the full path from ad to landing page to call to action. Record where the experience creates uncertainty, then decide which points need behavioral validation. [figure: Four labeled roles in a row: champion, economic buyer, technical evaluator, and skeptic, each with a one-line description of what that role checks before a purchase moves forward.] ## How do you use the results correctly? Simulated buyers can support a decision-specific experiment when tied to calibration and validation. Aggregate pattern matching is easier than person-level prediction. Segment differences, cultural effects, and prompt sensitivity need careful review. The output should state what was tested, which audience definition was used, what changed, what outcome was measured, and what remains uncertain. The marketing team still owns the decision. Real campaign results remain the final behavioral evidence. See prior studies for what a completed one looks like. --- # What Is Persona Simulation, and When Should You Trust Its Answer? URL: https://subconscious.ai/blog/use-cases/what-persona-simulation-when-should-trust-its-answer Published: 2026-08-25 Persona simulation combines AI and data into a query-able stand-in for a customer, user, or stakeholder, one a team can question, test messaging against, and use to anticipate reactions before a real launch. The category has moved from static profile documents to interactive platforms that respond the way a specific person would to a question, a message, or a scenario. The harder question for a buyer isn't whether a persona tool exists. It's whether a given answer from one is validated behavioral signal or a fluent, confident-sounding output that happens to confirm what the team already believed. Shipping a message, price, or launch decision on the strength of the wrong answer means discovering the mismatch only after the spend is committed. ## What separates a simulation from a persona document Slap a stock photo and three bullet points on a document, and you have a profile, not a simulation. The real test is whether you can question it and get back the same traits, opinions, and reactions each time you ask. Three things distinguish a persona simulation from a static profile: - Interactivity. A team can ask it questions directly and get a response in seconds rather than read a fixed document. - Data grounding. Its answers trace back to observed behavior, public records, or structured input rather than getting made up from a generic template. - Validation. Better platforms test the persona's responses against held-out real human data and publish how the outputs were checked, not just what they claim. ## How does a persona simulation get built? Most platforms follow the same four-stage flow: an input stage that gathers public information, customer interviews, behavioral data, or demographic profiles; a training stage that fits a persona model, often anchoring a large language model in that source data through retrieval or conditioning; a validation stage, where the better platforms test responses against held-out human data; and a use stage, where teams query the persona directly, run multi-persona panels, test messaging, or stress-test a decision. The category's core failure mode sits inside that validation stage, and it's a documented one rather than a hypothetical. Research on persona-conditioned language models has found consistent homogenization: distinct personas converge toward generic, averaged responses instead of preserving the variation a real population would show (Investigating Persona Collapse and Homogenization in Large Language Models, arXiv). A related study of LLM agents built to represent varied sentiment found stable but narrow response patterns, meaning the agents were internally consistent but didn't reproduce the real population's range of opinion (Stable Behavior, Limited Variation: Persona Validity in LLM Agents, arXiv). Both papers describe the same risk: a persona can sound confident and self-consistent while quietly collapsing toward the model's default voice instead of the person it claims to represent. ## What is a persona simulation useful for? Teams use interactive personas to test messaging by putting two versions of a claim or a tagline in front of a persona and asking which lands and why, to pressure-test a product concept before committing engineering time, to pre-test campaign creative or a pricing change before it reaches the market, to run a panel of simulated customers that delivers a focus group's qualitative feel without recruiting, scheduling, or moderating a live one, to rehearse a sales or investor pitch against a simulated buyer before the real meeting, to walk a persona through a customer journey and flag friction at each step, and to explore a churned-customer persona for why they left without running exit interviews. Persona simulation doesn't replace research; it replaces some of it and accelerates the rest. It's a reasonable substitute for directional questions that used to require a full survey or focus group, where getting a rough read matters more than statistical rigor, and for pre-testing creative before it goes to a larger study. It accelerates real research by handling early-stage divergence, generating and narrowing hypotheses worth testing formally, and late-stage validation, checking whether a concept lands with a target segment before a bigger study confirms it. Statistical research bound by regulatory or compliance demands, in-depth ethnographic fieldwork, and live customer feedback gathered at scale are still outside its reach. ## Four questions before trusting a persona's answer A buyer evaluating any persona simulation tool should push on four things: - Validation. Does the platform publish accuracy figures, state what they were measured against, and explain how the check was run? A vague claim of realism is a red flag; a stated benchmark with its source and denominator is a signal worth weighing. - Source data. What grounds the persona: public information only, internal data, or customer interviews? More grounding generally means better fidelity, but grounding alone doesn't guarantee variation across a simulated population. - Consistency. Ask the same question three times. A persona that contradicts itself across repeats isn't actually anchored to anything. - Specificity. Ask a niche question that only a specific customer segment would care about. A generic, platitude-shaped answer is the signature of a fluent model wrapping a persona label around itself, which is the same homogenization risk the research above documents directly. ## Where does the framework break down? Three failure modes account for most bad decisions made on top of a persona tool. Generic outputs, where the persona reads like an unconditioned language model rather than a specific customer, usually mean the underlying grounding work wasn't done. Confirmation bias, where a team keeps asking a persona questions until it agrees with the plan already in motion, produces exactly the answer the team went looking for rather than a test of it. And over-trusting a fluent response as proof is the mistake the validation research above exists to prevent: a persona tool is a research instrument, not a truth machine, and a confident answer still needs to be checked against real people before it changes a real decision. ## How this maps to a causal experiment The reliability question above is really a design question: was the persona's answer produced by a controlled comparison, or by a single fluent response to a single prompt? Subconscious approaches the same buyer decision as a controlled experiment on a simulated population, not a persona chat: a team defines the alternatives, runs the comparison across the population, and can validate the resulting study with real human participants. That validation step is what the homogenization research above argues is missing from an unchecked persona conversation. When the decision depends on scale rather than a single conversation, Subconscious can also run controlled studies against a person-level audience graph covering 800 million real people. That audience graph is a modeling resource, not a recruitable panel; running a study against it and recruiting real participants for validation are separate steps. Details on how a study moves from simulated comparison to human validation are on Subconscious's research page, and published replication results for specific studies are on the leaderboard. [figure: Three-band list: tasks a persona simulation fully replaces, tasks it only speeds up around a real study, and tasks it cannot do, like regulated research or fieldwork.] ## Before a persona verdict changes a decision Treat a persona simulation's answer as a hypothesis, not a verdict. Run the four-question check above on any platform under consideration. Then require that a directional finding gets checked against a controlled comparison before it changes a message, a price, or a launch call, using real-human validation when the decision is expensive enough to justify it. Teams that want to see what that comparison-plus-validation workflow looks like in practice can review how Subconscious works or read more about the company. [figure: Four-step chain: input feeds training, which feeds validation against held-out human data, which feeds use. Validation is marked where personas can collapse into generic answers.] --- # Do LLMs Understand Real-World Prices? A Pricing Benchmark for Synthetic Consumers URL: https://subconscious.ai/blog/methods-and-validation/llms-understand-real-world-prices-pricing-benchmark-synthetic-consumers Published: 2026-08-25 An LLM-based synthetic panel can produce fluent, plausible-sounding survey answers without ever grounding those answers in real prices. Before a pricing or packaging decision leans on that kind of panel, the panel's price reasoning needs its own check. ## The decision this bears on A brand or insights team weighing an LLM-persona panel against a calibrated, validated experimental design is really deciding how much price-sensitive risk to accept. An uncalibrated panel that sounds confident about cost can still produce systematically biased demand or willingness-to-pay estimates. Shipping a pricing decision on that basis risks wasted research spend and a go-to-market call built on a number the panel never actually understood. ## How does this benchmark test price reasoning? [figure: A list of four checks a buyer applies to a pricing method: is the estimate grounded, does it use reference examples to correct itself, does its bidding show restraint, and does the resulting choice behavior hold up.] A recent benchmark tested this directly by adapting a classic televised pricing-showcase game into a structured evaluation for large language models. Two models see the same showcase of consumer packaged goods, mostly everyday items like toothpaste and snack bars with a combined retail value around $20. Each model bids the total retail value; the closest bid without going over wins, and if both models overbid, neither wins. Each round follows the same five steps: generate a random showcase of three items, hand each model ten example prices from similar products as calibration reference, send the identical prompt to two models, parse the bid and its rationale from a strict JSON response, then compare both bids to the actual retail price to score the round. Naming this failure mode up front is what lets a buyer test each capability on its own. The design targets three related capabilities: estimating a real-world price, using reference examples to calibrate that estimate, and deciding how aggressively to act on it, though a single bid does not cleanly separate estimation from shading. ## Accuracy and restraint are not the same skill Dozens of models entered the tournament; two preliminary rounds narrowed the field to a small group of finalists competing across fifty rounds. The best-performing model in the finals estimated prices with an error noticeably below the error rate of human contestants on the original televised game, though the comparison is confounded: models bid on roughly $20 CPG showcases while contestants bid on showcases worth tens of thousands under real stakes; the weakest model's price estimates were only marginally better than random guessing, with almost no relationship between its bids and the actual prices. Accuracy alone did not decide the outcome. The game penalizes overbidding by even a cent with automatic disqualification, so rational play means bidding a lower quantile of the estimate's distribution, not a small shave off a point estimate. Human contestants on the original show overbid about 25% of the time, a descriptive rate under real stakes and prior-round selection, not a normative benchmark. Among the finalist models, the most conservative overbid only 2% of the time, while the most aggressive overbid 72% of the time. The model with the single best price accuracy in the field still overbid well above the human benchmark, which cost it standing on the overall leaderboard. Models varied in how well they used the ten reference prices they were given: models that incorporated the calibration examples tended to produce better-calibrated bids than models that largely ignored them, an observational pattern that does not isolate prompting effects from underlying model capability, though it echoes recent controlled research on prompting strategy. A controlled study of prompting approaches for choice-modelling tasks found that how calibration guidance and reference information get structured in a prompt materially changes the quality of a model's output, not just which model runs the prompt (Can large language models assist choice modelling? Insights into prompting strategies and current models' capabilities, arXiv, 2026-07-28). "Our findings reveal that proprietary LLMs can generate valid and behaviourally sound utility specifications, particularly when guided by structured prompts (CoT)." Sfeir, Nova, Hess, and van Cranenburgh, Journal of Choice Modelling (source) ## What this means for evaluating a synthetic-consumer method The historical benchmark numbers above describe one study's models and prompts at one point in time. They are not a live leaderboard and not a claim about any specific vendor's current product. Read them as evidence that price reasoning splits into distinct, testable parts, not as a ranking to shop from. A synthetic-consumer method built around raw LLM roleplay inherits whatever calibration and restraint its base model happens to have on a given day. A method built around calibrated experimental design paired with validation against real human data tests those parts on purpose: it asks whether an estimate is grounded, whether the model uses reference information to correct itself, and whether its resulting choice behavior holds up before it drives a pricing or packaging decision. Subconscious's causal action-testing and discrete-choice experiment design is built around that kind of calibration and validated design rather than raw model access. ## What are the limits of this benchmark? A number without its limits is marketing. This benchmark measures price estimation and bidding restraint in one bidding game. It does not measure willingness-to-pay in a real purchase context, brand-specific price perception, or how a synthetic panel performs on a causal question outside pricing. A team evaluating a synthetic-consumer method for its own pricing or packaging question still needs evidence specific to that question and, when the decision is large enough, a way to check a simulated result against real-human testing. Subconscious can test or validate studies with real human participants, and a team can move from a simulated experiment to that kind of validation without changing the underlying causal question. [figure: Two columns, roleplay and calibrated design, against four rows: grounding, reference use, restraint, choice validity. Roleplay is inherited/untested; calibrated is tested on purpose.] ## Next step Teams comparing synthetic-consumer methods for a pricing decision can see current published comparisons on the leaderboard or read about the validated experiment workflow on how we work. --- # Running Client Workshops on Evidence Instead of Opinions URL: https://subconscious.ai/blog/use-cases/running-client-workshops-evidence-instead-opinions Published: 2026-08-25 Agencies and consultancies run client workshops for one of two reasons. Either the client needs to feel like part of the strategy, or the facilitator has to shake a stuck decision free so the project can keep moving. Both paths usually end the same way: a board full of sticky notes, a few hours of talking, and no decision that survives the next stakeholder meeting. The failure is not the facilitation method. It is that workshops usually sit at the end of the process, after the room has already formed opinions, instead of at the point where those opinions could be tested against something. That gap has a name: the highest-paid person's opinion routinely overrides the room, a pattern documented across corporate decision-making research (Data-Driven Decision Making: Beware of the HIPPO Effect, Forbes). ## Where workshops actually break Facilitators who run these sessions recognize the same failure modes: - The HIPPO problem. The highest-paid person's opinion wins, and everyone in the room already knows it. The workshop becomes theater for a decision made before it started (What Is the Highest Paid Person's Opinion?, Growth Method). - False consensus. Everyone nods, the deck gets signed off, and two weeks later different stakeholders each remember the meeting differently, and their accounts don't match. - The research gap. Nobody in the room can say what customers would actually think, so the default move is commissioning research that stalls the project for weeks. - Creative paralysis. Too many options, no way to narrow them, so the facilitator calls a vote. The result reflects the room's politics, not the quality of the ideas. These failures cost real money: rework, scope creep, and a client relationship that erodes every time a "final" decision gets re-litigated. ## The decision this article is about The question a facilitator has to answer is not whether the workshop should use data. It is narrower: which specific moments in the session are testable claims about buyer response, and which are judgment calls the room needs to make together. Route the wrong ones to a vote, and the workshop reproduces the HIPPO problem with extra steps. Route the wrong ones to a test, and the room's energy goes into pressure-testing something nobody actually disagreed about. [figure: Four-step path: options generated untested; top options tested via controlled comparison instead of a vote; winning direction pressure-tested against the hardest objection; culture and alignment stay outside testing.] Subconscious supports the testable moments with causal action testing: controlled, randomized experiments run against a person-level audience graph covering 800 million real people, distinct from convening a live recruited group in the room. ## A decision path for the room Divergent phase: human only. Let the room generate options without interruption. Testing narrows; it does not generate. Early evidence here just anchors the group on whichever option happened to be tested first. Convergent phase: test the top options. Once the room has a shortlist, such as a tagline, a creative direction, or a price frame, this is the moment a controlled comparison earns its keep. Instead of a vote that rewards whoever argues loudest, run the alternatives as a causal comparison and let the room converge on a result instead of a compromise. Pressure-test phase: stress the winner. Take the direction the room converged on and test it against the hard question: what would make a buyer reject it, or mistake it for a competitor's message. This is where a workshop's confident answer either holds up or reveals a gap before the client sees it. Human-only moments. Culture conversations, team alignment, and deeply relational client dynamics are not measurement problems. A controlled test answers which option moves an outcome, not how a team wants to work together. Forcing evidence into that conversation reads as cold, not rigorous. The same logic carries across workshop types. Positioning and messaging sessions are the strongest fit: taglines and value propositions are exactly the kind of alternative a controlled comparison is built to test. Creative-direction and pricing workshops fit the convergent-and-pressure-test pattern well. Naming sessions, which are notoriously opinion-driven, benefit from replacing preference with a measured comparison of association and recall. Highly technical stakeholder sessions, such as engineers debating an API design, usually fall outside what a general buyer-facing test was built to answer. ## What this changes about the workshop itself Framed correctly, this is not a claim that a live audience reacts to material inside the meeting. It is a claim about method: a randomized, controlled comparison with a documented result, run on the specific options the room narrowed to, that the facilitator brings back to a later working session. The value is in the evidence, not the speed of getting it. When a client asks how confident the recommendation is, the honest answer follows a fixed grammar: the claim, the comparison it came from, the source, and its limitation. A causal action test is not a clinical trial, an observed usability session, or automatic proof of market performance. It estimates which tested action moves a defined outcome, for a defined audience, and that estimate can be checked further. Subconscious can also test or validate a result with real human participants, so a team can move from the workshop's controlled comparison to real-human validation without changing the underlying question. ## What are the limits of this approach? A controlled comparison does not suit pure brainstorming or a conversation whose goal is team alignment rather than a measured outcome. The audience graph a test draws on is not the same thing as a recruited panel of real respondents assembled for that specific session. Keep those two claims distinct when a client asks who was actually tested. [figure: Ranking of four workshop types by fit: positioning best, since taglines are buyer alternatives; creative/pricing fit well; naming fits by measuring recall; technical sessions like API design don't fit.] ## What does this mean for the facilitator? The workshops that hold up after the meeting ends are the ones where the room's biggest disagreements got tested instead of voted on. That does not require redesigning the session. It requires knowing, before the workshop starts, which of its moments are testable claims, and routing only those through a controlled comparison. See how the underlying method works for the mechanics, and what a completed comparison looks like for the shape of the output. The rest of the room's time stays exactly what it should be: people making a judgment call together. --- # Four Places to Test Employer-Branding Messaging Before It Ships URL: https://subconscious.ai/blog/buyers/validate-employer-brand-recruitment-messaging Published: 2026-08-25 An agency running employer-branding work usually ships EVP copy, job postings, career page sections, and campaign creative on internal workshop consensus, then waits out the 30-60 day posting window to find out. Testing each piece of messaging against the client's actual target-candidate segment before launch catches a mismatch while it still costs nothing to fix. ## Why the posting window is an expensive place to guess A job posting runs for 30-60 days before an agency gets a real signal on it. In one planning example, a posting collected 200 applications and none of them were qualified. Unclear expectations, generic language, requirements that discourage qualified applicants, only surface once the client is looking at the application pile (Netchex, Hiring Pitfalls to Avoid in 2026). By then the client has burned the full posting window and the ad spend on messaging that drew volume but not fit. The EVP is the same problem one layer up. It sets the frame for every posting, career page section, and campaign asset that follows, but it is normally written and approved internally, not checked against the people it is supposed to persuade (AIHR, Employee Value Proposition (EVP): All You Need to Know in 2026). ## What a message test actually needs to answer Each stage of employer-branding content is a distinct, testable message decision: does this EVP wording, this posting draft, this career page section, or this campaign concept move stated interest in the specific segment the client is hiring for. Job seekers tell you directly what they weigh when comparing postings, clarity on compensation, growth path, and day-to-day expectations chief among them (Indeed, Survey: How to Improve Your Job Postings to Attract Better Candidates). [figure: Four employer-branding message decisions shown in sequence: EVP wording, job posting variant, career page section, and campaign creative, each tested against a target-candidate segment before a posting goes live.] ## Four places to test employer-branding messaging before launch ### 1. EVP wording The employer value proposition anchors every other piece of employer-branding content, but it is usually validated by how it sounds in the room, not by how the target segment reacts to it. Testing two or three EVP drafts against the client's actual hiring segment, senior engineers, early-career marketers, mid-career finance professionals, whatever the role calls for, shows which wording moves stated interest. ### 2. Job posting variants A posting that emphasizes autonomy and growth reads differently to a senior engineer than one that emphasizes stability and benefits, and the difference shows up in who applies. Testing two posting variants against the target segment before either goes live surfaces which framing draws the right applicants. ### 3. Career page sections A career page has to answer why a candidate should work at this company instead of another one, and different sections carry that argument differently for different segments. Testing draft sections against the target audience before a design team invests in production shows which parts read as compelling and which read as generic filler. ### 4. Campaign creative Recruitment campaign creative, LinkedIn sponsored content, programmatic job ads, campus materials, competes for attention the same way consumer advertising does. Testing headline and image variants against the target candidate segment before spend goes out shows which concept holds attention and which reads as stock corporate messaging. ## Where this fits a causal test, not a panel The shape of this work is a message test: comparing two or more wording, layout, or creative variants for a defined audience segment and measuring which one moves stated interest. Subconscious runs that as a randomized experiment, discrete choice, monadic, or claims-style, and returns which variant moves the outcome. That is not a recruitable panel of candidates and not a simulation of any one person. Subconscious can run these studies against a person-level audience graph covering 800 million real people, and can also test or validate a study with real human participants without changing the underlying causal question being asked. ## Limitations and failure conditions There is no published employer-branding or recruitment-agency case result to point to yet, so treat the approach above as a testing method to apply to a client's own segment, not as a track record. The audience graph used to run these studies is built for study recruitment; it is not a simulated or recruitable pool of job candidates, and framing it that way overstates what it does. A message test also only tells you which wording or creative a segment prefers, it does not verify culture fit, screen for skills, or replace the interview and hiring process that follows an application. [figure: Two columns. Left: message test, variants measured against a target-candidate segment from an 800M-person audience graph. Right, crossed out: recruited candidate panel, simulated single applicant.] ## Next step An agency with a client EVP draft, a job posting variant, or a campaign concept ready to test can see how a causal experiment is set up for a defined candidate segment, review how a study moves from design to result, or book time to scope a test against an upcoming posting window. --- # Choosing Attributes and Levels for a Discrete Choice Experiment URL: https://subconscious.ai/blog/implementation-and-procurement/choosing-attributes-levels-discrete-choice-experiment Published: 2026-08-25 ## The decision before the study runs Before a conjoint or discrete choice study can go into the field, someone has to decide what it actually tests: the attributes (price, features, packaging, messaging, positioning, and similar variables) and the levels within each that respondents will see and trade off against one another. This is the second step in defining an experiment, ahead of standing up the market simulation and running the controlled comparisons. Get this step wrong in either direction and the study's answer is compromised. Leave out an attribute that actually drives the choice, and the experiment answers a different question than the one that matters. Include too many attributes, or combine levels into unrealistic packages nobody would encounter in the real world, and respondents disengage or answer inconsistently, degrading every estimate the study produces downstream. ## What is an attribute in a discrete choice experiment? An attribute is any dimension of the offer that could plausibly change a customer's decision. For a product, that might be price, a feature, packaging format, or a messaging claim. For a service or policy proposal, it might be coverage terms, delivery timing, or a benefit's wording. Each attribute needs multiple levels: the concrete variations respondents will actually see and compare, such as three price points or two versions of a feature. The discipline here is narrowing, not listing everything that could conceivably matter: a well-specified attribute list holds the small set of variables that plausibly change the outcome. ## How to narrow the list Three checks separate a usable attribute list from an overloaded one: - Does it plausibly drive the decision? If removing the attribute wouldn't change how a customer chooses, it's a candidate for exclusion rather than inclusion. - Are the levels realistic in combination? Attributes tested together should reflect offers a customer could actually encounter, not synthetic combinations that never occur in the market. - Is the list short enough to hold respondent attention? Every additional attribute adds cognitive load; a list that tries to cover every possible variable produces noisier answers about all of them, not more complete ones. The ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force report sets out this same tradeoff as a core design constraint: attribute and level selection has to balance realism and relevance against respondent burden, because an overloaded design degrades the statistical properties of every estimate that comes out of it. ## How does attribute selection fit into the experiment workflow? Subconscious's discrete choice experiments use McFadden-style discrete choice and Mixed Logit estimation to recover the causal effect of each attribute and level on the choice a customer makes. That estimation is only as good as the attribute list feeding it: the statistical methods task force report documents how the analysis stage depends on decisions made at the design stage. Subconscious's published validation record covers this modeling layer directly: our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters, drawn from a corpus of roughly 300 replicated studies across 9 domains, the mean is 0.73. See the causal fidelity paper. That figure describes replication across the corpus, not how accurately any single team selects its own attributes, which remains a judgment call shaped by domain and decision. [figure: A decision path narrowing candidate attributes through three checks (drives the decision, realistic levels, short enough for respondent attention) down to a locked experiment design.] ## What are the limitations of attribute selection? Attribute selection is a judgment call, not a mechanical procedure. Domain knowledge about what influences a customer's decision has to come from somewhere, whether prior research, stakeholder interviews, or a pilot round, before the list locks. No amount of downstream estimation sophistication corrects for a missing attribute upstream, and because levels must reflect realistic combinations, the same attribute list rarely transfers unchanged from one product or market to another. [figure: A chain shows an attribute list flowing through design into Mixed Logit estimation to causal effect estimates. A parallel path shows a missing attribute passing through unchanged, still absent from the final estimate.] ## Next step Once the attribute list and levels are locked, the next step is standing up the market simulation and running the experiment. Teams that need to confirm a result against real people can move from a simulated study to real-human validation without changing the underlying causal question. Review worked examples, or look at the underlying research behind the estimation methods. Teams ready to scope a study can start with a demo. --- # Why B2B Focus Groups Are Broken and What Replaces Them URL: https://subconscious.ai/blog/use-cases/why-b2b-focus-groups-broken-what-replaces-them Published: 2026-08-25 B2B focus groups fail because they seat one shared room for a decision that a buying committee of separate roles with separate objections actually makes, and structured per-role experiments replace them. A focus group puts eight to twelve people in a room, a moderator asks questions, and a team draws conclusions about how a larger population will respond. The format dates to wartime morale research in the 1940s, was adapted for consumer marketing in the 1950s, and has changed little since. For B2C decisions, that format holds up well: one person buying cereal or soap makes an individual, preference-driven choice, so a room of twelve target consumers can give a usable signal about a larger population. B2B buying does not work that way. The focus group is a mismatched tool for that decision. ## Why can't one focus group represent a B2B buying committee? A typical enterprise software purchase is not decided by one person. It moves through an Economic Buyer who controls budget and weighs ROI and risk, a Technical Buyer who evaluates integration complexity and security posture, one or more User Buyers who will run the product day to day, and often a Champion advocating for the deal. [figure: Diagram contrasting one shared focus-group room against four separate tests, one per buying-committee role: Economic Buyer, Technical Buyer, User Buyer, and Champion, each carrying its own objection.] A focus group seats four to eight of these people together, but in a real deal they are rarely in the same room making a joint call. They sit in separate meetings, weigh separate priorities, and shape each other's positions through a political process that a shared session cannot reproduce. An analysis of enterprise buying puts the typical committee at eight to thirteen stakeholders (Attainment, 2026). One ninety-minute session, run with whichever subset of that committee had a free hour, is asked to stand in for all of them. ## How long does a B2B buying decision take? Enterprise software purchases typically take six to eighteen months from initial awareness to signed contract, moving through problem recognition, solution exploration, business case, evaluation, and procurement. Concerns shift stage to stage: a champion who is enthusiastic in an early conversation can turn defensive once the CFO raises cost in a later one, and a technical buyer who was an ally during evaluation can introduce new security requirements during procurement. A single focus-group session is a snapshot of one stage, not the sequence. The people who actually decide are rarely the people with ninety free minutes to sit in a research session. Attendance in a scheduled focus group correlates with schedule flexibility, and schedule flexibility correlates with lower organizational seniority: the research skews toward whoever had a free calendar slot, not toward the people whose judgment moves the deal. ## What does B2B buyer research actually need? B2B research needs to represent the full buying committee rather than one role at a time, follow how a position shifts as a deal moves stage to stage, reach senior stakeholders without depending on their calendar, and run often enough to answer a new question as it comes up rather than before a major launch. Ship one message that happens to work for whichever stakeholder showed up in the room, and a team can discover post-launch that the technical or economic buyer's real objection was never tested: a deal that looked validated stalls on an objection nobody ran. ## Where a structured experiment fits Subconscious runs controlled experiments that compare messaging and claims across defined buyer segments with quantified uncertainty: separate experiments per role replace the one shared session. Testing a positioning statement against an Economic Buyer segment and a Technical Buyer segment as two separate comparisons, instead of one blended read, means a result that "resonates" in the room isn't secretly the median of four disagreeing audiences. Subconscious runs controlled studies against a person-level audience graph covering 800 million real people, a reach claim rather than a recruited panel: a study result comes from randomized comparisons across that audience graph. When a comparison result needs confirmation before a launch decision, a team can test or validate the same study with real human participants without changing the causal question. ## What this does not cover This is buyer-segment experiment design, not a buying-journey simulator: it does not model how one committee's position shifts meeting to meeting over a six-to-eighteen-month cycle, and a claim about a deal's trajectory needs to come from tracking that deal. It is also not a persona builder: the unit of comparison is a defined buyer segment tested with a message or claim, not a simulated individual with a backstory. [figure: Five sequential stages of a B2B buying decision, with one highlighted point marking where a single focus-group session sits against the full stretch of the timeline.] ## Next step Before running a message or price point through a scheduled session that can only seat one slice of the committee, Subconscious's use cases walk through how a buyer-segment experiment is scoped, and current experiment results show what a comparison across defined segments looks like end to end. A demo shows a committee-role comparison run against a specific message. --- # Competitive Win/Loss Analysis: Testing Objections Before You Trust a Thin Interview Sample URL: https://subconscious.ai/blog/use-cases/competitive-win-loss-analysis-testing-objections-before-trust-thin Published: 2026-08-25 A head of product marketing has a pricing narrative or battlecard objection ready to ship against a competitor. The problem: the exit interviews that would confirm the pattern haven't come in yet, and the few that did might be noise from whoever was willing to explain why they walked. Stated loss reasons in win/loss research run close to 85% inaccurate against what a buyer acted on (User Intuition), so a team working from 8 to 12 self-reported interviews a quarter can fix the wrong objection, ship a feature nobody needed, or hand reps a battlecard that never touches the real decision. [figure: Five-step path: thin interview sample feeds a controlled choice experiment against a named competitor, producing ranked objections compared against real interviews, then splitting into acting or gathering more data.] ## Why is win/loss data thin by default? The prospects a team most needs to hear from (the ones who chose a competitor or churned quietly) are the least likely to answer an exit survey. What comes back skews toward whoever is most frustrated or most polite, not the deals that shaped the pattern. Even the interviews that happen carry social-desirability bias: "we went in a different direction" is a real answer, but not an actionable one. ## What can a controlled experiment do before the interviews arrive? Subconscious runs controlled discrete-choice experiments against a defined buyer segment, testing which claims, feature framings, or price moves shift preference against a named competitor, reported as a causal effect with a confidence interval, not paraphrased quotes. Run against a segment standing in for a lost-deal or churn profile, it ranks which objection is worth a sales or product response before committing real budget to a fix. This is a hypothesis-generation step, not a replacement for real win/loss interviews: it tells a team what to test next with real people, not why any named deal was lost or why a specific account churned. ## Building the test - Describe the loss or churn profile. Start from what the CRM shows about a cluster of lost deals or churned accounts: company size, industry, the decision-maker's role, the competitor chosen. - Split into two or three segments. An enterprise evaluator who chose a named competitor is a different buyer than an SMB founder who churned after a free trial; test each separately. - Run the same choice experiment across segments, varying price framing, a specific feature's presence, and switching cost, and measuring which change shifts stated preference and by how much. - Compare the ranked result against the real win/loss interviews and NPS verbatims a team has. Where the two agree, act on the ranked factor with more confidence. Where they diverge, gather more real interviews before committing. ## Do simulated experiments replace real interviews? Neither replaces the other; the right approach depends on how much real data is available and how fast a decision is needed. ## Turning the result into action Sales enablement. Give sales the objection that ranked highest in the test, along with the counter-argument that moved stated preference in the same experiment. Roadmap input. When a feature gap consistently ranks as the deciding factor, that's a reason to investigate with real customers, not a green light to build. Positioning. If the test shows a message losing to a competitor's framing, not their actual product capability, that's a messaging fix, not an engineering one. ## Limitations A simulated experiment can only rank which factors move stated preference across a defined segment standing in for that profile. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, kept distinct from a recruited panel, but that audience graph doesn't ingest a company's CRM data and doesn't substitute for talking to its own lost or churned customers. Before a pricing or positioning decision ships, the ranked hypothesis needs validation: Subconscious can test or validate studies with real human participants, so a team moves from a simulated experiment to real-human validation without changing the causal question. See current research and methodology, read a case study of a study built this way, book time to scope a study against a specific competitive loss pattern, or read how a study like this gets built. --- # Testing Paywall Copy Before You Spend Live A/B Test Traffic URL: https://subconscious.ai/blog/use-cases/testing-paywall-copy-before-spend-live-b-test-traffic Published: 2026-08-25 A team with several paywall copy variants and one live A/B test slot should run a randomized experiment against defined reader segments first, then commit only the strongest variants to real traffic. [figure: Four stages left to right: write distinct paywall copy variants, test each against every reader segment, filter out variants that fail any segment, commit survivors to the one live A/B test slot.] ## Why the paywall decision is expensive to get wrong The paywall is a binary moment in a freemium product. A user hits the usage limit and, within seconds, does one of a few things: upgrades, dismisses the modal and stays on the free tier, or leaves the app frustrated. The copy in that modal shapes that split more than most decisions a product team makes that quarter. CTA phrasing and offer framing are documented factors in paywall conversion, and plan or pricing structure often outweighs copy in its effect on lifetime value (Qonversion, "Beginners Guide to Paywall A/B Testing"). Live paywall tests are the standard way to resolve which copy wins, but they carry real cost: live traffic, weeks to reach significance, contamination of other in-flight experiments, and a hard limit on how many variants a team can commit to at once (Adapty, "Paywall A/B Testing Mistakes to Avoid"). A controlled experiment run against defined audience segments before the live test does not replace the live test. It filters the variant list down to the ones worth spending live traffic on. ## Which reader segments decide whether paywall copy works? Paywall copy has to work for more than one reader at once, and each reader arrives at the paywall in a different state: A single piece of copy has to survive all of these readings at once. "Write clearer copy" is not an actionable instruction on its own, because clarity for one segment can read as a mismatch to another. ## Should paywall segments come from a generic template? The segment definitions above are a starting shape, not a fixed persona set. A defined segment should come from the team's own paid and free user data, not an assumed archetype: job function, company size, use-case depth, and frequency of use are the kinds of traits that separate a paying user from a free user, and those traits differ by product. A B2B workflow tool and a consumer creator tool will not share the same power-user profile, and copy tuned to one will not automatically work for the other. The free-tier segments, the evaluating user and the accidental visitor, are harder to characterize because there is less behavioral data on them. A rough profile built from available usage data is still more useful than an untuned, generic one. ## Running the pre-test before the live test - Write variants that stake out different strategic positions, not small wording tweaks of the same idea. A value-reminder framing, a scarcity framing, a social-proof framing, a direct-utility framing, and a minimal framing that gets out of the reader's way are different bets. - Run every variant against every segment. Ask each defined segment what it would do next on seeing the copy and what it would need to read before upgrading. This surfaces which variants fail, and the failures are often not the ones a team expects. - Look for the variant that survives across segments, not the variant that wins for any single one. Paywall copy earns more from not failing badly for any reader than from being optimized for one at the expense of the others. - Stress-test the survivors. Ask each segment which sentence would make it close the modal without upgrading, or which claim feels unearned. This finds the specific line costing conversion, which a team that has read the copy dozens of times stops seeing. - Commit only the surviving shortlist to the live A/B test. The output is a shortlist that has already survived a first pass, a different starting point than one chosen by internal debate alone. ## What this approach tends to surface Recurring patterns worth checking for directly: - Copy that leads with the features a paid plan opens up, rather than the task the user was interrupted from, tends to underperform copy that acknowledges the interruption first. - A price shown too prominently pulls attention away from the value proposition that should justify it; framing value before price tends to test better than leading with the number. - The secondary "not now" option is frequently under-designed. Both the accidental visitor and the annoyed power user rely on that exit path, and a hostile-feeling exit pushes them to churn instead of staying on the free tier. - Logo-based social proof using names a reader does not recognize tends to underperform specific, concrete usage detail about existing paying customers. None of these are guaranteed outcomes for a given product, but they are the kind of miss a segment-level pre-test is built to catch before the live test does. ## What does a paywall pre-test not replace? A pre-test experiment does not predict the live A/B test's exact result, and it is not a substitute for direct customer discovery or for observed churn and retention data once a variant ships. What it changes is which variants earn a live test slot, and how much analytics contamination and lost conversion a team absorbs while a live test runs its full duration. Subconscious runs this kind of experiment as a randomized comparison: each paywall copy variant is tested as a defined action against defined audience segments, with upgrade choice as the measured outcome. Subconscious can test or validate the same study with real human participants once a shortlist is set, without changing the underlying causal question. There is no published Subconscious benchmark or case result for paywall copy specifically. The case studies page covers the range of decisions this method has been used for, and the research program explains how replication and human baselines build trust in the method itself. If a paywall, pricing page, or trial-expiry flow is already on the roadmap, the fastest way to see this in practice is a live walkthrough against real copy variants. --- # Lock the Target Group and Stimulus Before Writing Survey Questions URL: https://subconscious.ai/blog/methods-and-validation/lock-target-group-stimulus-before-writing-survey-questions Published: 2026-08-25 Lock the target group, the decision question, and the stimulus before a single survey question gets written. Teams that start from a vague business ask instead of a structured plan end up fielding an instrument that produces clean-looking answers to the wrong question. ## The decision this page answers A research lead, or a product or brand manager without a dedicated research team, is about to scope a study on product, pricing, CX, brand, or UX. The choice: lock the target group, the stimulus, and the decision question into a structured plan before committing fieldwork budget, or write questions from a loose brief. ## Why is the wrong order expensive? A number on its own is a marketing claim. Publishing the failure mode next to it is what makes the number checkable. A fielded instrument built on a fuzzy target-group definition or an untested stimulus still returns answers that do not map to the actual decision. The team has already paid for recruiting, programming, and moderation by the time that becomes visible, and the usual fix is re-running the study or quietly shelving the results. ## What to lock before writing a question Four choices determine whether a survey measures the right thing, each made explicitly, in this order, before wording starts. If the target-group definition is still fuzzy at this stage, the task is to pressure-test it and surface the subsegments and assumptions that need evidence before fielding a study. The American Association for Public Opinion Research's best-practice guidance treats a clearly specified target population and a defined measurement objective as preconditions for a defensible survey, not optional refinements made after drafting. ## Where does simulated pre-testing fit? Subconscious is a causal behavioral platform: controlled experiments on simulated populations that test product, pricing, messaging, and go-to-market actions before capital gets committed to fieldwork. At this planning stage, it pressure-tests a target-group definition, a stimulus, and a decision question against a simulated audience before recruiting a real respondent. Subconscious can run controlled studies against a person-level audience graph covering 800 million real people, which is a distinct capability from recruiting real participants for a study. Naming this boundary is what lets a buyer verify the scale claim against actual capability. The audience graph supports simulated experiments at that scale. It is not a recruitable panel of 800 million people, and the two are never interchangeable. ## When does the plan still need real respondents? A limitation stated here is a limitation a buyer can plan around later. Use a simulated pre-test to compare proposed actions and identify assumptions that still need evidence. It does not replace human evidence when the decision carries regulatory weight, needs representative population statistics, or requires financial or compliance certainty that only a fielded study with real respondents can support. Subconscious can also test or validate studies with real human participants, keeping the causal question fixed when a team moves from a simulated experiment to real-human validation. That step matters only when the stakes justify it, not for every study. ## Failure conditions to watch for - The target group is still a guess. If the team cannot describe who is answering and what they already know, no amount of question polish fixes that. - The stimulus is described instead of shown. A vague verbal summary of a pricing change or feature produces answers about the summary, not the product. - The analysis plan gets written after the data arrives. Deciding the cut or comparison after seeing the answers is how a study ends up supporting whatever conclusion the team already believed. - A directional pre-test gets treated as a final measurement. A confident-sounding answer from a simulated pass is not automatically a certified statistic. ## Practical checklist before fieldwork opens - Write the target group as a specific population, not a demographic label. - State the decision question as a comparison between named options. - Build the stimulus as something a respondent could actually see or read, not a description of one. - Decide the analysis plan and the segment cuts before any answers exist. - Pressure-test the plan against a simulated pass, then decide whether the stakes justify real-human validation before locking the field instrument. More on how that review process works in practice is at how we work and in applied case evidence. To scope a causal comparison around a specific product, pricing, messaging, or market decision, book a study discussion. [figure: Four-step sequence: target group, decision question, stimulus, analysis plan. Each locks before the next; skipping to question wording early is the failure path.] --- # Best tips and tricks for drafting surveys for response quality URL: https://subconscious.ai/blog/methods-and-validation/best-tips-tricks-drafting-surveys-response-quality Published: 2026-08-27 A research or insights leader who owns survey design is really deciding one thing: whether to trust a clean-looking dataset or to build in the randomization that makes an answer causal. The standard fixes for response quality are well known: trim survey length, cut matrix grids, randomize item order, replace attention-check traps with commitment pledges, and screen on a vetted panel. All of it cleans the respondent side of a stated-preference instrument. It reduces noise in self-reports. It does not turn a self-report into a measurement of what actually drives behavior, because none of it randomizes the thing being tested. - Keep surveys to 7-10 minutes to reduce abandonment, and avoid matrix/grid questions, a known trigger for straightlining. - A percentage with no source named next to it is marketing. Commitment requests (asking respondents to pledge thoughtful answers up front) cut data-quality issues by more than half compared with traditional attention-check traps, per Qualtrics' own research on its product (qualtrics.com), not independently replicated. - Prolific and CloudResearch panels outperform MTurk, Qualtrics panels, and SONA student samples on attention checks and instruction-following, per a 2023 PLOS ONE panel comparison (journals.plos.org); panel quality shifts over time, so treat this as a baseline, not a current read. - Fraudulent or unusable responses commonly run 15-30% on nonprobability online panels, and AI-written open text now routinely defeats completion-time and attention-check filters built for a pre-LLM world (NORC, norc.org; CloudResearch, cloudresearch.com). - None of this substitutes for randomizing the choice itself. A respondent who passes every check is still just telling you what they say they'd do. ## What actually makes survey responses clean? Response-quality hygiene is a well-tested set of tactics, and it works on the problem it's built for: separating attentive humans from bots, speeders, and satisficers. Length matters. The standard advice caps most instruments at 7-10 minutes to reduce abandonment on longer surveys. Matrix and grid questions get cut or broken up because they invite straightlining, where a respondent picks the same column down an entire block without reading each item. Item order gets randomized to control for position effects. And the detection fight has moved past gotcha attention checks: Qualtrics' 2024-25 research found that asking respondents to commit to thoughtful answers up front cut data-quality issues by more than half compared with a control group using traditional attention-check traps (qualtrics.com). A number without the name of who ran the study attached is marketing copy. That's Qualtrics-run research evaluating its own product, not an independent replication, so read the magnitude as directional. Panel choice compounds all of it. A peer-reviewed comparison across MTurk, Prolific, CloudResearch, Qualtrics panels, and SONA student samples, published in PLOS ONE in 2023, found Prolific and CloudResearch respondents pass attention checks, follow instructions, and produce meaningful open-text answers at meaningfully higher rates than the alternatives (journals.plos.org). Naming a study's expiration date is what lets a buyer decide when to re-check it. Panel quality shifts over time, and that comparison predates the AI-generated response problem covered next, so treat it as a baseline on panel selection rather than a current read on bot detection. If the question is "how do I stop garbage responses from entering my dataset," this is the current best practice, and it's worth doing regardless of what comes next. ## Is attention-check vetting still working? Not as well as it used to, and the gap is widening. Attention checks and completion-time filters were built for a pre-LLM world, where bad responses looked like gibberish or random clicking. CloudResearch argues that AI-generated survey responses now read as coherent, contextually appropriate, and human, which means the old filters miss exactly the thing they were designed to catch (cloudresearch.com). NORC's literature review puts fraudulent or unusable responses in nonprobability online panels at roughly 15-30%, depending on panel composition and incentive structure (norc.org). The center of the fight has shifted from "catch the speeder" to "catch the bot that reads like a person," and that fight is not settled. But even a fully solved version of that fight, every bot caught, every satisficer filtered, every respondent vetted and committed, only gets you a clean set of stated opinions. It says nothing about whether those opinions predict behavior. ## Why doesn't a clean dataset answer why people choose? Because hygiene operates on the respondent, and causal identification requires a manipulation in the design itself. A respondent who passes every attention check, every commitment pledge, and every speed filter is still answering a question with no randomized condition attached to it: no control group, no varied price or message or feature set assigned at random, no way to separate what someone says moved them from what actually would. That's the say-do gap, and no amount of respondent screening closes it, because the screening never touches the part of the design that would let you isolate a driver of behavior from a stated preference. [figure: A branching decision path. Both branches start with screening and cleaning respondents. One branch asks a stated-preference question and ends in a self-report with no causal content. The other branch randomizes an attribute before the choice and ends in a causal effect estimate with a confidence interval on that effect within the simulated population.] ## What does a randomized discrete-choice experiment add? It adds the manipulation itself. Discrete choice experiments (DCE), Mixed Logit, and ICLV are estimators, statistical models for analyzing choice data, not causal methods on their own. The causal identification comes from randomly assigning which price, message, or attribute bundle each respondent sees before they choose, the same logic as an A/B test applied to preference structure. A standard multinomial logit also carries the IIA assumption (independence of irrelevant alternatives), which can distort substitution patterns when alternatives aren't truly independent; Mixed Logit relaxes that assumption, and ICLV adds a layer for latent constructs like trust or risk aversion that a single choice question can't observe directly. None of the three methods manufactures causality by itself. What makes an effect causal is that the treatment was randomized before the outcome was observed. ## How is causal accuracy actually verified? By checking whether a simulated study reproduces the direction and outcome of a real human study, and reporting how often that holds. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, described in the causal fidelity paper. An accuracy figure with no boundary stated around it is marketing copy. That number is a validation-set result, not a guarantee for a new, unseen market, and it comes with a real limitation worth stating plainly: some published studies used for validation could sit inside a model's training data, which is why the replication protocol exists as a check rather than a claim that the problem is solved. Method-by-method and study-by-study results are public on the leaderboard, so a buyer can see where replication holds and where it doesn't rather than taking a single aggregate figure on faith. More on how the discrete-choice and Mixed Logit estimators fit into the broader validation approach is covered in methods and validation. ## What should a senior buyer decide this quarter? Keep the hygiene. Cap survey length, cut the matrix questions, randomize order, use commitment requests over gotcha attention checks, and pick a panel with a track record like Prolific or CloudResearch over MTurk. None of that is wasted effort, and skipping it will corrupt any downstream analysis, causal or not. But treat it as necessary and not sufficient: before committing budget to a stated-preference study, ask whether the design randomizes the actual variable in question, price, message, feature, or whether it only asks people to rate or rank options they were never randomly assigned to see. If the answer is the latter, the cleanest dataset in the world still won't tell you which action moves the outcome, only what respondents say about it. A concrete next step: take your next planned survey and check whether any variable in it is randomly assigned across respondents before they answer. If none is, that's the gap between a clean dataset and a causal one, and it's the first thing to fix before the next fielding. For a closer look at how a randomized, discrete-choice design would apply to a specific decision, the team is a reasonable place to start that conversation. --- # Classifying Types of Conjoint Analysis URL: https://subconscious.ai/blog/methods-and-validation/classifying-types-conjoint-analysis Published: 2026-08-27 --- Title: Classifying Types of Conjoint Analysis: Why Format Isn't the Whole Decision --- # Survey scripting best practices in market research URL: https://subconscious.ai/blog/methods-and-validation/survey-scripting-best-practices-market-research Published: 2026-08-27 CAVEMAN MODE ACTIVE Fixed problems 1, 2, 3, 4, 5, 6, 7, 8. Cut unsourced completion/phone claims (not just numbers), cut Forsta vendor sentence, fixed Pew split-sample description, de-duplicated the PMC 89/52 finding across bullet and body, split the run-on, rewrote the opener to lead with the action decision, added the health-economics limitation into the figure caption itself. Everything else untouched. --- # Finally, a Brand Tracker with humble price tag URL: https://subconscious.ai/blog/implementation-and-procurement/finally-brand-tracker-humble-price-tag Published: 2026-08-27 --- A senior brand or insights lead evaluating brand trackers this quarter is not really choosing a price tier. They are choosing whether, six months from now, anyone on the team can say why a metric moved. A subscription tracker like Tracksuit or Latana costs a fraction of what Kantar, Ipsos, or Nielsen charge, but it runs the same underlying survey instrument those enterprise programs run, so it inherits the same blind spot: it can show that awareness moved, not why it moved. A humble price tag lowers the invoice. It does not buy attribution. - Enterprise brand trackers from Kantar, Ipsos, and Nielsen run $100K to $500K+ a year for multi-market programs; survey-based trackers more broadly run $25K to $150K a year (userintuition.ai). - Subscription challengers like Tracksuit ($99 to $299/month) and Latana ($1,000 to $3,000/month) cut the price by 10x to 100x, but the measurement underneath is still a repeated cross-sectional survey, not an experiment. - A cheaper subscription buys the same measurement uncertainty more frequently. It does not tell you whether a metric move came from your campaign, a competitor's stumble, seasonality, or sampling noise. - Attributing a metric move to a specific decision, such as a price change or a new message, requires a randomized experiment analyzed with discrete choice models, not a cheaper wave of the same instrument. ## How much does a brand tracker cost in 2026, and why is the price dropping? Enterprise programs from Kantar, Ipsos, and Nielsen have historically run $100K to $500K or more per year for quarterly or monthly multi-market waves, with survey-based trackers more broadly landing in the $25K to $150K range (userintuition.ai). That pricing put continuous brand health measurement out of reach for most mid-market teams, who could afford a wave once or twice a year at best. Since 2023, a wave of subscription trackers has repackaged the same core metrics, awareness, consideration, NPS, brand associations, into always-on dashboards at a fraction of the cost. Tracksuit prices its Essential, Advanced, and Pro tiers at $99, $199, and $299 per month (userintuition.ai), pitched explicitly as a fraction of the $1M+ a year enterprise brands have typically spent (Tracksuit). Latana sits between the two tiers, at $1,000 to $3,000 a month for enterprise configurations, with mid-tier annual contracts reported in the $20K to $50K range (g2.com). The buyer's procurement choice now looks real: a six-figure agency program, or a four-figure SaaS subscription promising comparable metrics at higher frequency. ## Is a cheaper brand tracker measuring something different than an expensive one? No. Both run the same repeated cross-sectional survey: ask a sample of respondents what they think or would do, plot the answers over time. Cutting the price from $500K a year to $299 a month changes the cadence and the invoice, not the instrument. A move in the resulting line, whether it comes from a $500K Kantar wave or a $99 Tracksuit dashboard, is still an uncontrolled correlation. Nobody ran an experiment; nobody controlled for what else changed that quarter. That matters because stated preference reliably diverges from actual behavior, a gap researchers call the say-do gap: what a respondent says they would do on a survey is not what they do when a real price or a real switching cost is on the table. If the underlying survey question already carries that kind of bias, running it more often at a lower price does not fix it. It just produces the same uncertainty on a tighter loop. ## What actually explains a metric move? Usually more than one thing, which is exactly why a single tracked line cannot answer the question on its own. A brand awareness score can move because of a campaign, a competitor's product recall, a seasonal pattern, or plain sampling noise in that wave's respondent panel, and a repeated survey has no mechanism to tell those apart. It reports the number. It does not report the cause. [figure: A diagram showing four separate candidate causes, a campaign, a competitor event, seasonality, and sampling noise, all pointing into a single observed metric-move box, illustrating that the observed number cannot be traced back to one cause without an experiment.] Answering "what moved it" requires isolating one variable and holding the rest constant, which is what a randomized experiment does and a tracking wave does not. ## What would it take to attribute a metric move to a cause? A randomized experiment on the decision itself, analyzed with a discrete choice model, rather than another wave of the same observational survey. In a randomized experiment, respondents (or a simulated population standing in for them) are exposed to controlled variations, say, two price points, two messages, two feature bundles, and their choices are estimated using methods like McFadden discrete choice, Mixed Logit, and ICLV. Naming this distinction is what lets a buyer check the method. These are estimators, not causal methods in themselves; the causal identification comes from the randomized manipulation in the experiment design, not from the estimator. Mixed Logit is worth naming specifically because it relaxes the independence-from-irrelevant-alternatives assumption a flat logit model carries, which matters whenever the real question is which competitor a switched customer would move to next. That structure is what lets a team say a metric moved because of the price change, not merely that it moved. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. That number is a validation-set result, not a guarantee for a new, unseen market, and it comes with a real limitation worth stating plainly: some of the published studies used for validation could have been present in a model's training data, which is exactly why the replication protocol exists, rather than a claim that the problem doesn't exist. The leaderboard tracks this in public rather than as a vendor claim. The misses sit on the public leaderboard next to the hits, so this interval's scope is stated directly. A confidence interval produced by one of these experiments covers the estimated effect within the simulated population that was run. It does not bound the real market unconditionally. For a deeper look at how the discrete choice methods work, see the methods and validation hub. ## Enterprise agency, subscription dashboard, or randomized experiment ## The real cost of a humble price tag The subscription price is the visible line item. The real cost sits one step downstream: the decisions a team makes on a metric move nobody can attribute to a cause. Take a team that sees consideration dip after a rebrand. They reverse the rebrand. The actual driver was a competitor's promotion that quarter. That team just spent real budget acting on noise a $299 dashboard reported with total confidence. A lower price tag doesn't change who pays for that mistake. It just makes the wrong call cheaper to generate and easier to justify, since "the dashboard moved" reads as evidence even when it isn't. That is the actual comparison a senior buyer should be running, not agency price against subscription price, but subscription price against the cost of a decision made on an unattributed move. For a walk-through of how this plays out in practice, see the comparisons hub. Next step: before the next renewal decision, take the last three metric moves your current tracker reported and try to write down, for each, the specific decision or event that caused it, without guessing. If none of the three can be attributed with any confidence, that is the answer to whether the tracker's price tag was ever the real cost. When you want to test a specific decision, like a price change, a message, or a feature bundle, against a randomized experiment instead of another wave, the team at /meet can walk through what that would look like for your market. --- # Comparing four methods of conjoint for pricing research URL: https://subconscious.ai/blog/comparisons/comparing-four-methods-conjoint-pricing-research Published: 2026-08-27 A pricing lead picking between CBC, ACBC, MBC, and volumetric conjoint is choosing a survey format, not a guarantee of accuracy. All four ask respondents to trade off features and price in a hypothetical scenario, and none of them checks whether that trade-off matches what people do when real money is on the line. The decision that actually determines pricing accuracy happens after the questionnaire is built: whether the resulting price sensitivity gets validated against a real behavioral outcome. - CBC (Choice-Based Conjoint): standard discrete choice tasks with price as one attribute among several. Fastest to field, easiest to analyze, and the default starting point for most pricing studies. - ACBC (Adaptive CBC): adds a build-your-own screening stage before an adaptive choice tournament, run through Sawtooth's Lighthouse Studio. Surveys typically take two to three times longer than standard CBC (Sawtooth Software). - MBC (Menu-Based Conjoint): respondents assemble their own bundle or subscription tier item by item, the right fit for configurable offers with add-ons (Sawtooth Software). - Volumetric conjoint: asks how many units a respondent would buy, not just which option they'd pick, which matters when demand volume drives the pricing decision more than choice share. - All four are stated-preference formats. The format you choose changes what respondents see; it does not, by itself, tell you whether the trade-off they report predicts what they'd actually pay. ## What do CBC, ACBC, MBC, and volumetric conjoint actually measure? Each method estimates the same underlying thing, a respondent's relative preference for price versus other attributes, through a different questionnaire structure. ## Which conjoint method should a pricing team pick? Pick based on the product structure and attribute count, not on an expectation that one method is more accurate than the others. A short, fixed feature set with a single price point points to CBC. A long attribute list, or a pricing study where precision on part-worths matters more than fielding speed, points to ACBC, with the tradeoff that it costs two to three times the interview length (Sawtooth Software). A subscription or configurable bundle points to MBC. A category where customers buy varying quantities, not a single unit, points to volumetric. None of these choices changes whether the resulting price sensitivity holds up against real behavior. ## Why do CBC and ACBC produce almost the same answer? Because they're estimating the same utility function with different collection mechanics, not different models of how people decide. Sawtooth's own comparisons find "very strong similarities" between CBC and ACBC part-worth utilities, with ACBC utilities showing somewhat more precision and price carrying slightly more relative importance than in standard CBC (Sawtooth Software). The same source notes that CBC hit rates gain little beyond 10-15 choice tasks, meaning the extra length ACBC adds mostly buys precision, not a different conclusion. If two methods that differ this much in respondent burden still converge on the same price sensitivity, the choice among them is a design question, not an accuracy lever. ## Does agreement between methods mean the price estimate is accurate? Publishing where two methods agree, and where that agreement stops, is what lets a buyer check the estimate rather than take it on faith. No. Agreement between CBC and ACBC only shows that the methods are consistent with each other; it says nothing about whether either one matches real purchase behavior. A meta-analysis of 28 stated-preference valuation studies found a median hypothetical-to-actual willingness-to-pay ratio of 1.35, meaning people overstate what they'd actually pay by roughly a third in a hypothetical survey setting, before any conjoint-specific design choice enters the picture (ResearchGate). This bias runs in one direction, stated willingness to pay comes in high, and it applies regardless of which of the four conjoint architectures collected the data. [figure: Bar chart showing actual willingness to pay as a baseline of 1.0 and median stated willingness to pay at 1.35, based on a meta-analysis of 28 stated-preference valuation studies.] ## How is trade-off accuracy actually validated? By checking the estimated price sensitivity against a real behavioral outcome, not by choosing among CBC, ACBC, MBC, or volumetric formats. McFadden discrete choice, Mixed Logit, and ICLV are estimators that turn choice data into part-worths and willingness-to-pay figures; they are not causal methods on their own. Causal identification comes from randomizing the price and feature manipulations respondents see, then analyzing the resulting choices with these models. A flat multinomial logit, the base case under standard CBC, imposes the independence-of-irrelevant-alternatives assumption, which can distort preference-share and substitution estimates; Mixed Logit relaxes that assumption at the cost of a more complex model. Subconscious runs randomized experiments on a simulation of the market and checks the result against a human baseline. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, a validation-set result on the studies in the public leaderboard. It is a validation result, not a guarantee for a new market. See the causal fidelity paper. That number also comes with a limit worth stating plainly: some of the human studies used for validation are published, and published studies can sit in a language model's training data; the replication protocol behind the leaderboard is built to address that risk, but it doesn't make the risk disappear. Stating where a result stops applying is what turns it into something a buyer can act on. A confidence interval from a simulated experiment covers the effect within the simulated population it was run on; it does not bound the real market unconditionally. For a longer treatment of what replication testing does and doesn't prove, see the methods and validation hub. ## What are the limits of this approach? A number without its limits is marketing; naming the failure mode is what makes it a working tool instead. None of this replaces the questionnaire-design decision covered above; it sits on top of it. Choosing CBC over ACBC because the study needs a shorter field time is still a legitimate call. What changes is what happens next: instead of treating the resulting price sensitivity as a finished answer, it becomes a hypothesis to check against a held-out human study or a behavioral outcome, with the hypothetical-bias direction named and the confidence interval scoped to the population it was estimated on. A method that produces a precise, internally consistent estimate that has never been checked against real behavior is not more accurate for being precise. Before locking a launch price off any conjoint output, pull the top two price points from the study and re-test them as a randomized experiment against a human holdout rather than trusting the questionnaire result on its own; the comparisons hub has more on how to structure that check. If it's useful to walk through your specific study design, the Subconscious team is available for that conversation. --- # AustralianSuper leads the superannuation market in early 2026 URL: https://subconscious.ai/blog/industries/australiansuper-leads-superannuation-market-early-2026 Published: 2026-08-27 A superannuation fund's chief marketing officer or head of product deciding whether to match AustralianSuper's 2026 brand spend needs to know one thing first: does any of this data prove members actually prefer AustralianSuper, or does it just prove most of them were never asked? As of April 2026, AustralianSuper tops every stage of the funnel Super Review tracks, with 58% unprompted brand awareness, 30% of members considering it, and 19% naming it their primary fund, while Australian Retirement Trust (the merged QSuper and Sunsuper fund) closes the conversion gap behind it (Super Review). A number without its limits is marketing copy. Publishing the limit next to it is what lets a buyer check the claim. None of those three numbers, alone or together, show that a specific attribute like fees, disclosed returns, or service quality is what causes a member to name AustralianSuper over the alternative of simply staying where an employer put them. - AustralianSuper leads by scale and leads every funnel stage tracked by Super Review in April 2026, with ART closing the gap behind it. - Every headline metric behind that lead (awareness, consideration, a fifth straight Roy Morgan trust award) is a stated-preference survey response, not a measured behavioral effect. - Market-wide switching intent fell in 2025 even as NPS improved, which is consistent with inertia, not with active preference forming. - Consolidation is shrinking the field fast: the top 24 funds hold about 96% of industry assets, and APRA's own performance test measures backward-looking benchmark performance, not why members choose a fund. - A fund cannot tell whether it is winning on merit or on default status until it runs a controlled experiment that removes the default and observes what members choose instead. ## What does market leadership look like for AustralianSuper in early 2026? Market leadership right now means scale, incumbency, and top-of-funnel recognition, not a demonstrated causal edge in what members would choose freely. AustralianSuper enters 2026 as Australia's largest superannuation fund by scale. It has now won Roy Morgan's "Most Trusted Superannuation Brand" award for a fifth consecutive year (Roy Morgan), and per Super Review's April 2026 funnel tracking, it outranks every competitor at every stage measured, from unprompted awareness through primary-fund status (Super Review). By every conventional trade-press metric, this is a fund that has won. What those metrics cannot separate is how much of that win comes from members actively choosing AustralianSuper versus members simply never leaving the fund their employer defaulted them into. ## Does AustralianSuper's funnel lead prove members prefer it? Naming a metric's failure mode is what lets a buyer check it before spending against it. No. A funnel lead built on awareness, consideration, and trust scores is a correlation between brand recognition and default-driven inertia, not proof that any specific fund attribute causes members to choose it. Superannuation in Australia is a category where most members are enrolled by an employer's default fund arrangement rather than by comparing options and switching. Investment Trends' 2025 survey of more than 8,000 members found switching intentions declined even as market-level NPS improved from -19% to -4% (Investment Trends). If members are switching less while rating the category more favorably, that pattern is at least as consistent with rising comfort in staying put as it is with rising conviction about any one fund. "Market leader" in this data is a description of who holds the most defaults and the most name recognition. It is not evidence of what members would pick if the default were removed and they had to choose. ## AustralianSuper vs Australian Retirement Trust: what the funnel shows and what it doesn't ## Why one funnel number hides two different causal stories A single "primary fund" percentage can be produced by two entirely different processes, and the survey has no way to tell them apart. One member never compares funds at all; the employer's default enrolls them, and they are counted as a primary-fund holder without ever making an active choice. Another member compares fees, returns, and service, actively decides, and ends up counted the exact same way. Both land in the same 19% figure. That is the difficulty a funnel number cannot resolve on its own, and it's why AustralianSuper's funnel lead can be entirely real as a measurement while telling a buyer almost nothing about which of the two processes is driving it. [figure: A branching diagram showing two separate paths that arrive at the same measured outcome. Top path: employer default enrollment leads to no active comparison, the member never leaves the default, and is counted as a primary-fund holder. Bottom path: the member actively compares funds on fees, returns, and service, then chooses or switches, and is counted the same way.] ## What is APRA's Your Future, Your Super test actually measuring? A regulator's test is only useful to a buyer once its blind spot is stated plainly. It measures backward-looking, risk-agnostic investment performance against a benchmark, not whether members prefer or benefit from a fund's specific attributes. Passing means a fund's returns cleared a bar over the measurement period; it says nothing about fees relative to service, disclosure quality, or engagement design, and nothing about whether trustees are optimizing for member outcomes versus optimizing for the pass/fail line itself. If the regulator's own accountability tool can't isolate what drives good outcomes for members, a brand funnel survey built on recall and self-report certainly can't either. That gap is exactly where causal experimentation is supposed to sit, and right now nothing in the public record fills it. ## Why did switching intent fall as trust scores rose? Most likely because a market with fewer real alternatives and stronger defaults gives members less reason to compare, not because they've concluded any one fund is causally superior. APRA-regulated fund count fell from 107 to 89 in FY23-24, and the top 24 funds, each over $20 billion, now hold roughly 96% of industry assets, with nine funds exceeding $100 billion each (Investor Daily). As the field consolidates, the practical cost of comparison shopping rises and the number of genuinely different options falls. Rising trust and falling switching intent in that environment (Investment Trends) is exactly what a say-do gap looks like: what members report feeling and what they'd actually do if forced to choose can move in different directions, and a stated-preference survey has no way to catch the difference. ## What would a causal test of fund preference look like? It would put a representative sample of members through a randomized experiment where fund attributes, including default status itself, are manipulated one at a time, then analyze the choices with discrete choice models such as McFadden's discrete choice model, Mixed Logit, and ICLV. Mixed Logit in particular lets you model substitution across funds without imposing the independence-of-irrelevant-alternatives assumption, the requirement in a flat logit model that the ratio of choice probabilities between any two funds stays the same regardless of what other funds are in the choice set, an assumption a consolidating market with one dominant scale leader is likely to violate. Run correctly, that design produces a confidence interval around each attribute's effect, one that describes the simulated population under test and doesn't stand in as a claim about the entire live market. The misses sit on the public leaderboard next to the hits, which is what makes the number checkable. Subconscious's best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. It is a validation result, not a guarantee for a market that hasn't been tested yet. Published studies can sit in a model's training data, so the protocol is built specifically to check for that contamination rather than assume it away. The leaderboard publishes those replication results study by study, and the methods and validation hub covers how the estimators and the experimental designs fit together. ## The next move for a fund's product or marketing lead Before committing budget to chase AustralianSuper's funnel numbers, check whether a comparable study already exists on the leaderboard; if it doesn't, the next real step is designing a randomized experiment that holds default status constant and tests the specific attribute (fee structure, disclosed return, engagement touchpoint) you're actually deciding on. That tells you which lever moves member choice, not just which fund gets recognized. If you want help scoping that design for your own member base, the team can walk through it. --- # Should I include adcepts in preference share simulations? URL: https://subconscious.ai/blog/methods-and-validation/should-i-include-adcepts-preference-share-simulations Published: 2026-08-27 A senior buyer weighing whether to greenlight a new product does not need a preference-share number that has been massaged by an ad execution nobody randomized. Leave adcepts out of the simulation. If the message itself carries value, test it on its own: randomize the creative as an attribute in a dedicated discrete choice experiment (DCE) and read its causal effect on choice directly, instead of bolting one unblinded execution onto a product conjoint and reporting the resulting lift as a forecast. - Don't fold an adcept into a product preference-share simulation. The lift it produces mixes concept value, execution quality, and pre/post order effects into one number you can't decompose (Wikipedia: Adcept). - Adcepts descend from Brand-Price Trade-Off methodology and Nielsen's BASES system, which corrects the resulting bias with proprietary factors, not a causal estimate (BASES overview). - A conjoint yields causal, component-specific effects only when every attribute, including message content, is randomized across respondents, not shown once to everyone (Hainmueller, Hopkins, and Yamamoto, 2014). - If message value matters to the launch decision, run it as its own randomized DCE with creative as an attribute, and report a causal effect with a confidence interval, not a before/after percentage. - The stated-intent overstatement documented by Jamieson and Bass in 1989 doesn't disappear because an adcept is inside the survey; BASES corrects it after the fact, a dedicated DCE avoids manufacturing it in the first place. ## What is an adcept, and why does it break a preference-share simulation? An adcept is exactly what the name suggests: a rough, halfway-finished ad, a "concept" version of "advertising," typically a picture board, a short video, or a piece of copy shown mid-survey to represent a message before it's polished into a real campaign (Wikipedia: Adcept). Researcher Richard Woods, cited on that same page, flags the core problem directly: when an adcept is too polished, respondents stop judging the underlying concept and start judging the specific execution in front of them, the picture, the actor, the tagline, none of which is the thing the buyer actually wants an answer about. That's the confound at the heart of this decision. A product conjoint is built to isolate the causal effect of each product attribute by randomizing them independently across respondents. Dropping one unblinded ad execution into the middle of that design, then re-running the choice tasks, doesn't add an attribute to the experiment. It adds an uncontrolled event that every respondent experiences identically, which is the opposite of what makes a conjoint identify anything at all. ## Where adcepts came from: Brand-Price Trade-Off and Nielsen's BASES Adcepts aren't a recent shortcut, they're inherited from Brand-Price Trade-Off (BPTO) methodology. When a conjoint tests an unfamiliar new product, respondents tend to penalize it simply because they don't recognize it yet, so vendors insert an ad execution mid-survey and compare "before adcept" share to "after adcept" share, treating the difference as the lift advertising will eventually buy. Nielsen's BASES system, the category leader since Burke Marketing built it in 1977 and Nielsen acquired it in 1998, formalizes this inside its Awareness-Trial-Availability-Repeat (ATAR) framework and claims sales forecasts within roughly ±10% of actual results across tens of thousands of launches (BASES overview). That accuracy claim is real, but it's earned by applying proprietary correction factors to stated purchase intent after the survey runs, not by a randomized manipulation that isolates the ad's own effect. The correction is doing the work the study design should have done. ## Why an unblinded exposure isn't a randomized experiment Hainmueller, Hopkins, and Yamamoto's 2014 paper in Political Analysis is the methodological line in the sand here: conjoint analysis produces causal, component-specific effects only when every attribute in the design, including message content, is fully randomized across respondents. Survey experiments that expose everyone to the same single treatment, rather than randomizing it, are limited to unattributable "catchall" effects, meaning you can observe that something changed, but you cannot say which part of what respondents saw caused it. An adcept shown once, mid-survey, to every respondent is precisely that catchall case. The "after adcept" lift BASES reports isn't wrong because the number is fabricated, it's underdetermined: it can't distinguish "this message concept has value" from "this particular picture board, video edit, or copy draft happened to test well that day." ## The stated-intent bias adcepts can't escape Jamieson and Bass documented in the Journal of Marketing Research in 1989 that respondents systematically overstate their purchase intent in surveys, which is exactly why BASES needs its correction layer in the first place. An adcept doesn't fix that bias, it inherits it: respondents are still stating intent in a hypothetical, unincentivized moment, now with the added confound of a single unblinded ad execution layered on top. BASES patches the result with proprietary adjustment factors calibrated against Nielsen's own historical launch database. That's a defensible business practice for a firm with tens of thousands of launches to calibrate against. It is not a causal estimate a buyer can inspect, replicate, or attribute to a specific message element. [figure: A decision path with two branches. One branch shows an adcept added mid-survey to a product conjoint, producing an unattributable preference-share lift. The other branch shows creative randomized as its own attribute in a dedicated experiment, producing a causal effect estimate on choice with a confidence interval.] ## Should I ever test messaging causally? Yes, but as its own experiment, not as an add-on to a product simulation. Build a dedicated DCE where the message or creative execution is itself a randomized attribute, alongside whatever product features you'd normally test, and let respondents choose across combinations where creative varies independently. That gives you a causal effect of the message on choice, reported with a confidence interval, rather than a single before/after percentage point. The confidence interval describes the effect within the simulated population you tested, it doesn't bound demand in the real market unconditionally. And if the study reports preference share or substitution patterns from a standard multinomial logit, name the assumption underneath it: independence of irrelevant alternatives (IIA), which forces substitution to be proportional across every alternative regardless of how similar two options actually are. Mixed Logit relaxes that assumption by allowing preferences to vary across respondents, which matters when creative or product options are close substitutes for each other. ## Adcept bolt-on vs. a dedicated randomized creative DCE ## How is this validated, and what are the limits? The way to trust any of this is to check it against real human behavior on studies the model wasn't built to answer. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, in the causal fidelity paper. That's a validation result, not a guarantee for a market you haven't tested yet, and it doesn't erase a structural risk: some published human studies used for validation may already sit in a model's training data, which is exactly why the replication protocol treats reproduction against a holdout as the check, rather than treating any single match as proof. The current standings across methods and study types are public on the leaderboard. None of this is estimator-specific: McFadden discrete choice, Mixed Logit, and ICLV are estimators applied to a design, not causal methods in themselves. The causal identification comes from randomizing the attribute, including creative, inside the experiment. More on how that design discipline gets applied across study types is in the methods and validation collection. If you're deciding right now whether to greenlight a launch simulation with an adcept baked in, the concrete next step is to split the study: run your product conjoint clean, with no ad execution inside it, then design a second, separate DCE where creative is a randomized attribute and read off its causal effect on choice as its own number. If you want a second set of eyes on that split before you run it, meet the team. --- # 5 Best Practices in Concept Testing: Launch With Confidence URL: https://subconscious.ai/blog/use-cases/5-best-practices-concept-testing-launch-confidence Published: 2026-08-27 Concept testing exists to answer one question before money moves: will this specific product decision change what people actually buy, and by how much. The best practice isn't a smarter survey question, a bigger panel, or a faster AI simulation. It's replacing "would you buy this" with a randomized discrete-choice experiment that isolates which concept attribute moves purchase intent, with a confidence interval attached to the answer. A VP of product deciding whether to greenlight a launch needs that causal answer, not a top-box score with a margin of error nobody reports. - The core practice: run a randomized discrete-choice experiment, not a rating-scale survey, so you can attribute intent shifts to a specific attribute rather than to sampling noise or question order. - The metric to drop: top-box purchase intent, because roughly half of "definitely buy" respondents and only 10-20% of "probably buy" respondents actually purchase (Analysis Group, Befurt & Silk). - The metric to add: a confidence interval on the effect of each concept attribute, scoped to the population you simulated or sampled, not the whole market. - The speed tradeoff: AI synthetic panels solve the 2-3 week fielding cycle but substitute one unvalidated simulation for one uncontrolled survey unless the vendor publishes replication accuracy against real human studies. - The launch gate: don't ship on a single concept score. Ship on a causal estimate of which attribute change moved intent, tested against a holdout. ## Why does stated purchase intent fail as a launch signal? Stated purchase intent fails because the scale you're scoring collapses a continuum of hesitation into a binary "will buy" signal that doesn't map to behavior. Quirks has flagged top-box scoring as marketing research's "top mistake," arguing that discarding the full scale in favor of top-2-box percentages throws away statistical power and reliability that the underlying continuum actually carries (Quirks). Analysis Group's Befurt and Silk quantified the gap directly: respondents who pick the top rating on a purchase-intent scale have roughly a 50% chance of actually buying, and respondents one notch down have only a 10-20% chance (Analysis Group). A concept that scores well on top-box is not a concept validated to sell. It's a concept that made people comfortable saying yes to a hypothetical. This matters because the stakes are asymmetric. An often-cited estimate, attributed to Clayton Christensen and reported by Inc., puts new-product failure at roughly 95 percent of the 30,000-plus products launched each year; the underlying methodology isn't published, so treat the figure as a directional stake, not a precise rate (Inc.). If the pre-launch gate is a survey metric with a documented say-do gap, that gate is not doing the job it was built for. ## What the traditional five-practice checklist gets right, and where it stops The standard concept-testing checklist is not wrong, it's incomplete. Define objectives, size the sample at 100-300 respondents per concept, choose monadic versus sequential monadic exposure, test price and packaging alongside the core idea, and iterate early. Each of these is a real improvement to survey design. None of them touches the underlying instrument. Sequential monadic designs, in particular, introduce order and fatigue effects that no amount of sample size fixes. A respondent's third exposure carries the residue of the first two. A bigger N gives you a tighter confidence interval around a biased number. That's the trap: better survey execution produces more precise estimates of the wrong thing. Stated intent, however cleanly measured, is a correlational proxy. It tells you what people say they'd do, not what changes their behavior when a specific attribute moves. ## Survey, synthetic panel, or randomized experiment: which fits your launch decision? ## Are AI synthetic panels a shortcut to validity? No. They're a shortcut to speed. Platforms like Aaru, Synthetic Users, Outset, and Attest compress the traditional 2-3 week fielding cycle into minutes by simulating respondent reactions instead of collecting them. Published performance against human benchmarks varies by vendor, and few name the protocol behind the number. Agreement with a human panel on a stated-preference question is not a causal estimate. It's a faster version of the same instrument. If the underlying question is still "would you buy this," a synthetic panel returns a faster, cheaper version of a number that already has a known say-do gap. Speed is being solved faster than validity is. Before trusting any panel, ask the vendor for a published replication rate against real human studies and the failure mode when the market being tested wasn't in the training data. ## The practice that isolates cause: randomized discrete-choice experiments This is the one practice the standard checklists skip. Instead of asking respondents to rate a fixed concept, a discrete-choice experiment randomly varies concept attributes (price, feature set, positioning) across choice tasks and forces a trade-off, then estimates which attribute change moved the choice. The randomization is what earns the word "causal." McFadden discrete choice, Mixed Logit, and ICLV are the estimators used to recover preference weights from that data; they are not, by themselves, causal methods. The causal claim comes from the randomized manipulation in the experiment design, not from the choice model. Mixed Logit also relaxes the independence-of-irrelevant-alternatives assumption that a flat multinomial logit imposes, which matters for any preference-share or substitution question. A standard logit will predict that a new concept variant steals share proportionally from every existing option, which is rarely how real substitution works. [figure: A four-step chain showing a concept attribute being randomized, a respondent choosing between options, a discrete choice model estimating the attribute's effect, and that effect being reported with a confidence interval.] ## How much replication accuracy is enough to trust a simulated concept test? Enough is a published number tied to a public protocol, not a vendor's internal claim. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. Full results are published in the causal fidelity paper. That number is a validation-set result, not a guarantee for a market you haven't tested yet, and it carries a real limitation worth stating plainly: some of the human studies used for validation were published before the underlying models were trained, so the replication protocol has to account for the possibility that a model has seen the original result rather than independently reproducing it. The full method comparisons, including how this stacks against other simulation approaches, are on the leaderboard and in the methods and validation hub. Ask any panel vendor claiming "80-95% agreement with human benchmarks" the same question: agreement on what protocol, against which holdout, published where. ## Where willingness-to-pay estimates still need a caveat Any willingness-to-pay number pulled from a concept test, stated-preference survey or discrete-choice experiment alike, runs high relative to what people actually pay unless the design is incentive-aligned, meaning respondents face a real consequence for their choice rather than a hypothetical one. This is documented hypothetical bias, and the direction is consistent: stated WTP overstates real WTP (Wiley Health Economics). A discrete-choice experiment doesn't eliminate this bias by default. It gives you a defensible way to isolate which attribute drives the WTP shift, which is a different and more useful claim than "respondents said they'd pay $12." ## A launch-ready concept testing checklist Before a concept moves from test to launch, a VP of product at the launch gate should be able to answer: - What attribute was randomized, and what was held constant? - What is the confidence interval on the attribute's effect, and what population does it cover? - Was the willingness-to-pay estimate incentive-aligned, or does it carry the standard upward hypothetical bias? - If a synthetic panel was used, what replication rate is published against real human studies? - Does the preference-share estimate rely on a flat logit's IIA assumption, or does it use Mixed Logit to relax it? A concept that clears every item on the standard five-practice list but can't answer these five questions has been surveyed, not tested. See how this plays out in practice in the comparisons hub and in published case studies. Next step: take the concept variant you're least sure about and design one randomized choice task around its single most important attribute, before you run the next full survey wave. If you want a second read on the design, talk to us. --- # AustralianSuper leads the funnel, CommBank leads on customer support URL: https://subconscious.ai/blog/industries/australiansuper-leads-funnel-commbank-leads-customer-support Published: 2026-08-27 A chief marketing officer at an Australian super fund deciding where next year's budget goes, brand spend or call-center headcount, is staring at two 2026 datasets that don't agree. AustralianSuper leads every stage of the funnel by a wide margin: awareness at 58%, consideration at 30% and primary-fund status at 19%, across four brand-tracker waves from December 2025 to April 2026 (Super Review). A separate mystery-shopper study, fielded in July 2026, found AustralianSuper answered only about 10% of test calls and was the one major fund excluded from the industry's call-center rankings, while smaller funds including CommBank Super are credited with a real service-quality edge (Financial Newswire). Neither survey answers the question a CMO actually needs answered: what does a bad call do to the odds a member switches. That takes a controlled experiment, not two scoreboards read side by side. - AustralianSuper tops every funnel stage Super Review tracked, with awareness at 58%, consideration at 30% and primary-fund status at 19% (December 2025 to April 2026). - AustralianSuper answered roughly 10% of mystery-shopper calls in a July 2026 study and was dropped from Super Consumers Australia's call-center rankings entirely (Financial Newswire). - CommBank Super and Australian Retirement Trust are credited with real service-quality and conversion advantages despite far lower brand awareness than AustralianSuper. - Naming this failure mode is what lets a marketing lead check the comparison before repeating it. Stated preference and observed call-handling behavior are being reported as if they measure the same thing. They don't, and the two datasets weren't even collected in the same period. - Neither dataset is a causal test of switching. That requires a randomized experiment analyzed with discrete choice models. ## Does AustralianSuper's funnel dominance mean its members are safe from switching? No. Funnel dominance measures what people recall and say they'd consider, not what they do when their fund fails them on a call. AustralianSuper's consideration score actually moved during the tracking window, from 24% in December to a peak of 31% in February, easing back to 30% by April (Super Review). Rest and Hostplus trail well behind at 48% aided awareness each, and on paper AustralianSuper looks unassailable at every stage from recall through primary-fund status. A funnel chart like that is a snapshot of stated preference. It tells you what a survey respondent said in a phone or online poll, not what a member does the next time they need help and can't get through. Those are different behaviors, and only one of them puts a dollar figure on retention risk. ## Does CommBank Super actually outperform AustralianSuper on customer support? A number posted without its limits is marketing copy, and this one carries its limit plainly. Only partially, and the public data can't fully quantify it. On the one metric in this story that reflects a real interaction rather than a stated opinion, whether the fund picked up the phone, Super Consumers Australia and CSBA ran 1,000 mystery-shopper calls across 20 funds in July 2026 and found an industry-average experience score of 49.9%, with no fund clearing 55% and none reaching SCA's own 80% benchmark (Super Consumers Australia). AustralianSuper did worse than that average by a wide margin: it answered only about 10% of calls attempted, meaning close to 90% went unanswered, so few that SCA excluded it from the call-center rankings rather than publish a number that would have anchored the bottom of the table (Financial Newswire). Coverage of the same dataset credits CommBank Super and Australian Retirement Trust with what Financial Newswire called "genuine competitive advantages" in service quality and conversion efficiency, despite awareness numbers far below AustralianSuper's, but neither the SCA release nor that coverage publishes a fund-by-fund score for CommBank Super, so the size of that advantage isn't public. What's confirmed is narrower and still decisive for a CMO: AustralianSuper's own answer rate was bad enough to disqualify it from comparison entirely. ## One fund, two scoreboards Holding four numbers in your head at once, on two different scales, from two different studies, is where this story gets misread. The figure below puts them on one axis. [figure: Bar chart showing AustralianSuper at 58 percent aided awareness, 30 percent consideration, 19 percent primary-fund status, and roughly 10 percent of mystery-shopper calls answered.] ## The say-do gap: why survey answers and behavior diverge Two different kinds of measurement are being collected on AustralianSuper right now, and coverage often treats them as if they're checking the same thing. The brand tracker asks people what they think and feel: aided awareness, consideration, primary-fund status. It's a stated-preference instrument, run across four waves from December 2025 to April 2026, mostly independent of any specific service failure. AustralianSuper's consideration score actually rose during that window, from 24% in December to 31% in February, before settling at 30% in April (Super Review). The mystery-shopper study measures something else entirely: what happens when someone actually tries to call. AustralianSuper answered about one in ten of the 1,000 test calls placed across the industry in July 2026 (Financial Newswire). Those two facts get placed side by side often, stated preference holding up, call-handling near zero, as if the gap between them proves something about how resilient the brand is, or isn't. It doesn't. A brand survey and a mystery-shopper call are different instruments, run in different months, on different sample sets. Neither is built to isolate the effect of one bad call on a member's later choice. Reading a causal story into the space between two separately-collected numbers is the mistake this article is arguing against. ## Where does the causal chain actually break? Right where the funnel stops. The brand-tracker stages, awareness, consideration, primary-fund status, read like a causal sequence because they're drawn as one, arrow after arrow toward eventual retention. They aren't. Each stage is a self-reported snapshot at a point in time, not a measured step in a chain of cause and effect, and the step that would actually determine switching, a member's response to a specific service failure, sits outside the funnel with no instrument pointed at it. If a member does leave AustralianSuper, the question of where they land matters for anyone modeling the market. A flat multinomial logit would assume independence of irrelevant alternatives, that a member's relative odds of choosing CommBank Super over Rest don't shift depending on which other funds are in the choice set. In a market with more than a dozen credible super funds, that assumption rarely holds, which is one reason Mixed Logit and ICLV, both of which let preferences vary across simulated members, are the more defensible estimators for a substitution question like this one. ## Why DCE, Mixed Logit, and ICLV aren't causal by themselves A benchmark number posted without its limits is just marketing copy. Discrete choice models estimate what respondents chose; they don't, on their own, tell you why. The causal identification comes from the randomized manipulation built into the experiment design, not from the estimator. A study built to answer this question would randomize a service attribute, say, call wait time or hold-abandonment probability, across a discrete choice experiment on a simulated population, then measure the causal effect of that attribute on stated switching intent and modeled primary-fund status. Mixed Logit and ICLV are the estimators suited to pulling that effect out of the choice data once the randomization has done the identifying work. On its best configuration, the method behind these estimators reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). The misses sit on the same public leaderboard as the hits, so the gap gets published instead of hidden. That's a validation-set result, not a finding about Australian super funds specifically: this market hasn't been tested yet, and because some published studies can sit inside a model's training data, the replication protocol behind that number exists specifically to check for that contamination rather than assume it away. Method performance across study types is public at the leaderboard; the underlying validation approach is documented in /blog/methods-and-validation. ## What should a super fund's marketing or CX lead do this quarter? Stop comparing the funnel number to the call-center number and start testing the link between them directly. That means designing a discrete choice experiment that includes a service-quality attribute, wait time, abandonment rate, resolution on first call, as a randomized variable, then reading the modeled effect on switching probability and primary-fund status against a holdout sample before trusting it. It also means treating AustralianSuper's funnel lead and its call-center exclusion as two separate facts until a causal test says otherwise, not as evidence that answers each other. A concrete next step: pull the attribute list from a published discrete choice study design in your category and add a randomized service-quality attribute to it before your next brand-tracker wave runs, so the two data sources stop talking past each other. If you want a second set of eyes on that design, book time with the team. --- # Willingness to Pay: What It Is and How to Measure It URL: https://subconscious.ai/blog/methods-and-validation/willingness-pay-measure Published: 2026-08-27 A pricing lead deciding what to charge for a new product needs an answer to one question: how much will customers actually pay, not what they say they would pay on a survey. Willingness to pay is the price at which a customer's choice tips from buying to walking away, a causal effect of price on behavior, not a number people report on a form. Measuring it well means running an experiment that manipulates price and observes the resulting choice, then checking that estimate against real behavior, because stated answers alone tend to overstate WTP by roughly three times on average, according to decades of hypothetical-bias research, including List and Gallet's meta-analysis of 29 payment experiments. - Willingness to pay is the price point where a customer's choice shifts from buying to not buying, a causal effect of price on behavior, not a self-reported number. - The three common measurement tools are Van Westendorp's Price Sensitivity Meter, Gabor-Granger sequential pricing, and conjoint or discrete choice analysis. - Stated WTP figures run high by a well-documented margin, roughly three times actual willingness to pay on average (List and Gallet), unless the design is validated against real payment behavior. - Conjoint analysis reduces hypothetical bias by forcing trade-offs, but it does not eliminate it without a validation step. - A trustworthy WTP number comes with a stated accuracy rate against real behavior and a confidence interval, not a point estimate alone. ## What does willingness to pay actually measure? Willingness to pay measures a causal effect: how much price has to move before a customer's choice moves with it. That is different from asking someone to name a number. A stated dollar figure is a survey answer. A WTP estimate, done right, comes from an experiment where price is randomized or systematically varied and the outcome is an actual choice, then the relationship between the two is estimated statistically. The distinction matters because the two produce different numbers, and the gap between them is large enough to sink a pricing decision. ## How do you measure willingness to pay? Three tools dominate pricing research, and they differ mainly in how directly they ask for a price. Van Westendorp's Price Sensitivity Meter uses four contextual questions (too cheap, a bargain, getting expensive, too expensive) instead of one direct ask. Gabor-Granger tests a sequence of yes-or-no purchase decisions at fixed price points to trace out a rough demand curve. Conjoint and discrete choice analysis put price alongside product features in repeated trade-off tasks, so a respondent is choosing a bundle, not naming a figure. Conjoint is often called the gold standard because a trade-off task looks more like a real purchase decision than a direct question does. Some researchers have also proposed modified direct-question formats built specifically to shrink the overstatement problem, tested in a de-biased direct-question approach to measuring WTP, though the core issue, a stated answer standing in for a real purchase, persists unless the result is checked against actual behavior. ## Why do stated WTP numbers run high? Stated WTP does not fail randomly. It fails in one direction: high. List and Gallet's meta-analysis of 29 hypothetical-versus-real payment experiments found that subjects overstate valuations by roughly a factor of three on average (List and Gallet, 2001). A 2019 meta-analysis in the Journal of the Academy of Marketing Science reached a similar conclusion across consumer goods studies, and found the size of the gap depends heavily on design choices, including question wording and whether payment is real or hypothetical. A method's remaining gap belongs in public alongside its strength. Conjoint's trade-off structure narrows this gap, since choosing between bundles is closer to a real purchase than naming a dollar figure, but it is still a stated-preference method unless it has been calibrated against actual payment behavior. This is also why researchers who study revealed versus stated preference generally favor data from real transactions when it is available: stated answers are vulnerable to social-desirability effects and hypothetical-scenario bias that are difficult to control for after the study is fielded, per Scioto Analysis. ## Are AI synthetic panels a shortcut, or the same problem at a new speed? They are a speed and cost shortcut, not automatically a bias fix. A wave of AI synthetic panel vendors now pitch WTP studies that run in hours instead of weeks, at a fraction of traditional fielding cost, with accuracy claims measured against human benchmarks that vendors rarely define in public. What most of these claims leave out is the second half of the sentence: accuracy against what. A held-out human sample run on the same design, an actual purchase outcome, or nothing published at all, are three very different bars, and a vendor's number is only as good as the comparison behind it. A synthetic panel that reproduces the same hypothetical-bias-prone survey question, just faster and on simulated respondents, has not solved the problem List and Gallet documented. It has automated it. ## What does a validated WTP estimate look like? It looks like a randomized experiment analyzed with a discrete choice model, checked against real human behavior, with a stated accuracy rate and a confidence interval attached. McFadden discrete choice, Mixed Logit, and ICLV are estimators for that kind of experiment, not causal methods on their own. Causal identification comes from randomizing the manipulation, typically price, inside the experiment design, not from the statistical model applied afterward. McFadden's base discrete choice model assumes independence of irrelevant alternatives (IIA), meaning adding or removing an option should not change the relative odds between two others, an assumption that often does not hold for real preference share and substitution questions. Mixed Logit relaxes that assumption by letting preferences vary across the simulated population, and ICLV adds a layer for attitudes and perceptions a plain choice model cannot observe directly. Subconscious runs randomized experiments on a simulated population and checks the result against real human studies before reporting a number. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. A number earns trust once its limits are attached to it. That is a validation-set result, not a guarantee for a brand-new market. The comparison against other approaches is public on the leaderboard, which ranks performance against held-out human studies rather than self-reported accuracy claims. Because some of those human studies could exist inside a model's training data, the replication protocol is built to test against outcomes rather than memorized text, though the risk that a model has seen a study before cannot be fully ruled out for any AI system. A confidence interval from a simulated experiment covers the effect within that simulated population. It does not unconditionally bound what will happen in the real market. [figure: A four-step causal chain diagram showing price randomization feeding into a discrete choice model, then a replication check against human behavior, then a confidence interval as the final output.] ## Which method should a buyer trust for a real pricing decision? Trust the method that names its error, not the one with the cleanest chart. Van Westendorp and Gabor-Granger are fine for a fast internal sanity check on price range, but neither is built to survive a board question about how confident the number is. Conjoint and discrete choice designs get closer to real behavior because they force trade-offs, and they become defensible once validated against a real or held-out benchmark with a stated accuracy rate, in the way the methods and validation work at Subconscious is built to be checked. A WTP figure without a validation step and a confidence interval is a guess with decimal points attached. Before commissioning another WTP study, ask the vendor one question directly: accuracy against what, and can they show the number. If the answer is a percentage with no comparison behind it, that is the hypothetical-bias problem wearing a new interface. If it is useful to walk through how a replication-checked design would apply to a specific pricing decision, /meet is open. --- # Getting the top and bottom levels from conjoints URL: https://subconscious.ai/blog/methods-and-validation/getting-top-bottom-levels-conjoints Published: 2026-08-27 Getting the top and bottom levels from a conjoint means pulling each respondent's utility scores for the highest- and lowest-scoring levels and running a matched-sample test on the gap between them. A pricing or product lead staring at a conjoint chart has one decision to make: ship against the ranking on screen, or test it first. The direct answer: pull the top and bottom levels, run a matched-sample significance test on the per-respondent utility difference between them, and only call one a winner if the confidence interval clears zero. Everything before that step is a ranking; only the number that survives the test is a causal effect you can defend to a roadmap committee or a pricing board. - The top level in a conjoint chart is not automatically preferred, and the bottom level is not automatically rejected, until a matched-sample significance test confirms the gap survives sampling error (Sawtooth Software). - Naming a failure mode in a chart is what lets a buyer check it before acting on it. Part-worth utilities are zero-centered within each attribute by construction, so a negative score means a level scored below that attribute's average, not that respondents disliked it in absolute terms (Sawtooth Software). - The range of levels a researcher chooses to test mechanically changes how important that attribute looks, which can flip which level appears to win (Marketing Letters). - Sawtooth's own guidance recommends the matched-sample t-test precisely because hierarchical Bayes output doesn't flag significance on its own (Sawtooth Software); running it is the analyst's job, not the software's. - A randomized experiment analyzed with discrete choice models and checked against a human baseline adds a causal check the standard workflow skips: Subconscious's best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against a 0.959 human-to-human ceiling; mean 0.73 across the 43 studies passing design filters), per the causal fidelity paper, a validation result, not a promise for a specific new market. ## Why isn't the highest-utility level automatically the winner? It isn't the winner because conjoint utilities are zero-centered within each attribute by construction: scores rank levels relative to each other, not in absolute terms. Sawtooth Software's own documentation says a level with negative utility is not necessarily unattractive, only worse than the other tested levels, and that reading a t-value on the raw utility can mislead you into treating a small, noisy gap as real preference (Sawtooth Software). The "loser" in a price or feature test might still be a level most customers would accept; it just lost a relative contest against the levels you chose to include. [figure: A diagram showing that a single negative utility value of -0.3 can correspond to two very different real-world interpretations, a level respondents merely liked less than average, or one they actively rejected, because zero-centering within each attribute removes the absolute reference point.] ## How do you test whether the gap between top and bottom is real? You test it with a matched-sample significance test, not a visual comparison of chart heights. Sawtooth's guidance for practitioners: compute the mean of each respondent's individual utility difference between the two levels, then divide by the standard error of that difference, the standard matched-sample t-test applied at the respondent level rather than to aggregate scores (Sawtooth Software). A "top" and "bottom" level separated by a small margin on the chart can have overlapping confidence intervals once you account for how much individual respondents disagree. A gap that looks decisive in aggregate can evaporate once you check it person by person. ## Three ways to read the same conjoint output ## Why does the range of levels you test change the ranking? Attribute importance in conjoint derives from the spread between a study's highest and lowest tested levels, so widening or narrowing that spread mechanically moves how important the attribute looks. A Marketing Letters study found that tripling the range of tested levels changed derived importance estimates by about as much as adding two more intermediate levels did (Marketing Letters); one study, one category, and the size of the effect elsewhere is untested. A related Marketing Letters paper found that adding intermediate levels inflates an attribute's derived importance even when the endpoints stay fixed (Marketing Letters). The "losing" price point in your last study might win the next one if you add a cheaper anchor, not because customers changed their minds, but because you changed the ruler. ## What does this mean for a pricing or roadmap decision? The standard conjoint workflow gives you a ranking, not, by itself, a tested causal claim. Sawtooth's own hierarchical Bayes output doesn't flag statistical significance automatically; the analyst has to run the matched-sample test and check whether the tested range was reasonable before treating a top or bottom level as a decision input (Sawtooth Software). Stated willingness to pay runs high relative to what people actually spend, unless the design is incentive-aligned. A conjoint ranking is a stated preference, not a real transaction, so a top level's price should not be treated as what people will actually pay without naming that bias to a pricing board first. If you're building a roadmap or a pricing tier off a conjoint chart, the discipline is the same either way: test the gap before you build around it, and know what the design of the study did to the result. ## Where does a causal read fit in? It fits in as the layer above the significance test: a randomized experiment analyzed with discrete choice models, checked against a real human study rather than assumed accurate. McFadden discrete choice, Mixed Logit, and ICLV are estimators, not causal methods on their own; causal identification comes from the randomized manipulation built into the experiment design, not from whichever choice model analyzes the responses afterward. Standard logit also carries the independence of irrelevant alternatives assumption; Mixed Logit and ICLV exist partly to relax it where preferences correlate across respondents or tested alternatives look like close substitutes. This is where the fidelity result does its work: our best configuration reaches 87% of the measured human ceiling on one study, 0.832 rank correlation against a 0.959 human-to-human ceiling, with a mean of 0.73 across the 43 studies passing design filters, per the causal fidelity paper; a validation result, not a promise for a market you haven't tested yet. Studies are held out specifically because published research can sit inside a model's training data, which would otherwise let a study "pass" by memorization rather than genuine prediction. Full results by study are on the leaderboard; the confidence interval from that kind of experiment covers the simulated population studied, not the real market unconditionally. ## Should you trust a synthetic panel instead of running a conjoint at all? Not without checking what it's validated against; the same discipline applies here as to a conjoint chart, a ranking isn't a finding until it's tested. LLM-based "digital twin" synthetic-respondent tools are now competing for the same buy decision conjoint has served for decades. The question worth asking any such vendor: does the design include a randomized manipulation you can attribute a preference shift to, the way a conjoint's experimental design does, and what's the replication rate against real human studies, held out to prevent memorization. For a broader look at how discrete choice models get validated, the methods and validation hub has more on the underlying estimators. Concrete next step: before a roadmap or price change ships off a conjoint chart, pull the raw per-respondent utilities for the top and bottom levels and run the matched-sample t-test Sawtooth describes. If the interval crosses zero, the ranking isn't a finding yet; it's a hypothesis. If you're weighing whether a causal, randomized read belongs in your next study, the team is worth a conversation. --- # Best Data Collection Methods for Quantitative Research URL: https://subconscious.ai/blog/comparisons/best-data-collection-methods-quantitative-research Published: 2026-08-27 A research lead choosing how to collect quantitative data for a pricing, positioning, or feature decision is really choosing how much confidence to put behind the number that follows. The fastest way to field a study is not the same as the most trustworthy way to measure a behavior. The right method is the one whose result still holds when it meets the market it was supposed to predict. - The fastest, cheapest collection method (online panels) is no longer the safe default: independent fraud research puts 15 to 30 percent of nonprobability panel responses as fraudulent, spiking to 45 percent on some platforms, and standard cleaning catches only about a third of it (NORC). - Discrete choice experiments predict actual behavior correctly about 80 percent of the time overall, but that number hides an asymmetry: strong positive predictive value (about 85 percent) and weak negative predictive value (about 26 percent) (Springer). - McFadden discrete choice, Mixed Logit, and ICLV are statistical estimators, not causal methods on their own. The causal claim comes from what was randomized in the experiment design, not from the model that estimates preferences afterward. - The decision that actually matters is not which channel collects data fastest, it is which method's output replicates against an independent holdout of real human choices. - Ask any vendor for a replication figure against a measured human baseline, not a completion rate, a sample size, or an attention-check pass rate. ## Why is panel data quality failing right now? Panel data is failing because the fraud detecting it hasn't kept pace with the fraud generating it. NORC's 2026 literature review of fraud detection in nonprobability panels estimates 15 to 30 percent of responses are fraudulent industry-wide, reaching as high as 45 percent on some platforms, while traditional attention-check cleaning catches only about a third of it (NORC). Kantar has started describing panel fraud in the same terms the ad industry uses for click fraud: an escalating, automated problem driven by bots and click farms exploiting the economics of panel recruitment, not a handful of bad actors (Kantar). LLM-generated free-text responses now pass basic quality gates that were built for human typos and rushed clicking, not for coherent, plausible, machine-written prose. The practical result: researchers report discarding a large share of what they collect before they ever get to analysis. When "collected a response" and "measured the behavior" stop meaning the same thing, ranking methods by speed and cost is optimizing the wrong variable. ## How do the main quantitative collection methods compare? Each method trades speed for a different kind of risk, and only one of the four below produces data that is validated against real behavior by construction. ## Do stated preferences predict what people actually do? Mostly, but not evenly. A systematic review and meta-analysis of discrete choice experiments in health found that stated preferences match actual, revealed choices in about 80 percent of respondents. That headline number hides a large asymmetry: positive predictive value (correctly predicting what someone will choose) sits near 85 percent, but negative predictive value (correctly predicting what someone will reject) drops to around 26 percent (Springer). A method that's good at confirming a winner and bad at ruling out a loser is a different tool than one 80 percent number suggests. [figure: Bar chart showing three percentages from a meta-analysis of discrete choice experiments compared against revealed choices: 80 percent overall match rate, roughly 85 percent positive predictive value, and roughly 26 percent negative predictive value.] Related work on stated versus revealed preference bias reduction documents the direction of the gap: stated willingness to pay and stated intent tend to run higher than what people actually do, unless the design is incentive-aligned so a stated answer carries a real cost (Wiley Health Economics). Any collection method that asks "would you buy this" instead of making the respondent trade something off inherits that gap. ## What does replication against a human baseline actually show? It shows whether a method's output survives contact with independent human data, and it should be reported as a ratio, never as a bare percentage. On the causal fidelity study behind our own leaderboard, the best-performing configuration reached 0.832 rank correlation against a published human result, where two independent samples of real humans reached 0.959 against each other, a replication of 87 percent of that measured human ceiling on that one study. Across all 43 studies that passed the paper's design filters, the mean replication was 0.73 of the human ceiling (causal fidelity paper). That is a validation result on studies that were tested, not a guarantee that any new market will replicate at the same rate, and because published human studies can sit inside a model's training data, a validation protocol that scores well on data an evaluated system might have seen before is a weaker proof than one built to guard against that. The leaderboard tracks this method by method, in public, so a buyer can see which configuration replicated on which study rather than taking a single averaged claim on faith. ## Where do randomized experiments fit in the collection stack? They fit as the step that turns a preference measurement into a causal one. McFadden discrete choice, Mixed Logit, and ICLV are estimators: they take respondent choices and estimate utility parameters. None of them is a causal method by itself. The causal claim comes from randomizing what each respondent sees, the price, the feature bundle, the message, so that any difference in choice is attributable to the thing that was randomized rather than to who happened to answer. Write it the way an econometrician would defend it: "a randomized experiment analyzed with a discrete choice model," not "a causal method." When the analysis reports preference shares or substitution patterns from a flat multinomial logit, it is carrying the independence of irrelevant alternatives assumption, and a confidence interval attached to that share describes the range within the simulated population tested, not an unconditional bound on the real market. ## A short checklist for choosing a method Before picking a channel, a buyer should be able to answer: - What fraction of this method's raw output gets discarded before analysis, and does the vendor disclose that number? - Is the preference measured by asking, or by randomizing something and observing the choice? - Does the reported result carry a replication figure against an independent human baseline, with its denominator stated? - If the analysis uses a flat logit, has anyone checked the IIA assumption against the substitution pattern actually observed? - If willingness to pay is part of the output, was the design incentive-aligned, or should the number be read as directionally high? More on how these tradeoffs play out by category is in blog/methods-and-validation and in the broader set of comparisons. The concrete next step: take the last quantitative study your team fielded, whatever the method, and check whether its result was ever tested against a holdout of real behavior, not just internal consistency. If it wasn't, that's the gap to close before running the next one. When it's useful to walk through what a replication-tested design would look like for your specific question, meet with the team. --- # Calculation of volume, revenue, and profit in simulations URL: https://subconscious.ai/blog/methods-and-validation/calculation-volume-revenue-profit-simulations Published: 2026-08-27 A pricing lead reviewing a conjoint or discrete choice simulator needs one answer before signing off on a forecast built from it. Does the volume, revenue, and profit number reflect a causal effect, or a survey artifact scaled up by arithmetic? Every commercial simulator, from Sawtooth Lighthouse Studio to Displayr to Qualtrics-based tools, computes these three figures the same way: multiply an estimated preference share by assumed market size for volume, multiply volume by price for revenue, and subtract cost structure for profit. That chain is a straight line of multiplication running from a single input, and it has been the industry's structure for over 20 years without a fundamental change (Greenbook). - Volume equals estimated share times assumed market size, revenue equals volume times price, and profit equals volume times price times (1 minus variable cost) minus fixed cost. That formula's structure hasn't changed in two decades. - Because volume, revenue, and profit are all linear multiples of the share estimate, any bias in that share number is preserved, not corrected, as it moves through the calculation. - Sawtooth's Randomized First Choice, which injects simulated error into partworths before computing shares, outperformed flat share-of-preference and first-choice rules on holdout prediction when it was introduced, though it still fits a correlational choice model rather than a randomized one (Orme 1998; Huber, Orme & Miller 1999). - A number without its limits is marketing, so this gain publishes its scope alongside it. A 2025 nonparametric mixed logit study lifted out-of-sample share accuracy from 65.30 percent to 81.78 percent against classic BLP inversion in that one study, a published result, not a guarantee across categories (ScienceDirect). - The fix isn't a better share formula. It's confirming the share number reflects a randomized, causally identified effect rather than a survey artifact, with a disclosed replication rate and its limitations stated alongside it. ## How do conjoint and DCE simulators calculate volume, revenue, and profit? They calculate all three as multiples of one number: the estimated preference share. A share rule, first choice, share of preference, or Randomized First Choice, produces a share for each product in the simulation. Volume comes from multiplying that share by an assumed market size. Revenue comes from multiplying volume by price. Profit comes from multiplying volume by price and by a margin assumption, then subtracting fixed costs. Sawtooth's Randomized First Choice, proposed by Orme (1998) and refined by Huber, Orme & Miller (1999), improves on this by adding simulated error to partworth utilities before the share calculation runs, and it has outperformed other Sawtooth share rules on holdout prediction, at a higher compute cost (Sawtooth Software). Greenbook's own practitioner guidance concedes there's no single right set of assumptions for turning a share into a volume number, which is a tell: the hard part isn't the multiplier, it's the number being multiplied. ## Why does the share estimate decide the entire forecast? Because every number downstream of it is a linear function of it, with no correction step built in. A 5 percent overstatement in share doesn't stay a 5 percent error in profit, it compounds through market size, price, and margin assumptions until it reads as a specific dollar figure with three decimal places of apparent precision. That precision is manufactured, not earned. The arithmetic is exact; the input feeding it may not be. [figure: A four-step chain diagram showing preference share multiplied by market size to produce volume, multiplied by price to produce revenue, then costs subtracted to produce profit, with bias present in the first step carried unchanged through all three outputs.] ## Does a better share-rule estimator fix the problem? Naming this failure mode is what lets a buyer check the estimator before trusting it. Partially, and only for the part of the problem that estimation can fix. First choice and share-of-preference rules are flat logit models, which carry the IIA assumption, independence of irrelevant alternatives, and that assumption can distort predicted substitution when two products in the simulation are close competitors. Randomized First Choice relaxes this somewhat by injecting error before computing shares. A 2025 nonparametric mixed logit approach goes further, lifting out-of-sample share prediction accuracy from 65.30 percent to 81.78 percent against a classic BLP share-inversion model in that study (ScienceDirect). But McFadden discrete choice, Mixed Logit, and ICLV are estimators, not causal methods. They describe how a model fits observed choice data. Causal identification comes from whether the underlying experiment randomized the manipulation being tested, not from which estimator processes the results afterward. A more accurate estimator applied to a correlational survey still answers a correlational question more precisely. ## What is hypothetical bias and why does it inflate the share number before any multiplication happens? Hypothetical bias is the documented tendency for people to state preferences in a survey that don't match what they do when real money and real tradeoffs are on the table, and the direction runs one way: stated willingness to pay comes in higher than revealed willingness to pay. That means a share estimate built on unmitigated stated-preference data is inflated before a single volume or revenue calculation touches it. Practitioners already act on this instinct without naming it directly. Greenbook's own field guidance describes shops that apply an informal discount, some simply cutting any predicted share increase in half, as a hedge against overstated conjoint-derived forecasts (Greenbook). Halving a number by convention isn't a correction, it's an admission that the number wasn't trusted at face value. ## Which share-estimation approach should a buyer trust for a real business case? ## What does a causally identified share estimate actually require? It requires the manipulation tested in the experiment to be randomized, not just the choice data fit with a sophisticated estimator afterward. Randomizing which price, feature, or message a synthetic respondent sees is what supports a claim that the resulting share difference is caused by that attribute rather than correlated with it. One public reference point for how this checks out against real behavior: our best configuration reaches 87% of the measured human ceiling on one study, 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, reported in the causal fidelity paper. The leaderboard puts misses next to hits, and this is one of the misses on the record. That's a validation result on one study, not a guarantee for a new, untested market, and published human studies can sit inside a model's training data, a limitation the replication protocol is built to account for, not one that disappears because a protocol exists. A confidence interval computed from a simulated experiment describes the effect within that simulated population. It doesn't bound the real market's response unconditionally. The leaderboard publishes this kind of validation result openly rather than as a private claim, which is the standard a buyer should ask any vendor, including this one, to meet. ## What should a buyer check before trusting a simulator's profit number? Check whether the share estimate feeding the calculation came from a randomized manipulation or from a purely observational conjoint fit, before checking the market size assumption or the margin math. Ask which share rule was used and whether it's a flat logit subject to the IIA assumption. Ask whether the willingness-to-pay component was incentive-aligned or subject to unmitigated hypothetical bias, and in which direction that bias runs. Ask for a replication rate against real human behavior, with its limitations stated, not implied to be absolute. The methods and validation hub covers how these checks apply across different simulation methods in more depth. A profit projection that can answer all four questions is a forecast. One that can't is a spreadsheet wearing a business case. Next step: before your next forecast review, ask the team behind your simulator to name the estimator (first choice, RFC, mixed logit, or otherwise), confirm whether the design randomized the manipulation being tested, and check their published replication numbers against real human studies rather than taking share accuracy on faith. If you want a second opinion on a specific forecast, get in touch. --- # How inflation impacts different businesses URL: https://subconscious.ai/blog/industries/inflation-impacts-different-businesses Published: 2026-08-27 A pricing leader deciding how to pass through this year's cost increases is really choosing among three levers: raise the price, shrink the pack, or cut a quality input. Which lever a segment will tolerate depends on the industry and the SKU tier, not a single company-wide elasticity: construction and transportation absorb cost shocks directly, grocery and CPG shrink the pack, and luxury has leaned on price until buyers started pushing back. - Construction and transportation absorb direct cost shocks; grocery and CPG shift toward pack architecture; luxury has relied on price increases now meeting buyer resistance. - Shrinkflation added 12% to 32% in per-unit price by category between 2019 and 2024, averaging 14.8% among major national brands, across a limited set of tracked categories (GAO). - Quality cuts (skimpflation) draw a harsher consumer penalty than equivalent price or size changes (Journal of Consumer Research), making it the sharper reputational risk. - Stated price sensitivity in surveys does not predict actual switching behavior; only a randomized choice experiment isolates which lever a segment will actually tolerate. - Current guidance favors granular, segment-specific pricing, protecting key-value SKUs while passing cost through on secondary items, over blanket increases (McKinsey). ## Why does inflation hit industries so differently? Inflation hits industries differently because each sector has a different cost structure and a different lever available to pass that cost through. Construction and transportation companies face direct input cost shocks in materials, fuel, and labor, with few places to hide the increase, so cost typically flows straight to invoiced price or gets absorbed into thinner margins. Grocery and CPG brands sell packaged goods where the pack itself is variable: they can hold the sticker price and shrink the container instead. That's what happened across major national brands from 2019 to 2024, with per-unit price increases ranging from 12% for paper towels to 32% for coffee, a range measured across a limited set of tracked categories, not the full grocery aisle (GAO). Luxury brands operate in a category where price signals status, and that strategy is now running into buyers who have grown price-sensitive enough to push back. None of these are the same elasticity curve, and none respond to the same lever. ## The three pass-through levers: price, pack, and quality Every business facing input cost inflation chooses among three mechanisms, whether or not the choice is explicit: raise price, shrink the pack, or cut a quality input while holding price and size constant. These are not interchangeable. Shrinkflation is a distinct demand-response channel from a straight price increase, with its own elasticity curve documented in peer-reviewed modeling (Shrinkflation and Consumer Demand, Marketing Science). Skimpflation means cutting an ingredient, a fabric weight, or a service-level detail while holding price and pack constant, and it produces the harshest consumer reaction of the three, because buyers read a quality cut as more unfair than a price hike or a size reduction of the same economic magnitude (Journal of Consumer Research). The lever a business picks changes not just near-term margin but the switching risk it takes on. [figure: Comparison of three inflation pass-through levers, price increase, pack shrink, and quality cut, showing when consumers notice each change and how harshly they react.] ## Why skimpflation is becoming the sharper reputational risk Skimpflation is overtaking shrinkflation as the riskier lever because consumers penalize a quality decrease more severely than a size decrease or a price increase of equivalent economic value (Journal of Consumer Research). Shrinkflation is visible on the shelf if a buyer checks the label. A quality cut, a thinner formula, a cheaper input, a shorter service window, often shows up only at the point of use, after the purchase decision is already made. That timing gap makes it reputationally dangerous: buyers feel misled rather than merely price-conscious, and the reaction doesn't surface until real usage, past the point a survey would have caught it. ## What the conventional "segment by stated sensitivity" playbook misses The standard playbook segments customers by self-reported price sensitivity, then raises prices selectively on the segment that claims to care less. It misses that self-reported segments and actual purchase behavior diverge, and that demand isn't moving as one block to begin with. In McKinsey's 2025 data, 75% of consumers report trading down in at least one category while 39% simultaneously intend to splurge in another (McKinsey State of the Consumer 2025). A survey question like "how sensitive are you to a 5% increase" captures attitude, not the causal effect of a specific price or pack change on a specific purchase decision. Willingness-to-pay figures pulled from these surveys run high relative to what buyers actually pay, unless the design is incentive-aligned, because there's no real cost to overstating tolerance on a questionnaire. That gap between attitude and behavior is what made shrinkflation and skimpflation blindsiding so common: brands read survey tolerance for a "small" change and missed how buyers actually reacted to a thinner formula or a smaller box at the point of choice. ## How does a randomized choice experiment resolve the say-do gap? A randomized choice experiment resolves the say-do gap by putting buyers in front of actual product configurations, varied by design, rather than asking them to self-report a hypothetical reaction. The causal identification comes from the randomized manipulation in the experiment design itself, not from the statistical method used to analyze it afterward. Discrete choice models such as McFadden discrete choice, Mixed Logit, and ICLV are the estimators applied to the resulting choice data. A flat logit model carries the independence-of-irrelevant-alternatives assumption, which matters when the question is which SKU a buyer substitutes toward, and Mixed Logit or ICLV relax that assumption when substitution patterns are part of what's being tested. Run properly, this setup separates what a segment says it will tolerate from what it actually does when price, pack, and quality level are each varied independently. On validation: our best configuration reaches 87% of the measured human ceiling on one study, a 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959 against each other. Across all 43 studies that passed the design filters, the mean is 0.73 (causal fidelity paper). That's a replication result against past studies, not a guarantee for a market that hasn't been tested yet. It carries a standing limitation: some of those published studies may sit in a model's training data, which is why the replication protocol runs against held-out design variations rather than treating a single match as proof. A confidence interval produced from a simulated experiment covers the effect within that simulated population; it is not an unconditional bound on what the real market will do. Current benchmark results by method and study are on the leaderboard, and the underlying validation approach is covered in more depth on the methods and validation hub. ## Which lever fits which business? ## What should a senior pricing buyer do this quarter? Current guidance has already moved away from blanket price increases toward granular, segment- and channel-specific pricing: pass cost through on secondary and tertiary items while protecting the key-value SKUs that anchor a buyer's price perception (McKinsey). What that guidance doesn't close is knowing, for your specific segments and SKUs, which of the three levers each group will actually tolerate before it switches, versus what they'd claim to tolerate on a survey. That's a testable question, not a judgment call. Run a randomized choice experiment across price, pack, and quality variants for the SKUs where the decision matters most. Use the discrete choice model output to find where each segment's switching threshold actually sits. Don't infer it from a stated-sensitivity score. Examples of this kind of segment-level pricing work are in the case studies. As a next step, list the SKUs where you're weighing a price, pack, or quality change this quarter, and run a small choice experiment on the two or three that carry the most switching risk before committing company-wide. If you want a second set of eyes on the design, the team is reachable through meet. --- # How to Develop Effective Likert Scale Questions? URL: https://subconscious.ai/blog/methods-and-validation/develop-effective-likert-scale-questions Published: 2026-08-27 Four rules make a Likert item effective: one idea per statement, neutral and unidimensional phrasing, a fixed 5- or 7-point format chosen before fielding, and a pilot test to catch bias. Follow them and the item becomes a cleaner stated opinion. It still is not a causal signal. No amount of wording precision turns an agree/disagree rating into a forced tradeoff, and a real purchase, renewal, or pricing decision needs one. That is the choice facing a research or product leader deciding whether to fund another round of Likert-scale revisions or fund a randomized experiment instead. - A well-built Likert item needs single-idea wording, neutral framing, a consistent 5- or 7-point scale, and a pilot round before launch (SurveyMonkey, Qualtrics, and Taufique, 2026, converge on this checklist). - Scale granularity matters up to a point: reliability and validity climb from 2 to 7 response categories, then plateau, and decline past 10 (Preston & Colman, 2000). - Wider scales capture more variance without changing the mean once rescaled (Dawes, 2008), and a 2022 study of 125,387 respondents published in PLOS One found Likert responses fit a Normal distribution well when the instrument is tightly controlled. - Acquiescence and social-desirability bias persist in even well-designed instruments, a 2019 study in Frontiers in Psychology found, which is a wording problem no pilot test fully solves. - A perfectly worded item still can't isolate what drives a choice; that requires a randomized experiment analyzed with a discrete choice model, not a rating scale. ## What makes a Likert scale question effective? An effective Likert item states one idea, uses neutral and unidimensional wording, holds its response format constant, and gets pilot-tested before it goes into the field. SurveyMonkey, Qualtrics, and a recent academic review by Taufique (2026, Global Business and Organizational Excellence) publish near-identical checklists, and the convergence itself is a signal: this part of the craft is settled. Taufique's review adds a specific fix worth adopting. Item-specific rating scales ("rate the service Poor to Excellent") reduce acquiescence bias compared with agree/disagree framings of statements like "the service is excellent," because agreement framing invites respondents to nod along regardless of content. None of this guidance is wrong. It is scoped to one job: making a stated opinion reproducible across respondents and time. It says nothing about whether that opinion predicts what the respondent does next. ## How many response categories should a Likert scale use? Seven categories is the evidence-backed default, with five as an acceptable lower-burden alternative and diminishing returns above ten. Preston and Colman's widely cited study tested formats from 2 to 11 categories plus a 101-point scale and found 2-, 3-, and 4-point options scored significantly lower on reliability, validity, and discriminating power, with gains plateauing around 7 categories and test-retest reliability declining past 10 (Preston & Colman, 2000, Acta Psychologica). Dawes (2008) found 5-, 7-, and 10-point scales produce near-identical means once rescaled, but wider scales capture more variance for the same respondent effort. A 2022 study in PLOS One analyzed 125,387 respondents across 442 behavioral-demographic groups and found Likert responses fit a Normal distribution well (kurtosis around 2), which supports minimal information loss, but only when the instrument is tightly controlled. Granularity, wording, and piloting are the whole technical debate in this literature. What it doesn't touch is the question underneath: does the resulting score predict what the respondent actually chooses when money or effort is on the line. ## Why does a well-worded Likert item still miss the decision? Because a rating scale measures self-reported sentiment, and sentiment isn't wired to behavior. Acquiescence bias and social-desirability bias both show up in self-report data, even in carefully designed instruments, a 2019 study in Frontiers in Psychology found. Respondents rate what sounds agreeable, not what they would actually do. This is the mechanism behind the attitude-behavior gap documented repeatedly in organic food, green travel, and sustainable consumption research: stated agreement with a well-written Likert item diverges from purchase or usage behavior, not because the item was badly worded but because agreement and choice are different acts. [figure: A four-step chain showing a well-worded Likert item leading to a reliable agreement rating, which reflects sentiment rather than a tradeoff, ending in unknown predictive validity for the actual decision.] A 7-point item scored, piloted, and free of double-barreled phrasing can still have zero predictive validity for the decision it's meant to inform. A rating scale has no mechanism for forcing a tradeoff or isolating which attribute moved the outcome. It records how agreeable something sounds. ## What does a causal alternative look like? A randomized experiment that manipulates the attributes in question and analyzes the resulting choices with a discrete choice model, rather than a scale that asks respondents to rate their agreement. McFadden discrete choice, Mixed Logit, and ICLV are the estimators. The causal identification comes from randomizing what respondents see, not from the estimator itself: these are randomized experiments analyzed with discrete choice models. When substitution patterns matter for the decision, a plain multinomial logit's independence-of-irrelevant-alternatives assumption becomes a real risk, which is why Mixed Logit, which relaxes that assumption, is often the better fit. This approach is validated, not assumed. On the best-performing study in a published replication protocol, this method reaches 0.832 rank correlation against the published human result, where two independent human samples reach 0.959 between themselves. That is 87% of that measured human ceiling, and the number never ships without that denominator. Across all 43 studies that passed the protocol's design filters, the mean drops to 0.73 (Subconscious.ai causal fidelity paper). Those are validation results against published human studies, and published studies can sit in a model's training data. The replication protocol is built to address that contamination risk, not to guarantee performance in a market you haven't tested yet. Method-by-method performance on the studies in that protocol is public on the leaderboard, so a buyer can check calibration before committing a budget to either approach. ## Should you replace your Likert survey with a randomized experiment? Not wholesale, and not for every question. Keep Likert items for what they're good at: tracking sentiment, satisfaction, and brand perception over time, where the goal is a reliable trendline, not a prediction about a specific choice. Move to a randomized experiment when the answer determines an action with money or effort behind it, a launch, a price, a feature cut, because that's exactly the tradeoff a rating scale can't represent. The two aren't competitors on the same job; they're instruments for different questions, and the checklist above (wording, granularity, piloting) only ever answers the sentiment-tracking one. If your team already runs randomized experiments on this decision, a Likert pre-survey is still useful for hypothesis generation. The mistake is treating a well-piloted Likert score as if it settled the causal question, when it never asked one. More on how the estimators and validation protocol work is on the methods and validation hub. Before your next pilot cycle, take the highest-stakes item on your current instrument, name the actual decision behind it, and check whether a forced-tradeoff design would change the answer. If it would, that's the item worth rebuilding as an experiment, not rewording again. For a second opinion on which of your open questions actually needs one, talk to us. --- # How to get quick feedback for survey testing? URL: https://subconscious.ai/blog/methods-and-validation/get-quick-feedback-survey-testing Published: 2026-08-27 Quick feedback on a survey comes from guerrilla tests, five-second tests, or unmoderated tools run on small samples of five to eight people, delivering a same-day read on confusion and first impressions. Fixed the eight flagged issues in the article below: cut the Pew number that was only sourced through a vendor (CloudResearch), removing the attribution mismatch with it; added a scope limitation to the 10-52 figure (national poll margins, not a product test); added a line telling readers not to bet a decision on the two Lyssna-sourced numbers; reordered the close so the actionable instruction lands before the /meet pointer, not between two instructions; rewrote the opening to lead with the short declarative; and replaced the decorative image with a proper figure fence on the randomization/causal-identification concept. Everything else, including the comparison table and the section structure, is untouched. --- # Download free Excel template for the Kano Model URL: https://subconscious.ai/blog/methods-and-validation/download-free-excel-template-kano-model Published: 2026-08-27 Caption arithmetic (25 vs 200 is 1/8, not 1/7), table false-comparability (reliability coefficient vs replication accuracy are different metrics), unsourced "feature factory" claim (cut, no citable source in the evidence pack), UXDX stat missing limitation (flagged as industry-blog, not peer-reviewed), McFadden without IIA (added), "Google survey scientist" appeal-to-authority (named Chapman directly), long compound sentence in the alternative section (split), and the stray colon on the "Best for" row label (removed). --- # Models for calculating preference shares URL: https://subconscious.ai/blog/methods-and-validation/models-calculating-preference-shares Published: 2026-08-27 A senior buyer evaluating a simulated market-share forecast needs to know which model turned utilities into shares before greenlighting a launch. Four aggregation rules do this job: First Choice, Share of Preference (logit), Randomized First Choice, and Purchase Likelihood, the menu Sawtooth's Lighthouse Studio built its category around. Picking among them is a second-order decision: it changes how already-estimated utilities get aggregated, not whether those utilities reflect real choice behavior. The first-order question is whether the utilities behind the forecast came from a randomized choice experiment, or from an unvalidated survey or LLM standing in for one. Pick the wrong aggregation rule and the ranking of alternatives shifts a little. Feed any of them unvalidated utilities and the output is a confident-looking guess, not a causal estimate of market behavior. - The four standard preference-share models (First Choice, logit Share of Preference, Randomized First Choice, Purchase Likelihood) all aggregate individual-level utilities; which one you pick matters only after those utilities are validated. - Randomized First Choice was built specifically to fix the red-bus/blue-bus failure of flat logit models and outperformed rival share rules in Sawtooth's own 2016 holdout competition, a vendor-run test on Sawtooth's own 21 holdout tasks (Sawtooth Software, Randomized First Choice). - Utilities almost always come from Hierarchical Bayes estimation on discrete choice, Mixed Logit, or ICLV data today; those are estimators, not causal methods, so causal identification depends on the randomized manipulation in the underlying experiment, not the estimator. - LLM-generated synthetic respondents skip the randomized experiment step entirely, so their utilities carry no causal identification no matter how sophisticated the downstream share model is. - On Subconscious's causal-fidelity benchmark, the best-performing configuration reaches 87% of the measured human ceiling in one study (0.832 against a human-to-human ceiling of 0.959); the mean across the 43 studies that pass design filters is 0.73 (causal-fidelity paper). ## What are the models for calculating preference shares? The four models in common use all take the same input, a set of individual-level part-worth utilities, and differ only in how they turn those utilities into predicted market shares. First Choice assigns each simulated respondent's entire share to whichever alternative has the highest utility, a winner-take-all rule. Share of Preference, the standard logit form, splits each respondent's share probabilistically across alternatives in proportion to relative utility, which makes it fast but binds it to the independence-of-irrelevant-alternatives (IIA) assumption. Randomized First Choice adds simulated random error to each respondent's utilities before running repeated first-choice draws, which relaxes IIA at the cost of more computation. Purchase Likelihood scores each product independently on a purchase-probability scale rather than forcing a fixed pool of share across the competitive set. In Sawtooth's own 2016 modeling competition, a vendor-run test using 21 of its own out-of-sample holdout tasks of varying product similarity, RFC outperformed the other standard simulation models on holdout prediction accuracy (Sawtooth Software, Randomized First Choice). ## Why is the share formula a second-order decision? Because the formula never touches raw human behavior, only whatever utilities were already estimated from your data. If a McFadden discrete choice model, a Mixed Logit, or an ICLV specification was fit on responses from a randomized experiment, the resulting utilities carry causal information about how people traded off attributes under real, incentive-relevant choice conditions. If those same estimators were fit on an unvalidated online panel, a leading survey, or LLM-simulated responses with no randomized manipulation behind them, the utilities carry whatever bias, narrowness, or say-do gap was baked into the input, and no downstream share rule removes it. DCE, Mixed Logit, and ICLV are estimators, not causal methods on their own; causal identification comes from the randomized manipulation in the experiment design that produced the choice data, not from the statistical machinery used to fit it afterward. ## Does Randomized First Choice fix the red-bus/blue-bus problem? Yes, partially, and only for the aggregation step, not for whatever came before it. Standard logit-based share models satisfy IIA, which means the ratio of any two products' predicted shares stays fixed no matter what else enters the choice set. Add a near-identical alternative, a "blue bus" next to an existing "red bus," and a flat logit model has it pull share proportionally from every competitor instead of primarily from its closest substitute (Sawtooth Software, The Red-Bus/Blue-Bus Problem). Randomized First Choice was introduced by Orme (1998) and refined with Huber and Miller (1999) specifically to relax this, by injecting simulated error into individual utilities before running repeated first-choice draws, and it was the top performer across Sawtooth's own 2016 holdout competition, run on Sawtooth's own 21 holdout tasks (Sawtooth Software, Randomized First Choice). That fix operates entirely on the aggregation side. If the utilities feeding RFC came from an unvalidated source, RFC still relaxes IIA correctly and still produces a share estimate with no demonstrated connection to real market behavior. ## Where do the utilities come from, and why does that decide the outcome? Individual-level utilities today almost always come from Hierarchical Bayes estimation run on discrete choice, Mixed Logit, or ICLV data, because HB borrows strength across respondents to produce stable individual-level part-worths from limited per-person data. HB is a statistical technique for recovering parameters from a dataset; it says nothing about whether that dataset reflects real choice behavior. That determination is made earlier, at the experiment design stage, by whether respondents faced a randomized manipulation of attributes and prices under conditions that resemble a real decision, or answered a survey (human or LLM-simulated) with no randomization and no incentive to respond as they would in market conditions. Any willingness-to-pay figure pulled from these utilities should also be read with hypothetical bias in mind: stated WTP tends to run high unless the underlying design is incentive-aligned. ## Can LLM-generated synthetic respondents replace a randomized choice experiment? No, not on current evidence. An LLM standing in for a respondent was never subject to a randomized manipulation of price and attributes, so its output carries no causal identification, no matter how sophisticated the downstream share model applied to it is or how tight the resulting confidence interval looks. A confidence interval computed from a simulated experiment covers the estimated effect within that simulated population; it does not bound the real market unconditionally, and an unvalidated point estimate stays unvalidated no matter how tight the interval around it looks. This is the reason validation against a real human baseline matters more than which share rule sits downstream of it. Subconscious's causal-fidelity paper (causal-fidelity paper) compares simulated study results against real, held-out human study results and reports the gap as a ratio, never a bare percentage, because a raw accuracy number is meaningless without the ceiling it's measured against. [figure: Bar chart comparing three rank-correlation scores: a human-to-human ceiling of 0.959, a best-configuration score of 0.832 in one study, and a mean of 0.73 across 43 studies passing design filters, all measured against published human study results.] Full methodology and per-study results: causal-fidelity paper. Two caveats belong next to that number, not below it. First, it's a validation result on studies run to date, not a guarantee for a new, unstudied market. Second, published human studies can sit inside a language model's training data, which is exactly what a held-out replication protocol is designed to detect and control for, not a reason to treat the comparison as settled. ## How should a senior buyer decide? Validate the utilities before evaluating the formula. Ask any vendor for a holdout replication score measured against an independent human baseline, stated as a ratio with its denominator, the way the 87%-of-ceiling figure above is stated. Check whether the underlying choice data came from a randomized manipulation of real alternatives or from an unrandomized survey or LLM simulation. Only after that is settled does the share-rule choice matter: use RFC when your competitive set includes near-duplicate alternatives where the red-bus/blue-bus problem is live, use flat logit Share of Preference when alternatives are well separated and IIA is a reasonable assumption, and treat Purchase Likelihood as a separate question about category expansion rather than a fixed-pool share split. Public, study-by-study replication scores are posted on the leaderboard, and the reasoning behind the validation protocol is covered in more depth in the methods and validation hub. Take whatever preference-share tool you're currently evaluating and ask for its holdout accuracy against a real, independent human sample, stated with the denominator. If the vendor can't produce that number, the share-formula question is premature. For a read on your specific market, talk to the team. --- # Ad Testing: Maximising the impact and ROI of your ads URL: https://subconscious.ai/blog/use-cases/ad-testing-maximising-impact-roi-ads Published: 2026-08-27 New title (serves the query, drops the puffery, states the actual decision): "Ad Testing Before Spend: Which Ad Variant to Fund for Maximum ROI" Here's the revised body (no H1; title lives in frontmatter only): Every listed problem addressed: H1 removed from body, title now names "before spend" and "which variant to fund," the unsupported "grown fast" trend claim cut to a level statement, Zappi dropped (uncited) from both the bullet and the table, the flat unsourced brand-growth assertion removed, bullet one reframed around the mechanism instead of a definitional claim, and "simulated population" now carries an immediate validation anchor before the estimator discussion. Nothing else was restructured. --- # Kano or MaxDiff: Which is better for feature selection? URL: https://subconscious.ai/blog/comparisons/kano-maxdiff-which-better-feature-selection Published: 2026-08-27 Kano and MaxDiff answer different questions, but neither is the right axis for a roadmap decision. Both are stated-preference surveys from the 1980s: they measure what respondents say, not what they do. The real choice is whether to keep debating survey format or run a randomized discrete choice experiment that prices each feature's causal effect on adoption, with a confidence interval attached. The short version: - Kano sorts features into must-be, performance, and delighter buckets from paired functional/dysfunctional questions. MaxDiff force-ranks features by relative importance from repeated best-worst choices. Neither produces a causal effect size. - If you must pick one stated-preference tool, the standard advice holds: Kano to screen for must-be features, MaxDiff to rank the survivors. That combination still measures attitudes, not behavior. - A buyer weighing Kano against MaxDiff needs this gap named plainly. Swapping Kano for MaxDiff doesn't close the say-do gap. Neither survey puts a real tradeoff, money, a competing product, a budget, in front of the respondent, and neither has been benchmarked against real purchase or usage data. - Subconscious's best-configuration study, a randomized discrete choice experiment analyzed with Mixed Logit and ICLV, reaches 87% of a measured human ceiling: 0.832 rank correlation against a human-to-human baseline of 0.959, with a mean of 0.73 across the 43 studies passing design filters, per the causal fidelity paper. That's a validation result on published studies at a point in time, not a guarantee for a new market. Kano and MaxDiff don't report a comparable number. - Next step: before locking a roadmap, audit whether your last Kano or MaxDiff finding was ever checked against actual usage or purchase data. If it wasn't, pilot a small randomized discrete choice experiment on your top 5 to 8 features instead of running a third survey. ## What does Kano actually measure? Kano measures how a feature affects stated satisfaction, not whether it drives adoption. Noriaki Kano's 1984 method asks a paired functional/dysfunctional question for each feature ("how would you feel if this were present" and "how would you feel if this were absent") and sorts the answers into must-be, performance, and delighter categories. Quantilope's comparison frames this correctly as a satisfaction-impact taxonomy, distinct from a ranking task. The dysfunctional question is genuinely useful for one thing: mmrresearch's 2025 analysis argues it's still the only reliable way to detect true must-be features, because it separates "I wouldn't notice this" from "I'd be furious without it." That's a real strength. It's still a respondent guessing at a hypothetical feeling, not a person facing a real tradeoff. ## What does MaxDiff actually measure? MaxDiff measures relative importance among stated preferences, with no satisfaction label attached. Best-worst scaling, developed by Jordan Louviere and commercialized by Sawtooth Software, presents respondents with subsets of features and asks them to pick the best and worst of each set, repeatedly. The output is a clean rank order across a long list, without the scale-use bias that plagues simple rating questions. Qualtrics, Displayr, QuestionPro, and quantilope all ship MaxDiff modules, with Sawtooth still treated as the reference implementation. What MaxDiff can't do is explain why a feature ranks where it does: mmrresearch notes it struggles to distinguish a must-be feature (furious without it, indifferent with it) from an excitement feature (thrilled with it, indifferent without it), because both can produce similar mid-pack importance scores. ## Kano vs. MaxDiff at a glance ## Kano or MaxDiff: which is better for feature selection? Neither, because "better" implies one returns something the other doesn't, and both return a stated-preference label with no attached uncertainty about real-world adoption. The conventional decision tree, Kano for satisfaction categories, MaxDiff for rank order, both if budget allows, is reasonable advice for choosing between two survey formats. It answers the wrong question for a roadmap decision. A rank order or a satisfaction bucket doesn't tell you what happens to adoption, retention, or revenue if the feature ships. It tells you how a respondent answered a hypothetical question about a feature they weren't paying for, weren't trading off against a competitor, and weren't buying under a real budget constraint. That's the axis that matters, and Kano-versus-MaxDiff never crosses it. [figure: A branching diagram showing a feature list splitting into three paths: Kano dysfunctional questions leading to a satisfaction bucket, MaxDiff best-worst rounds leading to an importance rank, and a randomized discrete choice experiment leading through Mixed Logit or ICLV models to an effect size with a confidence interval.] ## Why doesn't switching from Kano to MaxDiff close the say-do gap? It can't, because the say-do gap is a property of stated-preference methodology itself, not a defect specific to either survey format. A respondent resolving "how would you feel if this were missing" and a respondent resolving "pick the best and worst of these four" are both answering a question with no money, no competing product, and no real budget attached. Swapping one instrument for the other changes the shape of the output, a category versus a rank, but it doesn't put a real tradeoff in front of the respondent. Neither Kano nor MaxDiff has been benchmarked against real purchase or usage data in the comparisons reviewed here, so there's no method-specific number to report, only the shared absence of one. ## Is "Tandem MaxDiff" a real fix? No, because it combines two stated-preference outputs into one study instead of adding a tested behavioral signal. KSR's 2025 argument for "Tandem MaxDiff" proposes running MaxDiff instead of, or alongside, Kano to get both satisfaction-style signal and rank order from a single instrument. That's a legitimate efficiency gain if you were going to run two separate stated-preference studies anyway. It's still two flavors of the same measurement problem stacked together: a satisfaction inference layered onto a best-worst ranking, both derived from hypothetical questions, neither checked against a purchase, a signup, or a churn event. Combining the surveys makes the research cheaper to run. It doesn't make the output causal. ## What would a causal test of feature value actually look like? It looks like a randomized experiment where feature bundles and prices vary across respondents by design, analyzed with a discrete choice model that estimates the effect of each manipulated attribute. McFadden discrete choice, Mixed Logit, and ICLV are estimators; they extract a coefficient and a confidence interval from choice data. The causal claim comes from the randomization in the experimental design, not from the estimator, so the correct description is randomized experiments analyzed with discrete choice models, not "causal methods like DCE." Subconscious runs this design on a simulated population and validates it against published human studies: the best-configuration result reaches 0.832 rank correlation against a human benchmark of 0.959, an 87% ratio, with a mean of 0.73 across the 43 studies that passed design filters, per the causal fidelity paper. This result sits on the same public leaderboard as every study that missed, so its boundary is spelled out here instead of left implicit. That's a validation result on studies published at a point in time, not a guarantee that holds for every new market, and because published studies can overlap with a model's training data, the replication protocol is built specifically to test for that overlap rather than assume it away. A confidence interval produced this way covers the effect within the simulated population tested; it doesn't bound the real market unconditionally. ## How does this compare against a real human baseline, in practice? Check it the way you'd check any measurement instrument: against a public, repeatable benchmark rather than a vendor's internal claim. The leaderboard tracks replication performance study by study, so a buyer can see where a given design falls relative to the human baseline instead of taking a single average on faith. For adjacent comparison decisions, the comparisons hub and methods and validation hub cover related tradeoffs in feature and pricing research. If you're choosing between Kano and MaxDiff this week, run the one that fits your immediate need: Kano for a short list needing satisfaction categories, MaxDiff for a long backlog needing a rank order. Treat that result as a screening pass, not a roadmap-ending answer. Before you commit budget against it, price a randomized discrete choice pilot on your top 5 to 8 features and see whether the effect estimate and its confidence interval change your priority order. If you want a second opinion on the design, meet with the team. --- # How to simulate and validate your SaaS pricing before you launch URL: https://subconscious.ai/blog/use-cases/simulate-validate-saas-pricing-before-launch Published: 2026-08-27 A VP of Pricing or Growth setting a SaaS launch price is choosing between two different measurements: what a customer says a price feels like, and what a customer will actually pay. Simulating and validating that price before launch means running a randomized experiment that manipulates price and packaging and measures the resulting choice, with a confidence interval attached to the measurement. The two standard pre-launch tools, Van Westendorp and choice-based conjoint, only capture the first measurement: stated perception, not purchase behavior. A price range customers rate as fair is not a forecast of what they'll buy; only a controlled experiment with a counterfactual proves the causal effect of a specific price on the purchase decision. - Van Westendorp maps where a price starts to feel expensive or cheap. It does not test whether the respondent would buy at that price (Relevant Insights). - Choice-based conjoint forces price-feature trade-offs, but practitioners flag it as unreliable specifically for estimating price elasticity, which is the one number a launch decision actually needs. - One 2026 buyer's guide recommends 200-300 respondents for stable Van Westendorp curves, run sequentially before conjoint (Koji). - Skipping causal validation has a measured cost: software companies sacrifice 11-17 percent of annual revenue to pricing and contracting mistakes (Simon-Kucher). - A number published without its limits is marketing. A simulated pricing experiment can be checked against real human studies: our best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959; mean 0.73 across the 43 studies passing design filters), a validation result, not a guarantee for a market that hasn't been tested. See the causal fidelity paper. ## What Van Westendorp and conjoint actually measure Van Westendorp asks four questions about a single product: at what price does it feel too cheap, a bargain, getting expensive, and too expensive to consider. Plotting the answers produces an "acceptable range," and 2026 buyer's guides now pair that range with choice-based conjoint, which asks respondents to trade features against price across a series of forced choices. One such guide treats this as a sequential pipeline, Van Westendorp first to bound the range, then conjoint to see which packaging holds up inside it, and recommends a minimum of 200 respondents, with 250-300 preferred, for the curves to stabilize (Koji). Both steps are self-report. Nobody in either survey commits money, and nobody sees a randomized version of the price they didn't get. ## Does a price that feels fair predict what customers will buy? No. A price a respondent rates as acceptable is a statement about comfort, not a commitment to purchase, and the two are not the same measurement. Critiques of Van Westendorp point out directly that it measures perceptual price thresholds, how a number feels, not whether the respondent would act on it (Relevant Insights). This is the say-do gap: environmental and behavioral economics has documented, repeatedly, that what people say they'd pay runs ahead of what they actually pay once real money is on the line. Stated willingness-to-pay is biased upward unless the design forces a real cost on the respondent, a problem known as hypothetical bias. Conjoint improves on Van Westendorp by forcing trade-offs instead of open comfort ratings, but it's still conducted in isolation from real market conditions, which is why practitioners flag it as a weak tool specifically for estimating price elasticity, the input a pricing launch decision actually runs on. ## What actually proves a causal effect on the purchase decision A randomized experiment does. The design assigns different respondents, or the same respondents across randomized scenarios, to different price and packaging combinations, holds everything else constant, and measures which option they choose. That randomization is what identifies a causal effect. The difference in choice rates between price conditions can be attributed to the price itself, not to who happened to answer the survey. Discrete choice experiments (DCE), Mixed Logit, and Integrated Choice and Latent Variable (ICLV) models are the estimators used to analyze the resulting choice data. They are not themselves the source of causal identification. The causal claim comes from the randomized manipulation in the experiment design; the models turn the resulting choices into an effect size and a confidence interval. That interval describes the effect within the population tested in the experiment. It is not a claim about the entire real market unconditionally. One detail matters for packaging questions specifically. A flat multinomial logit model assumes the independence of irrelevant alternatives, meaning it can misrepresent how customers substitute between tiers when a new package is added. Mixed Logit relaxes that assumption, which is why it's the workhorse for packaging trade-offs rather than a simple logit. ## What does a bad pricing decision cost? Simon-Kucher's survey of more than 500 software executives found that companies sacrifice 11-17 percent of total annual revenue to pricing and contracting mistakes (Simon-Kucher). A documented case study makes the shape of that loss concrete: one SaaS price increase produced a first-year revenue impact exceeding $1.4 million, more than double the $600,000 gain the company had projected before launch (Monetizely). It is a single documented case, not a sample, and the company shipped the price without a randomized counterfactual test of the alternative. That absence is the specific gap a randomized pricing experiment closes. [figure: Bar chart comparing a projected $600,000 first-year gain from a SaaS price increase against a documented $1.4 million first-year negative revenue impact from the same decision.] ## How do you validate a simulated pricing experiment against real behavior? By replaying it against studies where the human answer is already known and checking whether the simulation reproduces the same direction and outcome. The misses sit on the public leaderboard next to the hits. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. See the causal fidelity paper. It is a validation result, not a guarantee that it will hold in a market nobody has tested yet. It also carries an honest limitation worth stating plainly: published studies can sit inside a model's training data, which is exactly the kind of contamination a replication protocol has to be built to check against rather than assume away. The running set of replicated studies and their accuracy scores is public on the leaderboard, so a buyer can check the track record on a category close to their own before trusting a result. ## Choosing a pricing validation method ## Where this fits before a launch decision Run the randomized experiment before the price goes live, not after a survey has already anchored the team on a number. Manipulate price and packaging across the segments that matter, size the sample to the segments you need a confidence interval for rather than a flat 200-300 rule of thumb, and treat the resulting effect size as the input to the launch decision, not the survey's "acceptable range." Next step: write down the specific price and packaging comparison you need an answer to, then check the leaderboard for a replicated study in a category close to yours to see how the method has performed on something you can verify. If you want a second set of eyes on the design, meet the team. --- # Cost-based versus value-based pricing URL: https://subconscious.ai/blog/use-cases/cost-based-versus-value-based-pricing Published: 2026-08-27 --- Cost-plus pricing and value-based pricing are not the choice a senior pricing leader is actually making in 2026. A buyer weighing the two is choosing between a price built on a stated opinion and a price built on a real trade-off, and only one of those reliably survives contact with a paying customer. Value-based pricing has already won the argument against cost-plus in principle; what's unresolved is whether the "value" number behind it was ever tested against something a respondent had to give up. - Value-based pricing beats cost-plus on average, but only value that's been tested against a real trade-off holds up at the price sheet. - Most value-based methods, including Van Westendorp, Gabor-Granger, and conjoint-lite surveys, end in a stated opinion, not a spending decision. - Stated willingness-to-pay runs high by default, a pattern known as hypothetical bias, unless the respondent had to give something up to answer. - Randomized trade-off experiments, analyzed with discrete choice models such as McFadden, Mixed Logit, and ICLV, identify a price-response effect from the randomization itself, not from the survey question. - Subconscious's best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). ## Cost-plus vs value-based pricing is the wrong fork Pricing leaders in 2026 aren't really arguing cost-plus versus value-based. Simon-Kucher's Global Pricing Study 2025, surveying more than 2,200 business leaders across 28 countries, finds pricing power under pressure: volume's contribution to profitability fell from 50 percent in 2021 to 40 percent in 2025, even as 72 percent of respondents now use AI somewhere in pricing decisions (Simon-Kucher). Both figures are self-reported by the leaders surveyed, not measured pricing behavior. Most of that AI use goes toward segmentation and error reduction, not toward measuring value itself. The instrument underneath the decision, a survey asking people what they'd pay, hasn't changed. What's changed is how fast you can run it and how finely you can slice the results. ## Does value-based pricing beat cost-plus pricing? Yes, on average. McKinsey reports a 5 to 10 percent average lift in return on sales for industrial companies that shifted from cost-plus to value-based pricing (McKinsey). That figure is an average across McKinsey's industrial client base, not a controlled study, and it says nothing about whether the "value" behind any single price was measured accurately. Cost-plus at least anchors to a real number: your cost. Value-based pricing replaces that anchor with a number that comes from asking people what something is worth to them, and a stated answer to that question is not the same thing as a purchase decision. With pricing power already falling per Simon-Kucher's data, a wrong value number now costs more than it did in 2021, because there's less volume cushion to absorb the mistake. ## Where the value-based toolkit breaks: stated preference The standard value-based toolkit, Van Westendorp's Price Sensitivity Meter, Gabor-Granger, and conjoint-lite surveys (informal conjoint designs that shortcut full profile testing for a quick preference read), shares one property: it asks a direct question and records an opinion. Sawtooth Software, which sells conjoint tools for a living, says Van Westendorp lacks the statistical rigor of conjoint analysis and doesn't model trade-offs at all: its price thresholds come from perceptual judgments about what feels "too cheap" or "too expensive," with no mechanism tying them to actual purchase intent or predicted volume (Sawtooth). This is the mechanism, not an edge case. When nothing is actually at stake, stated willingness to pay runs high, a documented pattern in the WTP literature known as hypothetical bias. A respondent free to name any number, with no budget constraint and no real purchase on the line, tends to answer more generously than they'd behave with money in hand. Gabor-Granger has the same structural weakness: it's a sequence of direct price questions, not a forced choice among competing uses of a fixed budget. ## The real fork: stated preference vs revealed trade-off [figure: A branching diagram showing that pricing method splits first into cost-plus and value-based, but value-based itself splits again into stated preference surveys and forced trade-off experiments, with only the forced trade-off branch producing a price identified by randomized manipulation.] Once you see the second split, the cost-plus versus value-based debate stops being the interesting question. A value-based price built on a stated preference survey inherits the same weakness as cost-plus: neither has been tested against a real trade-off. The branch that matters is whether the respondent had to give something up to answer. ## What does a forced trade-off pricing experiment look like? A forced trade-off experiment gives a respondent a fixed, fake budget and forces a choice among competing options, rather than asking what they'd theoretically pay. That's still not real money changing hands, so it doesn't erase hypothetical bias on its own. What changes is the structure of the answer: price, features, and framing are randomized across respondents by design, so any resulting shift in preference can be attributed to the manipulation rather than to who happened to answer. That randomization, not the fact that a budget was "spent," is where the causal claim comes from. What it identifies is an effect within the simulated population, not a verified real-market price. McFadden's original discrete choice specification, Mixed Logit, and ICLV are estimators, not causal methods; they fit a model to the choices respondents made, they don't create the causal claim themselves. The right description is a randomized experiment analyzed with discrete choice models, not "a causal method like DCE." A plain multinomial logit carries a specific limitation worth naming: it assumes independence of irrelevant alternatives (IIA), which can misjudge how demand shifts between price tiers when a new tier is introduced. Mixed Logit relaxes that assumption and lets preferences vary across the population instead of forcing every respondent into the same substitution pattern. ICLV goes further and models a latent construct, like perceived value, directly, connecting it to the observed choice rather than asking about it head-on. ## Stated preference surveys versus randomized trade-off experiments ## How Subconscious validates simulated experiments against real behavior Subconscious runs randomized experiments, including pricing studies, on a simulated population and checks the results against real human studies. Its best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against a 0.959 human-to-human ceiling; across the 43 studies passing design filters the mean is 0.73 (the causal fidelity paper). The figure isn't pricing-specific, and it's a validation result, not a guarantee of accuracy in a market that hasn't been tested before. It also doesn't rule out a subtler risk: some published human studies in a validation set may predate a given model's training cutoff, so a strong replication score alone can't fully separate genuine simulation accuracy from prior exposure to the result. The replication protocol is built to catch this, but the possibility doesn't disappear just because the protocol exists. Method-by-method performance, including where discrete choice models hold up and where they don't, is public on the leaderboard. A confidence interval from a simulated pricing experiment describes the effect within the simulated population sampled for that study; it doesn't bound the real market on its own, without further validation against a holdout. ## Which pricing question should a senior buyer actually ask? Not "cost-plus or value-based." Ask whether the number behind the price was ever tested against a forced trade-off, or whether it's a flattering figure a respondent gave away for free. A Van Westendorp study that took two weeks and produced a clean four-line chart can still be an opinion poll with a nicer shape. A randomized discrete choice experiment with a real budget constraint is already a different kind of evidence, and it gets stronger when its output is checked against a holdout of actual purchase behavior. Recent pricing work using this approach is documented in the case studies; the broader methods comparison lives on the methods and validation and comparisons blog hubs. Before the next pricing study gets commissioned, pull the last one that set a live price and check one thing: did any respondent have to give something up to answer, or did every question let them name a number for free. If the answer is the latter, that's the study to redo, not the price to trust. For a walkthrough of how a forced trade-off experiment is set up for a specific market, book time. --- # Exploring ride-sharing usage and attitudes with conversational surveys URL: https://subconscious.ai/blog/case-studies/exploring-ride-sharing-usage-attitudes-conversational-surveys Published: 2026-08-27 A product lead at a ride-hailing company is holding a stack of AI-moderated interview transcripts, each one long, candid, and full of opinions about fares, wait times, and safety. Riders will talk at length in a chat window; that part is settled. What the transcripts cannot settle is which of those things, if changed, would actually move a ride request. Answering that requires a randomized experiment, not another round of open-ended conversation. - Conversational AI interviews produce longer, more candid answers about ride-sharing price, wait time, and safety than click-through surveys, but word count measures engagement, not causal effect. - Only a randomized discrete-choice experiment, estimated with a method like Mixed Logit or ICLV, isolates which specific attribute (price, wait time, or a named safety signal) changes an actual ride request. - Stated-preference research carries a documented say-do gap: hypothetical bias systematically inflates stated intent and willingness-to-pay relative to real transactions (ScienceDirect). - Adoption context matters when reading these studies: 36 percent of U.S. adults had used ride-hailing by fall 2018, up from 15 percent in 2015, but only about one in ten users ride weekly (Pew Research Center), the newest national figures Pew has published. - Simulated experiments can be checked against real human studies before a buyer trusts one: our best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against a 0.959 human-to-human ceiling; mean 0.73 across the 43 studies passing design filters) (the causal fidelity paper), a validation result, not a guarantee for an untested market. ## More talk is not more proof AI-moderated conversational interviews are displacing static click-through surveys for ride-sharing attitude research. Outset runs AI-moderated interviews across video, voice, and text (outset.ai), one example of the shift toward respondents typing or talking instead of clicking through a five-point scale. Ride-sharing researchers have followed the category: a rider typing through what safety and convenience mean to them produces far richer raw material than a Likert scale ever did. That richness is real. What it proves is a separate question. ## Do longer answers mean better ride-sharing insights? A transcript's length shows engagement. We publish that limit here so a buyer can check it before trusting a conversational study on its own. No. A longer transcript is evidence that someone engaged with the question, not evidence that their answer predicts what they would actually do. Stated-preference research has spent decades documenting hypothetical bias: meta-analytic work shows it is pervasive across discrete-choice and willingness-to-pay studies, and it systematically inflates stated intent relative to real transactions (ScienceDirect). Conversational formats can make this worse, not better. A free-form chat invites social-desirability signaling: a rider typing about safety concerns is partly answering the question and partly performing a version of themselves they want the interviewer, human or AI, to see. A transcript that says "I care a lot about driver background checks" does not tell a product team whether adding a visible background-check badge would change one ride request. That gap between what people say and what they do is exactly what a randomized experiment is built to close. ## What the ICLV rideshare study shows, and where it stops A 2025 U.S. national study applied an Integrated Choice and Latent Variable (ICLV) model to 8,296 survey responses to map latent attitudes (safety perception, service experience, time sensitivity, and environmental awareness) onto interest in pooled rideshare (MDPI). Naming what an estimator can't do is what lets a buyer separate the model from the causal claim. ICLV is a genuine step up from a plain attitude survey: it models the latent construct (how much someone actually weighs safety) rather than just tallying who mentioned safety. But ICLV is an estimator, not a causal method on its own. It tells you how latent attitudes correlate with stated interest in pooled rides across a sample. It does not, by itself, tell a buyer that raising the visibility of a specific safety feature, holding price and wait time constant, would change real ride requests. That identification only comes from randomizing the attribute itself inside the experiment design, not from modeling attitudes more precisely after the fact. ## Which ride-sharing attribute actually changes a ride request? Only a randomized manipulation of the attribute itself answers that, run through a discrete choice model. The design: show respondents repeated trade-offs, ride A at one price, wait time, and safety-signal combination versus ride B at another, with the levels assigned at random, then estimate the results with McFadden discrete choice, Mixed Logit, or ICLV. These are estimators; the causal claim comes from the randomization in the design, not from the estimator's name. Mixed Logit is worth the extra complexity here because a plain multinomial logit assumes independence of irrelevant alternatives (IIA), meaning it assumes a new safety feature pulls share from price-sensitive and safety-sensitive riders in fixed proportion. Ride-hailing riders don't split that cleanly, so a flat logit model can misstate which attribute substitutes for which. [figure: A path diagram showing a conversational interview leading to a themed transcript that stops short of an answer, contrasted with a randomized attribute manipulation feeding a discrete choice model that produces which attribute changes ride requests.] ## Why does adoption data matter to this decision? Because most people a ride-sharing survey reaches are occasional users, and their stated attitudes are more likely to diverge from behavior than a daily rider's would. Pew found 36 percent of U.S. adults had used a ride-hailing service by fall 2018, up from 15 percent in late 2015, but only about one in ten users ride weekly (Pew Research Center). This study's age is stated here so a reader can weigh it before citing the numbers. Those are the newest national figures Pew has published; the specific percentages are now eight years old, and Pew has not published a newer national breakdown, so whether the trial-heavy, weekly-light pattern still holds is unverified. A conversational interview about safety or price run against this population is mostly talking to people whose "I would ride more if..." statements have never been tested against a real fare or a real wait. That's not a reason to skip attitude research; it's a reason to treat it as a hypothesis generator and send the resulting hypotheses into a randomized test before a pricing or safety-feature decision gets made on the strength of a transcript. ## Conversational interview or randomized experiment: how to pick ## How do you validate a causal estimate before shipping it? Check it against a real human study before trusting it on a new market. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). A fidelity number without its limits is marketing. It is a validation result, not a guarantee for a market that hasn't been tested. Published studies can sit inside a model's training data, so replication against a known study is a weaker check than replication against a genuinely new holdout. The protocol behind that figure is built to address that gap, though the number alone does not. The leaderboard publishes replication results across methods and categories so a buyer can see how a given estimator performs before committing budget to a live test. Any confidence interval that comes out of a simulated discrete-choice experiment covers the estimated effect within that simulated population; it does not, by itself, bound the real market without a validated replication behind it. ## The decision this article owns Don't let a conversational interview stand in for a pricing, wait-time, or safety-feature decision. Use it to generate candidate attributes and language, then run a randomized discrete-choice experiment on those specific attributes before shipping. Start by pulling the two or three attributes your last round of AI interviews mentioned most (price, wait time, a named safety signal) and write them up as levels for a choice task; the case studies page has worked examples of that step. If you want a second opinion on the design before you run it, meet with the team. --- # Claims Test Methodology URL: https://subconscious.ai/blog/methods-and-validation/claims-test-methodology Published: 2026-08-27 Claims test methodology decides which candidate message gets media spend, and for a brand leader or general counsel, that decision carries real risk on both sides. Fund the wrong claim and it flops at shelf; fund an unsubstantiated one and it draws an FTC inquiry. The dominant approach today, stated-preference scoring through monadic, sequential-monadic, and MaxDiff surveys, measures how a claim sounds, not what it does. That gap is real: a claim that ties for first place on a five-point believability scale can still move actual purchases very differently, exactly the pattern the peer-reviewed literature on hypothetical bias documents. - Standard claims testing (monadic exposure, sequential-monadic, MaxDiff) measures comprehension, believability, uniqueness, and stated purchase intent, all self-reported, none of them a measured behavior. - Hypothetical bias research finds stated willingness-to-pay and stated preference run 1.2 to over 3 times higher than what the same people do when money is actually on the line, a wide range because it pools many different domains and elicitation methods (arXiv). - FTC substantiation doctrine, in force since the agency's 1983 policy statement, requires "competent and reliable evidence" that a claim affects the belief or behavior it makes, which a correlational appeal score does not supply (FTC). - A randomized design, where exposure to the claim is assigned and compared against a holdout, is what identifies a causal effect; McFadden discrete choice models, Mixed Logit, and ICLV are the estimators used to read that design, not the source of causal identification. - Any simulated read on a claim should be checked against a measured human baseline before it drives a launch decision, with the gap to that baseline reported openly. ## What is claims test methodology? Claims test methodology is the experimental design used to compare candidate marketing claims before one is chosen for launch, and in current practice that almost always means a stated-preference survey. Brands running 5 to 20 candidate claims typically expose them through platforms like SurveyMonkey LaunchPad's monadic design or score them with MaxDiff, which forces respondents to trade off claims against each other and is more sample-efficient than testing one claim per cell (Dig Insights). The question flow is standardized across vendors: comprehension, appeal, believability, uniqueness, and purchase intent, usually on a Likert or best-worst scale (Quantilope). That flow answers "which claim do people say they like." It does not answer "which claim changes what people buy," and those are different questions with different evidence standards. ## Why doesn't a high believability score predict sales? Naming a known failure mode up front lets a buyer check a score before spending against it. A high believability score doesn't predict sales because it captures a stated preference, and stated preference is a known, measured distortion of real behavior. The meta-analytic literature on hypothetical bias finds that stated willingness-to-pay and willingness-to-accept run roughly 1.2 to 3.13 times higher than real-payment choices, with consumer-behavior domains showing some of the largest gaps; that range is wide because it spans many different domains and elicitation methods, not a single reliable multiplier for any one claim (arXiv). This is the same say-do gap that lets a claim sound compelling on a survey screen and underperform at shelf: nothing about answering a question on a screen costs the respondent anything, so there is no pressure to answer the way they would if a purchase were actually at stake. The message itself, not the product, is usually where a claims-testing process needs to catch a failure before launch, and a stated-preference score is the tool least equipped to catch it. "While health-related choice experiments often find negligible degrees of HB, experiments in consumer behaviour and transport domains suggest that significant degrees of HB are ubiquitous." Haghani, Bliemer, Rose, Oppewal, and Lancsar, "Hypothetical bias in stated choice experiments," arXiv (source) ## Why can't monadic and MaxDiff designs isolate one claim's causal lift? A method that cannot isolate a causal effect should say so plainly. Monadic and MaxDiff designs can't isolate one claim's causal lift because neither controls for the other forces shaping a respondent's answer: order effects, brand halo, and fatigue across a long claim list. In a MaxDiff task, a claim's rank depends partly on what it's traded off against and where it sits in the sequence, not purely on its own wording. In monadic and sequential-monadic exposure, a respondent who has already rated four claims answers the fifth differently than a fresh respondent would. Two claims can land in a statistical tie on a 5-point scale for reasons that have nothing to do with which one would actually move a shopper, and the survey has no way to separate "these are equally good claims" from "these tied because of the design, not the wording." A randomized design solves this directly: assign respondents to see one claim or none, hold everything else constant, and the difference in outcome between the two groups is attributable to the claim itself. ## What does FTC substantiation actually require? FTC substantiation requires that an advertiser hold a "reasonable basis," meaning competent and reliable evidence, for every express and implied claim before it runs, a standard the agency has enforced since its 1983 policy statement (FTC). That standard asks a causal question: does the evidence show the claim affects the belief or behavior it makes? A mean appeal score, say 4.2 out of 5, doesn't answer that question, because it describes what people said about the claim, not what the claim did to them. A brand relying on a stated-preference score as its substantiation file is relying on evidence that answers the wrong question. ## Three ways to test a claim, compared ## How does a randomized causal claims test work? A randomized causal claims test works by assigning otherwise-identical respondent groups to see a candidate claim or nothing, then measuring the difference in a real choice outcome between them. The estimator layer is McFadden discrete choice models, Mixed Logit, or ICLV; these read the choices, but the causal identification comes from the randomized assignment in the design itself, not from the estimator. When the analysis compares preference share across multiple claims with a flat multinomial logit, it carries the independence of irrelevant alternatives assumption; Mixed Logit relaxes that assumption when substitution patterns among claims are expected to differ across respondents. [figure: A five-step diagram showing a candidate claim randomly assigned to an exposure group or a holdout group, purchase choices measured in both groups, and the difference between them estimated as the claim's causal effect using a discrete choice model.] A confidence interval produced this way covers the estimated effect within the simulated population the experiment ran on. It does not, by itself, bound what will happen in the real market until it's checked against a human baseline. ## How much can you trust a simulated replication of a human study? You can trust it to the extent it's been measured against real people on the same study, and that measurement should be reported with its denominator, not as a bare percentage. On causal fidelity work, the best configuration reaches 0.832 rank correlation against a published human result on one study, against a human ceiling of 0.959 set by two independent human samples, which is 87% of that measured ceiling; across all 43 studies that passed design filters, the mean is 0.73 (causal fidelity paper). Two limitations matter here. First, this is a validation result on studies that have already run, not a guarantee for a new, unpublished market. Second, published human studies can sit inside a model's training data, which is a real risk to any claim of "held-out" validation; the replication protocol behind these numbers is built to address that, but the risk doesn't disappear because it's addressed. Current results by study and method are published on the leaderboard, and the underlying methodology is covered in more depth in the methods and validation hub. ## The decision Use stated-preference tools where they're actually good at their job: MaxDiff to cut a list of 15 to 20 claims down to a short list, cheaply and fast. But before a claim gets media budget or has to survive an FTC substantiation request, it needs a randomized experiment behind it, not another appeal score. If two claims tied on purchase intent in your last test, ask whether you'd actually bet the launch budget that they'd perform identically at shelf. If the answer is no, the survey score already told you it isn't sufficient evidence. As a concrete next step, pull your last claims-test dataset and look for any two claims within half a point of each other on purchase intent. That pair is exactly the case a stated-preference score can't resolve and a randomized design can. If you want to see how that design would run on your specific claim set, the team is reachable at meet. --- # Sample Size Calculator and Guide to Survey Sample Size URL: https://subconscious.ai/blog/methods-and-validation/sample-size-calculator-guide-survey-sample-size Published: 2026-08-27 A research director sizing a discrete choice or conjoint study is really deciding how much to trust the number that comes out the other end, and that decision starts before the study is even fielded. A standard sample size calculator, the kind built into SurveyMonkey or Qualtrics, answers only one question: how tight will the confidence interval be around a single population proportion. It says nothing about whether a choice-based study's preference estimates reflect what people actually do. Getting the sample size right and getting the decision right are two separate problems, and a buyer who conflates them ships a precisely measured, confidently wrong result. - Generic calculators like SurveyMonkey's and Creative Research Systems' surveysystem.com run the Cochran margin-of-error formula: for an unbounded population at 95% confidence and a ±5% margin, that math returns n≈384, and it answers a single-proportion question, not a choice question. - Choice-based conjoint and discrete choice experiments (DCE) estimate multiple part-worth utilities per attribute, so they need Sawtooth Software's Johnson and Orme rule of thumb, n·t·a/c ≥ 500, plus practical floors of 300 respondents per study and 200 per subgroup. - A properly sized sample narrows your confidence interval; it does not close the gap between what people say they'd pay and what they actually pay, a gap hypothetical bias research puts at roughly 2-3x for stated willingness-to-pay, with reported bias magnitudes from 25% to 300%. - The only way to know a preference is real, not just precisely estimated, is to check the design against a human baseline before you trust it. - Subconscious's replication protocol, detailed on the public leaderboard, reports how often simulated studies reproduce the direction and outcome of real ones, which is a different number than anything a sample size calculator can produce. ## What a sample size calculator actually tells you A sample size calculator tells you how confidently wrong you can afford to be, not whether the thing you measured is true. The Cochran formula behind SurveyMonkey's tool and surveysystem.com's long-standing calculator takes three inputs, confidence level, margin of error, and population size, and returns the number of respondents needed to estimate one population proportion within that margin. Plug in 95% confidence, a ±5% margin, and an unbounded population, and both tools return roughly the same answer: n≈384. That's the correct math for "what share of our customers prefer X," a single descriptive number. It was never built to size a study that estimates several attribute-level utilities at once, which is what a conjoint or discrete choice experiment does. ## Why conjoint and discrete choice studies need a different formula A choice-based study needs more respondents, tasks, or alternatives than a descriptive survey because it's estimating a set of relative utilities, not one proportion. Sawtooth Software, the field standard for CBC design, publishes Johnson and Orme's heuristic: n·t·a/c ≥ 500, where n is respondents, t is tasks per respondent, a is alternatives per task, and c is the largest number of levels in any attribute. Sawtooth also recommends a practical floor of 300 respondents per study and 200 per subgroup you intend to analyze separately. A more formal, power-based alternative comes from the PMC guide to DCE sample sizing in healthcare, which derives sample requirements from statistical power rather than a rule of thumb, and academic work continues to refine Orme's heuristic because it can underestimate the sample a given design actually needs. ## How many respondents do you need for a discrete choice experiment? Start with Johnson and Orme's n·t·a/c ≥ 500 and treat 300 respondents as a floor, not a target. If your design has 4 tasks, 3 alternatives per task, and a maximum of 5 levels on any one attribute, the formula wants n ≥ 500 × 5 / (4 × 3), or roughly 209 respondents, but Sawtooth's practical floor still puts you at 300 for the full study and 200 for any subgroup you plan to cut the data by. For anything higher stakes than a rough directional read, especially where you need defensible statistical power rather than a rule of thumb, the PMC guide's power-based approach is the more rigorous path. Either way, this number answers "will my part-worth estimates be stable," not "are those part-worths real." ## Three ways to size a study, compared Best for: pick the generic calculator when you're estimating one proportion off a descriptive survey. Pick Sawtooth's rule of thumb or the PMC power method when you're sizing a conjoint or DCE and need the study to hold up statistically. Pick a causal, baseline-validated design when the decision riding on the result is expensive enough that "statistically stable" isn't the same bar as "actually true." ## What a bigger sample size can't fix A larger n narrows the confidence interval around your estimate; it does not move that estimate closer to what people actually do. This matters most for willingness-to-pay questions, where stated preference research consistently runs high. Hypothetical bias research reports stated WTP running roughly 2-3x higher than revealed WTP on average, with documented bias magnitudes ranging from 25% to 300% depending on the study. Doubling your sample size in that situation buys you a tighter interval around a number that's still wrong in the same direction, by the same margin. Any willingness-to-pay estimate your team relies on should carry that caveat explicitly, because the bias runs one way: stated numbers overstate real willingness to pay, and a bigger sample makes the overstatement more precise, not less real. ## Does a tighter confidence interval mean the preference is real? No. A confidence interval tells you how much sampling noise surrounds your estimate; it says nothing about whether the underlying preference would hold up if people actually had to choose. This is where the sample-size question and the causal question split. A confidence interval from a simulated or fielded experiment covers the estimated effect within that specific study population, not the real market unconditionally, and a flat logit model carries the independence-of-irrelevant-alternatives (IIA) assumption, which can distort preference-share and substitution estimates when alternatives aren't truly independent, a limitation Mixed Logit is specifically designed to relax. None of that shows up in a sample size calculator's output. It shows up when you check the design against what real people actually did. [figure: A branching diagram: a single-proportion survey uses the Cochran margin-of-error formula; a choice-based or DCE study uses Johnson and Orme's rule of thumb or power-based DCE guidance; both paths still require checking the design against real human behavior before trusting the result.] ## How causal validation checks a design against real behavior Validation means running the design as a randomized experiment, analyzing it with discrete choice models, and checking the result against a holdout of real human data, not against a wider or narrower confidence interval. Subconscious's replication protocol shows our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, detailed in the causal fidelity paper. That figure is a validation-set result, not a guarantee for a market that hasn't been studied yet, and it comes with a caveat worth stating plainly: some published human studies used for validation could sit inside a model's training data, which is exactly the kind of contamination risk the replication protocol is built to catch, not a problem it pretends doesn't exist. Method choice matters too. McFadden's discrete choice model, Mixed Logit, and ICLV are estimators, not causal methods on their own; causal identification comes from the randomized manipulation in the experiment design itself, which is why the accurate description is randomized experiments analyzed with discrete choice models. Study-by-study replication performance, broken out by method and category, is published on the public leaderboard. More on how these methods compare in practice is in the methods and validation hub. Before you field your next choice-based study, run it through Johnson and Orme's rule of thumb yourself, confirm you're above Sawtooth's 300-respondent floor, and then ask the harder question a calculator can't answer: has this design, or one like it, been checked against real human behavior. The leaderboard is a reasonable place to look for that second answer. If you want a second read on your specific design, meet the team. --- # Synthetic Respondents: What They Are and When to Trust One URL: https://subconscious.ai/blog/methods-and-validation/synthetic-respondents-what-they-are-when-trust-them Published: 2026-08-27 A synthetic respondent is a language model conditioned on a demographic or behavioral profile, then asked to answer survey and experiment questions the way that profile of person would. The decision a research lead faces is whether a given vendor has proven, against real human behavior, that its output can carry a real decision. Trust a synthetic respondent when its output has been scored in a blind replication of a published randomized experiment, against a measured human-to-human ceiling, with the misses published alongside the wins. - A synthetic respondent is a model-generated stand-in for a survey or experiment participant, built to answer as a defined population would. - Accuracy claimed against a vendor's own survey tests imitation. The vendor wrote both the test and the answer key. - The credible standard is blind replication of a published randomized experiment, scored against a measured human-to-human agreement ceiling of 0.959. - Subconscious's best configuration reaches 0.832 rank correlation on the Hainmueller immigration conjoint - 87% of the 0.959 ceiling measured on that same study - and separately averages 0.73 across the 43 published studies that passed design filters (causal fidelity paper). - Independent research is mixed: LLM agents reproduce qualitative economics results but understate variance and shift with prompt wording. ## What is a synthetic respondent, and when should you trust one? A synthetic respondent is a language model prompted with a persona - age, income, prior purchase behavior, stated preferences - and asked to complete a survey or a discrete choice task as that person. Every vendor in this category makes some version of that model. The difference between them is what happens after the model answers. Trust follows from validation. A synthetic respondent that writes fluent, plausible answers has proven only that the model can write fluent, plausible answers. Trust begins when a vendor replicates a published randomized experiment blind: the study is held out, meaning the model was not scored against it before this run, and the vendor reports the correlation between the simulated result and the real one. A buyer who skips that step has evaluated a demo. It has not been validated as an instrument for a real decision. ## How did the market get here? The category spans several distinct products built for different jobs. Simile and Aaru ground simulated markets in real behavioral and trading data rather than persona role-play alone. Synthetic Users and similar tools generate personas for qualitative product research, mostly aimed at directional usability findings rather than causal estimates. Survey incumbents including Qualtrics and SurveyMonkey have added AI panel features on top of existing survey infrastructure, positioned as a faster version of the same product they already sell. None of this is dishonest by default. It means the category label "synthetic respondents" covers products built for different jobs, validated to different standards, and a buyer has to ask which standard applies before comparing prices. ## What does independent research say about synthetic respondents? Independent research supports the idea that LLM agents can reproduce real behavioral patterns, and separately warns that the same agents distort the details a serious decision depends on. John Horton's NBER working paper found that LLM agents conditioned on demographic profiles reproduced the qualitative findings of classic behavioral-economics experiments, though he cautions against trusting the exact magnitudes (NBER working paper 31122). Bisbee, Clinton, Dorff, Kenkel, and Larson, publishing in Political Analysis, found the opposite failure mode in a different task: synthetic feeling-thermometer responses understated the variance present in real human samples, which produced artificially precise standard errors, and the results shifted with prompt wording and model version (Political Analysis, 2024). A model can get the direction of an effect right and still get its precision wrong in a way that misleads a buyer who reads only the point estimate. A more recent result narrows the gap for a specific task. Maier and colleagues report that semantic similarity rating reached 90% of human test-retest reliability on purchase-intent replication, with response distributions that looked realistic rather than compressed (arXiv, 2025). Read together, these three results show real directional capability alongside real limits in variance and prompt sensitivity. A buyer should ask which of these failure modes applies to the specific decision at hand. ## Why a self-reported accuracy number does not prove anything The conventional pitch in this category is a single accuracy percentage, benchmarked against a survey the vendor ran itself. That number answers a narrow question: can the model imitate the answer the vendor already expected. It does not answer whether the model predicts behavior in a study the vendor did not design and was not scored against ahead of time. A published human-to-human ceiling exists precisely to keep this claim honest. When two independent samples of real humans answer the same study, they agree with each other at 0.959 rank correlation (causal fidelity paper). No simulation should be reported as beating that number, because it means the simulation is claiming to agree with humans more than humans agree with themselves. Any vendor number that arrives without a stated human ceiling and a stated study set is not comparable to anything. ## What does blind replication against a human ceiling actually measure? [figure: Bar chart comparing three rank-correlation figures: a human-to-human agreement ceiling of 0.959, a best-configuration score of 0.832 on one study, and a mean score of 0.73 across 43 published studies.] Subconscious's own validation follows this discipline. On its best configuration, run against the Hainmueller immigration conjoint, it scores 0.832 rank correlation against the published human result, 87% of the 0.959 human-to-human ceiling measured on that same study. Across the 43 published studies that passed design filters, the mean drops to 0.73 (causal fidelity paper). Those numbers, and every miss behind them, are published on the leaderboard rather than filtered down to a favorable headline. One limitation applies to any replication protocol in this category, including this one: published studies can sit inside a model's training data, so a strong replication score is not automatically proof the model reasoned rather than recalled. Design filters and holdout selection lower that risk. They do not remove it, and a buyer evaluating any vendor's leaderboard should ask what the filter excludes. ## How do you tell a validated simulation from roleplay? Ask for the protocol before the pitch: which published studies were replicated blind against a stated human baseline, and whether the losses are public. The comparison below maps the honest differences across the category. A vendor that cannot answer these questions, or that only offers a comparison against its own survey, belongs in one of the top three rows. See case studies for examples of the replication protocol applied to specific decisions, and the methods and validation hub for the underlying estimators. ## Where caveats still apply Randomized experiments, not the estimator, are what produce a causal claim. McFadden discrete choice, Mixed Logit, and ICLV are estimators used to analyze a randomized experiment's results; identification comes from the manipulation in the design, not from the statistical model layered on top. When a discrete choice task uses a flat logit specification, preference-share and substitution results carry the independence-of-irrelevant-alternatives assumption, which can distort estimated switching between similar options; Mixed Logit relaxes that assumption at the cost of more parameters to estimate. Willingness-to-pay estimates from any hypothetical choice task run high relative to what people actually pay, unless the design is incentive-aligned - meaning respondents' choices carry real stakes. A confidence interval produced from a simulated experiment covers the estimated effect within the simulated population under study, not the real market unconditionally; treat it as a bound on the model's estimate, not a guarantee about the world. None of these caveats argue against using synthetic respondents. They argue for asking a vendor to state them. A vendor unwilling to name its own method's limits has skipped the validation protocol. A concrete next step: pull the study list behind any vendor's accuracy claim, check whether it includes a stated human-to-human ceiling and a denominator, and check the leaderboard for a protocol that publishes both the hits and the misses before you buy against a real decision. If you want to walk through how the replication protocol applies to your specific market, get in touch. --- # What reviewers should ask about synthetic market research data URL: https://subconscious.ai/blog/methods-and-validation/reviewers-should-ask-about-synthetic-market-research-data Published: 2026-08-27 A research director vetting a synthetic-respondent vendor has one decision to make: sign off on the data or send it back. That decision turns on a single question: can the vendor produce a calibration study, a case where their synthetic estimate was checked against a real human study and came with an error number attached? Ask for that study and that number before anything else on the checklist. Buyers are being asked to sign off on this data with no shared standard for what "validated" means, even as synthetic respondents move from conference-talk novelty to a paid line item at research vendors. If a vendor can't produce a calibration study, the rest of the standard checklist (panel size, LLM version, prompt design) is secondary. - The question that matters is not "is this sample representative" but "was this specific causal claim checked against a held-out human study, with what error, on what kind of decision." - Demographic representativeness doesn't predict causal accuracy. A 2024 study in Political Analysis found 48% of coefficients from ChatGPT-generated survey data differed significantly from real survey data, and the effect's sign flipped in 32% of those divergent cases, in one comparison built on a single model's outputs (Political Analysis). - General-purpose LLM panels deviate from human benchmark data by 0.87 to 0.88 standard deviations; a calibrated model deviated by 0.07, in a single replication of one benchmark (Greenbook). - Silicon samples, Sarstedt and colleagues' term for LLM-generated survey respondents, are on firmer ground for upstream pretesting and pilot work than for confirmatory claims a decision will actually rest on (Psychology & Marketing). ## Why representativeness checklists test the wrong thing The conventional buyer checklist asks about panel provenance, demographic balance, model transparency, and data handling, the questions ESOMAR's buyer-vetting guidance also centers. That's the right list for vetting a traditional panel and the wrong list for vetting a causal claim. A synthetic sample can match census age, income, and geography and still get the underlying relationship backwards. That's not hypothetical: it's what a 2024 study in Political Analysis found when researchers compared ChatGPT-generated survey responses to real ones on matched questions, where the demographics lined up but nearly half the coefficients didn't (Political Analysis). Representativeness is a property of the sample. Causal accuracy is a property of how the model responds to a manipulation, an intervention, a price change, a feature add. Those are different things to test, and a checklist built for the first one won't catch a failure in the second. ## Why do synthetic panels get demographics right and the causal effect wrong? They get demographics right because matching a population's composition is a sampling problem, and they get causal effects wrong because predicting how that population would respond to a specific change is a different, harder problem that most LLM-based panels have never been checked against. Greenbook's replication of the Paxton and Yang benchmark makes the gap concrete: general-purpose models like GPT and Gemini deviated from human survey answers by 0.87 to 0.88 standard deviations, while a model calibrated specifically for this task deviated by 0.07, in this single replication of one benchmark, and the uncalibrated models systematically inflated top-two-box scores and correlations between variables (Greenbook). That inflation matters for a buyer because it doesn't just add noise, it adds a consistent upward bias that would make a mediocre concept look validated. [figure: Bar chart showing general-purpose LLMs deviating from human survey data by 0.87 to 0.88 standard deviations, compared to 0.07 standard deviations for a calibrated model.] ## What should you actually ask a synthetic research vendor? Ask for a calibration study: a specific instance where the vendor ran the same randomized experiment on a synthetic population and on a real human baseline, and report the error between the two. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). That number describes a validation set, not a guarantee for a market you haven't tested yet, and it's worth pressing any vendor on the same distinction: what set was this measured on, and how often is it re-run. It's also worth asking directly whether any of the published studies used for calibration could have been in the underlying model's training data, since a model that has memorized a study's published result isn't demonstrating replication, it's demonstrating recall. A credible vendor will have a protocol for that risk, not a denial that it exists. ## Two ways to vet a synthetic sample Neither check replaces the other. A vendor should pass both, but only the second one tells you whether the number they're handing you is safe to make a decision with. ## Where is it safe to trust a synthetic respondent today? It's safest for early-stage, upstream work, screening concepts, drafting a questionnaire, exploring which messages are worth testing further, and riskiest for confirmatory claims a real budget decision depends on. Sarstedt and coauthors, writing in Psychology & Marketing, make this the center of their guidance: silicon samples are defensible for pretesting and pilot studies, and risky when treated as a substitute for a confirmatory study (Wiley). That maps onto the calibration question directly. A vendor with no calibration study might still be useful for a first pass at message testing. The same vendor without a calibration study has no business producing the number that decides which action a team takes with real money behind it. ## What a calibration number needs to specify A calibration number is only as useful as what it says it covers. A confidence interval from a simulated experiment describes the effect within that simulated population; it doesn't bound the real market unless the vendor can show the simulated population tracks the real one on the dimensions that matter for this decision. If a substitution or preference-share question runs through a standard multinomial logit, it carries the independence of irrelevant alternatives assumption, meaning adding or removing an option shouldn't change the relative odds between the others, an assumption that breaks down often enough in real markets that it's worth asking whether the vendor's estimator relaxes it. Mixed Logit does; a flat logit doesn't. And the estimator itself (McFadden discrete choice, Mixed Logit, ICLV) isn't what makes a claim causal. Identification comes from the randomized manipulation built into the experiment design; the estimator just describes the response once the intervention has run. ## Checking the vendor's track record before you sign Ask to see the vendor's calibration results across more than one study, not just the case study they lead with in a sales deck. Subconscious publishes its accuracy scores across studies on a public leaderboard, which is the kind of artifact worth requesting from any vendor claiming causal validity: a running, checkable record rather than a single anecdote. Compare it against how the vendor talks about their own numbers. A vendor that reports a single flattering case study without a holdout methodology behind it is asking you to trust an anecdote. A vendor that publishes ongoing accuracy against human baselines, and names the limitation on the studies where it fell short, is giving you something you can actually audit. The concrete next step: before your next synthetic research engagement, ask the vendor for their calibration study, the human baseline it was checked against, the error number, and whether that number covers the class of decision you're about to make. If they can't produce it, treat the output as a pretest, not a confirmatory result. More on how the underlying replication protocol works is on the methods and validation page. If you want to walk through a specific decision with someone who can show the calibration behind it, reach out. --- # How to get the most out of video interviewing in surveys? URL: https://subconscious.ai/blog/methods-and-validation/get-most-out-video-interviewing-surveys Published: 2026-08-27 A research buyer weighing video interviewing for an upcoming study is really deciding how much weight to put on what respondents say on camera versus what a randomized experiment shows they actually choose. The direct answer: video interviewing produces richer testimony and higher respondent satisfaction, but the camera itself changes the data. Respondents round numbers more and give more socially desirable answers on video than they do on a web survey. So the way to get the most out of it is to use it for hypothesis generation and stakeholder buy-in. Then validate the resulting claims with a randomized experiment that measures choice, not self-report. - Use video interviews to surface language, objections, and hypotheses, not to measure demand. - Expect on-camera bias: a 1,067-respondent study found live video interviewees round more numerical answers and give more socially desirable, less sensitive responses than web-survey respondents. - AI-moderated scale changes volume, not causal validity: more interviews still means more stated preference, not a randomized manipulation. - To find out what people will actually do, run a randomized discrete choice experiment, not a bigger interview program. - Before trusting any simulated or synthetic result, ask for its replication accuracy and how that number is defined. ## Does video interviewing change what respondents say? Yes. The same peer-reviewed study vendors cite for "video improves engagement" also found that being on camera changes the substance of the answer. In a study of 1,067 respondents comparing live video, prerecorded video, and text web surveys, live video interviewees rounded more numerical answers and gave more socially desirable, less sensitive responses than web-survey respondents, even though they reported higher satisfaction with the experience (mda, 2023). A cited number is only useful once its boundaries are on record. That sample was drawn for survey-methodology research, not a commercial buying decision, so the exact bias sizes may not carry over to a product or pricing study, but the mechanism behind them, a person watching changes the answer, does. Peer-reviewed guidance on live video design attributes this to interviewer presence itself: a moderator on screen introduces time pressure and social presence effects that a text or async format doesn't have (Survey Practice). So the richness a video interview adds and the bias it introduces come from the same source: a person is watching. Warmer testimony and skewed testimony are not two separate effects to weigh against each other. They're the same camera. "those interviewed via live video were less likely to select the same response for all statements in a battery (non-differentiation) and reported higher satisfaction with their experience but provided more rounded numerical (presumably less thoughtful) answers and selected answers that were less sensitive (more socially desirable)" Conrad and colleagues, Methoden, Daten, Analysen (source) ## Two vendor camps, and a new one scaling with AI Video interviewing in market research has split into two camps for years, and neither solved the core trade-off between depth and scale. Voxpopme built async, self-recorded video depth: respondents film themselves answering prompts on their own time, which scales better than live moderation but loses the follow-up questions a moderator would ask. Discuss.io built live-moderated video panels: real depth through real-time follow-up, but bound by how many interviews a human moderator can run in a day. The disruption is AI moderation. Listen Labs, founded in 2023, has conducted over one million AI-moderated video and voice interviews (VentureBeat), a company-reported figure relayed through VentureBeat, not independently audited. That volume changes the scale side of the trade-off. It does not change the depth-versus-truth question the mda study raised: an AI moderator asking better follow-up questions still can't stop the camera from changing the answer. ## Does AI-moderated video interviewing at scale fix the causal gap? No. Scaling video interviews with AI moderation multiplies the volume of testimony, not the causal validity of what it tells you. A million AI-moderated interviews is still purposive, stated-preference data: respondents opted in, nobody was randomly assigned to a condition, and there's no counterfactual against which to measure what they would have said or done otherwise. Qualitative methodologists have been raising a related warning for years about sample size in interview studies specifically: a systematic 15-year review found that qualitative interview sample sizes are predominantly characterized as insufficient, and justifications for "how many interviews is enough" are routinely absent or ad hoc (BMC Medical Research Methodology). Scale doesn't resolve that problem, it just makes the ungrounded claim bigger. More interviews narrow the confidence interval around what people said. They do nothing for the gap between what people said and what they'll actually do. [figure: Chain diagram with two trend lines. Interview volume climbs steadily upward. The presence of a randomized manipulation stays flat at zero. The two lines never converge no matter how many interviews are added.] ## Video interviews versus a randomized experiment ## When does video interviewing earn its place in a survey program? It earns its place early, before you've decided what to test. The standard advice for running a good video interview still holds: ask open-ended questions instead of yes-or-no ones, keep prompts tight, pilot before fielding, and start with respondents who'll actually record comfortably on camera (Survey Practice). What that advice leaves out is the handoff. A good video interview program produces a shortlist of hypotheses and language, not a demand estimate. The mistake is letting a well-run interview study, or a well-run AI-moderated one, stand in as the evidence for a decision that has real budget behind it. That's where the study needs a second stage, one built to measure choice under a randomized manipulation rather than to collect a better-worded opinion. ## How does a randomized experiment answer the question video can't? A randomized experiment answers it by manipulating the thing you're actually deciding about (a price, a feature, a message) and measuring which action changes the outcome, instead of asking respondents to describe their own reaction to it. Subconscious runs these as randomized experiments on a simulation of the market, then checks the simulation against real human behavior. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). A raw performance number is marketing without its limits stated plainly. That's a validation-set result, not a guarantee for a market that hasn't been tested before. Published studies can also sit inside a model's training data, which is why the replication protocol tracks holdout studies rather than trusting any single published number blind. The public leaderboard tracks these runs model by model, so a buyer can check current accuracy before putting a number from a deck in front of their own stakeholders. The choice data from these experiments is analyzed with McFadden discrete choice models, Mixed Logit, and ICLV: estimators that recover preference structure from the choices people made, not causal methods on their own. The causal claim comes from the randomized manipulation in the experiment design, not from the estimator. A plain logit model carries the IIA assumption, that adding or removing an alternative doesn't change the relative odds between the others, which breaks down fast in real substitution patterns; Mixed Logit relaxes that assumption, which is why it's the standard choice for preference-share and substitution questions. The leaderboard puts every miss on the same page as every hit, and this confidence interval is one of the misses worth stating. Every effect reported this way carries a confidence interval that covers the effect inside the simulated population it was measured in, not a bound on the real market until it's been checked against a human baseline. More on how the estimators and the randomized designs fit together is in the methods and validation hub. Before fielding another round of video interviews, pull the transcripts from the last study and tag every load-bearing claim as either "said" or "chose." If most of the claims driving the decision are "said," that's the gap a randomized experiment closes, not a bigger interview budget. When you're ready to run a randomized experiment against your own market, meet is the place to start. --- # Creating a correlation matrix for conjoint simulations URL: https://subconscious.ai/blog/methods-and-validation/creating-correlation-matrix-conjoint-simulations Published: 2026-08-27 A senior insights buyer evaluating a conjoint simulator needs to know what a correlation matrix can and cannot tell them before they trust its forecast. A statistic published without its limits is marketing copy. The direct answer: a correlation matrix confirms your experimental design is statistically well-behaved, or that simulated shares aren't tangled together by a modeling artifact. It cannot confirm the simulator predicts what real buyers would choose. That requires comparing simulated results against replicated human studies, not auditing the matrix's off-diagonal values. - A correlation matrix run at design time (Sawtooth's rms cor and rel effec) confirms the choice tasks in your survey aren't collinear enough to blow up your effect estimates. - A correlation matrix run at output time, across simulated shares, flags when two concepts move together so tightly the simulator is likely double-counting them, the Red-Bus/Blue-Bus problem. - Naming a gap in a diagnostic is what lets a buyer check it before trusting a forecast. Neither check tells you whether the underlying choice model matches how real people choose. That's a separate validation step against a human baseline. - Fixing double-counting is a modeling choice, First Choice or Randomized First Choice simulation instead of plain Share-of-Preference logit, not something you find by reading a matrix more carefully. - Correlated price and quality attributes in your design are often intentional, conditional pricing exists to avoid dominated, unrealistic profiles. ## Two places a correlation matrix shows up in conjoint work Market simulators take part-worth utilities from a conjoint study and convert them into simulated choice shares, so a team can test a hypothetical product scenario before it ships (Sawtooth Software, 2019). Practitioners build correlation matrices at two separate points in that workflow, and it's easy to conflate them because both produce a matrix of numbers that looks like the same kind of check. The first point is design time: does the survey itself produce clean, independent signal. The second is output time, after the simulator has run: do the simulated shares move independently, or does the model appear to be counting the same preference twice. A matrix that looks clean at one point says nothing about the other. [figure: Two boxes labeled design-time and output-time, each showing what a correlation matrix measures at that stage, with neither box connecting to human-behavior validation.] ## What Sawtooth's design-time diagnostics actually measure Sawtooth Software, the dominant CBC vendor, reports two statistics from a respondent's design correlation matrix: rms cor, the root-mean-square of its off-diagonal elements, and rel effec, relative estimation efficiency calculated from the inverse of that matrix. Both are core design-quality statistics in Sawtooth's CBC output. These numbers answer one question: did the experiment collect signal clean enough to estimate stable part-worths. They say nothing about whether the resulting simulator, once you feed it new product scenarios, predicts what people would actually pick. A well-designed experiment can still sit inside a poorly validated simulator. ## What the output-time correlation matrix catches: the Red-Bus/Blue-Bus problem A correlation matrix of simulated shares exists to catch one specific failure: near-identical concepts inflating each other's combined share because the model can't tell they're substitutes rather than distinct options. Sawtooth's classic illustration repaints half a bus company's fleet blue and adds it back into a Share-of-Preference simulation. The result nearly doubles the bus company's predicted total share, because the logit model double-counts the two nearly identical bus options instead of treating them as substitutes for the same trip (Sawtooth Software, Red-Bus/Blue-Bus). This is the independence-of-irrelevant-alternatives (IIA) assumption failing. A flat Share-of-Preference logit assumes every option is equally substitutable with every other option, so adding a near-clone of an existing product inflates the total share of that product line rather than splitting share the way it would in a real market. A correlation matrix of simulated shares will show the red bus and blue bus moving together almost perfectly, which is the signal practitioners look for. But catching the symptom isn't the same as fixing the cause, and it doesn't tell you whether the model was ever right about substitution in the first place. ## Fixing substitution bias is a modeling decision, not a matrix-reading exercise Once a correlation matrix flags a Red-Bus/Blue-Bus pattern, the fix lives in the choice model, not in re-reading the matrix more carefully. Sawtooth's own guidance is direct: First Choice simulation models are not subject to IIA bias at all, and Randomized First Choice shows much less IIA bias than standard Share-of-Preference logit models (Sawtooth Software, 1999). That's the practical decision point for a buyer evaluating a simulator: which share-prediction method does it run by default, and can you switch it. A simulator that only offers plain Share-of-Preference logit will keep producing this distortion regardless of how disciplined its correlation diagnostics look. ## Correlated price and quality attributes are sometimes the point Not every correlation in a design is a defect to be designed away. If price and quality attributes are varied independently, the experiment can generate profiles no real buyer would recognize, a premium feature bundle offered at a bargain price, for instance, which the market itself would never present. Sawtooth's 2025 guidance on price treatment recommends conditional or incremental pricing specifically to reintroduce the price-quality correlation buyers expect from real markets, avoiding these dominated, unrealistic profiles (Sawtooth Software, 2025). So a design-time correlation matrix showing some correlation between price and quality levels isn't automatically a red flag. The right question is whether that correlation was put there deliberately, to match how the market actually prices products, or whether it leaked in as an accident of a poorly randomized design. ## Does a clean correlation matrix prove your simulator is accurate? An honest answer here costs a claim, and it goes on the page anyway. No. A clean correlation matrix, at either the design stage or the output stage, proves that the numbers are well-behaved. It does not prove the choice model behind those numbers matches how real people choose. This is the gap that matters for a buyer making a real purchase decision. Red-Bus/Blue-Bus is diagnosed by looking at share correlations, but it's caused by a model-validity failure: a Share-of-Preference logit that was never checked against observed human substitution behavior. That model can produce a mathematically tidy correlation matrix at every stage and still misrepresent how buyers would actually split their choices between a new product and an existing one it resembles. Auditing the matrix optimizes the wrong layer. It tells you the arithmetic is internally consistent, not that the arithmetic describes the market. ## How do you actually validate a conjoint simulator against real behavior? You compare the simulator's predictions against a held-out human study and measure how often it reproduces the same direction and outcome, not against its own correlation diagnostics. That's a different kind of check: it requires an independent human baseline, run separately from the simulation, and a defined replication protocol for scoring agreement. This is also where method choice matters, precisely stated. McFadden's discrete choice model, Mixed Logit, and ICLV are estimators used to analyze choice data, not causal methods in themselves. Causal identification comes from the randomized manipulation built into the experiment design, the same randomization that lets a study distinguish "people chose this product" from "people chose this product because of this specific attribute." Written accurately, the claim is randomized experiments analyzed with discrete choice models, not "causal methods like DCE." Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, measured against a set of held-out human studies (the causal fidelity paper). The misses sit on the same public leaderboard as the hits, so two of them are worth stating directly. Two limitations matter here. First, it's a validation-set result, not a guarantee for a new, unseen market; a simulator that replicates well on past studies can still miss on a market structure it hasn't been checked against. Second, published human studies can sit inside a model's training data, which would inflate apparent replication if the model had simply memorized the answer; the replication protocol is built to guard against this, but the underlying risk doesn't disappear just because a protocol exists. Results and methodology are published on the leaderboard, which lets a buyer check replication performance directly instead of taking a vendor's summary number on faith. More on how this validation work is structured is on the methods and validation hub. ## Where each check belongs Before trusting any simulator's forecast, ask for its replication rate against held-out human studies, not just its design diagnostics or a clean share correlation matrix. If a vendor can't produce that number with a defined method, treat the forecast as unvalidated regardless of how disciplined its matrices look. If you want to walk through how a specific validation compares against your current simulator setup, meet with us. --- # Bring your own respondents URL: https://subconscious.ai/blog/implementation-and-procurement/bring-own-respondents Published: 2026-08-27 A pricing lead deciding how to staff next quarter's conjoint study faces a real fork: buy sample from a marketplace panel like Cint, Dynata, Prolific, or CloudResearch, or route the survey to the company's own CRM list, customer panel, or app users. Using your own respondents solves who answers a survey, not what the answer is worth: a verified customer list still produces stated preference data, filtered by who chose to respond, unless the study underneath it is built as a randomized experiment. The decision that actually protects a launch bet is the experimental design, not the respondent source. - Bringing your own respondents (BYOP) means routing a survey or conjoint study to a company's own CRM list, customer panel, or app users instead of a third-party marketplace panel. - A number without its limits is a marketing claim. BYOP buys data hygiene: known identities, lower bot risk, higher engagement. It does not buy causal validity. - Panel fraud is real and getting worse: NORC puts nonprobability sample fraud at 15 to 30 percent industry-wide, spiking to 45 percent on some platforms and 90 percent in social-media-recruited samples (NORC). - A verified respondent list run through a rating-scale or ranking survey still returns stated preference filtered by selection bias, not a causal estimate of what a price or feature change will do. - The actual fix is a randomized experimental design analyzed with discrete choice models, independent of where the respondents came from. ## What "bring your own respondents" actually means BYOP means a company supplies its own respondent pool, CRM contacts, an existing customer panel, or app users, instead of paying a marketplace for sample. Platforms like Qualtrics now treat this as a first-class option next to marketplace panel, not a workaround. A parallel market of "branded panel" vendors (Rival, Reach3) and AI-moderated qualitative tools has made owning a respondent base cheap enough to default to, not a workaround. That's why BYOP has moved from a niche choice to a live procurement decision for buyers who used to just default to a marketplace panel. ## Why buyers are choosing BYOP over marketplace panels now The shift is trust-driven, not cost-driven. GRIT's 2025 Insights Practice Report flags rising data-quality concerns, attributed to synthetic respondents and Gen Z survey fatigue, now among the industry's top five priorities (GRIT via NewtonX). CloudResearch estimates 30 to 40 percent of online survey responses are fraudulent or unusable, and finds that most current fraud is still coordinated human click-farm activity, not AI bots (CloudResearch). NORC separately estimates roughly 40 percent of nonprobability survey interviews in 2025 were fraudulent (NORC). These are nonprobability-sample estimates with contested measurement methodology, not a single agreed industry rate, so read them as directional. A known CRM list sidesteps that specific problem because the buyer already knows who is on it. ## Does bringing your own respondents solve the panel fraud problem? Yes, largely, for fraud specifically. A CRM list of verified customers removes the click-farm and bot risk that drives NORC's 15 to 90 percent fraud range, because identity is already known rather than self-reported to a marketplace. That is a genuine, measurable improvement in data hygiene. But fraud and validity are different failure modes, and fixing one says nothing about the other. ## What bringing your own respondents doesn't fix Naming a failure mode is what lets a buyer check it before they rely on it. Removing bots from your list guarantees you're talking to real people. It does not guarantee that what you're measuring predicts a real decision. A self-selected, engaged BYOP list run through a rating scale or ranked-preference survey still returns stated preference: what respondents say they'd do, filtered by who chose to answer at all. That's correlation with a selection bias baked in, not a causal estimate of what a price change, feature cut, or message swap will actually do to choice. A CRM-sourced conjoint that asks customers to state a willingness to pay runs into the same standard problem regardless of respondent source: stated willingness to pay runs high relative to real purchase behavior, unless the choice itself is incentive-aligned. [figure: A flow diagram showing a CRM list leading to a verified real respondent, which then splits into two paths: a rating or ranking survey producing stated preference with selection bias, or a randomized experiment producing a causal estimate of choice.] ## BYOP, marketplace panel, or randomized experiment: how the three compare ## Where does CRM data fit inside a causal research design? CRM respondents are most valuable as a human baseline, not as the whole study. McFadden discrete choice, Mixed Logit, and ICLV are estimators, not causal methods on their own; the causal identification comes from randomizing what respondents are exposed to inside the experiment, not from where the respondents were recruited. A randomized experiment run partly on synthetic populations and validated against your own CRM respondents as a holdout gets you both: known, real customers and a design that isolates the effect of the change you're testing. Subconscious's own validation protocol reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. The misses sit on the public leaderboard right next to the hits. That is a validation result, not a guarantee for a brand-new market. It also doesn't eliminate the risk that a published human study sat inside a model's training data, a risk the replication protocol is built to test against, not one the number pretends away. Results by domain are public on the leaderboard, and a confidence interval reported from a simulated experiment covers the effect within the population simulated, not the real market unconditionally. ## What should a buyer check before betting a launch on BYOP data? Ask whether the study varies something, not just asks about it. If the survey shows every respondent the same product, price, or message and asks them to rate or rank it, the design cannot separate cause from the respondent's prior opinion, no matter how clean the list is. If it randomizes what different respondents see, whether run on your CRM, on synthetic populations, or a blend, that's the design that produces a defensible causal estimate. For deeper background on how simulated and human-run studies are evaluated against each other, see the methods and validation hub, and for how BYOP-style approaches stack up against panel and simulation alternatives more broadly, see comparisons. Before the next study goes out, pull up the current survey instrument and check one thing: does any question vary a price, feature, or message across randomized groups, or does every respondent see the same stimulus and get asked to rate it? If it's the latter, the respondent source, however clean, will not turn the result into an answer you can bet a launch on. If you want a second opinion on a specific study design, talk to us. --- # Case Study: Willingness to Pay For Ingredients URL: https://subconscious.ai/blog/case-studies/case-study-willingness-pay-ingredients Published: 2026-08-27 A packaged-food pricing lead needs one number before approving a clean-label launch price: how much extra shoppers will actually pay for the claim. Not what they said in a survey. That number belongs in a pricing model only after it's been checked against real purchase behavior and carries a confidence interval, not when it's a single conjoint run handed straight to finance. Most published ingredient-claim studies skip that check. - Ingredient-claim WTP studies (clean label, organic, free-range, functional claims) are usually run once as a conjoint or discrete choice experiment, and the point estimate gets reported as the price a brand can charge. - Stated WTP runs high: a meta-analysis of 28 stated-preference studies found a median hypothetical-to-actual ratio of 1.35, meaning real buyers typically pay less than survey respondents claim they will (Murphy et al., Environmental and Resource Economics). - Published examples show the spread this creates: clean-label yogurt WTP of $2.54 to $3.53 per 32oz unit (PMC), free-range poultry premiums of 43 to 93 percent depending on category (DCE food research review), and organic/natural premiums that shift with what information respondents saw right before the choice task (Wallace & Huffman). - A number without its limits attached reads as marketing. A WTP number becomes a pricing input only after it's checked against real behavior and scored with a confidence interval. On the leaderboard, our best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against the published human result, where two independent human samples reach 0.959), with a mean of 0.73 across the 43 studies passing design filters: a validation result, not a guarantee for any new market or claim (the causal fidelity paper). - The buyer's action before approving a premium: ask what the estimate was validated against, not how large the sample was. ## The decision: pricing an ingredient claim before you commit to it The decision a pricing lead owns here is narrow and expensive: approve a premium on a reformulation or launch based on a WTP figure, or send the study back for validation first. Getting it wrong in either direction costs money. Either a launch priced below what the market would bear, or a claim that never earns back the reformulation spend because the premium was never real. Ingredient-claim WTP studies are one of the most common applied uses of discrete choice research in food, almost always run to justify a price move before it happens. Trust the study that was checked against purchase data, not the one with the largest sample size. See more examples in the case studies hub. ## What the standard ingredient-claim case study measures The conventional version of this study runs a conjoint or DCE panel once, estimates a per-unit dollar premium for the claim, and reports it as the price the brand can charge. A 2021 mixed-logit study on 250 yogurt consumers is a representative example: clean labeling was worth $2.54 to $3.53 per 32oz unit, and it also reduced the choice penalty consumers assigned to poor texture relative to longer-ingredient-list yogurts (PMC). Belgian DCE work on free-range poultry found premiums ranging from 43 to 93 percent depending on the product category, a wide enough band that picking the wrong end of it changes the launch math (DCE food research review). Wallace and Huffman studied natural, organic, and conventional foods. Premiums moved materially depending on which information treatment respondents saw right before the choice task (Wallace & Huffman). Conventional write-ups tend to bury that detail under model-fit statistics. ## Why does a one-shot conjoint overstate what shoppers will pay? A one-shot conjoint overstates WTP because stated preference is not the same behavior as spending real money. The gap runs in a known direction: stated WTP typically runs high, at a median ratio of 1.35 to actual spending (Murphy et al., Environmental and Resource Economics). Add the Wallace-Huffman finding that premiums shift with the information respondents saw right before the choice task, and a single conjoint fielding isn't measuring a stable number. It's measuring a framing-sensitive snapshot. The estimators used to produce it (McFadden discrete choice, Mixed Logit, ICLV) recover preference structure from that snapshot. They aren't causal methods themselves. The causal claim has to come from a randomized manipulation inside the experiment design, not from the estimator's model fit. ## How does a WTP number get checked before it reaches pricing? It gets checked by running the estimate through a replication step against real human behavior and attaching a confidence interval, not by reporting the point estimate alone. The chain has enough steps that it's easy to compress into "we validated it" without saying what that means, so it's worth laying out in full. [figure: Five-step chain showing a willingness-to-pay estimate moving from randomized experiment design through simulated choices, comparison to a human holdout study, a replication accuracy score, and finally a confidence interval before it is usable for pricing.] Replication, in the sense the leaderboard tracks it, is scored against the published human result and a measured human-to-human ceiling: the best configuration reaches 87% of that ceiling on one study (0.832 against a 0.959 ceiling), with a mean of 0.73 across the 43 studies passing design filters. That is a validation result, not a guarantee that any new market or claim will replicate at that rate (the causal fidelity paper). Naming a failure mode is what lets a buyer check it before relying on the number. Published human studies can sit inside a model's training data, which would make a replication look stronger than it is. The replication protocol is built to test against that risk, but the risk itself doesn't disappear because a protocol exists to check for it. ## What does the confidence interval actually cover? The scope of a confidence interval is worth stating plainly so a buyer can check it before using the number. It covers the effect within the simulated population the study was run on, not the real market unconditionally. A confidence interval attached to a simulated experiment tells a pricing lead how much the estimate would vary if the simulated study were repeated, given the population and design used. It doesn't extend that guarantee to shoppers who weren't represented in the design. It doesn't correct for hypothetical bias on its own, either. That requires an incentive-aligned design or a validation step against real behavior, a separate check from the interval itself. ## Does the premium hold up when claims compete on the same shelf? A pricing lead needs to see a modeling limit next to the claim it affects. Only if the model used to estimate substitution accounts for it, which a flat multinomial logit does not do well. A standard logit model carries the independence of irrelevant alternatives (IIA) assumption, which implies that adding a new competing claim (say, a free-range option next to a clean-label one) pulls share from every existing option in fixed proportion, a pattern real shelves rarely follow. Mixed Logit relaxes that assumption by allowing preferences to vary across the simulated population, which is part of why it shows up in the yogurt and food-research work cited above, but relaxing IIA is not the same as validating the resulting share estimate against real substitution behavior. Any preference-share or substitution claim built on a flat logit needs the IIA assumption named next to it, not left implicit in the model choice. ## What should a pricing lead ask for before approving a premium? Ask for the validation source, the confidence interval, and the bias direction before the sample size. Concretely: was the WTP estimate checked against a real-behavior holdout, or only reported off the conjoint itself? What does the confidence interval cover, the simulated population or a claimed bound on the real market? Was the design incentive-aligned, or does the number need a hypothetical-bias discount before it goes into the P&L? And if the study compares multiple claims on the same shelf, was substitution modeled with an approach that accounts for IIA, or with a flat logit that assumes it away? A study that answers all four without hedging is a decision input. One that answers none of them is a survey artifact dressed as one. Before approving a premium, pull the confidence interval and the validation source for any WTP number your team has been handed. Check whether the design controlled for information order the way the Wallace-Huffman work flagged. To talk through how a validated ingredient-claim study gets built, reach the team. --- # Mini-lecture: Conjointly's guide to sample selection URL: https://subconscious.ai/blog/methods-and-validation/mini-lecture-conjointlys-guide-sample-selection Published: 2026-08-27 A research director deciding whether to follow Conjointly's sample-selection guidance is really deciding one thing: how many respondents a study needs, and whether that respondent pool will tell the truth. Conjointly's guide, like the rest of the category, answers only the first half. It sizes a sample against a target confidence interval; it says nothing about whether the people filling out that survey exist, pay attention, or would act the same way once money or a real choice is on the line. A study can hit its target N exactly and still produce a causal effect that isn't real. - Conjointly's sample-size guidance sits in the same lineage as Sawtooth's Johnson rule of thumb (n·t·a/c ≥ 500), with 300 respondents as the common default and roughly 200 per subgroup when segments are reported separately. - Academic discrete choice experiments show no real consensus on N: a systematic review of healthcare DCEs found sample sizes from 10 to 3,727, median 294. - The bigger risk now sits upstream of sample size: opt-in online panels carry roughly double the error of probability-based panels, and AI agents defeat the attention checks panels use to certify a respondent as human. - A precisely sized sample of unreal respondents still produces a precise, unreal effect. Sample-size math and respondent-reality checks are two different questions, and most guides only answer the first. - The check that closes the gap is replication against a real human study, not a larger N. ## What Conjointly's sample-size guidance recommends Conjointly's guidance, like most in the category, treats sample selection as a statistical power problem. The reference point most of the field converges on is Sawtooth's Johnson rule of thumb for choice-based conjoint studies: n·t·a/c ≥ 500, where n is respondents, t is tasks per respondent, a is alternatives per task, and c is the largest number of levels in any attribute. In practice this collapses to a familiar default: roughly 300 respondents, with about 200 per subgroup if the plan calls for reporting results by segment (Sawtooth Software). This is useful math. It tells a buyer how much precision a given N buys, and how that precision degrades as tasks, alternatives, or attribute levels increase. What it does not do is say anything about who fills out those 300 or 200 surveys. ## Why does academic practice show such a wide range of "right" sample sizes? Because there isn't one right sample size, only a right size for a given effect and a given population, and researchers disagree on both constantly. A systematic review of discrete choice experiments in healthcare found published sample sizes ranging from 10 to 3,727 respondents, with a median of just 294 (PMC). That spread is not evidence of sloppiness. It reflects that the "correct" N depends on the effect size researchers expect to detect, the design's efficiency, and how finely they plan to slice the results. What the spread does undercut is the idea that a single formula, applied mechanically, settles the sample-selection decision. Johnson's rule and the 300-respondent default are reasonable starting heuristics, not proof that a study's findings are real once the target N is hit. ## Why does sample size say nothing about whether your respondents are real? Because N measures how many observations you have, not what those observations are worth, and the value of each observation has been dropping. Pew Research compared six online panels across 29,937 U.S. adults and 28 benchmark variables and found that opt-in samples carry roughly twice the average error of probability-based panels (Pew Research Center). That gap exists before anyone touches a sample-size formula. Layered on top of it: CloudResearch's 2025 analysis found that AI agents now pass thousands of standard attention checks with near-perfect accuracy, defeating the exact screens researchers rely on to certify a respondent pool as clean (CloudResearch). A sample-size formula has no term for either of these problems. It assumes the respondents behind N are what they claim to be. [figure: Bar chart showing opt-in panel error at roughly double the error of probability-based panels.] ## Sample size versus sample reality These are two separate audits, and a buyer needs both before trusting a result. ## What does causal validation check that a sample-size formula can't? It checks whether the effect a study finds actually reproduces what real people do, not just whether the sample was big enough to estimate it precisely. Subconscious runs randomized experiments on a simulated population and analyzes them with discrete choice models, McFadden discrete choice, Mixed Logit, and ICLV among them. These are estimators, not causal methods in themselves; the causal identification comes from the randomized manipulation built into the experiment design, not from the choice of model. Mixed Logit and ICLV also relax the independence-of-irrelevant-alternatives assumption baked into a flat logit, which matters directly if the question is about substitution between options rather than a single average effect. The validation number that matters here is replication accuracy: how often a simulated study reproduces the direction and outcome of the original human study, measured against a set of held-out human studies. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). Two caveats belong next to that number, not after it. First, it is a validation-set result, not a guarantee for a new, unstudied market; every new study still needs its own check. Second, published human studies can sit in a model's training data, which would inflate a replication score if left unaddressed; the protocol behind the 0.832 and 0.73 figures holds out studies and populations to control for this, but the concern does not fully disappear just because a protocol exists. A confidence interval computed from a simulated experiment covers the effect within that simulated population; it does not, on its own, bound what the real market will do. The leaderboard publishes results across methods and studies so a buyer can see how replication accuracy holds up outside a single case. ## A decision checklist for a sample-selection call - Start with Johnson's rule or Conjointly's 300-respondent default to size the study; it's a reasonable floor, not a finish line. - Ask which panel the respondents come from and whether it's probability-based or opt-in; opt-in carries roughly double the error, before any other issue shows up. - Ask what stops an AI agent from completing the survey as a human; standard attention checks no longer do this reliably. - Ask whether the resulting preference estimates have been checked against real human behavior, not just internal statistical consistency. - If the study reports willingness-to-pay, treat stated figures as directionally high unless the design is incentive-aligned; hypothetical bias runs in a known direction. ## Where does this leave a buyer choosing between guides? It leaves the sample-size guide as necessary but incomplete, and the reality check as the part most guides skip. Conjointly's mini-lecture, Sawtooth's rule of thumb, and the healthcare DCE literature all answer "how many." None of them answer "are these respondents, and their stated preferences, real." A study can be correctly sized and still be measuring something that doesn't exist outside the survey tool. The methods and validation hub covers how to check the second half of that question once the first half is settled. Next step: before running a new choice study, pull the panel's own quality documentation and ask two questions directly, what share of respondents failed an attention check in the last quarter, and how the panel screens for AI-generated responses. If you want a second opinion on a specific market or design, meet the team. --- # Criticisms and counter-criticisms of Kano model URL: https://subconscious.ai/blog/methods-and-validation/criticisms-counter-criticisms-kano-model Published: 2026-08-27 A product leader deciding whether to greenlight a feature for the next roadmap cycle needs a straight answer on whether Kano's basic, performance, and delighter categories can be trusted for that decision, or whether the mounting criticism against the model means it should be retired. Short answer: the instability and wording problems raised since 2015 are real and partially fixable, but they aren't the main issue. Kano's dual-response survey was built to record how satisfied someone says they'd feel about a feature in isolation, not to isolate the causal, traded-off effect that feature has on what the person actually chooses when price and competing options are on the table. - The sharpest published critique, Chapman and Callegaro's 2022 Sawtooth Software Conference paper, finds Kano's dual-response items behave like unreliable survey questions, with category assignment unstable below roughly 200 respondents. - Terry Grapentine's 2015 Quirks piece reaches a parallel conclusion from a psychometrics angle: no validated scale, and answers shift with wording, and he recommends conjoint-type trade-off methods instead. - Most practitioners haven't switched anyway. Slevitch's 2025 systematic review finds hospitality and tourism studies still default to the original 1984 scoring table despite years of proposed categorization fixes. - Every fix on offer, larger samples, adaptive dual-response scoring, AI-assisted classification, repairs measurement noise in the self-report design. None of them puts the feature into an actual trade-off against price or a competitor. - A randomized trade-off experiment addresses that gap directly by making the choice itself the measurement, not a follow-up satisfaction rating. ## What is the core criticism of the Kano model? The core criticism is that Kano's dual-response items, the "how would you feel if this feature were present" and "how would you feel if it were absent" pair, function as low-quality survey questions rather than a validated measurement instrument. Chapman and Callegaro (2022) argue the response scale is multidimensional rather than unidimensional, meaning the same answer can reflect different underlying attitudes, and that category assignment (must-have, performance, delighter, indifferent) becomes unreliable below roughly 200 respondents. The Quant UX Blog's assessment makes the instability concrete: studies run at N=20, N=30, even N=100 are highly likely to produce different category assignments if repeated, with stable aggregate answers only showing up around N=200 and above. For a study that's typically run once, on a convenience sample, to decide which feature gets built, that's a real problem before any deeper issue is even on the table. ## The counter-criticism: why product teams keep using it anyway The standard defense is that Kano is simple, cheap, and communicates in three buckets an executive can act on without a statistics background, and that the sample-size problem is a known, fixable limitation, not a fatal flaw. That's the conventional treatment: run a bigger sample, tighten the item wording, and the categorization holds up. It's also, per Slevitch's 2025 review, what most teams actually do: despite a growing menu of categorization fixes published over the past decade, most hospitality and tourism studies in his systematic review still default to the original 1984 scoring table. The model survives less because the criticism has been answered and more because switching costs more than tolerating the noise. ## Where the 2025 fixes still fall short Slevitch's review catalogs the current wave of patches, including an adaptive dual-response variant and AI-assisted scoring, alongside the older fixes of bigger samples and cleaner item wording. Each one targets the same failure point: getting a more stable category out of noisy self-report data. None of them changes what the respondent is actually being asked to do, which is rate a feature they've often never used, alone, with no price attached and no competing option in view. A more stable estimate of an unstable question is still an estimate of the wrong thing for a roadmap or pricing decision. ## The flaw underneath the scoring table This is the structural issue the sample-size debate skips past. Satisfaction categorization was never built to isolate the causal, traded-off effect a feature has on what someone actually chooses; it was built to sort self-reported reactions into buckets. A person can honestly report that a feature would delight them and still not pick the product that has it once price or a competing feature enters the decision. Kano's survey design has no mechanism for surfacing that, because nothing in the question is ever varied or traded off. [figure: Two-column comparison. Left column, Kano dual-response item: respondent rates a single feature in isolation, no price shown, no competing option shown, output is a self-reported satisfaction category. Right column, randomized trade-off experiment: the feature is randomly varied against price and competing features across repeated choice tasks, output is a causal effect on choice with a confidence interval.] ## Why delighters are the hardest thing to survey The delighter category is definitionally a forecast of reaction to a surprise, and forecasting reactions to things people haven't experienced is precisely what stated-preference surveys do worst. Grapentine's conclusion follows from this: Kano lacks psychometric validation, scale wording measurably shifts outcomes, and he recommends conjoint-type trade-off methods for product design questions instead of satisfaction ratings. The same weakness shows up around pricing. When a delighter is really a premium feature, asking people to state what they'd pay for it runs into the same problem willingness-to-pay questions always have: stated willingness-to-pay comes in high relative to what people actually spend, unless the design incentive-aligns the choice, meaning it has real consequences for the respondent rather than a hypothetical answer. ## Is Kano compatible with randomized, causal experiment design? Not as it's currently practiced, but the underlying question it's trying to answer, does this feature change what someone chooses, is testable with randomized experiments analyzed with discrete choice models rather than dual-response items. McFadden discrete choice, Mixed Logit, and ICLV are estimators, not causal methods on their own; the causal identification comes from randomly varying the feature, price, and alternatives within the experiment design, not from the estimator that analyzes the resulting choices. When a flat multinomial logit is used, it carries the independence-from-irrelevant-alternatives assumption, which is part of why Mixed Logit and ICLV extensions exist: they relax that assumption when preferences vary across people or when a new option pulls share unevenly from existing ones. Whether this kind of design actually reproduces real buying behavior is a testable claim, not an assumption. In a validation study, our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, a result reported in the causal fidelity paper, not a guarantee for a new market the design hasn't been tested against. That number also doesn't resolve whether any of the original studies in that validation set could have been part of a model's training data before the test; the replication protocol is built to check for that rather than assume it away. And a confidence interval reported from a simulated experiment covers the effect within the simulated population that was fielded, not an unconditional bound on the real market. The public leaderboard shows how different designs perform against held-out human studies, which is the level of scrutiny a Kano scoring table has never been put through. ## What replaces it: a decision framework The choice isn't Kano versus nothing; it's which tool matches the decision in front of you. More detail on how these designs are validated is on the methods and validation blog. If a Kano survey is already scheduled for an upcoming roadmap review, the concrete next step is to add one randomized trade-off task to the same fielding: the feature in question against price and a real competing option, then check which one actually predicts a pick rather than a stated satisfaction rating. Compare the result against the public leaderboard before treating either number as final. If it's easier to walk through setting one up, meet the team. --- # What is TURF Analysis and When to Use It? URL: https://subconscious.ai/blog/methods-and-validation/turf-analysis-use Published: 2026-08-27 Summary of changes: rebuilt the intro's two overloaded sentences into short declaratives (buyer/decision split; "use it / stop / no mechanism" split); fixed the 93% sentence's grammar and definition placement; scoped the confidence interval to the simulated population in the bullet list; added a one-line disclosure that the TURF-assumption sources are vendor/practitioner literature, framed as a credibility point rather than hidden; recast the "which action" list without comma-as-dash; retitled the Subconscious section as a direct question in the owned lexicon; and added a concrete segment-based symptom that tells a reader when to demand Mixed Logit over a plain logit. No other sections were restructured. --- # How to analyse time series results URL: https://subconscious.ai/blog/methods-and-validation/analyse-time-series-results Published: 2026-08-27 A brand or insights lead watching a tracker move after a launch, price change, or repositioning needs one thing: proof that the decision caused the move, not just proof that the move is real. Wave-over-wave significance testing can only rule out noise; it cannot rule in your decision as the cause, because the tracker never observes what the metric would have done without it. Getting a real answer means treating the tracker's job (did the number move) and the causal job (did my decision move it) as two separate questions, answered with two different designs. - A statistically significant wave-over-wave move only means the shift is unlikely to be sampling noise. It says nothing about what caused it. - Panel conditioning (respondents changing their answers simply from repeated participation) is a built-in confound in every longitudinal tracker, not an edge case. - Small subgroup cuts carry margins of error wide enough to manufacture "significant" moves out of nothing. - A clean pre-trend in your tracking data does not validate that the trend would have continued unchanged without your decision. - Attributing a shift to a specific action requires a designed comparison against the same population, before and after, not an annotated chart. ## What does it mean to analyze time series results from a brand tracker? In practice, "analyzing time series results" means three things done in sequence: plotting the tracked metric wave over wave, running a test against the margin of error to flag whether a shift is statistically distinguishable from noise, and segmenting by cohort, region, or subgroup to see where the shift concentrates. Most teams add a fourth step by habit: annotating the chart with whatever launch, price change, or competitor move happened nearby, and treating that proximity as the explanation. That fourth step is where the analysis quietly stops being statistics and starts being narrative. The first three steps are legitimate description. They tell you a number moved and where. None of them tell you why. ## The standard playbook, and where it stops Longitudinal consumer research today runs almost entirely on repeated cross-sectional or panel surveys fielded at regular waves. Kantar alone maintains a brand-equity database spanning 21,000 brands across 540 categories in 55 markets (Kantar). The broader shift industry-wide has been toward "agile" continuous tracking with dashboards that auto-flag wave-over-wave movement the moment it clears the margin of error. Standard industry guidance calls for 300-400 respondents per wave for topline metrics, 600-1,000+ for subgroup cuts, and at least 12 periods before trusting a trend line. What changed in that shift is automation, not the underlying logic. The design is still repeated observation of the same or similar population over time. It is not a controlled comparison against a counterfactual, and no amount of dashboard speed changes that. ## Why doesn't a statistically significant move prove your decision caused it? A significant move clears the noise bar, but clearing the noise bar is not the same as ruling out every other explanation for the shift. Attributing a tracked change to a specific cause requires something closer to the parallel-trends assumption used in difference-in-differences analysis: that absent your decision, the tracked group would have followed the same path as some comparison group or its own prior trend. That assumption combines restrictions on unobserved potential outcomes with restrictions on how the "treatment" (your decision) was assigned, which makes it non-standard and not directly verifiable from the observed data alone (NBER). A clean historical trend line does not settle this. Parallel pre-trends in tracking data are neither necessary nor sufficient proof that the parallel-trends assumption will hold going forward, so a smooth chart before your decision does not validate a causal read of what happens after it (World Bank). You never observe the counterfactual path the tracked group would have taken without your action. Annotating the chart with "this is probably why" is a plausible story, not a proof. ## The confound sitting inside every wave Beyond the counterfactual problem, tracking studies carry a mechanical confound: repeated participation in the same longitudinal survey measurably changes how respondents answer in later waves, independent of any real underlying change. This panel conditioning effect is documented in the General Social Survey and holds across repeated-panel designs generally (NIH/PMC). A metric can drift purely because your panel has answered the same questions five times before, with no connection to your decision, your competitor, or the market at all. ## How much noise hides inside a small subgroup cut? A lot more than the "statistically significant" label on a dashboard suggests, and the gap is easiest to see in numbers rather than description. [figure: Bar chart showing margin of error of approximately 14 percent at a sample size of 50 respondents, dropping to approximately 3 percent at a sample size of 1,000 respondents.] This is why subgroup cuts (the ones standard guidance says need 600-1,000+ respondents) are where trackers most often manufacture false positives: a topline sample built for ±3% precision gets sliced into a regional or cohort cut running closer to ±14%, and the dashboard applies the same significance flag to both (Kantar). ## What a designed experiment adds that a tracker structurally cannot A tracker tells you a number moved. Only a designed experiment, run against the same population before and after the decision, with the decision itself as the randomized intervention, can tell you whether your decision moved it. That is the difference between description and causal attribution: description asks whether the line changed; a designed experiment builds a holdout so you can compare what happened against what would have happened without the change, instead of inferring it from an annotated chart. This is a design choice, not a bigger tracker. No amount of additional waves or tighter panel management turns a repeated observational survey into a randomized comparison. The intervention has to be built into the design from the start. ## How this looks in practice The mechanics: run a randomized experiment against a synthetic population built to mirror your market, analyzed with discrete choice models (McFadden discrete choice, Mixed Logit, ICLV) to estimate preference structure. DCE, Mixed Logit, and ICLV are estimators, not causal methods on their own. Causal identification comes from the randomized manipulation built into the experiment design. The estimates from that design get validated against a human baseline. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. That number is a validation-set result, not a guarantee for a new market, and it comes with a caveat: some published human studies used for validation could exist in a model's training data, and the replication protocol is built to address that risk rather than pretend it away. Results across studies are tracked on a public leaderboard, and the broader method context sits in the methods and validation hub. ## Tracker or designed experiment: which fits the decision in front of you? The two designs answer different questions, and confusing them is the core mistake. Neither design replaces the other. The tracker's job is surveillance: flag that something is worth investigating. The experiment's job is attribution: determine which action drove the outcome, before you spend the budget or after you need to explain the result to the board. ## What should a senior buyer do next? Before your next wave closes, name the specific decision you want credit or blame for, and ask whether your current design can separate that decision's effect from panel conditioning, sampling drift in the relevant subgroup, and any coincident event on the calendar. If the honest answer is no, that's the signal to commission a designed experiment against the same population before the decision ships, not to add another annotation to the chart after it does. If you want to see how a randomized experiment against a synthetic population compares against your next tracker wave, talk to the team. --- # New data quality safeguards against fraudulent survey responses URL: https://subconscious.ai/blog/methods-and-validation/new-data-quality-safeguards-against-fraudulent-survey-responses Published: 2026-08-27 A research director evaluating a fielding vendor for 2026 needs to know what a "fraud-free" certification actually covers. New safeguards, device fingerprinting, IP and geo verification, behavioral biometrics, AI-response classifiers, confirm that a respondent is a real, attentive human. They do not confirm that the human's stated answer predicts what they will actually do, and that second question is a different quality bar than most fraud-detection vendors are building for. - Fraud screening (fingerprinting, IP/geo, biometrics, AI classifiers) verifies identity and attention, not behavioral validity. - Fraud rates are severe: NORC's 2026 literature review pooled a 61% fraudulent-response rate across 13 studies, with social-media-recruited samples hitting 94-95%. - In a Stanford-affiliated study of roughly 800 Prolific participants, nearly a third admitted using ChatGPT or similar tools to answer surveys, a self-reported number that likely understates the true rate, and that AI-assisted text passes speed and straight-lining checks by looking polished rather than repetitive. - Passing every fraud filter still leaves the say-do gap: a verified, attentive human giving a fast, sincere answer to a hypothetical question can still fail to predict what they do in the real market. - The test that closes that gap is replication against observed behavior, not screening at intake, and it's a separate step most vendors don't run. ## What do fraud safeguards actually verify? They verify that a response came from a distinct, geographically plausible human who was paying enough attention to avoid obvious tells, nothing more. Panel vendors have layered device fingerprinting, IP and geo verification, behavioral biometrics, and AI-vs-AI detection classifiers into what's being marketed as the 2026 baseline for trusted data. Each of those checks answers a version of the same question: is this a real, attentive person and not a bot or a duplicate account? None of them ask whether that person's answer, once verified as sincere, actually predicts what they'll do outside the survey. That's a fraud question and a validity question, and conflating the two is the mistake a buyer pays for later, when a "clean" sample still produces numbers that don't hold up. ## How big is the fraud problem right now? Bad enough that fraud, not sampling error, is now the leading data-quality risk in nonprobability research. NORC's 2026 review pooled 13 studies and found fraudulent-response rates ranging from 13% to 97%, with a pooled proportion of 61% (NORC, 2026). Social-media-recruited samples were the worst case in that review, at 94-95% fraud, and one of the reviewed studies found only 3 of 981 collected responses were real (NORC, 2026). That NORC roundup also cites broader industry estimates of 15-30% fraud generally, spiking to 45% on some platforms, plus a 2025 estimate of roughly 40% of global nonprobability interviews as fraudulent (NORC, "The Fraud Problem Reshaping Survey Research"). The range across these estimates is wide because methodology and recruitment channel change the number as much as fraud itself does, which is exactly why a single reported "fraud rate" is hard to act on without knowing which population produced it. [figure: Bar chart of survey fraud rate estimates ranging from 15 percent as an industry baseline low to 95 percent for social-media-recruited samples, with NORC's pooled estimate across 13 studies at 61 percent] ## Generative AI raised the bar for what "fluent" means Fluent, on-topic writing used to be a decent proxy for a real, engaged respondent. A Stanford-affiliated study of roughly 800 Prolific participants (Xu, Zhang, Alvero, November 2024) found nearly one-third admitted using ChatGPT or a similar tool to help answer survey questions (Stanford Report, 2024). That text passes speed checks and straight-lining flags because it isn't fast or repetitive. It's polished. Cint has made the same point directly: AI-generated open-ends can look more coherent and higher-quality than genuine human responses, which inverts the heuristic researchers have relied on for years, that fluency signals authenticity (Cint, 2025). AI-vs-AI classifiers are a reasonable response to that specific problem. They still only answer whether the text was LLM-generated, not whether the underlying preference it describes is one the respondent would act on. ## Two failure modes, one industry building for only one Fraud and validity fail in different ways, and stacking more identity checks addresses only one of them. A bot or a duplicate account fails the first test: is this a real, distinct human paying attention? A verified, sincere, attentive human still fails the second test whenever their fast, self-reported answer to a hypothetical doesn't match what they'd actually choose with money or time on the line. That gap predates any bot problem and has nothing to do with fraud. A sample can be 100% fraud-free by every fingerprinting, geo-verification, and AI-classifier standard on the market and still produce results that don't replicate against real-world choice, because none of those layers test whether the response reflects actual behavior. The industry's 2026 "new baseline" hardens the first failure mode almost exclusively. The second one is the harder problem, and it's the one that determines whether a finding is safe to act on. ## What does replication accuracy mean, and what does it not cover? Replication accuracy is the buyer-facing test that fraud screening skips, and it comes with real limits worth knowing before citing the headline number. Subconscious's best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. A number without its limits is marketing. It is a validation result, not a guarantee for a new market. Two limitations matter here. The misses belong on the leaderboard next to the hits. First, this is a validation-set result, not a guarantee that a new, unstudied market will replicate at the same rate. Second, published human studies can sit inside a model's training data, which would inflate replication scores if left unaddressed. The replication protocol is built to account for that risk, but it's worth asking any vendor about directly rather than assuming it away. The underlying method is randomized experiments analyzed with discrete choice models, specifically McFadden discrete choice, Mixed Logit, and ICLV. The causal claim comes from the randomized manipulation in the experiment design, not from the estimator itself. DCE, Mixed Logit, and ICLV are ways of estimating preferences from choice data; they don't make a study causal on their own. The leaderboard publishes replication results by study and method, so a buyer can check this claim against specific cases rather than take a single headline number. ## Fraud screening vs. replication validation: what each proves ## What should a research director check before trusting a sample? Ask the vendor two separate questions, not one. First, what fraud screening was applied, and what rate did it catch, given that pooled fraud rates run as high as 61% and platform-specific spikes reach 94-95% (NORC, 2026). Second, and separately, has this finding, or one structurally like it, been checked against a real human-behavior holdout, and at what replication rate. A vendor who can only answer the first question has proven the sample is clean, not that the result is true. Reviewing published replication results by method on the leaderboard before fielding gives a concrete way to check the second question against real cases rather than a vendor's own summary. Background on the discrete choice methods behind those studies is on the methods and validation blog hub. ## The next study you run Before the next fielding decision, separate the fraud question from the validity question in the vendor scorecard. Log the fraud-screening stack and its catch rate in one column, and whether the finding replicates against observed behavior in a second. A strong first column does not stand in for the second. A sample passing every identity and attention check is a necessary condition for trustworthy data, not a sufficient one. If a study's finding matters enough to act on, check whether anyone has run the replication test, not just the fraud test. For a walkthrough of how that replication check works on a specific study, book time with the team. --- # UX Testing vs Market Research for Evaluating Software URL: https://subconscious.ai/blog/comparisons/ux-testing-vs-market-research-evaluating-software Published: 2026-08-27 A VP of Product deciding whether a redesign ships next sprint needs a method for evaluating the software, not just a preference between UX testing and market research. UX testing shows whether five to twenty users can complete a task; market research shows what a sample says it wants or would pay, and neither randomizes the variable in question. So neither answers the question the ship decision depends on: which design, price, or message changes behavior, with a confidence interval attached. - UX testing (UserTesting, Maze, Lyssna) measures whether a small group of users can complete a task; it does not test whether a specific design decision changes behavior at scale. - Market research (Qualtrics, SurveyMonkey) measures stated preference through surveys; stated intent consistently diverges from actual purchase behavior. - Neither method randomizes the variable under debate, so neither can claim the design, price, or message caused the outcome observed. - A randomized experiment run on a simulated market and checked against real human behavior isolates that one variable and reports a confidence interval around its effect. - The right choice depends on the question: diagnosing an existing interface calls for usability testing; deciding between options that don't exist yet calls for a randomized experiment. ## UX testing and market research answer different questions, but neither proves causation UX research is built to answer "does this work and is it intuitive," using qualitative methods: personas, journey maps, moderated usability sessions. Market research is built to answer "will people want this and pay for it," using quantitative surveys, segmentation, and purchase-intent questions. The two disciplines have split into separate tool categories and separate vendor camps. UX Studio's own comparison of the two describes them as typically run as distinct workstreams across the product lifecycle rather than as one integrated method (uxstudioteam.com). Most guidance tells the buyer to run both anyway: usability testing to catch friction, market research to validate demand. Running both doesn't close the underlying gap, because both traditions report what a small, self-selected sample said or did in one uncontrolled setting. Neither isolates the effect of the one variable the buyer controls, on behavior, with a number attached to how confident to be in that estimate. CB Insights' analysis of failed venture-backed companies found poor product-market fit was the leading cause of failure, cited in roughly 42-43% of cases (cbinsights.com). The stat doesn't prove descriptive research caused those failures; it's a correlation, not a test of what would have moved the market before the money was spent. That test is exactly what UX testing and market research aren't built to run. ## Does the five-user usability rule still hold? Only at the discovery rate it assumes, and that rate is often optimistic. Jakob Nielsen and Thomas Landauer's 1993 model holds that five users surface about 85% of usability problems, built on an average 31% per-session problem-discovery rate (measuringu.com). [figure: Bar chart showing 5 users needed at a 31 percent problem-discovery rate versus about 18 users needed at a 10 percent discovery rate.] At lower discovery rates, the math changes fast: reaching the same 85% coverage at a 10% discovery rate takes about 18 participants, not five (measuringu.com). A buyer treating "we ran five usability sessions" as a decision gate is relying on an assumption about their own study's discovery rate that nobody checked. ## Why do stated-preference surveys mislead on purchase intent? Because what people say they'd buy and what they actually buy diverge, and the gap is measurable. A study by DecTech with Warwick University's Behavioural Science Group, using 52 weeks of actual sales data across 600 stores, found revealed-preference methods predicted real-world purchase outcomes 1.5 times more accurately than the best-performing stated-preference survey method (dectech.co.uk). That result comes from one retail category, 600 stores, and one 52-week window; it's evidence that the say-do gap is real and measurable, not a general law about every survey. The gap itself is structural to the survey format: a respondent answering a purchase-intent question pays no cost for being wrong, so the answer reflects self-image and social desirability as much as actual demand. Any willingness-to-pay figure taken from an unincentivized stated-preference survey carries hypothetical bias: stated WTP runs directionally high, not as a price to plan around. ## UX testing vs market research vs a randomized experiment ## What a randomized experiment on a simulated market proves A randomized experiment answers a narrower, harder question than either UX testing or market research: which specific intervention, this design against that one, this price against that one, this message against that one, changes choice, and by how much, within a stated confidence interval. The causal identification comes from the randomized manipulation built into the experiment design, not from the statistical method used to analyze results afterward. Naming a method correctly is what lets a buyer check the claim behind it, so the terminology gets corrected here rather than left loose: McFadden discrete choice, Mixed Logit, and ICLV are estimators that turn choice data into effect sizes; they are not causal methods on their own, and the more accurate description of the work is randomized experiments analyzed with discrete choice models. When a flat logit model is used to estimate preference share or substitution patterns, it carries the independence of irrelevant alternatives (IIA) assumption; Mixed Logit relaxes that assumption when substitution patterns are expected to vary. Subconscious runs these randomized experiments against a simulated population and checks the result against real human studies. A performance number by itself is marketing copy. The study it was measured against, and what it does not cover, is what turns it into evidence a buyer can check. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market. Details are in the causal fidelity paper. Results by method and study are published on the leaderboard, so a buyer can check accuracy by method rather than take the headline number on faith. ## Where this method still has limits Three caveats worth holding onto. First, published human studies can sit inside a model's training data, which would artificially inflate replication accuracy if not controlled for; the replication protocol is built to check for this, but the risk doesn't disappear just because a number exists (see the causal fidelity paper). Second, a confidence interval from a simulated experiment covers the effect within the simulated population that was tested, not the real market unconditionally; it's evidence to weigh, not a bound to cite unconditionally. Third, 87% of the measured human ceiling (0.832 against a 0.959 human-to-human ceiling; mean 0.73 across the 43 studies passing design filters) is a validation-set result. A new market, a new category, or an unusual audience can fall outside that validation set, and the honest move for any specific new question is to check the leaderboard and the method used for a comparable study, not to assume the headline number transfers. ## Which method should the senior buyer choose? It depends on whether you're diagnosing an existing interface or deciding between options that don't exist yet. If the product is built and the question is "where do people get stuck," a moderated usability session still does that job well and fast, as long as it's sized for the discovery rate observed, not the assumed 31%. If the question is "which of these designs, prices, or messages will change behavior, and how confident should I be," neither a usability session nor a purchase-intent survey answers it, because neither randomizes the variable in question. That's a randomized experiment, run against a large enough simulated population to produce a confidence interval, then checked against a human baseline before the buyer trusts it. Name the one variable in play for the decision in front of you: this design versus that one, this price versus that one, this message versus that one. Check whether the study about to run randomizes it. If it does, it's a causal experiment, whatever tool built it. If it doesn't, it's a description of a sample, and the decision still has to be made on top of that description. Compare methods and studies on the leaderboard before choosing one, or talk to the team about how the randomized-experiment method applies to your specific decision. --- # Classification of quality issues in survey sample URL: https://subconscious.ai/blog/methods-and-validation/classification-quality-issues-survey-sample Published: 2026-08-27 A pricing manager deciding whether to trust a survey sample before setting a launch price is really weighing two separate quality questions, and most tooling shipping in 2026 answers only one. The first is respondent-level: was this a real, attentive human, not a bot, a speeder, or a duplicate account. The second is causal: does that real human's answer predict what they will actually do. Fraud-detection tooling has gotten very good at the first question and has almost nothing to say about the second, which is the more expensive one to get wrong. - Survey sample quality splits into two separate checks: respondent-level (was this person real and attentive) and causal (does an honest answer predict real behavior). - Fraud screens (attention checks, digital fingerprinting, speed and consistency filters) catch fake or inattentive respondents; they say nothing about whether a real respondent's answer predicts real behavior. - The costliest classification error isn't a fake respondent. It's a real, verified one answering a hypothetical question in a design that was never built to predict behavior. ## What quality issues does 2026 survey tooling actually classify for? Nearly all of it classifies for fraud, not measurement validity. NORC estimates that roughly 40 percent of nonprobability survey interviews in 2025 were likely fraudulent, an estimate the firm puts at around 2 billion interviews, extrapolated from panel-level fraud rates rather than a direct industry census, with valid response rates on some panels falling from about 75 percent to about 10 percent (NORC). Vendors like Qualtrics, CloudResearch, Prolific, Research Defender, and Veridata sell respondent-level screens in response: digital fingerprinting, speed and consistency checks, duplicate-IP detection, attention-check pass rates. These tools answer one question well: is the person on the other end of this survey real and paying attention. They were built for a real and worsening problem. LLM-driven bots have passed attention checks at up to 99.8 percent in proof-of-concept testing while writing coherent, polished open-ended responses, even though most current fraud volume is still human click-farm labor rather than bots (NORC). None of these tools, however, evaluate whether the survey question itself, answered honestly, tells you anything about future behavior. ## The academic root: Total Survey Error's two branches The respondent-versus-measurement split isn't new; it comes from Total Survey Error, the framework Groves and Lyberg formalized in Public Opinion Quarterly. TSE divides survey error into a representation branch (coverage, sampling, nonresponse error, roughly: did you reach the right people) and a measurement branch (construct validity, questionnaire design, processing error, roughly: did the question and the answer mean what you think they mean) (Groves & Lyberg). Most quality frameworks built since inherit this structure loosely. The problem is that 2026's fraud-detection tooling has almost entirely colonized the representation branch and left the measurement branch, where hypothetical bias lives, mostly unaddressed by comparison. ## Why fraud detection dominates the current conversation Fraud is dominating because it's acute, visible, and now coordinated across the industry. NORC frames fraud as an existential threat to the nonprobability panel model rather than a solvable edge case (NORC). In late 2025 and 2026, MRS, ESOMAR, the Insights Association, and SampleCon began coordinating shared fraud definitions and standards across the industry (MRS). This is legitimate, necessary work. A dataset that's 40 percent fraudulent, per NORC's estimate above, is unusable regardless of what else is true about it. But coordination on fraud definitions doesn't touch the second problem, and a buyer who reads "data quality" as solved once fraud is filtered is solving the cheaper half of the problem. ## Can a fraud-clean sample still be causally wrong? Yes. A sample can pass every fraud, bot, and attention-check filter and still be causally wrong, because passing those filters only proves the respondent is a real, attentive human, not that the question they answered predicts what they'll do. A verified, 100 percent human respondent answering "would you buy this at $12" is still producing stated-preference data, and stated preference is subject to hypothetical bias regardless of how clean the panel is: stated willingness to pay tends to run higher than what the same respondent would actually pay in a real transaction. No digital fingerprint or speed check touches that gap, because it isn't a fraud problem. It's a design problem: nothing about the question forced a real tradeoff the way an actual purchase decision would. [figure: A four step chain running from verified respondent through hypothetical question to stated answer to actual behavior, with the say-do gap marked between stated answer and actual behavior, after all fraud checks pass.] The two classification systems are answering different questions, and treating them as one continuum is the mistake: ## How a causal validation approach classifies quality differently A causal approach asks a different question. Not whether the respondent is real, but whether the design forces a real tradeoff, the kind that produces answers that predict behavior. Then it checks that prediction against a held-out human result. That means running a randomized experiment, where the manipulation itself identifies a causal effect, and analyzing it with discrete choice models: McFadden discrete choice, Mixed Logit, and ICLV are estimators for that analysis, not causal methods in themselves. The causal identification comes from the randomized manipulation in the design, not from the estimator applied afterward. Subconscious runs this kind of study on a simulation of a market, then checks it against a real, held-out human study. Across that validation set, our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, documented in the causal fidelity paper. That's a validation-set result, not a guarantee for a new market you haven't tested yet, and it comes with a caveat worth stating plainly: some of the published human studies used to validate it could sit in a model's training data, which is exactly the kind of contamination the replication protocol is built to catch rather than pretend doesn't exist. Results published on the public leaderboard carry a confidence interval scoped to the simulated population tested, not the real market unconditionally. More on how these studies are built and checked is on the methods and validation hub. ## What should a buyer check before trusting a sample's quality? Check both branches, not one. First, confirm the fraud screen: what bot, speed, and duplicate-detection methods were applied, and what fraud rate they found, given NORC's finding that some panels' valid response rates have fallen to about 10 percent. Second, and separately, confirm the design: was the question a hypothetical stated-preference item, or a randomized experiment with a forced tradeoff, and was it checked against a held-out human result rather than only against internal consistency. A spotless dataset that never answers the second question hasn't been validated. It's just been cleaned. A reader can start by pulling one recent stated-preference study from their own program and asking whether it forced a real tradeoff or a hypothetical one. That single check surfaces most of the gap described above without any new tooling. For a closer look at how a specific market or decision would hold up under a randomized, replicated design, meet with the team. --- # What Promotion Mechanics Are Used in FMCG/CPG? URL: https://subconscious.ai/blog/industries/promotion-mechanics-used-fmcg-cpg Published: 2026-08-27 A revenue growth management lead choosing the mechanic for next quarter's promo calendar decides which lever moves volume without training shoppers to wait for the next deal. FMCG and CPG brands draw from a short list of mechanics: straight price discounts, BOGO, multi-buy, bonus packs, coupons and rebates, loyalty-point multipliers, and bundles. Choosing among them is the decision this article answers: which mechanic to test before it goes on the calendar. That choice usually runs on category habit and manufacturer defaults: a category manager copies last year's calendar because testing every mechanic in-market is slow and expensive. This article treats the decision differently: pick and test the mechanic before the trade budget commits, not after the redemption report lands. Randomized experiments analyzed with discrete choice models answer the question before the calendar prints: which mechanic changes behavior, and by how much. - No mechanic wins universally. The right one depends on category, price tier, and shopper segment, which is why testing beats copying last year's calendar. - Randomized experiments analyzed with discrete choice models (McFadden DCE, Mixed Logit, ICLV) isolate which mechanic caused a shift in choice, not just which one correlates with higher scan sales. - Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. A rank-correlation number without its limits is marketing copy. The 87% figure holds on the study it was measured against, and it isn't a guarantee for a market or category the model hasn't been checked against. - Stated preference for deep discounts tends to run higher than what shoppers actually redeem, so any willingness-to-pay read needs a hypothetical-bias caveat unless the design pays out real money. ## What promotion mechanics do FMCG and CPG brands actually use? Straight price discounts (10 percent off, $1 off) are the default because they're simple to execute and easy to measure. BOGO and multi-buy offers (buy two, get one free; three for $5) trade margin for basket size and tend to pull volume forward rather than grow the category. Bonus packs (20 percent more, same price) protect the shelf price point while still moving units, which matters in categories where a visible price cut signals lower quality. Coupons and rebates shift the discount to a subset of shoppers willing to clip or clip digitally, which segments the spend but adds friction. Loyalty multipliers and bundles are newer additions, useful where retailer data partnerships make targeting possible. Each mechanic has a different mix of trade cost, margin impact, and shopper perception. The mechanic that lifts volume in salty snacks without cannibalizing full-price sales can flatten volume in a category where shoppers already stockpile on deal, like paper goods or laundry care. ## How do you test promotion mechanics before committing trade dollars? Run a randomized experiment, not a bigger regression on historical scanner data. A discount that runs during a holiday week and one that runs in a slow month aren't comparable: redemption rate and lift both mix the mechanic's effect with distribution, seasonality, and competitor activity happening in the same window. Separating "this mechanic caused this outcome" from "this outcome happened while this mechanic was running" takes a randomized manipulation. [figure: A four step chain: random assignment of promotion mechanics to simulated shoppers, observed choices under each mechanic, estimation of effect size with a discrete choice model, ending in a causal claim about which mechanic drove the outcome.] Present each simulated shopper segment with a different mechanic, or combination of mechanics, as if it were a real shelf choice, then analyze the resulting choices with an estimator built for discrete alternatives. McFadden discrete choice models give a fast baseline read on which mechanic wins share across a large candidate set. Mixed Logit adds shopper-level variation, so a value-driven segment and a loyalty-driven segment can show different sensitivities to the same discount depth instead of being averaged into one number. ICLV goes further and folds in latent attitudes, like perceived price fairness or quality erosion, that explain why a mechanic underperforms even when the discount is deep enough on paper. A flat discrete choice model carries the IIA assumption: it treats every mechanic in the test as substituting proportionally for every other one, whether the alternative is a BOGO or a modest coupon. That rarely matches how shoppers actually trade off similar options. Mixed Logit and ICLV loosen this by modeling shopper-level heterogeneity, but the assumption doesn't disappear entirely. It only relaxes within segments. Treat preference-share output from a flat logit as directional for close substitutes, not a fixed law. A deeper walkthrough of when each estimator fits lives in methods and validation. ## How accurate are simulated promotion tests against real shopper behavior? Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. Publishing the ceiling next to the score is what lets a buyer check the method against their own category. It is a validation result, not a guarantee for a promotion mechanic in a market or category the model hasn't been checked against. Published human studies can sit inside a model's training data; the replication protocol is built to catch that overlap, not to let the number wave it away. Current results across categories are visible on the public leaderboard, which is the place to check before treating any single study as settled. ## Does the say-do gap distort promotion mechanics research? Yes, when the question asks shoppers what they'd pay or which discount they'd wait for, because stated willingness to pay runs higher than what people actually redeem at checkout. This is hypothetical bias, and it shows up specifically as shoppers claiming they'd hold out for a deeper discount than the one that actually changes their basket. A discrete choice design that presents mechanics as competing shelf options, rather than asking shoppers to name a price, narrows this gap because it forces a real trade-off instead of an open-ended guess. It still isn't the same as an incentive-aligned design, where a real purchase is on the line. Any willingness-to-pay figure from a standard DCE should carry that direction of bias explicitly, not get reported as a clean number. A confidence interval attached to a simulated promotion test covers the effect within that simulated population under the tested conditions; it isn't a bound on what will happen in the real market until the same design has been checked against a human holdout. ## Building a promotion-mechanics test before the calendar locks Start with the mechanics actually under consideration for the category, not a hypothetical long list; three to five candidates is enough for a first pass. Randomize which mechanic each simulated shopper segment sees, run the McFadden discrete choice baseline to rank them, then move the top two or three into Mixed Logit to see whether a value segment and a loyalty segment disagree. If a mechanic underperforms in a way the numbers alone don't explain, an ICLV pass on attitudes like price fairness usually shows why. Check the result against the leaderboard for category precedent, and look at case studies for how other CPG teams structured a similar test before treating the read as final. Take the two mechanics most under debate for the next promo cycle and run them through this sequence before the trade budget commits. For help scoping the test, book time with the team. --- # What are Omnibus Surveys? URL: https://subconscious.ai/blog/implementation-and-procurement/omnibus-surveys Published: 2026-08-27 A VP of pricing weighing a price increase on a flagship SKU wants a fast read on how customers will react. Omnibus surveys deliver that read cheaply and quickly. A number without its limits is marketing. They do not prove what a price change, a new feature, or a new claim does to demand, because nothing on the shared instrument is manipulated and there is no control group to compare against. - An omnibus survey is a shared questionnaire where multiple clients each buy a few questions on one fielded instrument, run on a fixed cadence by the provider. - Pricing runs from roughly $1.50 per completed response self-serve (YouGov) to $1,000 for each of the first three questions and $750 after that (SSRS). - The standard switch point to a custom survey is roughly five to six questions, per Versta Research; below that, omnibus is usually cheaper. - Omnibus and custom survey both measure stated opinion, the say-do gap between what someone reports and what they would actually do; neither manipulates price, feature, or claim, so neither can prove a causal effect on demand. - Proving what a specific change does to demand requires a randomized experiment with a control condition, not another question on a shared instrument. ## What is an omnibus survey, and how does it work? An omnibus survey is a single fielded questionnaire that a provider builds from many clients' questions stitched together, then runs against one panel or sample on a set schedule. SSRS's Opinion Panel Omnibus, for example, fields twice a month in a fixed Friday-through-Monday window, and clients buy in by the question rather than commissioning their own fielding (SSRS). Ipsos runs Online, Face-to-Face, and Online Overnight versions of the same model and has moved its Omnibus product onto a self-serve digital platform, live in Australia, France, Germany, Italy, the UK, and the US, with the full product available in 100+ markets (Ipsos). You get a general-population read fast, on someone else's schedule, sharing the questionnaire with clients whose topics you don't control. ## How much do omnibus surveys cost, and how fast do they run? YouGov fields omnibus across 70+ markets with self-service pricing from roughly $1.50 per completed response and results available in as little as 24 hours (YouGov). SSRS prices its omnibus at $1,000 for each of the first three questions and $750 per question after that, with a predictable twice-monthly fielding window (SSRS). Cheap per question, fast to field: that combination is why omnibus remains the default first call for a directional read. It says nothing yet about whether the question being asked is one a survey, shared or custom, can actually answer. ## When does a custom survey beat omnibus? Once a buyer needs more than a handful of questions, or a narrow or low-incidence audience, the math flips. Versta Research's guidance puts the tipping point at roughly five to six questions, the level where a short custom survey becomes price-competitive with omnibus while adding targeting and design control that a shared instrument can't offer (Versta Research). Providers like TGM Research and Kadence lean into this and position omnibus explicitly as a triage step: run a cheap question first to see whether a topic justifies the budget for a full custom instrument. That framing is useful, but it only resolves cost and scope. It doesn't resolve whether either format, omnibus or custom, can tell you what will happen if you actually change the price, the feature, or the claim. ## Can an omnibus survey tell you what people will actually do? Naming this failure mode is what lets a buyer check it. No. An omnibus question captures a stated opinion at a point in time. That's the say-do gap: what someone reports and what they would actually do are not guaranteed to match, and omnibus has no way to close it, because nothing in the design manipulates price, feature, or claim and there's no control group to compare against. That gap is easy to miss because the survey report looks the same either way, a table of percentages, so it's worth making the missing step explicit. [figure: Two sequential paths. The omnibus path runs from a shared-instrument question to a stated opinion, with no manipulation and no control group, ending at cannot isolate a causal effect. The experiment path runs from a control condition to an intervention applied to one arm, a comparison against control, ending at a causal effect with a confidence interval.] ## Does sitting next to unrelated questions affect your data? This kind of miss belongs on the record next to the hits. It can, structurally, even before any effect on data quality is measured. Omnibus stacks each client's items into one instrument, and even when topic order is randomized, a respondent answers your question in the context of whatever unrelated, sometimes burdensome, topic came right before it. That ordering is outside your control on a shared instrument, and it's one more variable that neither omnibus nor custom, unrandomized survey design accounts for. It doesn't change the causal gap above, but it's a reason to treat even the stated-opinion number with some caution when it comes off a shared instrument. ## What proves a causal effect instead of a stated opinion? A randomized experiment with a control condition, not a bigger or better-worded survey question. The design applies an intervention, a specific price, feature, or claim, to a treatment condition and withholds it from a holdout, then compares choices between the two using discrete choice models: McFadden discrete choice, Mixed Logit, and ICLV. Those are estimators, not causal methods on their own; the causal identification comes from randomizing the intervention across conditions, not from the estimator doing the counting. Where a flat multinomial logit is used for preference-share or substitution questions, that estimate carries the IIA assumption (independence of irrelevant alternatives); Mixed Logit relaxes it, which is why it's the standard choice when substitution across options matters. Validated against human baseline studies, this kind of simulated, randomized design reaches 87% of the measured human ceiling on one study (0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959), with a mean of 0.73 across the 43 studies that pass design filters. Publishing where the method falls short is what makes the number usable. That is a validation result, not a guarantee for a market that hasn't been run yet, per the causal fidelity paper. Because some of those published studies could sit in a model's training data, that risk is worth naming directly rather than assuming it away; it's also why replication results, and the studies they're checked against, are published openly on a running leaderboard rather than asserted. The broader case for treating discrete choice modeling as an estimator layered on a randomized design, not a causal method by itself, is laid out in more depth in methods and validation, alongside worked examples in case studies. ## What should a buyer do next? Pull your last few omnibus briefs and check the verb in each question. If it asks what people think, prefer, or intend, an omnibus or custom survey both work, and Versta's five-to-six-question threshold still tells you which one to buy. If it asks what happens when you change the price, the feature, or the claim, that's a causal question, and no shared questionnaire slot, omnibus or custom, answers it; you need a control condition and a randomized intervention instead. For a second read on a specific decision, talk to us. --- # Gabor-Granger or Van Westendorp? URL: https://subconscious.ai/blog/comparisons/gabor-granger-van-westendorp Published: 2026-08-27 A pricing lead choosing between Gabor-Granger and Van Westendorp for an upcoming study is really choosing between two ways of asking a hypothetical question. Gabor-Granger asks respondents to say yes or no to a sequence of set prices; Van Westendorp asks them to name the price where a product feels too cheap, cheap, expensive, or too expensive. Neither one tests what a real price change would do to real demand, which is the actual decision at stake. - Gabor-Granger and Van Westendorp are both stated-preference surveys: they ask about price in isolation, under hypothetical conditions, and return a self-reported number rather than a tested behavioral response. - Gabor-Granger builds a demand curve from sequential purchase-intent questions and points to a revenue-maximizing price; Van Westendorp triangulates an acceptable price range and a quality floor from four open-ended questions. - Standard practitioner advice treats them as complementary: Van Westendorp early, when the acceptable range is unknown, and Gabor-Granger later, once the range is known and a single price point is needed. - Naming a failure mode here lets a buyer check it before they commission the study. Both carry documented distortions: stated purchase intent runs higher than real buying behavior, and Van Westendorp's line-crossing logic has been criticized for lacking a theoretical foundation and for lowballing the market. - The decision that actually matters is whether to trade either stated-preference survey for a randomized pricing experiment that reports a causal effect with a confidence interval, not which four questions to ask. ## What do Gabor-Granger and Van Westendorp actually measure? Both measure stated intent, not behavior. Gabor-Granger shows respondents a sequence of set price points and asks a yes/no purchase-intent question at each one, building a demand curve and identifying the price that maximizes projected revenue. Van Westendorp asks four open-ended questions (too cheap, cheap, expensive, too expensive) and finds where the resulting curves cross to triangulate an acceptable price range and a "too cheap" quality floor. Both methods date to the 1960s and 70s, predate modern experimental design, and remain the two defaults taught by nearly every market research shop, including SurveyMonkey, Sawtooth Software, Drive Research, QuestionPro, and Conjointly, whose comparison guides describe the two methods in nearly identical terms. ## When should you use each one, according to standard advice? The standard advice is a decision tree, not a validation claim. Use Van Westendorp early in a product's life, when the acceptable price range is unknown, especially for new or unfamiliar offerings. Use Gabor-Granger once the range is roughly known and the question has narrowed to a specific revenue-optimizing price, especially for existing products or line extensions. Most guides recommend running both in sequence, then patching known weaknesses with Newton-Miller-Smith or Rayner interpolation extensions. That advice answers which questions to ask. It does not answer whether the answers mean what buyers actually do at those prices, and none of the major guides publish a benchmark checking their outputs against real purchase behavior. ## Gabor-Granger vs. Van Westendorp, side by side ## Why do both produce numbers you can't defend in a real pricing decision? Both fail for the same underlying reason: they ask about price with nothing real at stake. Respondents answering a Gabor-Granger sequence or a Van Westendorp questionnaire are imagining a purchase, not making one, and stated purchase intent consistently runs higher than what people actually do once money changes hands. That gap has a name, hypothetical bias, and its direction is well established even where its exact size for a given market isn't. Van Westendorp compounds the problem by asking about price in isolation from competitors and product context, which the research community has criticized directly: the method's line-crossing logic "has been criticized and largely discredited for lacking a solid theoretical foundation and a track record of predictive success" (Wikipedia), and it tends toward "lowballing," producing optimal prices lower than what the market would actually bear (Relevant Insights). Even Sawtooth, which teaches the method, notes it doesn't natively predict purchase behavior, which is why practitioners patch it with Newton-Miller-Smith extensions rather than validating it against ground truth (Sawtooth Software). A patch on a stated-preference question is still a stated-preference question. [figure: A four-step chain showing how a stated pricing question, asked with nothing at stake, produces a self-reported number that is inflated by hypothetical bias and carries no confidence interval or causal claim.] ## What does a causal alternative look like? A causal alternative randomizes the price itself and measures the resulting change in choice, rather than asking about price directly. The causal claim comes from the randomization, not from the statistical model used to read it out: McFadden discrete choice, Mixed Logit, and ICLV are estimators for reading the results of a randomized experiment, not causal methods on their own. Where the analysis relies on a flat multinomial logit, preference-share and substitution estimates carry the independence-of-irrelevant-alternatives assumption, which can misstate how buyers switch between close substitutes; Mixed Logit and ICLV relax that assumption at the cost of added modeling complexity. This is also where a real validation number matters, and where it's still bounded by real limits. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market, and current model-by-model figures are published on the leaderboard rather than left as a single quoted number (see the causal fidelity paper). Publishing this caveat is what lets a buyer weigh the figure instead of just quoting it. That figure also comes with an honest caveat: some published human studies could exist in a model's training data, and the replication protocol is built to screen for that contamination risk rather than pretend it doesn't exist. A confidence interval from a simulated experiment describes the effect within the simulated population tested; it doesn't bound the real market unconditionally, which is exactly why it's a starting estimate to validate against real behavior, not a final answer. More on how that validation works is in the methods and validation series. ## So which one should you run? Neither, as your primary source of a price. If you're this far into comparing Gabor-Granger and Van Westendorp, you already have a rough price range and a specific decision in front of you: what happens to demand if price moves. That's a question about behavior under a change, which is what a randomized experiment is built to answer, not what an isolated, hypothetical survey question was ever designed to answer. This sits alongside the rest of our comparisons coverage of legacy stated-preference methods. Before commissioning either study, run one gap-check on data you already have: take a past Gabor-Granger or Van Westendorp result and compare it against actual conversion or renewal behavior at the prices you tested. If the stated number and the behavioral number diverge by more than a few points, that's hypothetical bias showing up in your own numbers, and it's the reason to test the next price change as an experiment instead of a question. If you want to see how a randomized pricing experiment is set up before running one, get in touch. --- # Comparing different approaches to claims' diagnostics URL: https://subconscious.ai/blog/comparisons/comparing-different-approaches-claims-diagnostics Published: 2026-08-27 Title picked: "Comparing Claims Diagnostics: What Each Method Actually Answers" Article body below, per brief. --- # Nokia Rebrand — Old vs. New? URL: https://subconscious.ai/blog/case-studies/nokia-rebrand-old-vs-new Published: 2026-08-27 A brand leader deciding whether to greenlight a rebrand needs one thing from the Nokia story: proof, not anecdote, that the new mark changes what buyers actually do. Nokia's February 2023 redesign replaced its 60-year-old wordmark and Nokia Blue with a lighter, disconnected geometric mark from Lippincott, timed to Mobile World Congress and meant to signal a shift from "phone brand" to B2B network-technology company (It's Nice That). Design press largely endorsed the strategic logic; social media reaction was harsh and near-instantly compared the new mark to Kia's 2021 rebrand (Creative Bloq). Neither company ran a controlled test before shipping the new logo, so everything written about either rebrand since, including this article's own sources, is a description of what happened after the fact, not proof of what the logo caused. - Nokia's old mark (Nokia Blue, block wordmark) is gone, replaced by lighter geometric letterforms from Lippincott, unveiled February 26, 2023 ahead of Mobile World Congress. - Design critics praised the B2B repositioning logic; general social sentiment was hostile and drew immediate comparisons to Kia. - Kia's 2021 rebrand is the reference case for "rebrand risk": a cited 27 percent brand-recognition drop and roughly 30,000 monthly searches for "KN car." - Kia's later sales recovery is routinely used to argue the backlash didn't matter, but that recovery coincided with a new EV lineup and a market-wide rebound, so it doesn't isolate the logo's effect. - The fix for both cases is the same: test the rebrand as a randomized choice experiment before it ships, not after. ## What exactly changed in Nokia's 2023 rebrand? Nokia retired its signature blue and the solid block wordmark it had used for roughly six decades, replacing them with thinner, disconnected geometric letterforms designed by Lippincott (It's Nice That). CEO Pekka Lundmark framed the change as a repositioning away from "phone brand" toward network infrastructure and B2B technology, timed to land just before Mobile World Congress. That framing matters for what comes next: the audience Nokia says it's rebranding for is enterprise buyers, not the people who post about logos on social media. ## Why did people immediately compare it to Kia? Because the visual language landed in the same territory: both moves swapped a heavy, literal wordmark for a lighter, more abstract geometric one, and viewers reached for the most recent comparable case they had. Creative Bloq documented the comparison directly, noting that social response was harsh even though the strategic rationale drew praise from critics (Creative Bloq). Transform magazine's assessment of the Lippincott work was similarly favorable on strategy, describing a considered rationale for the B2B shift even as public sentiment diverged sharply from critic sentiment (Transform). The split itself is the tell: design press and the general public were evaluating different things, one on strategic fit, the other on gut reaction to a familiar shape changing. ## Kia's rebrand: the number behind the comparison Kia's 2021 logo change is the case everyone reaches for, and it comes with real numbers attached. Verasight survey data attributed a 27 percent drop in brand recognition to the redesign (Verasight). The new mark was widely misread as "KN," generating roughly 30,000 monthly Google searches for "KN car" (SlashGear). Those are observational numbers, a survey and a search-volume count taken after the fact, and they measure confusion, not whether the logo itself changed anyone's purchase decision. ## Did Kia's later sales recovery prove the logo didn't hurt it? No. Kia's sales grew after 2021, but that growth arrived alongside a new EV lineup and a broader auto-market recovery, so crediting (or clearing) the logo for it is a correlation error, not a finding. This is the exact shape of the problem that makes rebrand case studies hard to reason about from prose alone: three separate causes moved at once, and only one outcome, sales, was observed. [figure: Diagram showing three arrows, labeled new EV lineup launched, auto market recovery post-2021, and logo redesign, all converging into a single box labeled observed sales change, illustrating confounded causes feeding one outcome.] The standard version of this story, both for Kia and now for Nokia, treats "the backlash faded" as evidence the rebrand worked. It's closer to a null result: nobody measured the logo's isolated effect at all, which means the claim that it didn't matter is unproven, not disproven. ## Live-market reaction vs. a controlled choice test The two approaches to answering "did the rebrand work" produce different evidence, on different timelines, from different populations. ## Why isn't social sentiment the buyer Nokia is targeting? Because a Twitter reply and a B2B procurement decision are made by different people, on different timelines, for different reasons. Nokia's stated goal was enterprise recall and consideration among network-technology buyers, not favorable sentiment among consumer phone users who remember the old brand. Stated social reaction is also a self-selected sample: the people motivated to post are disproportionately the ones with the strongest (usually negative) reaction, not a representative slice of the buying population. Treating that sample as market feedback measures noise, not the effect the rebrand was actually meant to produce. ## How do you test a rebrand before it ships? Run a randomized experiment where buyers choose between logo and positioning variants under controlled conditions, then analyze the choices with discrete choice models, McFadden's foundational logit, Mixed Logit, or ICLV, rather than reading sentiment after the fact. The causal identification comes from the randomization in the experiment design, not from the estimator; McFadden's model, Mixed Logit, and ICLV are ways to recover preferences and effect sizes from those randomized choices, not causal methods on their own. Mixed Logit is worth naming specifically because it relaxes the independence-of-irrelevant-alternatives (IIA) assumption that a flat logit imposes, which matters when a rebrand test includes several design variants a buyer might substitute between rather than a single up-or-down choice. This is the same logic behind Subconscious's validation work: a study replicated on a simulated population is checked against the original human study's direction and outcome. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). That figure is a validation-set result, not a guarantee for a new market or a new rebrand, and it comes with a standing caveat: some published human studies used for validation may overlap with a model's training data, which is why the replication protocol exists rather than treating held-out comparisons as a clean guarantee. The leaderboard publishes those replication results so buyers can check the method's track record before trusting it with their own decision, and the methods and validation hub covers how the discrete choice models above are applied and checked. ## What should a buyer do before the next rebrand? Write the choice test before the design is final, not after the launch generates headlines. Decide which two or three logo and positioning variants are actually on the table, define the buyer population the rebrand is meant to move (Nokia's case: enterprise network-technology purchasers, not consumer phone users), and run a randomized choice experiment across those variants before any of them ship. If a rebrand decision is on your desk now, that test is the concrete next step, not another round of reading social sentiment or comparable case studies. More rebrand and positioning decisions like this one are broken down in case studies. If it would help to think through the specific variants worth testing, get in touch. --- # Conversations, Not Checkboxes: A Smarter Way to Run Surveys URL: https://subconscious.ai/blog/methods-and-validation/conversations-not-checkboxes-smarter-way-run-surveys Published: 2026-08-27 A research lead deciding whether to move survey budget from static forms to AI-moderated conversation is really deciding whether the answer they get was ever tested against a real decision. Conversational interviewing fixes inattention: people write more, and write it more carefully, than they do clicking through a grid. It does not fix hypothetical bias: a more articulate transcript is still someone describing what they'd do, not doing it. The smarter run is a randomized discrete choice experiment validated against real behavior, not a friendlier interface. - Conversational AI interviewing (Listen Labs, Outset, and other vendors) produces richer, longer open-text answers than static forms, but that's a fix for inattention, not for hypothetical bias. - A data quality gain published without its failure mode is marketing copy. The same 1,800-participant study that found richer chatbot answers also found inflated false-positive coding errors tied to respondent acquiescence, saying more isn't the same as saying something more accurate. - Peer-reviewed work on stated versus revealed preference shows the say-do gap is structural: a friendlier interface makes the hypothetical answer more articulate, not more true. - A randomized discrete choice experiment, not a conversation, manipulates an attribute at survey scale and checks whether the response moves the way an actual choice would; field experiments and incentive-aligned designs do this too, a conversation does not. - leaderboard publishes that check against held-out human studies: our best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959), with a mean of 0.73 across the 43 studies that pass design filters, a validation-set result rather than a guarantee for a new market, detailed in the causal fidelity paper. ## What's wrong with checkbox surveys, and does conversation actually fix it? Checkbox surveys suffer from satisficing: straightlining, speeding, low-effort clicking that quietly corrupts a dataset. Conversation genuinely fixes this part. AI-moderated interviewing is now the fastest-growing corner of the market research industry. Vendors including Listen Labs and Outset pair LLM-moderated conversation with follow-up probing, Listen Labs alongside quant formats like Likert scales and MaxDiff inside a single session, and both are extending "conversation" from one-off studies toward ongoing tracking of customer experience. The mechanism behind the growth is real. A 1,800-participant web-survey field experiment found that LLM chatbot probing produced more detailed, informative open-ended answers than standard forms (arXiv 2504.13908). People engage longer with a conversation than with a grid, and they say more when something follows up on what they just said. That is a genuine data quality gain over a checkbox form. The miss sits on the record right next to the hit, where a buyer can weigh both. It is also not the whole story: the same study found that chatbot probing inflated false-positive coding errors, driven by respondent acquiescence bias. More detail didn't just mean more truth. Some of it was agreement dressed up as insight. "Our findings reveal that AI chatbots perform moderately well in live coding even without survey-specific fine-tuning, despite slightly inflated false positive errors due to respondent acquiescence bias." Barari and colleagues, arXiv:2504.13908 (source) ## Why a more articulate answer still isn't a causal one Inattention and hypothetical bias are different diseases, and conversation only treats one of them. Satisficing happens when someone hasn't engaged with the question at all. Hypothetical bias happens after full engagement: the respondent has thought carefully, answered sincerely, and still described what they'd do rather than done it. No amount of follow-up probing turns a stated intention into a revealed choice, because the person was never given a real tradeoff with a real cost. Naming a known gap lets a buyer check the claim before they spend on it. This gap is documented, not speculative. A peer-reviewed comparison in Health Economics found that stated-preference survey responses systematically diverge from revealed, actual behavior, and treated closing that gap as a methods problem requiring bias-reduction techniques, not an interface problem (de Corte et al., 2021). A separate natural experiment using a 2008 US tax rebate compared what people said they'd do with the money against what they actually spent it on, and found the same divergence when the stakes were real money rather than a hypothetical scenario (CEPR/VoxEU). Neither paper is about survey UX. Both are about what happens when you ask someone to narrate a decision instead of putting them through one. ## Two failure modes that look identical from the outside [figure: A two-column comparison. Left column, inattention: caused by low engagement, fixed by conversational probing and follow-up questions. Right column, hypothetical bias: caused by a fully sincere but hypothetical answer with no real cost attached, fixed only by a randomized experiment checked against real behavior.] ## Conversational interviewing versus a randomized discrete choice experiment ## How does a randomized experiment answer the question conversation can't? A randomized experiment answers it by manipulating an attribute across respondents and measuring whether the response moves the way a real choice would, instead of asking someone to describe their reasoning. McFadden discrete choice, Mixed Logit, and ICLV are estimators, not causal methods on their own; they're the statistical models used to read the results. The causal identification comes from the randomization in the design, not from which model fits the output afterward. Mixed Logit is worth naming specifically because it relaxes the independence-of-irrelevant-alternatives assumption that a flat multinomial logit carries, which matters directly for any preference-share or substitution question: if you're asking which action drives a shift in share between options, that IIA assumption determines whether the answer is trustworthy. The result of running that design against a simulated population, then checking it against a real human study, is captured in the causal fidelity paper: the best configuration reaches 87% of the measured human ceiling on one study (0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959), with a mean of 0.73 across the 43 studies that pass design filters. It is not a claim that any new market will replicate at that rate; it's a track record on the studies used to build it, published on the leaderboard rather than asserted. A published number carries its limits, so a buyer can verify it directly. One honest limitation belongs alongside it: some of those held-out human studies are published research, and published research can sit inside a model's training data. The replication protocol is built to guard against that contamination risk. It doesn't make the risk disappear, and no article claiming validation against a human baseline should pretend otherwise. ## Where does this leave a buyer deciding between them? It leaves the decision where it actually sits: conversation for finding the question, a randomized experiment for answering it. If the study is exploratory, mapping what customers are even thinking about before a category exists in your data, a conversational interface is the right tool and the engagement gains are real. If the study is meant to inform a decision with money behind it, which price holds, which message drives the outcome, which feature change moves the metric, a transcript full of well-articulated reasoning is not evidence that the reasoning would survive a real choice. That's the difference between a stated preference and a randomized intervention, and it doesn't close with a better chat interface. Methods and validation background for readers building this into a research stack is collected on the methods and validation blog. As a next step without talking to anyone: pull one recent decision your team validated with a conversational study, check whether it involved a randomized manipulation of an attribute with a measured effect size, and if it didn't, that's the study worth rerunning as a discrete choice experiment before the next budget cycle. If you want a second opinion on a specific design, the team is reachable through meet. --- # Conversational surveys explained in 6 minutes URL: https://subconscious.ai/blog/methods-and-validation/conversational-surveys-explained-6-minutes Published: 2026-08-27 A research or insights leader evaluating an AI-moderated interview tool needs one distinction before signing a contract: a conversational survey produces a longer, more textured stated opinion, not a validated prediction of behavior. The format swaps static Likert and open-end items for an LLM that asks adaptive follow-up questions, which is how Outset.ai, Listen Labs, Conveo, Strella, and a growing list of vendors now pitch qualitative and quantitative research alike. That gets a buyer more words per respondent and a higher completion rate. It does not, by itself, get them evidence that those words predict what the respondent will actually do. - What it is: a chat or voice interview where an LLM asks adaptive follow-ups instead of fixed survey items, marketed on completion rate and response depth. - Why adoption is accelerating: embedded adaptive interviews report completion rates far above linked email surveys, at a moment when traditional response rates keep falling. - What it doesn't fix: the say-do gap. Longer self-report is still self-report; nothing about the format attaches a real cost or trade-off to the answer. - A new risk it adds: the AI moderator's own question-selection pattern is itself an uncontrolled interviewer effect, on top of the say-do gap. - The fix: randomize the decision, not just the follow-up question, and check the result against a held-out human study. ## What is a conversational survey? A conversational survey is an interview in which an LLM asks adaptive follow-up questions based on what a respondent just said, instead of presenting the same fixed set of items to everyone. Outset.ai claims more than 500,000 AI-moderated interview hours across 10,000-plus studies in 85-plus countries, a vendor self-report on its own competitor-comparison page with no independent audit (Outset.ai). Listen Labs, Conveo, Strella, User Intuition, and Quals.ai run comparable products, and legacy platforms like Qualtrics and SurveyMonkey are adding conversational modules of their own. The pitch is consistent across vendors: richer, more human-sounding answers, gathered faster than a moderated human interview. ## Why adoption is accelerating now Adoption is accelerating because response rates to conventional surveys have been declining for years and conversational formats convert noticeably better on paper. Linked email surveys convert at roughly 6 to 15 percent, while embedded adaptive AI interviews report completion rates in the 40 to 70 percent range in early-2026 benchmark data (getperspective.ai). The two figures aren't measuring the same population: an embedded interview reaches someone already inside a panel or a live session, while a cold email invite has to earn the click first, so the gap overstates what switching format alone would buy. Vendors also advertise responses several times longer than matched open-end fields. For a research team measured on response volume, that gap is still enough to justify a switch on its own. ## What a longer answer actually measures A longer answer measures engagement and articulateness, not decision validity. None of the standard conversational-survey quality metrics are validated against what the respondent actually chose afterward: word count, completion rate, a vendor's proprietary "emotional intelligence" score. They measure how much a person said and how fluently, not whether saying it predicts doing it. That gap between stated opinion and actual behavior is the say-do gap, and it predates conversational formats. Adaptive follow-ups increase the volume and texture of an opinion; they do not attach a real trade-off or cost to it, so a respondent can still rationalize, anchor to what sounds socially acceptable, or answer a hypothetical as if it were free. [figure: A flow diagram showing an adaptive AI interview leading to a longer stated opinion, which stops at a break labeled say-do gap, while a separate parallel track shows a randomized experiment with a held-out group leading to a measured counterfactual and then the actual decision.] ## Does a richer answer predict an actual decision? Not on its own, and current evidence points to a second problem stacked on top of the first. AAPOR's 2026 guidance states that generative AI is reshaping survey research faster than existing methodological and governance standards can keep pace, and it specifically flags unresolved questions about the interviewer effects an AI moderator introduces on its own (AAPOR, PDF). That concern isn't hypothetical: an AI moderator makes its own choices about which follow-up to ask next, and that choice is a form of interviewer effect, just one AAPOR says the field doesn't yet have standards to measure. Separately, academic work on reinforcement learning for adaptive follow-up selection (AURA) is trying to reduce satisficing and raise response depth algorithmically, but the authors treat this as an active, unsettled research area, not a finished standard (arXiv). So the AI moderator is not a neutral pipe for collecting deeper opinion; it has its own question-selection bias, layered on the say-do gap the format was supposed to fix. ## Self-report or randomized experiment: what you're buying The honest way to tell them apart is to ask what varies and what's measured against it. ## How a randomized experiment closes the say-do gap A randomized experiment closes the gap by making the trade-off inside the question match the trade-off in the market, then analyzing the resulting choices instead of asking respondents to narrate their preferences. Subconscious runs randomized experiments analyzed with discrete choice models, specifically McFadden discrete choice, Mixed Logit, and ICLV; these are estimators that recover preference structure from a randomized manipulation, not causal methods on their own. The causal identification comes from the randomization built into the experiment design, not from the estimator that reads it out. Publishing the assumption behind a number is what lets a buyer check it before trusting the number. A plain multinomial logit model carries an independence-of-irrelevant-alternatives assumption, and Mixed Logit is one way to relax that assumption when substitution between options matters, so a preference-share number from a flat logit should be read with that constraint in mind. Replication accuracy is the number worth pressing a vendor on. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. It's a validation result, not a guarantee for a market that hasn't been tested yet, and because some of the underlying published studies could plausibly sit inside a model's training data, that number alone doesn't rule out memorization; the replication protocol behind it, and the ongoing public results on the leaderboard, matter more than the headline figure. ## Where conversational interviews still earn a place They earn a place upstream, in discovery, not downstream, in the decision. An adaptive interview is a reasonable way to surface language, objections, and hypotheses before a study is designed; it's a poor way to validate which of those hypotheses will actually move a purchase, signup, or churn decision. The methodical version of that workflow, generate hypotheses conversationally, then test the ones that matter with a randomized experiment, is covered in more depth in the methods and validation hub. Before renewing a conversational-survey contract, ask the vendor for one number: their predictive-validity or replication rate against real purchase, usage, or churn data, not completion rate, not word count, not a proprietary sentiment score. If they don't have one, the tool is measuring engagement, not decision validity. From there, book time to see how the replication protocol works against your own market. --- # How to manually calculate partworth utilities URL: https://subconscious.ai/blog/methods-and-validation/manually-calculate-partworth-utilities Published: 2026-08-27 Buyers evaluating a conjoint vendor's output have one real decision to make: whether to trust the part-worth utilities enough to act on them, not whether the underlying regression ran correctly. You can manually calculate partworth utilities from a rating-based (CVA) study in three steps: compute each attribute's utility range from the regression coefficients, zero-center the part-worths within each attribute, then rescale across attributes to get relative importance. A number without its limits is marketing. Publishing this one lets a buyer check it against the data itself. That check confirms the estimator was applied correctly to the data it was given. It says nothing about whether the preferences it recovered are the ones respondents would actually act on. - The manual process is standard: compute attribute-level utility ranges, zero-center part-worths within each attribute, rescale across attributes for relative importance. - OLS regression handles rating-based CVA designs; hierarchical Bayes (HB) is now standard for choice-based (CBC) designs because CBC tasks are too sparse per respondent for OLS to solve individually. - A correct calculation proves the estimator was applied correctly. It does not prove the responses predict real-world behavior. - Well-designed paired discrete-choice conjoint has matched real referendum outcomes in published research; ranking and rating tasks degraded badly by comparison. - Subconscious's best configuration reaches 87% of a measured human ceiling on one study: 0.832 rank correlation against a published human result, where two independent human samples reach a rank correlation of 0.959 with each other. Mean across the 43 studies that passed design filters is 0.73 (Causal Fidelity paper). That is a validation result, not a guarantee for a new market. ## How do you manually calculate partworth utilities? You run the process every conjoint textbook and vendor help center describes the same way: fit a regression of ratings or choices on dummy-coded attribute levels, compute the utility range within each attribute (highest coefficient minus lowest), zero-center the part-worths so they sum to zero within each attribute, then divide each attribute's range by the sum of all attribute ranges to get relative importance weights (QuestionPro, Part Worths - Conjoint Analysis). For a simple CVA study with dummy-coded levels and individual-level data, this is a spreadsheet-tractable OLS regression. Academic guides from WU Vienna and MIT Sloan teach the identical sequence. None of these guides claim the output is anything more than a correctly transformed version of the input data. ## Why most CBC studies use hierarchical Bayes instead of OLS Because modern choice-based conjoint designs don't give OLS enough data per respondent to solve. Hierarchical Bayes was introduced to conjoint estimation around 1995 specifically to recover individual-level part-worths from CBC designs where each respondent sees too few profiles for an aggregate or per-person OLS fit to converge (Sawtooth Software, Hierarchical Bayes Estimation). HB borrows statistical strength across respondents to stabilize individual estimates, which is why it dominates CBC pipelines today while OLS remains fine for the older rating-based CVA format. This is a real methodological choice, and vendors are right to explain it. It is also a choice entirely internal to the estimator. The misses belong on the record next to the hits. Naming this gap is what lets a buyer test for it before acting on the output. Neither OLS nor HB has any mechanism for checking whether the choices respondents made in the survey resemble the choices they'd make with real money and real consequences. ## Auditing the arithmetic vs validating the preferences These are different checks, and a buyer who only runs the first one has not touched the second. ## What does a correct part-worth calculation actually prove? A correct regression proves arithmetic, and saying that plainly lets a buyer separate it from a claim about behavior. It proves the arithmetic is right, not that the preferences are real. Reverse-engineering a vendor's coefficients and finding they "check out" confirms internal consistency between the estimator and the input data. It says nothing about external validity, which is whether the input data (the choices respondents made in a survey) resembles the choices those same people would make outside it. A buyer who stops at the spreadsheet has validated the layer of the claim that was never in question. The layer that matters, whether the recovered preferences predict real behavior, sits one step further out and requires a different kind of check entirely. [figure: A left-to-right chain showing survey response feeding an estimator, producing part-worths, where a manual arithmetic check stops, followed by an unverified gap before real-world choice.] ## Do conjoint results predict real-world behavior? Sometimes. It depends on task design, not arithmetic. A 2015 PNAS study by Hainmueller, Hangartner, and Yamamoto compared conjoint-derived effects on support for immigrant naturalization against actual outcomes from Swiss municipal referendums and found close correspondence, but only for paired discrete-choice tasks (PNAS, 2015). Ranking and rating tasks, the same tasks that OLS was built to handle, degraded badly by comparison in the same study. That result was earned through design discipline and an out-of-sample comparison against real behavior, not through auditing a regression. It is also a single validated domain (referendum voting), which is a limitation worth stating plainly rather than generalizing past. ## Where does the say-do gap hide inside a correct regression? A number on its own is marketing. Publishing where it fails is what lets a buyer verify it. It hides in the survey responses themselves, before the estimator ever runs, which is why no amount of recomputation catches it. Hypothetical bias, the tendency for stated willingness to act to run higher than actual behavior, is a documented and unresolved problem in the stated-choice literature. A part-worth calculation has no step that detects this. It takes whatever the respondent said as ground truth and transforms it faithfully. The transformation can be flawless and the ground truth can still be wrong. ## How does Subconscious validate part-worths against real behavior instead of just the math? By running randomized experiments analyzed with discrete choice models on a simulated population and checking the result against a measured human ceiling before treating it as usable. Subconscious's methods are McFadden discrete choice, Mixed Logit, and ICLV. These are estimators, not causal methods. The causal identification comes from the randomized manipulation built into the experiment design, not from the choice of model. Flat multinomial logit assumes independence of irrelevant alternatives, which is one reason Mixed Logit and ICLV are used for preference-share and substitution questions rather than a base logit alone. On the replication protocol, the best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against a published human result, where two independent human samples reach a rank correlation of 0.959 with each other. Across the 43 studies that passed design filters, the mean is 0.73 (Causal Fidelity paper). Full results by study are on the leaderboard. That is a validation result on studies already measured, not a guarantee for a new market. Published human studies can sit inside a model's training data, so the replication protocol is built to check for that rather than assume it away. The general validation approach is covered in the methods and validation hub. If you're deciding whether to trust a set of part-worth utilities, don't stop at recomputing the coefficients. Ask the vendor what independent behavioral benchmark, if any, the output was checked against, and whether that check used paired discrete-choice tasks or ratings and rankings, since the PNAS results show that distinction matters more than the estimator does. If you want a second read on how a study's replication protocol was structured before you commission it, book a working session with the team. --- # How to get the most out of open-ended questions URL: https://subconscious.ai/blog/methods-and-validation/get-most-out-open-ended-questions Published: 2026-08-27 An insights director staring down three thousand open-ended verbatims before a launch decision needs one thing: to know which comments describe the real reason people chose what they chose, and which are filler. The direct answer is that coding the text better, even with a well-built AI pipeline, only cleans up self-report. To find out which claimed reason actually moved the choice, you need a randomized discrete-choice experiment on the same population, with a confidence interval attached to the answer. Verbatims tell you what respondents say moved them; only a randomized manipulation tells you what did. - Open-ended text captures stated reasons filtered through recall bias, social desirability, and satisficing. AI coding organizes that text; it does not remove the bias underneath it. - 95 percent of market researchers now use AI tools regularly or experimentally for tasks like verbatim coding, and 66 percent rely on AI built into their research software, per Qualtrics' 2026 Market Research Trends report, based on a Q3 2025 survey of 3,000+ researchers. - At least nine distinct AI coding methods are in active commercial use, with no single validated standard among them, per Displayr's 2026 catalog. - The fix is not a better codebook. It is a randomized experiment on the same population, analyzed with Mixed Logit or ICLV, that tests which of the stated reasons actually carried causal weight. - A fidelity score published without its ceiling is just marketing, so we show both together. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper); current studies and their scores are public on the leaderboard. ## The open-end problem no coding tool fixes Panel-scale data collection has buried analysts in verbatims faster than any team can read them. That's the honest starting point: most of that text was never going to be signal. A meaningful share of any open-ended corpus is satisficer noise, typed answers like "good," "fine," or a copy-pasted sentence from three questions earlier, produced by respondents trying to finish the survey, not answer the question. That noise exists before any AI model touches the data. Coding it faster does not make it truer. ## What's the fastest way to make sense of thousands of open-ended verbatims? Right now, the fastest way is AI-assisted coding, and the market has already standardized on using it, just not on how. Qualtrics' 2026 Market Research Trends report, based on a Q3 2025 survey of more than 3,000 market research professionals across 14 countries, found 95 percent now use AI tools regularly or experimentally, with 66 percent relying on AI embedded directly in their research software, up from 62 percent in 2024 (Qualtrics, 2026 Market Research Trends). Tools from Qualtrics Text iQ, Ascribe/Forsta, and others now run sentiment scoring, automated theme extraction, and human-in-the-loop code assignment. Displayr's own catalog counts nine separate AI methods in active use for coding open-ends, a vendor listing rather than an independent survey of practice (Displayr, 2026). Nine competing approaches, even by a vendor's own count, is evidence of a fragmented practice, not a validated one. ## Where AI-assisted coding still breaks Publishing exactly where a method breaks lets a buyer test it before betting a launch decision on it. It still breaks at the same place human coding always broke: turning what someone typed into what someone meant. A JMIR AI study directly compared LLM-generated thematic summaries against human-coded ones in qualitative health care research and found LLMs can approximate human themes but do not reliably replicate them without independent validation (JMIR AI, 2025). A 2025 review of LLM-assisted thematic analysis goes further, warning that heavy reliance on automation risks "premature closure," where the model settles on a plausible-sounding theme structure before the actual variation in the data has been explored (arXiv, 2511.14528). Hallucination and prompt sensitivity compound the problem: change the prompt, get a different codebook, from the same verbatims. "Participants recognized potential efficiency and scalability gains, but highlighted risks related to bias, contextual loss, reproducibility, and the rapid evolution of LLMs." Ornelas and colleagues, ISERN workshop study on LLM-assisted thematic analysis, arXiv:2511.14528 (source) ## The gap between what people say and what moves them None of that fixes the deeper issue: a theme's prevalence in a codebook is not the same measurement as its causal weight in a decision. A verbatim that fifty respondents typed and a discrete-choice effect that shifted fifty respondents' actual choices both arrive as a number, which makes them easy to mistake for the same kind of evidence. They are not, because they were produced by different processes. [figure: Two parallel process chains. The top chain runs from respondent recall to typed reason to AI-coded theme to a prevalence percentage. The bottom chain runs from random assignment to discrete choice to a Mixed Logit or ICLV estimate to a confidence interval.] ## Can you get a causal answer straight from what people typed? No. Self-report, however well coded, has no random assignment in it, and without random assignment there's no way to separate what caused a choice from what a respondent believes or claims caused it. Discrete choice models, McFadden's original conditional logit, Mixed Logit, and ICLV, are estimators, not causal methods on their own. The causal identification comes from the randomized manipulation built into the experiment design: respondents are randomly shown different combinations of attributes and prices, and the model estimates which attribute actually shifted the choice. (A standard multinomial logit also carries the independence-of-irrelevant-alternatives assumption, which is why substitution patterns are usually checked against a Mixed Logit before they're trusted.) That's the structural difference between "why respondents say they chose it" and "why they chose it." ## What the replication numbers do and don't cover The misses sit on the leaderboard next to the hits. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). It is a validation result, not a guarantee for a new, unpublished market question. Current studies and their individual scores are tracked in the open on the leaderboard, so a buyer can check the number against a live record rather than a claim in a deck. ## Open-ended coding vs a randomized experiment: what each actually proves ## How to combine both, and what to do next Use the open-ends for what they're good at: generating the list of candidate reasons, in respondents' own language, before you spend anything on testing. Then take that list into a randomized experiment on the same population and let the discrete-choice model tell you which one actually moved the choice, with a confidence interval that covers the effect within that simulated population, not a bound on how the real market will behave in every future condition. Treat the verbatims as hypotheses, not conclusions, and treat the experiment's confidence interval, not the theme's word count, as the number you brief the launch decision on. For a worked example of this pairing in practice, see the methods and validation archive and a related case study. Concrete next step: pull last quarter's top three open-ended themes, write each as a testable claim, and run them as attributes in a randomized discrete-choice experiment before the launch decision locks. If you want a second set of eyes on the design, meet the team. --- # How to compare results across segments with crosstabs URL: https://subconscious.ai/blog/methods-and-validation/compare-results-across-segments-crosstabs Published: 2026-08-27 An insights director holding a banner table where Segment A converts 15 points higher than Segment B, with a budget reallocation decision riding on that gap, needs more than the crosstab to act. A crosstab that compares results across segments tests association within one observational sample: a starred cell means the split differs here, not that segment membership caused it. Only a randomized experiment, run separately inside each segment and holding everything else constant, identifies whether the intervention actually produces the gap. Act on the crosstab without that second step and you're one confounder away from optimizing for the wrong segment. - Crosstabs test association, not causation: a starred cell means the split differs in this sample, not that segment membership caused the difference (Qualtrics). - Simpson's Paradox is the documented failure mode: a trend that holds in every subgroup can reverse once the subgroups are combined, driven by an uneven confounder (MoreSteam). - Auto-generated crosstab cuts multiply false positives: hundreds of cuts run without correction turn "p<.05" into noise (Greenbook). - Stated survey answers carry hypothetical bias, and because overclaiming need not be equal across segments, a rate difference can reflect who overclaims rather than what a segment would actually choose. - The fix is structural: randomize the intervention inside each segment. A bigger table or a stricter p-value threshold doesn't touch the underlying problem. ## What does a crosstab actually measure? A crosstab measures the association between a segment variable and an outcome inside one observational sample. The standard workflow hasn't changed much in twenty years: pull a variable, cross it against segment, and flag cells that clear a chi-square or Z-test. Qualtrics, SurveyMonkey, and Kantar all document this as the complete method (Qualtrics). That workflow answers one question well: do these two groups look different in this dataset. It does not answer whether segment membership is the thing producing the difference, because nothing in a crosstab controls for what else moves alongside segment membership, age correlating with tenure, region correlating with channel mix, tier correlating with usage frequency. The table reports a correlation. A buyer reading it as a cause is making an inferential leap the table itself never licenses. ## Why can a "significant" segment difference be the wrong signal? A significant crosstab cell can be the wrong signal because the segment variable is standing in for a confounder that's unevenly distributed across the groups being compared. This is Simpson's Paradox: a trend visible in every subgroup can reverse when the subgroups are pooled, precisely because segment membership correlates with something else doing the causal work (MoreSteam). A classic version: a product outperforms a competitor in every region when read separately, then loses overall once the regions are combined, because the region with the worst comparison also happens to carry most of the volume. Nothing in a chi-square test catches this. The test only tells you a cell differs from what chance would predict, not whether the variable you crossed is the one responsible. [figure: Diagram showing Segment A outperforming Segment B in two separate subgroup comparisons, then the combined total reversing to favor Segment B.] ## The say-do gap hiding inside segment splits Crosstabs are built on stated survey responses, and stated responses diverge from actual behavior: this is hypothetical bias, and there's no reason to expect the overclaiming to be equal across segments. If one segment tends to overclaim interest more than another, a "Segment A rates this 15 points higher" finding can be an artifact of who overclaims, not what Segment A would actually pick if the choice cost something. A crosstab has no mechanism to separate the two. It reports the stated number as if it were the behavior. ## How many crosstab cuts can one dataset support before "significant" stops meaning anything? A dataset can support far fewer significant cuts than most banner tables run, because every additional test raises the odds of a false positive unless the analysis corrects for it. Greenbook flags this directly: repeated significance testing across many crosstab cells in a single study inflates the false-positive rate when no correction is applied, and that's a common gap in standard banner-table workflows (Greenbook). AI-assisted platforms make this worse by volume, auto-generating hundreds of cuts per study, which increases both the false-positive count and the pressure on analysts to trust the automated star rather than check the underlying cell size. Cell size is the part the automated star doesn't show: a cell with too few respondents can flip significance on a handful of answers, and whether chi-square is even the right test gets shakier once segment sizes are unequal to begin with. The fix isn't a bigger p-value threshold, it's checking the underlying n before trusting the flag. ## What does a randomized experiment inside each segment fix? Publishing where a randomized experiment stops is what lets a buyer check the claim against the design. A randomized experiment fixes the confounding problem, but it identifies the effect of the intervention within the segment, not the effect of segment membership itself; segment is never randomized, so it's never a causal variable in the design. Instead of splitting an existing sample and comparing stated answers, the design randomly assigns the same intervention (a price, a message, a feature) within each segment's own sample, then compares outcomes between the randomized arms inside that segment. That isolates what the intervention does inside Segment A from what it does inside Segment B: if the effect size differs by segment, that's a real, causally identified difference in how each segment responds to the intervention, not evidence about what caused the segments to exist or differ in the first place. McFadden discrete choice and Mixed Logit are the estimators used to size the resulting effect from the randomized choices. McFadden's standard specification assumes independence of irrelevant alternatives (IIA); Mixed Logit relaxes that assumption by allowing preferences to vary across respondents, but neither estimator is what makes the result causal. The causal claim comes from the randomization in the experiment design, not from the choice model applied afterward. More on how that identification works sits on the methods and validation hub. [figure: Flow diagram showing a shared starting path that splits into two branches: one ends at a significance-tested association, the other continues through randomization to a causal effect estimate with a confidence interval.] ## Crosstab vs randomized experiment by segment ## Where does replication accuracy fit into this decision? Replication accuracy tells a buyer how often this method has agreed with real human behavior before, which is a different question from whether any single result is right. A replication number without its limits is marketing, and the misses sit on the public leaderboard next to the hits. On the causal fidelity paper, our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. That figure describes a validation result, not a guarantee for a market that hasn't been tested yet, and it comes with a real methodological caveat: some of the human studies in that set were published before the underlying models were trained, so the replication protocol has to account for the possibility that a model recalled an answer rather than derived it, not pretend that risk away. The leaderboard tracks replication accuracy by domain, so a buyer can check the number for a category close to their own before trusting it for a new decision. ## What should a buyer do with this week's banner table? Take the one segment split currently driving a budget or messaging decision, and ask whether it would survive a randomized version: same intervention, same two segments, assignment randomized within each. If nobody can answer that without more data, the crosstab hasn't yet earned the decision it's being used to justify. Design the experiment first: randomize the intervention inside each segment, hold the rest constant, and compare the treated and control arms within each group before reallocating anything based on the original table. For a look at how that kind of segment-level experiment gets designed and validated, the team walks through it on a call. --- # Availability adjustment for conjoint preference shares URL: https://subconscious.ai/blog/methods-and-validation/availability-adjustment-conjoint-preference-shares Published: 2026-08-27 A pricing lead deciding whether to raise price off a conjoint simulator's forecast is really asking one question: does availability adjustment make the preference share trustworthy? A number without its limits is marketing. It does not. Availability adjustment scales simulated preference shares down for market realities the design can't see on its own, mainly which products shoppers actually know about and can actually buy, so the topline lines up with observed market shares. It fixes that one number. It does not check whether the attribute-level effects that produced it are correct. It doesn't tell you which feature actually moved choice. It doesn't tell you how much price actually mattered. - Availability adjustment corrects the topline for unequal awareness and distribution. It does not validate the causal drivers underneath the topline. - Sawtooth's Lighthouse Studio and Displayr apply availability adjustment inside a fixed four-step sequence: choice rule, availability, scale factor, calibration, and the order changes the answer. - Calibrating a simulated share to match a known market share can hide a wrong attribute effect, because the calibration step is built to absorb whatever gap exists, not to expose it. - A flat logit share estimate assumes independence of irrelevant alternatives (IIA); that assumption, not the adjustment, is what breaks when profiles are close substitutes. - The way to check attribute-level accuracy is replication against real human behavior, not a closer topline match. ## What availability adjustment corrects, and what it assumes Raw simulated preference share assumes every profile in a conjoint design is equally known and equally in stock, which no real market is. A new entrant with zero brand awareness gets simulated as if shoppers already know it exists on the shelf next to the incumbent. Availability adjustment corrects that by down-weighting or excluding profiles the analyst flags as unavailable or unfamiliar, so the simulated share reflects actual distribution and awareness instead of a fully-stocked hypothetical. Sawtooth's Lighthouse Studio implements this literally: its simulator can search for utility adjustments until simulated shares match a target share the analyst types into a Target Share column (Sawtooth Software, Share Adjustment). That target is set by the analyst, not derived by the model. It is a deliberate correction to one output, not a check on the mechanism that generated it. ## The standard four-step stack, and why order changes the result Sawtooth's own framework, published by Orme and Johnson (2006), codifies four sequential adjustments. First, change the choice rule. Then apply availability. Then tune a scale factor. Then calibrate the residual gap directly to known shares (Orme and Johnson, 2006). Displayr's documentation for calibrating conjoint simulators to market share mandates the identical order: availability before scaling, scaling before calibration. Running the steps out of sequence changes the resulting shares (Displayr). The scale factor step exists because first-choice-rule simulators reliably overstate the sharpness of real differences between products, especially for the already popular ones. Sawtooth's default exponent is 1. Analysts lower it specifically to flatten that overstated spread (Sawtooth Software, Share of Preference Options). ## Does calibrating the topline validate the attribute effects underneath it? The misses go on the record next to the hits, so a buyer can check them. No. Calibration adjusts the aggregate number to match a target; it says nothing about whether the feature-level effects that generated it are real. The calibration constant in that four-step stack is engineered specifically to absorb whatever gap exists between the raw simulated share and the target share. Consider a simple illustration. Say the true price elasticity implies a premium option should hold 30 percent share, but a mis-specified model puts it at 45 percent. Say a second, correctly specified model puts it at 32 percent. Calibration nudges both toward the same observed 33 percent market share: the first model absorbs a 12-point correction, the second a 1-point correction, and both simulators now report 33 percent. The topline is identical. The price elasticity feeding it is not. Availability adjustment fixes the marginal number, not the causal validity of the preference estimates underneath it. A calibrated topline is not proof that the attribute effects driving your actual decision are real. [figure: Diagram showing two parallel paths, one with correct attribute effects and one with wrong attribute effects, both passing through a calibration constant and arriving at the same final topline share number.] ## Why representativeness in the design matters as much as the adjustment Adjustment happens downstream of a deeper problem. Political-science methodologists Hainmueller, Hangartner, and Yamamoto (2015), writing in the Proceedings of the National Academy of Sciences, found that attribute-level effects (AMCEs) from a conjoint design matched real-world outcomes only when the profile distribution used in the design mirrored real-world attribute correlations, not the uniform-random distribution most conjoint tools default to. Naming this failure mode is what lets a buyer check the design before they lean on it. That result comes from a single political-science study; it has not been replicated across commercial market categories, so treat it as a documented risk, not a universal law. Stack it underneath a calibrated topline anyway and the failure mode compounds: a share number can match market history for reasons that have nothing to do with the mechanism your pricing or roadmap decision actually depends on. Availability adjustment cannot fix a design that never had representative attribute correlations to begin with; it operates entirely on the output side. ## What is the IIA assumption, and why does it matter for preference shares? The independence of irrelevant alternatives (IIA) assumption is what makes a flat logit's first-choice preference share possible to compute at all, and it's also what makes that share wrong when two profiles are close substitutes. A standard McFadden discrete choice model assumes a shopper's relative preference between any two options doesn't depend on what else is in the set, so adding a near-identical competitor pulls share proportionally from every existing option instead of mostly from its closest substitute. That's the same mechanism behind Sawtooth's scale factor guidance: first-choice simulators overstate sharpness for popular products precisely because the model isn't representing substitution correctly. Mixed Logit relaxes IIA by allowing preference parameters to vary across simulated respondents instead of assuming one fixed set of tradeoffs for everyone, and ICLV goes further by modeling the latent attitudes behind those tradeoffs directly. Neither availability adjustment nor scale-factor tuning relaxes the IIA assumption; they reshape the output of whichever estimator is running underneath. ## What actually validates the causal estimates behind a preference share? Before a buyer commits budget to a pricing or roadmap call, the question that matters is whether the attribute effects behind the forecast are real, not whether the topline number matches. That validation doesn't come from the estimator. Causal identification in a conjoint study comes from the randomized attribute manipulation built into the experiment design, not from the estimator applied afterward. McFadden discrete choice, Mixed Logit, and ICLV are estimators for reading that randomization: they describe how the choices are modeled, not why the result is causal. The estimator doesn't make the result trustworthy; the random assignment of attributes across profiles does. The way to check whether the resulting attribute effects are trustworthy, separate from whether the topline share happens to match, is replication against independently observed human choice behavior. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. It is a validation result, not a guarantee for a new market. A validation number earns trust only when its own blind spots are published alongside it. It also doesn't erase a real limitation: published human studies used for validation can sit inside a model's training data, and the replication protocol is built to address that risk rather than pretend it doesn't exist. Results are tracked openly on the leaderboard rather than reported only in a vendor's own case study, which is worth checking against whatever validation number a conjoint vendor gives you for their own tool. ## A decision checklist before you size a decision on a calibrated share Before a pricing or roadmap decision leans on a calibrated conjoint forecast, four questions separate a real causal estimate from a curve-fit: - What is the attribute-level replication rate against real human behavior, not just the topline share match? - Did the design's profile distribution mirror real-world attribute correlations, or was it uniform-random? - What scale factor was applied, and does the analyst know why it was needed? - Was the Target Share value in the calibration step set independently, or reverse-engineered from the market answer already known? If a vendor's answer to any of these is "we calibrated the topline," that answer describes calibration, not causal validation. Pull the adjustment log from your last simulator run, check whether the calibration step is larger than the attribute effect your decision depends on, and if it is, ask for the replication evidence before you commit budget. To see how attribute-level replication is checked before a topline number ever gets calibrated, get in touch. --- # Market Research Brief - Free Template + Examples URL: https://subconscious.ai/blog/implementation-and-procurement/market-research-brief-free-template-examples Published: 2026-08-27 A senior insights buyer picking a market research brief template has one real decision to make: whether the standard seven-field format is enough, or whether the brief also needs to specify how the resulting number gets checked against reality. Free templates covering that seven-field format exist from FlexMR, Fieldworkhub, Milanote, HolaBrief, Vision One, and Conjointly, and any one of them works as a starting point. None of them ask you to name the causal claim you're testing or the validation standard the vendor must clear before you trust what comes back. - A standard brief template (background, objective, audience, methodology, timeline, budget) is table stakes; free versions exist from FlexMR, Fieldworkhub, and others, and any competent vendor expects one. - A well-scoped brief still buys an unreliable answer if the study relies on stated preference alone: a published vaccination-behavior study found the negative predictive value of stated "no" answers was just 26%, meaning most stated refusals didn't hold up against actual behavior. The number is specific to vaccination behavior; the underlying gap between a stated answer and real behavior is not (PMC). - Panel fraud and bot contamination force researchers to discard a large share of fielded survey data: one industry estimate puts the average at 38% of collected data, up to 70% in some studies. That figure comes from a vendor blog without a published methodology, so treat it as directional, not precise (User Intuition). - The fix is not a longer template. It's two added fields: the causal claim being tested, and the validation standard the vendor must clear before the finding ships. - A senior buyer's real decision is whether to accept a vendor's stated-preference number on faith or require it be checked against a documented replication standard, such as the one published on the leaderboard. ## What does a standard market research brief template include? A standard brief template asks for seven things: background context, a single-sentence objective, the target audience, the methodology, deliverables, a timeline, and a budget. That structure is necessary but not sufficient: it says nothing about whether the study's output reflects real behavior. Two rules sharpen the template itself. FlexMR's guidance on the objective line: describe the knowledge gap you need closed, not the method, and keep it to one sentence so the brief doesn't drift into scope creep (FlexMR). Fieldworkhub's sequencing rule: start from the business decision the brief exists to support, then work backward into research questions (Fieldworkhub). ## Why isn't a well-scoped brief enough to trust the answer? A well-scoped brief can still commission a study that gets the underlying behavior wrong, because scoping and validity are different problems. A published comparison of stated and revealed preference in vaccination behavior found the two matched in 80% of respondents overall. That headline number hides a lopsided failure pattern: positive predictive value (a stated "yes" turning into real behavior) was 85%, while negative predictive value (a stated "no" holding up) was just 26% (PMC). In plain terms, when respondents said they wouldn't do something, they were wrong roughly three times out of four. Vaccination behavior is not pricing or product positioning, and the exact miss rate won't carry over to every market research question. But the mechanism is domain-general: a self-report about future behavior is not the behavior itself, and a textbook-perfect brief has no field that asks a vendor to check the gap between the two. "In 80% of the respondents the stated and revealed preferences corresponded. The positive predictive value is 85% but the negative predictive value is 26%." Lambooij and colleagues, BMC Medical Research Methodology (source) [figure: Bar chart showing three values from a stated versus revealed preference study on vaccination: overall correspondence 80%, positive predictive value 85%, negative predictive value 26%.] ## Why does data quality also belong in the brief? Data quality belongs in the brief because a clean scope says nothing about whether the fielded responses came from real people. One industry account puts the average discard rate to AI-generated panel fraud and bots at 38% of collected data, with some studies losing up to 70% (User Intuition). That figure comes from a vendor blog post without a published methodology or sample size, so treat it as directional, not precise. A brief optimized purely for scoping clarity doesn't touch this problem. It specifies who to study and what to ask. It says nothing about whether the fielded sample is real people, or whether the method used on real people produces answers that hold up against behavior. ## What should a causal-scoped brief include instead? A causal-scoped brief adds two fields to the standard template: the causal claim being tested, and the validation standard the vendor must clear. Naming the causal claim means stating what you're trying to identify, not just what you're trying to measure. For example: "does price framing X change purchase intent, independent of who sees it" rather than "understand reactions to price framing X." That distinction matters because discrete choice models, Mixed Logit, and ICLV are estimators, not causal methods on their own. Identification comes from the randomized manipulation built into the experiment design; the estimator just fits the model to the choices that design produced. A brief that specifies "use a discrete choice survey" without specifying what's being randomized and against what baseline is specifying a technique, not a causal claim. ## What validation standard should a vendor have to clear before you trust the number? A vendor should have to show its method's output correlates with a measured human baseline, with the ratio and its denominator stated, not a bare percentage. On one published study, the best-performing configuration reached 87% of the measured human ceiling. That's a 0.832 rank correlation against the published human result, against a ceiling of 0.959 measured between two independent samples of real humans answering the same study. Across all 43 studies that passed the design filters, the mean was lower: 0.73 of ceiling (causal fidelity paper). That's a validation result on studies already run, not a guarantee for a new market you haven't tested. It also carries a standing caveat worth putting in your brief: published human studies used as a validation benchmark may already sit in a model's training data. That's why replication protocols compare against held-out results rather than treating any single match as proof. The leaderboard publishes this validation ratio study by study, method by method, so a buyer can check a vendor's number against others before trusting it. Background on how discrete choice, Mixed Logit, and ICLV get validated against human baselines is covered on the methods and validation hub. ## Standard brief template vs. causal-scoped brief Both columns start from the same seven fields. The difference is two additional lines and the discipline to ask a vendor to show its work against a human baseline before the number goes into a decision memo. Take your existing brief template and add two lines to the objective section: the causal claim you're testing, stated as an intervention and an outcome, and the validation standard your vendor has to report before you accept the result. If you want a second pair of eyes on how to phrase the causal claim itself, talk to the team. --- # Causal AI: What It Is and What It Buys a Business Decision URL: https://subconscious.ai/blog/methods-and-validation/causal-ai-what-it-is-what-it-buys-business-decision Published: 2026-08-28 Three price points sit on a CPG pricing lead's desk before a shelf reset; none of last year's sales data covers any of them. Causal AI answers what happens if we act, where predictive analytics answers what usually happens. What causal AI buys that lead is an estimated effect for the action on the table, with a confidence interval, derived from a randomized experiment rather than a correlation in historical data. - The causal claim comes from randomizing which price, claim, or pack a respondent sees inside an experiment. McFadden discrete choice, Mixed Logit, and ICLV are the estimators that read the result afterward; the randomization is what makes the read causal. - On one published study, the best configuration reaches 0.832 rank correlation against a human baseline of 0.959, 87% of that measured ceiling; across the 43 studies passing design filters, the mean is 0.73 (causal fidelity paper). - Causal AI is not causal discovery from observational data alone, and a validated replication score is not a guarantee for a market that hasn't been tested. - The concrete case in this article: a pricing lead choosing among three shelf prices, walked through Judea Pearl's three rungs of causal knowledge - seeing, doing, imagining. ## What is causal AI, and how is it different from predictive AI? Causal AI estimates the effect of an action a business has not yet taken, using a randomized experiment rather than historical correlation. Predictive AI - sales forecasting, marketing mix models, most machine learning classifiers - describes what has usually happened given the conditions on record. Judea Pearl and Dana Mackenzie organized this distinction into three rungs of a ladder: seeing (association), doing (intervention), and imagining (counterfactual) (The Book of Why, 2018). Predictive AI lives on the first rung; causal AI operates on the second, by randomizing the action inside an experiment, and can extend to the third, by estimating what an action not taken would have produced. The tool category matters less than which rung the method actually reaches. [figure: A three-step list showing Pearl's ladder of causality - seeing, doing, imagining - each paired with what it can and cannot tell a pricing lead choosing among three shelf prices.] ## The decision: three prices, one shelf reset The pricing lead's problem has a fixed shape: three candidate prices, a shelf reset date, and no history at any of the three price points. Historical volume data exists for the prices that were actually charged in the past, confounded by promotions, seasonality, and competitor moves at the time. None of it says what would happen at a price never charged. The question that matters is which of these three prices, if charged, maximizes volume and margin together. That is a rung-two question, and it requires a rung-two method. ## Rung one: what historical sales data and MMM can see Historical sales analytics and marketing mix models are built for association: they describe which past prices correlated with which past volumes, under the conditions that happened to hold at the time. That is useful for monitoring a trend or forecasting near-term demand inside the range of prices already charged. It cannot choose among three untested prices, because extrapolating a correlation outside the range it was measured in mistakes an association for an effect. A price move confounded with a promotion, a competitor stockout, or a seasonal shift produces a correlation that has nothing to do with the price itself, and rung one cannot separate the two. ## Rung two: where causal AI operates This is where causal AI operates: randomize which of the three prices a respondent sees inside a simulated market, then estimate the effect of price on choice with a confidence interval. McFadden discrete choice, Mixed Logit, and ICLV are the estimators that produce that effect from the experiment's data; none of them is a causal method on its own. The causal claim comes from the randomization in the experiment design - price assignment did not depend on anything else about the respondent or the market - and the estimator reads the effect out of data that randomization already made interpretable. Preference-share and substitution questions between the three prices raise one more issue. A flat logit model carries the independence-of-irrelevant-alternatives assumption: it can misstate substitution patterns when the alternatives are not equally distinct from each other. Mixed Logit and ICLV relax that assumption, at the cost of more parameters to estimate. The confidence interval that comes out of this experiment covers the effect within the simulated population tested, not the real market unconditionally. ## Rung three: the counterfactual - what the price you didn't pick would have done Rung three asks what would have happened under an action not taken, and a well-designed randomized experiment answers it for all three prices at once, not just the one eventually chosen. Because each arm was assigned independently, the same dataset that identifies the effect of the chosen price also estimates what the other two would have done, which is what the next reset needs. Advertising measurement does a version of this after the money is spent: GeoLift builds a geo-level counterfactual by weighting untreated regions to approximate what the treated region would have done without the spend (GeoLift methodology). Running the experiment before the reset gets that same comparison for all three prices while the budget is still uncommitted. ## How does causal AI compare to predictive AI and causal-inference libraries like DoWhy and EconML? Causal AI, predictive AI, and causal-inference libraries answer different parts of the same problem, and picking the wrong one for a given decision produces a confident answer to the wrong question. ## What causal AI is not Causal AI is not causal discovery from observational data alone. Causal discovery libraries infer a plausible graph structure from patterns in existing data, and that graph is a hypothesis, not a randomized test. Causal AI is also not the estimator. McFadden discrete choice, Mixed Logit, and ICLV are statistical tools for reading an effect out of experimental data; none of them manufactures a causal claim by itself, and applying one to observational data does not make the result causal. A related failure mode is trusting stated preference on its own. Asking people what they would pay runs high against what they actually pay, with a median hypothetical-to-actual ratio of 1.35 across 28 studies (Murphy et al., Environmental and Resource Economics, study), unless the design is incentive-aligned so the stated answer has a real consequence. No validation score is a guarantee that a method proven on past studies will hold in a market it has never been tested against. That gap is why the validation record has to be published rather than asserted. ## Why does the validation record matter more than the method name? It matters because a method's name - discrete choice, Mixed Logit, ICLV - says nothing about whether its output tracks real human behavior; only a replication test against real humans answers that. Subconscious's best configuration reaches 0.832 rank correlation against a published human result on one study, where two independent samples of real humans reach 0.959 - 87% of that measured human ceiling. Across the 43 published randomized studies that pass its design filters, the mean rank correlation is 0.73 (causal fidelity paper), and every study behind that mean is published at the leaderboard, misses included. One caveat travels with these numbers: a published validation study can sit inside a model's training data, and a model scored on a study it has already seen is not a fair test. The protocol filters for that; the field has not solved it. More comparisons against this standard are in the methods and validation hub. ## What should a pricing lead actually do before the shelf reset? Check whether the method proposed for the decision has ever been randomized and scored against real human behavior, and ask to see the misses, not just the wins. A pricing lead can do this without engaging a vendor: list the three prices, define the outcome metric before running anything, and confirm the test assigns respondents to prices at random rather than reading last year's sales by tier. A method's score across dozens of prior studies predicts its reliability on a new decision better than its name does, so compare any proposal against the leaderboard. To talk through which price to test first, book time. --- # Nike leads the US sneaker market, Skechers and Jordan convert best URL: https://subconscious.ai/blog/industries/nike-leads-us-sneaker-market-skechers-jordan-convert-best Published: 2026-08-28 A brand strategist deciding where to point next quarter's marketing budget needs one fact fast: Nike still leads the US sneaker category on the metrics buyers actually track. Nike holds the top spot in aided awareness, consideration, and preferred-brand share among major US sneaker brands, while Skechers is closing the gap and Jordan converts a small footprint into an outsized preference rate. None of those three facts tells the strategist which lever, price, style, an endorsement, or distribution, would actually move a buyer tomorrow. - Nike leads major US sneaker brands on aided awareness, consideration, and preferred-brand share, though Skechers is gaining ground on Nike specifically in consumer consideration (YouGov). - Nike's global athletic-footwear share fell to roughly 22.9 percent in 2025, a third straight year of decline (Front Office Sports). A separate measure, Heuritech's tally of the Nike-Adidas-New Balance grouping, put Adidas at about 26 percent of that three-brand group in 2025, versus Nike's 34 percent and New Balance's 14 percent (SGB Online). - Skechers posted the steadiest growth in social visibility of any major sneaker brand in 2025, up 22 percent year over year, per Heuritech's Global Footwear Insights study (SGB Online). - Jordan's high preferred-brand ratio among a small aware base looks like efficient conversion, but a funnel ratio can't separate real preference strength from who was left in the denominator. - The decision a buyer actually needs to make, whether to move on price, style, an endorsement deal, or distribution, requires a randomized experiment, not a self-report funnel, to identify which lever moves people and by how much. ## What "Nike leads the sneaker market" actually measures Nike's lead is a brand-tracking result: highest aided awareness, highest consideration, highest self-reported "preferred brand" share among major US sneaker brands, per YouGov's ongoing tracking, which also shows Skechers gaining consideration share against Nike specifically (YouGov). That lead sits inside a US sneaker market projected near $25.6 billion in 2025, with running and walking categories driving most of the growth, categories where Nike has not kept pace (Footwear Magazine). That figure is a forecast made before the year closed, not an audited total, so it can shift as final 2025 data comes in. The erosion shows up globally too: Nike's athletic-footwear share fell to about 22.9 percent in 2025, a third consecutive annual decline, while GlobalData forecasts Adidas continuing to take share through the year (Front Office Sports; GlobalData). Leading a funnel and losing share are not contradictory. They are two different measurements, and only one of them describes what a buyer would do if something changed. ## Why do Skechers and Jordan convert their aware base better than Nike? Because the ratio's denominator is smaller, not necessarily because either brand is stronger. Nike's awareness base includes nearly everyone who owns a television or a phone, so its "preferred brand" percentage has to clear a huge, unfiltered pool of casual, indifferent, and even hostile respondents. Skechers and Jordan have smaller aware bases, and the people who remain in that smaller pool have already self-selected toward the brand, so a higher share of them say they prefer it. Heuritech's Global Footwear Insights study shows Skechers gaining the steadiest ground in social visibility of any major brand in 2025, a different metric from the preferred-brand ratio, but a real signal (SGB Online). Visibility and conversion ratios track different things, and a rising ratio on a smaller base and a rising ratio on a larger base are not directly comparable without knowing who dropped out along the way. [figure: Two funnels side by side: Nike's wide awareness base narrowing to a moderate preference percentage, and Jordan's narrow awareness base narrowing to a high preference percentage, showing how the ratio depends on who was asked.] ## What the funnel can't tell a buyer A funnel percentage is recall and self-report, not behavior. A respondent who names Skechers as their preferred brand hasn't told anyone whether that preference comes from price, comfort positioning, an athlete tie-in, retail distribution, or just a recent ad they happened to see. The funnel has no counterfactual built in: it cannot say what happens to Nike's preference share if Nike cuts price 15 percent, what happens if Jordan adds a mid-tier SKU, or what happens if Skechers increases media spend. Those are the actual questions a strategist is trying to answer before committing budget, and a stacked bar of aware, considered, purchased, preferred has no mechanism for attributing movement to a specific, changeable lever. ## Which lever would actually move a buyer? Randomized experiments, not more survey questions A randomized experiment answers this by varying the attributes that matter, price, style, endorsement, distribution, across simulated buyers and observing which changes shift choice. The causal identification comes from the randomized manipulation itself, not from the statistical model used to analyze the results. McFadden's discrete choice model, Mixed Logit, and ICLV are estimators that turn randomized choice data into attribute-level effects; they are not causal methods on their own, and a study is only as trustworthy as the randomization behind it. One caveat matters for any substitution question, such as which brand a Nike buyer would move to: a flat logit model carries the independence of irrelevant alternatives assumption, meaning it treats a new option as pulling share from every existing brand in fixed proportion. Mixed Logit relaxes this by letting preferences vary across simulated buyers, but the assumption still shapes any single-level substitution answer, so a well-built study should say where it applies and where it doesn't. More on how these models are used is in methods and validation. ## How accurate are simulated experiments compared to real human studies? On Subconscious's validation set, our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. It is a result on a held-out validation set, not a guarantee for a new, unstudied market. That number also comes with a standing limitation worth naming directly: some published human studies could sit inside a model's training data, which would make replication look easier than it is. The protocol is built to catch that by testing on held-out studies and by publishing results in the open on the leaderboard rather than asserting accuracy once and leaving it unverified. Any confidence interval that comes out of a simulated experiment covers the effect within that simulated population, not the real US sneaker market unconditionally. ## Funnel tracking vs a randomized discrete choice experiment More comparisons like this one live in the comparisons hub. ## What a lever test would actually look like for Nike, Skechers, and Jordan Nike's decline in global share while it still leads on awareness suggests price and distribution are worth testing directly, not assumed from the funnel: does a price move recover share faster than a distribution push, and among which buyers. Skechers' steady growth and rising consideration point toward comfort and price positioning as candidate levers, but a randomized test would be needed to separate the two rather than crediting either from a correlation. Jordan's small aware base with a high preference rate raises a specific, testable question: would a mid-tier SKU or a wider distribution footprint convert more of the buyers who are aware but haven't purchased, or would it dilute the scarcity that produces the high preference rate in the first place. Each of these is a hypothesis a funnel can suggest but not settle. A buyer who wants the answer can start by checking the leaderboard for a comparable category study and reading the replication protocol before scoping a test. If the next step is a lever test built for this specific decision, the team can help scope it. --- # How To Interpret Marginal Willingness To Pay URL: https://subconscious.ai/blog/methods-and-validation/interpret-marginal-willingness-pay Published: 2026-08-28 A pricing lead deciding whether to ship a feature at a premium is really asking how to interpret a marginal willingness-to-pay (WTP) number from a conjoint study. Read it as a price only when it carries three things: a confidence interval built on the experiment's randomized design, a competitive-choice context, and a discount for the gap between what people say they'd pay and what they actually pay. A bare coefficient ratio, the attribute coefficient divided by the price coefficient, has none of the three and is not a price yet. - The standard ratio-of-coefficients WTP estimator can have an undefined standard error, because the price coefficient's own sampling distribution can span zero (ScienceDirect, 2023). - Algebraic WTP calculations that ignore competitive alternatives and the "None" option systematically overstate what customers will pay (Sawtooth Software). - A meta-analysis of 28 matched hypothetical-and-real-payment studies found stated WTP overstates real purchase behavior by a median of 1.35x, and as high as 3x in some studies (Environmental and Resource Economics, 2005). - The same choice data can produce materially different WTP numbers depending on whether the model is specified in preference space or WTP space. - A defensible WTP figure needs a delta-method or bootstrap confidence interval, a simulation against a named competitive set, and an explicit hypothetical-bias discount, not a single point estimate. ## What a marginal WTP number actually captures A marginal WTP estimate comes out of a discrete choice model, usually McFadden's multinomial logit, Mixed Logit, or an Integrated Choice and Latent Variable (ICLV) model, fit to a set of randomized product-profile choices. The analyst takes the coefficient on the attribute of interest and divides it by the (negative) coefficient on price. The result is read as "the dollar amount customers will pay for this feature." That reading assumes the ratio behaves like an ordinary number with a stable variance. It doesn't, because it's a ratio of two estimated quantities, not a directly estimated quantity itself. DCE, Mixed Logit, and ICLV are estimators for recovering these coefficients from choice data; the causal claim in a WTP study comes from the randomized manipulation of attributes in the experiment design, not from the estimator itself. ## Why the standard ratio has no valid confidence interval A WTP figure earns trust only when its limits ship with it. The price coefficient is estimated with sampling error. That error can put real probability mass on both sides of zero. Divide by it, and the result behaves like a Cauchy distribution: undefined variance, sometimes bimodal, sometimes with no meaningful mean. Hensher and colleagues showed in 2023 that the naive ratio estimator has no valid standard error under standard assumptions unless the cost parameter is reparameterized, for example with an exponential transform that forces it to stay negative (ScienceDirect, 2023). A WTP figure reported without a delta-method or Krinsky-Robb confidence interval isn't a conservative simplification. It's a number whose error bars, if computed honestly, might not exist in the form the report implies. [figure: A five-step chain showing how a price coefficient with sampling error that spans zero turns a simple coefficient ratio into a Cauchy-like distribution with undefined variance, corrected by reparameterizing the cost term or using a delta-method confidence interval.] ## Why does WTP inflate without a competitive set? Naming this failure mode is what lets a buyer check the number before pricing off it. WTP inflates because a ratio calculated from two attributes in isolation ignores that a real buyer is choosing among competing products, including the option to buy nothing at all. Sawtooth Software's technical papers on this problem show that traditional algebraic and two-product WTP calculations systematically overstate willingness to pay, because they never force the simulated respondent to weigh the feature against real alternatives or a "None" option. Sawtooth's recommended fix is to simulate share-of-preference against a full competitive set and bootstrap the resulting confidence interval, rather than solving the ratio algebraically (Sawtooth Software). That kind of share-of-preference simulation is typically built on a logit model, which carries the independence of irrelevant alternatives (IIA) assumption: adding or removing one competitor shouldn't change the relative odds between two others. When that assumption doesn't hold, for example when two competitors are close substitutes, a flat logit share simulation will misstate how share actually moves, and Mixed Logit is the standard correction. ## How much should you discount stated WTP for hypothetical bias? This gap gets published so a buyer can judge the number before acting on it. Treat a stated WTP figure as running about a third higher than what customers would actually pay at the median, and up to three times higher in some studies, unless the survey design is incentive-aligned. A meta-analysis of 28 stated-preference studies that compared hypothetical and real-payment elicitation for the same goods found a median ratio of hypothetical-to-actual value of 1.35, meaning stated WTP overstates real purchase behavior by about a third on average, with some studies in the sample running as high as 3x (Environmental and Resource Economics, 2005). This is on top of, not instead of, the confidence-interval and competitive-set problems above. ## Why the same data can produce two different WTP numbers This split is spelled out here so a buyer comparing two vendors' numbers knows which model produced each one. Two analysts can run the identical choice data through two legitimate model specifications and get materially different WTP distributions, because WTP can be estimated in "preference space," where you estimate ordinary utility coefficients and then divide, or in "WTP space," where the price sensitivity is reparameterized so WTP is estimated directly. Kenneth Train and Melvin Weeks showed in "Discrete Choice Models in Preference Space and Willingness-to-Pay Space" (2005) that the two specifications do not converge on the same WTP distribution from the same choices, because the distributional assumptions placed on the coefficients differ between the two parameterizations. Neither model is wrong; they encode different assumptions about how price sensitivity varies across the population, and a buyer comparing two vendors' WTP numbers should ask which specification produced each one. ## What a defensible WTP figure requires A defensible WTP figure requires three things reported together: a confidence interval from the delta method or bootstrap (Krinsky-Robb), a share-of-preference simulation against a named competitive set that includes a "walk away" option, and an explicit discount or caveat for hypothetical bias. ISPOR's Good Research Practices Task Force reports on conjoint analysis, published between 2011 and 2016, formalized the delta-method and Krinsky-Robb correction methods for health-economics discrete choice experiments. A point ratio without a valid interval isn't a publishable result in that field, and it isn't a defensible pricing input in any other. ## How does a causal experiment change what a WTP number means? It changes what the confidence interval is allowed to claim. A WTP interval from a randomized experiment covers the effect within the simulated population that was run, not the real market unconditionally. And a simulated choice is still a stated preference, not a purchase: any WTP figure that comes out of one should be discounted and reported with its interval, not treated as a bare number. Subconscious runs randomized experiments, analyzed with discrete choice models such as McFadden discrete choice, Mixed Logit, and ICLV, on a simulation of the market, and validates the resulting studies against real human behavior. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, reported in the causal fidelity paper. That's a validation-set result, not a guarantee for a new market you haven't tested, and because published human studies can sit in a model's training data, the replication protocol is built to control for that risk rather than assume it away. The current results across studies are public on the leaderboard. None of this removes the hypothetical-bias problem named above. More on how the validation protocol works is in the methods and validation hub. If you're evaluating a vendor's WTP output, ask for three things before you price anything off it: the confidence interval and how it was derived, the competitive set the share simulation ran against, and whether the number is stated or incentive-aligned. If a vendor can't produce any of the three, treat the figure as a ranking signal, not a price. For a closer look at how a specific WTP study was built and validated, book time with the team. --- # How to perform smart sampling and data checking? URL: https://subconscious.ai/blog/methods-and-validation/perform-smart-sampling-data-checking Published: 2026-08-28 A research director who owns a sampling and data-checking protocol is choosing between two different guarantees: that respondents are real, and that the resulting choice model is causally right. Bot checks, attention checks, and rule-of-thumb sample sizes deliver the first guarantee. Only replication against a real human behavioral benchmark delivers the second. - A claim needs its limits stated next to it or it's just marketing. Passing a bot or attention check confirms a respondent is human. It does not confirm the study design recovers a true preference parameter. - CloudResearch ran 6,000 controlled trials. AI-generated responses passed attention checks 99.8% of the time, showing the standard screen no longer isolates the threat (CloudResearch). - NORC's 2026 literature review estimates 30-40% of raw online-panel responses are fraudulent or unusable, an estimate that varies by panel and vertical rather than one fixed rate; some teams discard up to 70% of what they collect (NORC). - Fully human, fully attentive nonprobability panels still carried average bias over 10 points against known population values, per Pew's 2016 benchmark, still the most recent large-scale study of its kind (Pew Research). AAPOR's 2013 task force report found no accepted basis for treating such panels as projectable at all (AAPOR). - DCE sample-size rules of thumb size a coefficient's precision, not its truth. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. It is a validation result, not a guarantee for a new market, per the causal fidelity paper: the number worth asking any vendor to match. ## What a data quality check is actually testing Naming a failure mode plainly is what lets a buyer check it before they rely on it. Every fraud screen, speeder flag, and captcha answers one question: is this response coming from an attentive human. That's necessary, not sufficient. The NORC and CloudResearch figures above describe the scale of that one question: how many respondents are fake, and how well the standard test still catches them. Some vendors pitch layered fraud detection across recruitment, registration, in-survey behavior, and post-survey review. All of that answers one question: was the respondent real. It doesn't answer whether the study's design and estimator recover a real causal effect. ## Does passing a bot check mean your data is valid? This is worth stating directly instead of leaving it implied in a sales pitch. No. A bot check confirms the respondent was human and attentive. It says nothing about whether their stated choices, run through a discrete choice model, predict what people actually do. A study can fail the second test while passing the first cleanly. [figure: Two-column comparison showing a bot check verifies respondent authenticity while replication verifies the model reproduces real behavioral outcomes.] The evidence for that gap comes from fully human, fraud-free panels, not fraudulent ones. Pew Research's 2016 benchmark, still the most recent large-scale study of its kind, compared nine nonprobability samples against known population values and found average bias exceeding 10 points, including 15.1 points on Hispanic respondents and 11.3 on Black respondents (Pew Research). Every one of those respondents would have passed a bot check. AAPOR's 2013 task force report goes further: there's no generally accepted theoretical basis for treating nonprobability online panel results as projectable to the general population at all (AAPOR). Cleaning the panel doesn't fix a design that was never validated against real behavior. ## Why fraud-free panels still produce biased answers The bias Pew and AAPOR document is structural, not contamination. It comes from who opts into online panels and how they answer hypothetical questions, not from bots slipping through. More screening can't fix that: screening removes fake respondents, not the gap between what a real, attentive respondent says they'd do and what they actually do when money or effort is on the line. Vendors selling layered fraud detection solve the NORC problem. They don't solve the Pew and AAPOR problem, and a buyer evaluating a sampling protocol needs to know which one is on the table. ## What do DCE sample-size rules of thumb leave out? A precise number is not the same claim as a correct one, and a buyer should be able to tell them apart. They tell you a coefficient will be statistically precise. They don't tell you it's correct. Practitioners commonly reach for Orme's 300-respondent minimum or Hensher, Rose, and Green's 50-per-alternative rule and treat either as a sign the data is ready. Both are heuristics for a coefficient's standard error, not tests of whether the design recovers a true causal preference. A well-powered Mixed Logit estimated on biased or unvalidated stated-preference data still produces a precise wrong answer: a tight confidence interval around a number that doesn't reflect real behavior. ## What actually proves a choice model is causally right? Replication: rerun the design against a real human study's outcome and check whether the model reproduces the direction and result, not just a plausible-looking coefficient. On a validation set of past studies, this reaches 87% of the measured human ceiling (0.832 against a 0.959 human-to-human ceiling; mean 0.73 across the 43 studies passing design filters), a validation-set result, not a guarantee for a brand-new market (the causal fidelity paper). That caveat matters twice over: some published studies used for validation could overlap with a model's training data, and the public leaderboard exists to control for exactly that risk, ranking approaches against held-out human studies selected to avoid training-data overlap rather than pretending the risk isn't there. That's the place to compare vendors on this claim, not on fraud-detection marketing. McFadden discrete choice models, Mixed Logit, and ICLV are estimators, not causal methods on their own. Causal identification comes from the randomized manipulation built into the experiment design; the estimator fits a model to the choices that randomization produced. A confidence interval from a simulated experiment covers the effect within the simulated population the experiment ran on. It doesn't bound the real market unconditionally; that link has to be established through replication against the population the study is meant to represent. ## How do you build a sampling and checking protocol that catches both problems? Run fraud screening and causal validation as two separate steps. They catch two different failure modes. - Screen for bots and inattentive respondents with layered detection, not a single attention check. Standard attention checks no longer catch AI-generated responses reliably (CloudResearch). - Size the study with a formal power calculation tied to the effect size that matters for the decision, not a rule of thumb like n=300. - Before trusting the output, check whether the design and estimator have been replicated against a real, held-out human study, and ask for the replication rate, not just the fraud rate. - Treat any willingness-to-pay estimate as directionally inflated unless the design is incentive-aligned; stated WTP runs high when nothing real is at stake. - For preference-share or substitution questions modeled with a flat logit, check the independence of irrelevant alternatives (IIA) assumption before trusting the result. A flat logit pulls share from every competitor proportionally, an assumption that breaks when two options are close substitutes. Mixed Logit and ICLV relax this by letting preferences vary across the simulated population. More on how the estimators and validation protocol fit together: methods and validation. ## Where willingness-to-pay estimates still mislead A WTP figure from a stated-preference discrete choice study runs high, in a known direction, unless the design pays out real money or otherwise makes the choice consequential. That's a separate failure from panel fraud or panel bias: even a perfectly sampled, perfectly human, direction-validated panel will overstate what people would actually pay when the question is hypothetical. A protocol that stops at fraud detection and rule-of-thumb sample size will never catch this, because WTP inflation shows up in the size of the effect, not in whether the respondent was real or the model was precise. Pull the last study your team shipped and check one thing: does the write-up compare the model's output to a real human behavioral benchmark, or does it stop at the fraud rate and the sample size. If it stops there, that's the gap to close before the next wave. Want the replication protocol behind that 87% figure walked through for your market? Book time with the team. --- # Busting Market Research Automation Misconceptions URL: https://subconscious.ai/blog/methods-and-validation/busting-market-research-automation-misconceptions Published: 2026-08-28 {"opener": "The biggest misconception in market research automation is treating human panels and synthetic respondents as the real choice, when neither produces causal evidence without randomized attribute manipulation and a human baseline check."} A senior buyer picking between synthetic respondent platforms and traditional human panels is choosing between two flawed defaults, not a right answer and a wrong one. Neither a fraud-contaminated human panel nor an LLM persona answering an unrandomized prompt can tell you what actually caused a choice. The fix isn't picking a side of the human-versus-synthetic debate; it's requiring randomization and a human baseline check before you trust either population's answers. - Human panels aren't the trustworthy default anymore: NORC estimates roughly 40 percent of nonprobability survey interviews in 2025 were likely fraudulent. - Synthetic personas don't fix that: researchers who already use AI tools still say they don't trust AI-generated participants, by a wide margin. - "Validate against a holdout sample" checks correlation with another uncontrolled sample, not causal identification. - McFadden discrete choice, Mixed Logit, and ICLV are estimators; the randomized manipulation inside the experiment is what makes it causal. - The buyer question that matters isn't who answered, it's whether the design can isolate which attribute moved the decision. ## Is human versus synthetic even the right fight? No. It's the fight the market is having, but it's the wrong one. Real capital is chasing the synthetic side of it: Simile raised $100 million in February 2026 with backing from Index Ventures, Fei-Fei Li, and Andrej Karpathy, and Aaru raised more than $50 million in December 2025 with Accenture as an investor. ESOMAR issued 2025 guidance requiring disclosure and holdout validation whenever synthetic data is used. All of that money and all of that guidance still assumes the deciding factor is which population answered the survey, human or simulated. It isn't. A panel of verified, real humans answering an unrandomized questionnaire and a chatbot persona answering the same unrandomized prompt fail the same way: both produce a stated preference with no controlled variation behind it, so neither can tell you what a specific price, feature, or message change caused in choice behavior. ## Why don't human panels count as the trustworthy baseline anymore? Because the baseline itself is contaminated. An estimated 40 percent of nonprobability survey interviews in 2025 were likely fraudulent, roughly 2 billion fraudulent interviews industry-wide, according to NORC at the University of Chicago. That estimate covers nonprobability panels specifically, not every survey method NORC tracks, but it's the segment most market research automation tools draw from. Worse, the industry's main fraud filter no longer works: AI agents completing surveys now pass 99.8 percent of attention checks, according to CloudResearch, the exact test panels use to catch inattentive or fraudulent respondents. That figure describes attention-check evasion, not an overall fraud rate, but it means the checklist researchers have relied on for a decade no longer separates real attention from bot behavior. A "verified human panel" claim is only as good as a verification method that bots now pass at nearly the same rate as people. ## What does the researcher trust gap actually measure? It measures that researchers already sense the problem, even without a causal framework to name it. In a 2026 survey covered by Development Corporate, 97 percent of surveyed researchers said they already use AI tools in their work, yet only 8 percent said they trust AI-generated participants outright, and 64 percent remain skeptical or opposed. The survey reflects self-reported sentiment, not a controlled trust experiment, but the gap between adoption and trust is the tell. Researchers are adopting AI for speed while withholding trust for the decisions that matter, which is a reasonable instinct pointed at the wrong target: the missing piece isn't more verification of the persona, it's randomization in the design. [figure: Bar chart showing 97 percent of researchers use AI tools, 8 percent trust AI-generated participants, and 64 percent remain skeptical or opposed, per a 2026 survey covered by Development Corporate.] Academic review of the underlying method is blunter. A 2024 study in Political Analysis examined whether large language models can stand in for human survey respondents and found systematic limits to treating LLM output as a replacement for individual-level human data, not just a bias to correct for (Political Analysis, Cambridge Core). That finding sits underneath the trust gap: researchers are right to withhold trust, they just haven't named why. ## Does a holdout sample prove causation? No, and this is the misconception the "disclose and validate" checklist doesn't correct. Validating a synthetic run against a human holdout sample checks whether the two uncontrolled samples correlate, not whether either one identifies what caused a choice. Correlation between a panel's stated answer and a persona's simulated answer tells you the two methods agree, or don't; it says nothing about which attribute, price, feature, or message, drove the outcome, because nothing in an unrandomized survey isolates that variable. Causal identification requires a randomized manipulation inside the experiment design itself. In Subconscious's own validation set, our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper). That's a validation-set result, not a guarantee for a market that hasn't been tested yet, and it comes with a separate limitation worth stating plainly: published human studies used for this kind of check can already sit inside a model's training data. A replication protocol has to control for that risk directly; pretending it doesn't exist is worse than disclosing it. ## Human panel, synthetic persona, or randomized experiment: how the three compare ## Where McFadden discrete choice, Mixed Logit, and ICLV fit They're estimators, not causal methods on their own. Write it plainly: randomized experiments analyzed with discrete choice models, never "causal methods like DCE." McFadden discrete choice models, Mixed Logit, and ICLV all model the choices a respondent makes; what makes the resulting effect causal is the randomized manipulation of attributes inside the experiment design that precedes the modeling. Mixed Logit adds one practical advantage over a flat multinomial logit: it relaxes the independence of irrelevant alternatives (IIA) assumption, which matters directly for preference-share and substitution questions, since a flat logit can predict share shifts that don't reflect real substitution because it treats each alternative as unrelated to the others. ICLV adds latent constructs, useful when the attribute you care about (trust, perceived quality) isn't directly observable. None of the three, on its own, turns an unrandomized survey into a causal read. And if a vendor quotes a willingness-to-pay number from either a human panel or a synthetic run, ask whether the design is incentive-aligned; stated WTP runs high otherwise, a say-do gap, not a defect unique to synthetic data. A confidence interval from a randomized simulated experiment covers the effect within the population tested; it doesn't, by itself, bound what happens when the product ships to the real market. ## A decision checklist for the senior buyer Before signing with any market research automation vendor, human panel or synthetic, ask for three things: - Was any attribute (price, feature, message) randomized across respondents, or was every respondent shown the same unrandomized survey? - What confidence interval covers the reported effect, and for which population does it apply? - Was the result checked against a held-out human study, and can you see the study, not just the match rate? If a vendor can't answer all three, you're being sold correlation dressed as insight, regardless of whether the respondents were verified humans or LLM personas. The public leaderboard tracks method-by-method replication scores if you want to compare providers directly, and the methods and validation archive covers the estimator-level detail behind DCE, Mixed Logit, and ICLV in more depth than fits here. Your next step doesn't require a call: pull your last three vendor reports and check whether any of them named a randomized attribute manipulation and a confidence interval, or just a sample size and a trust badge. If none did, you know what to ask for next time. When you're ready to test a specific pricing, feature, or messaging decision against a randomized design, book time with the team. --- # How Key Driver Analysis identifies what matters most URL: https://subconscious.ai/blog/methods-and-validation/key-driver-analysis-identifies-matters-most Published: 2026-08-28 A CX or insights leader looking at a Key Driver Analysis report has to decide whether to spend next quarter's roadmap budget on whatever attribute sits at the top of the ranking. KDA answers that ranking question with regression or Shapley regression run on survey data you already collected: it scores each attribute by how much of the variance in an outcome like NPS or loyalty it explains among the customers who happened to answer your survey. That score tells you what already correlates with satisfaction in the market as it exists today, not what would happen to satisfaction if you changed one attribute and held everything else constant. - Key Driver Analysis ranks survey attributes by their statistical association with an outcome, using multiple regression, relative importance analysis, or Shapley regression. - Shapley regression fixes the coefficient instability caused by intercorrelated attributes, but it still runs on observational, cross-sectional data. - A KDA ranking describes what explains satisfaction under the status quo. It cannot tell you what happens if you change an attribute, because nothing in the survey was manipulated. - Randomized choice experiments, analyzed with discrete choice models such as McFadden's model, Mixed Logit, or ICLV, introduce controlled variation so an effect can be identified rather than inferred. - A validated simulation protocol can reproduce the direction and outcome of real human studies at a measured rate, reported on the public leaderboard. ## What does Key Driver Analysis actually measure? KDA measures how much of an outcome's variance each attribute explains among customers who already lived through some mix of those attributes. In practice this means multiple regression, relative importance analysis, or Shapley Value regression run on a satisfaction or NPS survey. Shapley regression, the version most vendors now recommend, treats each attribute as a player in a cooperative game and assigns it credit equal to its weighted-average contribution to R² across every possible subset of attributes (Displayr). The output is a ranked list. Nothing about the underlying data changes: it is still a snapshot of customers who never experienced a controlled version of any attribute. ## Why Shapley regression became the industry standard Standard multiple regression breaks down when attribute ratings intercorrelate, and in satisfaction surveys they almost always do. Attributes routinely correlate above 0.80, the threshold generally flagged as dangerous for regression, which makes coefficients unstable and prone to flipping sign between runs (JumpData). Shapley regression was adopted specifically to stop that instability by distributing importance across correlated attributes rather than assigning it to whichever one happens to win the regression. It is a real fix for a real problem: unstable coefficients. It is not a fix for what the coefficients mean. ## The fix that never touches the real problem Shapley regression solves a computational problem, not a design problem. The data feeding it is still cross-sectional and observational: customers rated attributes as they already existed in the market, in whatever combination happened to occur. Nothing was assigned to them at random. Practitioners in the field have flagged this directly, arguing that KDA output is frequently over-interpreted as causal when it is fundamentally correlational, vulnerable to confounding and reverse causation regardless of which regression variant produced the ranking (Youssefnia, citing Scherbaum, Putka, Naidoo & Youssefnia, 2010). A stabilized coefficient is still a coefficient. It describes what varied together in your sample, not what would happen if you deliberately changed one thing and left the rest alone. [figure: Flow diagram showing how Key Driver Analysis moves from observed customer survey data through regression or Shapley regression to a ranked attribute list, without any point at which an attribute is deliberately varied.] ## Stated importance vs. derived importance: a debate that misses the causal question A parallel debate in CX research pits stated importance, asking customers directly what matters to them, against derived importance, inferring it from regression on their ratings. Comparative studies remain sparse and their results conflict, so neither camp has settled the question of which better predicts behavior. Both sides are arguing about a downstream detail. Whether you ask customers to rank attributes or infer the ranking statistically, you are still describing preferences within the status quo. Neither approach introduces a version of the market where one attribute changed and the rest held still, so neither can tell you what would happen if it did. ## What can a randomized experiment tell you that Key Driver Analysis can't? A randomized experiment can tell you what happens to the outcome when you actually change an attribute, because the attribute was assigned independently of everything else a respondent might bring to the choice. This is the distinction between backward-looking KDA, which explains satisfaction that already occurred, and forward-looking conjoint or discrete choice experiments (DCE), which predict what a future change would do. DCE, Mixed Logit, and ICLV are estimators, not causal methods in themselves; the causal identification comes from the randomized manipulation built into the experiment design, not from the statistical model fit to the results. That is why the accurate description is randomized experiments analyzed with discrete choice models, not "causal methods like DCE." Mixed Logit is also worth naming for a separate reason: it relaxes the independence of irrelevant alternatives (IIA) assumption baked into a flat multinomial logit, which matters if you plan to read preference-share or substitution results out of the model. A flat logit assumes adding or removing an alternative affects all other alternatives proportionally; real substitution patterns rarely behave that way. ## How accurate is a simulated experiment compared to a real one? Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. That figure is a validation-set result, not a guarantee for a market you haven't tested yet, and it comes with a caveat worth stating plainly: some published human studies used for validation could theoretically overlap with a model's training data, which is exactly why a held-out replication protocol, rather than a single retrospective comparison, is the check that matters. Results across studies and categories are tracked on the public leaderboard, which is worth checking against the category closest to your own decision before you trust a number for it. ## Key Driver Analysis vs. randomized choice experiments ## Which method should you run for your next roadmap decision? Run KDA when you already have survey data and want a quick, low-cost read on what correlates with satisfaction today; treat the output as a hypothesis generator, not a budget justification. Run a randomized choice experiment when the decision has real cost if you're wrong, a pricing change, a feature investment, a positioning shift, because that is the only design that tells you what happens if you actually change the attribute rather than what already happened to correlate with it. Case studies of teams using this second approach are on case studies. Before committing budget to the top attribute in your next KDA report, check whether that attribute has ever been tested with a controlled change, in your own data or in a published experiment; if it hasn't, that's the gap a randomized experiment closes. When you want a second read on a specific decision, book time with our team. --- # Mini-lecture: Conjointly's guide to inflation URL: https://subconscious.ai/blog/methods-and-validation/mini-lecture-conjointlys-guide-inflation Published: 2026-08-28 Inflation re-entered the forecasting conversation in 2026, and a pricing or insights leader deciding how to respond needs one thing Conjointly's mini-lecture on inflation doesn't deliver: a causal number. Conjointly's guide walks through the standard toolkit for tracking and reacting to price pressure, sentiment indices, Van Westendorp, Gabor-Granger, conjoint pricing studies. That toolkit tells you what people say about prices; it does not tell you what a specific price increase does to demand, with a confidence interval attached. This mini-lecture covers the same ground Conjointly's does, and adds the step it skips: a randomized experiment that returns a causal price elasticity instead of a stated-preference guess. - Conjointly's guide, like most pricing-research vendors in 2026, recommends higher-frequency sentiment tracking plus Van Westendorp, Gabor-Granger, or conjoint studies to find new price points as costs rise. - Those tools measure stated intent, not an actual reallocation of spending, and the underlying sentiment signal is itself documented as biased (ECB working paper 2642). - The decision this article owns: replace the pricing guess with a randomized experiment, analyzed with discrete choice models, that returns a causal elasticity and a confidence interval. - This benchmark sits on the public leaderboard so a buyer can weigh the misses next to the hits. On the one published Subconscious causal-fidelity benchmark, the best configuration reaches 87% of the measured human ceiling on a single study (0.832 against 0.959); the mean across all 43 studies that passed design filters is 0.73. - Next step: name the price move you actually need decided this quarter, and price a randomized test of it before committing budget to another sentiment read. ## What Conjointly's mini-lecture gets right The conventional advice is not wrong, it is incomplete. Tracking inflation sentiment at higher frequency than a quarterly index makes sense when prices are moving fast: the Conference Board, the University of Michigan survey, and the NY Fed's Survey of Consumer Expectations exist precisely because quarterly cadence is too slow for a live cost shock. Layering a Van Westendorp or Gabor-Granger study on top to probe new price points, sometimes alongside a conjoint-based portfolio trim of underperforming SKUs, is standard practice across pricing-research vendors in 2026. None of that is a mistake. The mistake is stopping there and treating the output as a demand forecast. ## What sentiment trackers and stated-preference tools can't tell you A causal number means little without its limits attached. Sentiment and stated-preference studies measure how people talk about prices, not how they would reallocate spending at a specific price point, and the talk itself is a documented biased signal. ECB research on consumer inflation expectations finds that average consumer inflation perceptions run systematically above actual inflation, with an attention asymmetry: price increases get weighted more heavily than decreases when people form their estimate (ECB working paper 2642). A Van Westendorp curve built on top of that biased perception inherits the bias. Any willingness-to-pay number it produces should carry its own caveat: stated willingness to pay runs high relative to incentive-aligned or revealed-preference designs, because a survey respondent faces no real budget constraint when answering. A sentiment index or a stated-preference curve can tell you that people feel prices are rising faster than they are. It cannot tell you what share of demand a $2 increase moves versus a $1 increase, because it never ran the counterfactual. ## Why is inflation a live forecasting problem again in 2026? Inflation stopped being a settled, post-2023 story this year because both the direction and the monitoring toolkit are under active revision. PIIE flags renewed upside risk to US inflation in 2026, reframing it as an open forecasting problem rather than a resolved one (PIIE, 2026). RBC Economics published a 2026 practitioner framework for which price indicators to watch and how often, which is itself evidence that the standard monitoring cadence needed retooling this year (RBC Economics, 2026). For a buyer, that combination means the sentiment-tracking half of the conventional playbook is getting more attention right now, not less, which makes it easy to mistake a faster read of sentiment for a better read of demand. ## Can a language model replace the consumer panel? Not on its own, at least not yet, and the Bank of England has already tested it. A 2026 Bank of England working paper ran GPT-3.5 against the Bank's own Inflation Attitudes Survey and found the model replicates some household-level regularities, income, housing tenure, food-price sensitivity, but concluded the model "lacks a consistent model of consumer price inflation" (Bank of England, 2026). That result came from matching aggregate perception patterns, not from a causal validation design: no randomized manipulation, no held-out study, no confidence interval on the match. It shows a language model can echo how people describe inflation. It does not show a language model, or any synthetic panel, can substitute for a designed experiment that isolates what a price change does to choice. ## The one decision: measure a causal elasticity, not a sentiment score The decision this article is arguing for is narrow: run a randomized experiment on the specific price move under consideration, and analyze it with a discrete choice model, so the result is a causal elasticity with a confidence interval rather than a stated preference. McFadden discrete choice, Mixed Logit, and ICLV are estimators. They are not, on their own, causal methods; the identification comes from randomizing the price or attribute levels each simulated respondent sees, not from the model that reads the resulting choices. That distinction matters when the model is a flat multinomial logit, which assumes independence of irrelevant alternatives: add a near-substitute SKU at the new price point and a flat logit can still hold implausibly stable shares between the existing options. Mixed Logit relaxes that assumption by letting preference weights vary across the simulated population, which is why it is the workhorse for substitution questions, not the plain logit. ## How much should a buyer trust a simulated elasticity? Naming a failure mode is what lets a buyer check the claim before relying on it. Enough to act on, with the limitation stated plainly, not enough to treat as a guarantee for a market that hasn't been tested. The only published fidelity result here is a ratio against a measured human ceiling, not a bare percentage: on the best-performing configuration for a single study, the simulated result reaches a 0.832 rank correlation against the published human result, against a 0.959 correlation between two independent samples of real humans, an 87% ratio to that measured ceiling. Across all 43 studies that passed the paper's design filters, the mean drops to 0.73 (causal fidelity paper). [figure: Bar chart with three bars: human-to-human baseline correlation at 0.959, best single-study configuration at 0.832 (87% of the ceiling), and mean across 43 studies at 0.73.] A benchmark only earns trust when the misses are published next to the hits. Two limitations belong next to that number, not below it. First, published studies used for validation can sit inside a model's training data; the replication protocol holds out entire studies rather than samples drawn from within a study specifically to catch that failure mode, but the risk is not eliminated by naming it. Second, a confidence interval produced by a simulated experiment covers the effect within that simulated population and design; it is a validation result on past studies, not a guarantee that it transfers to a market that has never been tested. The current full set of tested markets and methods is public on the leaderboard, which is the right place to check before assuming a given category has been validated. Sentiment trackers and stated-preference studies will keep their place for watching direction and screening rough price bands. But the decision a senior buyer actually has to defend this year, how demand responds to a specific dollar move on a specific SKU, needs a randomized experiment analyzed with a discrete choice model, not another mood read. Start by writing down the exact price change under consideration and the elasticity you'd need to see before approving it, then check the methods and validation hub for how that kind of test is structured before running it. If it's easier to talk through the specific decision first, book time. --- # Comprehensive Guide to Pricing Strategies URL: https://subconscious.ai/blog/use-cases/comprehensive-guide-pricing-strategies Published: 2026-08-28 A VP of Pricing weighing a renewal increase needs one thing from a research method: a number that matches what customers actually pay, not what they say they would pay. The right method isn't the fastest or cheapest one on the shelf. It's the one that has been checked against real purchase behavior, with a stated error rate and direction, before the price goes live. - Van Westendorp and Gabor-Granger give a fast price range for a single product but ignore competitors, brand effects, and actual purchase prediction, according to a practitioner guide to the method (Business Initiative). - Choice-based conjoint (Sawtooth CBC, Displayr, Qualtrics) is the standard for price-feature tradeoffs because it forces a choice instead of asking a direct price question (Sawtooth Software). - Stated willingness-to-pay questions run about 18 percent above real WTP in mobile product studies on average; the same paper reports a broader meta-analysis of 77 studies putting the bias at 21 percent, worse for expensive and specialty products. The bias is upward, not random (MDPI, 2026). - AI simulation vendors that skip fielded experiments haven't publicly disclosed a randomized control or a replication check against real human data, so that bias goes uncorrected rather than measured. - The right ranking criterion for a pricing method is how much it overstates willingness to pay and whether that overstatement has been checked against real behavior, not how quickly it produces a chart. ## What is the biggest risk in choosing a pricing research method? The biggest risk is picking a method whose error you can't see. Hypothetical stated WTP runs about 18 percent above real WTP on average in mobile product studies; the same paper also reports a broader meta-analysis of 77 studies across 47 papers, putting the average hypothetical bias at 21 percent, with the gap widening for higher-value and specialty goods (MDPI, 2026). That's exactly the segment where a pricing mistake costs the most: enterprise tiers, add-on SKUs, anything with a renewal cycle attached. A method that doesn't disclose its own bias isn't neutral. It's just quiet about being wrong. ## Van Westendorp and Gabor-Granger: fast, but blind to competition Van Westendorp's Price Sensitivity Meter is still useful for a quick read on an acceptable price range for one product formulation. According to a practitioner guide to the method, it does not account for competitive pricing, brand perception, or actual purchase-behavior prediction, and it's typically limited to one product at a time (Business Initiative). Gabor-Granger has the same shape: a direct demand curve from direct price questions, with the same hypothetical-bias problem as any stated-preference method. Both belong in the toolkit for a same-week sanity check on a single SKU. Neither should carry a renewal-pricing decision on its own. ## Why choice-based conjoint became the default for multi-attribute pricing Choice-based conjoint (CBC) is positioned by Sawtooth as the current standard for pricing and preference research because it forces tradeoffs among attributes instead of asking about price directly (Sawtooth Software). Discrete choice modeling is the preferred approach specifically when pricing is the primary research objective, since it can model brand-by-price interaction effects that single-product methods like Van Westendorp can't capture (Ibbaka). One caveat worth naming: when a conjoint's preference-share output comes from a flat multinomial logit, it carries the independence-of-irrelevant-alternatives (IIA) assumption, meaning it treats the ratio of preference between any two options as unaffected by a third. That assumption breaks down when a new price tier is a close substitute for an existing one, which is common in SaaS packaging. ## Can AI simulation panels replace fielded pricing studies? Not without a replication check, no. A newer wave of vendors (Aaru, Simile, Synthetic Users) promises instant synthetic panels without fielding a real experiment. The problem isn't synthetic data itself. It's that generating responses without a disclosed randomized control or a check against real human data means the hypothetical bias documented above doesn't get caught. It gets inherited. The alternative is a randomized experiment run on a simulated population, analyzed with discrete choice models, and checked for replication against a real human study before it's trusted. Randomized experiments analyzed with discrete choice models such as McFadden discrete choice (DCE), Mixed Logit, and ICLV get their causal identification from the randomized manipulation in the experiment design, not from the estimator itself. Subconscious's published causal fidelity results reach 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, per the causal fidelity paper. That's a validation-set result, not a guarantee for a market that hasn't been studied yet, and it comes with a real limitation: published human studies can sit inside a model's training data, so replication protocols have to design around that risk rather than pretend it doesn't exist. The current accuracy record across studies is public on the leaderboard, which is the kind of disclosure a synthetic panel without a replication step hasn't produced. [figure: Two parallel chains: one runs from a hypothetical price question straight to an uncorrected price decision, and one runs from a randomized choice experiment through a discrete choice model and a replication check against a human baseline to a price decision with a confidence interval covering the effect within the simulated population, plus a stated limitation.] ## Ranking methods by validated accuracy, not speed ## Why picking the wrong pricing method costs more in 2026 The stakes for picking the wrong method have gone up, though the figures below come from one vendor-published research source and haven't been independently replicated. Seventy-nine percent of IT buyers saw renewal price increases in the past 12 months, with a median increase of 7.8 percent, which the source attributes largely to AI-SKU bundling (Zylos Research, 2026). A poorly handled increase triggers a 10 to 15 percent churn spike; grandfathering existing customers plus clear value communication produces 26 percent higher gross retention instead (Zylos Research, 2026). Usage-based pricing adoption jumped from 27 percent of SaaS products in 2023 to 42 percent in 2026, adding structural complexity that a single-question survey tool wasn't built to test (Zylos Research, 2026). A pricing structure with three price points and a usage tier needs a method that can hold multiple tradeoffs at once and report its own error rate. Van Westendorp can't do the first. An unvalidated AI panel can't do the second. ## How should a senior buyer choose among these methods? Rank candidates by their measured overstatement of willingness to pay against real behavior first, and by budget and timeline second. That order matters because the direction of the error is known and consistent: stated preference and hypothetical WTP questions run high, worst on the exact products where a mistake costs the most (MDPI, 2026). A comparison table of methods and their turnaround times is a menu. A comparison of what each method costs you in accuracy, with the number and its source attached, is a decision framework. If a vendor can't tell you their measured bias against real purchase behavior, or their replication rate against a held-out human study, that absence is itself the answer. Before the next pricing committee meeting, pull the current numbers from the leaderboard and check whether the method under consideration for the renewal decision has a published replication rate against real human data. If it doesn't, ask why not. To see how a randomized pricing experiment is set up for a specific SKU, meet the team. --- # What is implicit testing? (with examples) URL: https://subconscious.ai/blog/methods-and-validation/implicit-testing-examples Published: 2026-08-28 A marketing leader deciding whether to greenlight a campaign, a package redesign, or a price change will eventually hear a pitch for implicit testing: measure how fast someone reacts to a logo or a word pair, and you'll see what they "really" think. Implicit testing is a reaction-time method, most often a version of the Implicit Association Test (IAT), that scores how quickly a respondent pairs a brand, ad, or product with positive or negative concepts, on the theory that faster pairings mean stronger automatic associations. It tells you which associations fire fastest in someone's head. The miss gets stated next to the number so a buyer sees both. It does not tell you what that person would actually choose, at what price, against which alternative, which is the only question a launch decision needs answered. - Implicit testing measures reaction-time latency to reveal associative strength, not a choice or a purchase. - A 2009 correlational meta-analysis by Greenwald et al., covering 122 studies and roughly 14,900 subjects, found IAT scores correlate with behavior at only .274; a correlational finding, not a causal one (meta-analysis). - A re-analysis found IAT adds only b = .08 of incremental predictive validity once explicit survey answers are already in the model (re-analysis). - NielsenIQ has publicly called commercialized reaction-time testing "modern snake oil," arguing vendors overstate what the underlying academic science supports (NielsenIQ). - A randomized discrete-choice experiment gives a causal estimate of behavior change with a confidence interval, which is the input a launch decision actually requires. ## How does implicit testing work? Implicit testing measures the milliseconds it takes a respondent to associate a stimulus, a brand logo, an ad, a package, with a positive or negative concept. The assumption is that faster response times reflect stronger, more automatic associations that a person either can't or won't report in a normal survey question. Vendors position this as a fix for the say-do gap, the well-documented divergence between what people say they'll do in a survey and what they actually do, which researchers attribute partly to limited self-awareness and unconscious rationalization (cloud.army). The pitch: skip the rationalizing, measure the reflex. ## What are examples of implicit testing? The category covers a handful of related tools, all built on the same reaction-time logic: - IAT (Implicit Association Test): the original academic instrument, pairing a target concept with positive or negative attributes and timing the response. - SIAT (Single-Category IAT) and MIAT (Multi-category IAT): variants that test one brand or several brands against a fixed set of attributes, common in commercial ad and packaging pretests. - Reaction-time ad and brand-tracker modules: sold as add-ons inside broader research suites from firms like Zappi, System1, Kantar, and Split Second Research, usually bundled alongside a standard explicit survey. In every case, the output is a latency score or an implicit association index, not a choice, a purchase intent estimate, or a market share number. ## Does implicit testing predict what people actually do? Weakly, on its own, and barely at all once you already have explicit survey data. The most-cited meta-analysis of IAT predictive validity, Greenwald et al. (2009), covering 122 studies and roughly 14,900 subjects, found an average correlation of .274 between IAT scores and behavioral, judgment, and physiological outcomes (meta-analysis). A later re-analysis asked a sharper question: once you control for what an explicit survey already tells you, how much additional predictive power does the IAT contribute? The answer was b = .08, which the authors concluded is not a practically meaningful addition (re-analysis). [figure: Bar chart showing two statistics: a .274 correlation between IAT scores and behavioral outcomes across 122 studies, and a .08 incremental predictive validity of IAT beyond explicit measures.] ## Is implicit testing "modern snake oil"? Naming a competitor's criticism instead of burying it is what lets a buyer check the claim against the source. That's the term NielsenIQ used in print, and it's aimed specifically at commercialized reaction-time methods like System1's, not at the original academic IAT. NielsenIQ argues that the science behind these commercial products has drifted from what the underlying academic research actually supports, and that brands making campaign or spend decisions on reaction-time deltas are making costly mistakes as a result (NielsenIQ). The critique lands because it comes from inside the research industry, not from a company selling a competing method. ## Implicit testing versus a randomized experiment Both implicit testing and stated-preference surveys are proxies. Neither puts a respondent in front of a real tradeoff and measures what they choose. ## What does a causal alternative look like? Randomized experiments analyzed with discrete choice models. McFadden discrete choice, Mixed Logit, and ICLV (Integrated Choice and Latent Variable) are the estimators; they're not what makes the result causal. Identification comes from randomly assigning respondents to different prices, features, or messages and observing what they choose, the same logic as a randomized controlled trial, applied to a market decision instead of a drug. That's a materially different claim than a reaction-time delta: it's an estimate of behavior change with a confidence interval, and that interval covers the effect within the tested population, not a guarantee about the real market outside it. Subconscious runs these randomized experiments on a simulation of a market and validates the results against real human studies. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, published with methodology in the causal fidelity paper. That's a validation-set result, not a guarantee for a new, untested market, and because some published studies could plausibly sit inside a model's training data, the replication protocol is built to test against that risk rather than assume it away. Current replication results by study are public on the leaderboard. More on how the estimators and the experiment design fit together is in the methods and validation hub. ## When does implicit testing still make sense? Stating where a method still works is part of publishing its limits. For low-stakes, early-stage screening, ruling out a logo or tagline with an obviously negative gut association before spending on production, implicit testing is a cheap filter. What it isn't is evidence for a launch, price, or campaign-spend decision. The .274 correlation and the .08 incremental validity number are ceilings on what reaction time can tell you about behavior; they don't move regardless of how the test is packaged or which vendor sells it. A buyer weighing implicit testing against a randomized experiment is really choosing between an association-strength score and a causal estimate of what people would do, and only one of those answers the question a launch decision is actually asking. Next step: pull the incremental-validity number for whatever implicit testing tool is on the table now. b = .08 is the published baseline. Ask the vendor what their design adds beyond that before signing off on it. If the decision warrants a causal estimate instead, get in touch. --- # 5 Best Practices of Online Survey Design URL: https://subconscious.ai/blog/methods-and-validation/5-best-practices-online-survey-design Published: 2026-08-28 A senior buyer fielding another market survey needs one answer before writing a single question: the five instrument-design practices that 2026's guides converge on are neutral wording, a funnel structure, avoidance of double-barreled and matrix questions, mobile-first design, and pilot testing before launch. Every major methodology playbook (SurveyMonkey, Qualtrics, Dynata, Sawtooth Software) agrees on this checklist, and following it produces a cleaner instrument. What it does not produce is proof that a respondent's stated answer predicts what that respondent will actually do. - The five instrument-design best practices for 2026 online surveys are neutral wording, funnel structure, avoiding double-barreled and matrix questions, mobile-first layout, and pilot testing. - These practices reduce noise in what a respondent says; they do not close the gap between stated intent and actual behavior. - Two of the most common respondent-authenticity screens, trap questions and speed checks, still pass the large majority of bogus respondents, 84% and 87% respectively, per a 2020 Pew Research Center study, still the most-cited benchmark for these two screens. - A meta-analysis of 77 stated-preference studies across heterogeneous product and policy domains found an average 21% gap between what people say they will do and what they later do, with stated willingness-to-pay running even higher than that average (Catalog of Bias, Schmidt and Bijmolt). - Closing that gap requires a randomized experiment analyzed with discrete choice models and checked against a human baseline, not better survey wording. ## What are the five best practices for online survey design? The five practices that practitioner guides agree on in 2026 are: write neutral, non-leading wording; build a logical funnel from broad to specific questions; avoid double-barreled and dense matrix questions; design mobile-first; and pilot test before fielding. Neutral wording has the deepest empirical backing. Pew Research Center recommends against agree/disagree question formats specifically because less-informed respondents show disproportionate acquiescence bias, and a forced choice between two substantive alternative statements performs better (Writing Survey Questions, Pew Research Center). Funnel structure keeps a respondent from anchoring on a narrow frame before you've asked the broad question. Double-barreled and matrix questions get cut because they ask two things at once and respondents answer the easier one. Mobile-first design and pilot testing exist because most fielded surveys are now taken on a phone, and no amount of careful drafting substitutes for watching a real person get stuck on question four (Masterclass in Survey Design Best Practices, Sawtooth Software). Follow this checklist and you get a well-built instrument. That is a real, necessary outcome. It is also a narrower claim than "valid data," and the two get conflated constantly. ## Why instrument quality is not the same as respondent authenticity Good wording tells you nothing about whether the person answering is real. One case makes the scale concrete: after a study's survey link leaked from its closed panel onto public social media, suspected fraud jumped from 17.4% of responses in the study's first 12 days to 83.1% of all new responses, 1,475 of 1,774, once the link went wide (Gordon et al., 2024, Health Expectations). That's one self-selected sample after a public link leak, not a base rate for panel-recruited work, but it shows the mechanism: once a link escapes a controlled panel, respondent authenticity can collapse fast. A perfectly worded questionnaire does not stop this. It has no mechanism to. ## Why passing the fraud screen still isn't validation The two screens buyers are told to trust for authenticity, trap questions and response-speed checks, catch almost none of the fraud they're meant to catch. [figure: Bar chart showing 84 percent of bogus respondents passed trap questions and 87 percent passed response-speed checks, according to Pew Research Center.] That's from Pew's own study of bogus respondents in online polls (Assessing the Risks to Online Polls From Bogus Respondents, Pew Research Center, 2020). If the standard design-level quality checks miss most fake respondents, the deeper question, whether a real, honest respondent's stated answer predicts their actual behavior, hasn't even been reached yet. ## Can a well-designed, fraud-screened survey predict what people actually do? No. Even an instrument with neutral wording, funnel logic, and a clean fraud screen still asks a respondent to predict their own future behavior, and self-prediction is a different measurement than observation. A meta-analysis of 77 stated-preference studies across heterogeneous product and policy domains puts the average hypothetical bias, the gap between what respondents say and what they later do, at 21%; stated willingness-to-pay studies tend to run even higher than that average (Hypothetical Bias, Catalog of Bias, Schmidt and Bijmolt meta-analysis). Design best practices reduce the noise in a self-report instrument. They were never built to close this specific gap, because the gap isn't a wording problem. It's the difference between asking someone what they'd do and randomly assigning them a choice, then watching what they pick. A well-designed survey, screened for fraud and polished for clarity, is still a well-designed guess. ## What closes the say-do gap: randomized experiments, not better wording Closing the say-do gap requires a different design element entirely: randomized manipulation of the choice itself, not better question wording. A randomized experiment assigns respondents, human or simulated, to different versions of a choice and estimates the effect of each attribute on the decision, typically analyzed with McFadden discrete choice, Mixed Logit, or ICLV models. These are estimators, not causal methods on their own. The causal identification comes from the randomization built into the experiment design, not from the statistical model that reads the results afterward. Subconscious runs these randomized experiments on a simulated market and checks the results against a measured human baseline. On the single best-configured study in that validation set, the simulation reaches a 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959 against each other, putting the simulation at 87% of that measured human ceiling. Across all 43 studies that passed the paper's design filters, the mean is lower, 0.73 (Subconscious, Causal Fidelity paper). That is a validation result on studies that were already run, not a guarantee that holds automatically for a market nobody has tested. Published human studies can also sit inside a model's training data, which is exactly why the replication protocol re-runs against independent human samples instead of trusting one published number. Current standings across the full set of studies are on the public leaderboard. ## Survey hygiene and causal validation, compared ## What a senior buyer should decide before fielding the next survey The decision most buyers now face isn't instrument design versus fraud screening, most fielded work needs both. It's whether the question at hand can be answered by self-report at all, or whether it requires watching a choice happen under controlled variation. A low-stakes, directional question, wording for a support flow, a UI label, still gets answered fastest by a well-piloted survey. A decision that moves real budget, a price point, a launch message, a positioning claim, is different. If that decision rests on what people say they'll do, it sits exactly on top of the 21% gap. Studies behind that distinction are in the methods and validation archive. Concrete next step: before fielding another wave, write down what decision the data needs to support, and check whether that decision would change if stated intent and actual behavior diverged. If it would, that's the trigger to design a randomized experiment instead of another survey. If you want to talk through where your next decision sits on that line, meet with the team. --- # What is Conjoint Analysis? (with examples) URL: https://subconscious.ai/blog/methods-and-validation/conjoint-analysis-examples Published: 2026-08-28 A product lead deciding whether a conjoint result is solid enough to greenlight a launch needs a straight answer, not a survey lecture. Conjoint analysis is a survey method that shows people product profiles built from varying attributes and prices, then infers what they value from the trade-offs they pick. Three common examples: a phone maker tests camera quality against battery life and price, a health system tests drug formulary trade-offs, a SaaS company tests feature bundles against price tiers. A number without its limits is marketing. Running one tells you what people say they'd trade off. It does not by itself tell you what they will do. - Conjoint analysis shows respondents product profiles with varying attributes and prices, then infers relative feature importance and willingness-to-pay from the trade-offs they choose. - Common examples: pricing and feature trade-offs in consumer tech, drug formulary and treatment-preference research in health economics, and bundle-versus-price decisions in SaaS. - The misses go on the public leaderboard right next to the hits. A conjoint result is a stated-preference estimate. It only predicts real behavior once it's been checked against real-world or held-out outcomes, and that check is not automatic (EJHE 2018, eClinicalMedicine 2024). - The three standard estimators, McFadden discrete choice, mixed logit, and ICLV, differ in what they assume about consistency and latent attitudes, not in whether the underlying experiment is causal. - Before greenlighting a launch off a conjoint, ask for the replication number against a held-out human sample, one built to guard against published studies leaking into training data. The public comparison is on the leaderboard. ## What is conjoint analysis? Conjoint analysis is a survey method that shows respondents product profiles built from varying attributes and levels, then infers relative feature importance and willingness-to-pay from the trade-offs they choose, instead of asking direct rating questions. It traces to Paul Green and V. Srinivasan's 1970s marketing applications of Luce and Tukey's mathematical psychology, and it's now a standardized category: Sawtooth Software is the reference implementation for CBC, ACBC, and hierarchical Bayes estimation, Qualtrics folds it into its enterprise XM suite, and ISPOR's Good Research Practices Task Force has codified design and analysis standards for discrete choice experiments (DCEs), the health-economics name for the same conjoint method (ISPOR task force). Most vendor guides stop at "design attributes, run the survey, read the utilities," and treat that as the finish line. ## What are some conjoint analysis examples? Common examples include pricing and feature trade-offs, health-plan or treatment-preference research, and bundle-versus-price decisions: - A research team benchmarked conjoint results against real Swiss naturalization referendum outcomes and found paired designs tracked real behavior best, but that design choices materially changed accuracy (PNAS, Hainmueller et al., 2015). - A phone maker tests camera quality, battery life, and price against each other to prioritize the next feature. - A health system runs a discrete choice experiment on drug formulary or treatment-preference trade-offs, now routine in pricing and access decisions. - A SaaS company tests feature bundles against price tiers before a pricing change. ## How do you know a conjoint result predicts real behavior? Naming this gap is what lets a buyer check the number before it drives a decision. Running a conjoint answers "what did people say they'd trade off." It doesn't answer "does this predict what they'll do," and treating the two as the same question is where launch decisions go wrong. A published number is only useful with its limits attached, so here are the field's own. A 2018 meta-analysis of health-related DCEs found pooled sensitivity of 88% against real-world choices but specificity of only 34% (AUC 0.60). DCEs are good at confirming who will act, weak at flagging who won't, which is exactly the segment that determines ROI on a launch (European Journal of Health Economics, 2018). A 2024 follow-up meta-analysis of ten studies found "reasonable" external validity for opt-in choices specifically, but accuracy still varies by intervention type, setting, and analysis method, not a uniform guarantee (eClinicalMedicine, 2024). Stated willingness-to-pay compounds the problem: absent bias-mitigation techniques, it commonly runs two to three times higher than revealed willingness-to-pay (arXiv, Haghani et al., 2021). Conjoint estimates can also shift with which attributes respondents happen to attend to during the task, a generalization risk separate from sample size or design efficiency (arXiv, 2024). [figure: A four-step path: run the conjoint survey, get utilities and preference shares, check the result against a held-out real outcome, then make the launch decision. The output of step two looks complete on its own but has not yet been validated.] Subconscious runs randomized experiments analyzed with discrete choice models on a simulation of the market, then scores that against real human results, not survey output alone. Our best configuration reaches 87% of the measured human ceiling on one study, 0.832 rank correlation against the published human result, where two independent human samples reach 0.959; the mean across all 43 studies passing design filters is 0.73 (causal fidelity paper). That's a replication accuracy figure against a measured ceiling, not a guarantee for a market we haven't tested. Because published studies can end up in a model's training data, the protocol scores held-out configurations built to guard against that overlap. Current numbers, by method and study, are public on the leaderboard. ## Which estimator fits your decision? It depends on what assumption you can defend: independence of alternatives, heterogeneous preferences across respondents, or an unobserved attitude driving the choice. ## What to do with a conjoint result before you greenlight a launch Ask whoever ran it for the replication number against a held-out human or real-world outcome, one built to guard against training-data overlap with published studies, not just the utilities table. If nobody can produce one, you have a stated-preference survey, not a validated prediction, and either is fine as long as you know which one you're deciding on. Check the estimator against what you actually need: preference-share and substitution calls need a mixed logit or a defensible IIA assumption, not a flat logit run by default. For how these estimators compare across live studies, see methods and validation. If you want to walk through a specific launch decision, meet with us. --- # The ROI of quality checks: never run consumer panels without them URL: https://subconscious.ai/blog/methods-and-validation/roi-quality-checks-never-run-consumer-panels-without-them Published: 2026-08-28 Fraud screens catch bots. They don't catch a wrong pricing decision. Below is the article. --- A VP of insights greenlighting a six-figure panel study has one real decision in front of them: whether a passed fraud screen is reason enough to trust the number that comes back. It isn't. Attention checks, digital fingerprinting, and speed traps confirm a respondent is a distinct, awake human. They say nothing about whether that human's answer predicts what they'll actually do when money is on the line. The return on a quality check that only authenticates the respondent is capped by the same say-do gap that has always undermined stated-preference research. The return that actually protects a launch or pricing call comes from replacing the stated-preference guess with a randomized experiment that reports its own confidence interval on the decision. - A screen that checks authenticity says nothing about the decision on the other end, and that gap belongs on the record before the number does. Fraud screening (attention checks, digital fingerprinting, speed traps) verifies the respondent is real. It does not verify the answer is a causal prediction of behavior. - Raw survey fraud runs 15 to 45 percent of responses depending on platform, and an AI agent can now pass as human 99.8 percent of the time against every detection method tested. - Even a fully clean, human, attentive panel still overstates stated willingness to pay and purchase intent by roughly 1.35x to 3x, worse for novel or complex products. - The ROI that matters isn't "fraud avoided." It's avoiding a pricing or feature decision that self-report data got backwards. - Randomized experiments analyzed with discrete choice models report a confidence interval on the decision itself. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73. ## What do fraud screens on a consumer panel actually verify? They verify that the entity answering is a distinct, attentive human, nothing more. The industry baseline, ESOMAR/GRBN's Guideline on Online Sample Quality, specifies digital fingerprinting, geo-IP validation, machine-learning speeder detection, and "red herring" attention questions (shop.esomar.org). Every one of those checks operates on the respondent's behavior while taking the survey, not on the content of their answer. A respondent who clicks at a normal pace, passes every geo-IP check, and answers the red herring correctly can still tell you they'd pay a 20 percent premium for a feature they'd never actually buy. Authentication and validity are different claims, and procurement checklists routinely treat the first as a proxy for the second. ## How much fraud is actually in panel data right now? Enough that authentication alone is now a losing bet. NORC's literature review puts raw survey fraud at 15 to 30 percent of responses industry-wide, spiking to 45 percent on some platforms (norc.org). In 2025, Dartmouth's Sean Westwood built an AI agent that evaded every bot-detection method tested 99.8 percent of the time, producing a demographically-tailored survey response for about $0.05 against roughly $1.50 for a real panelist, and showed that injecting just 10 to 52 fake responses could flip the outcome of a major national poll (govt.dartmouth.edu). "These aren't crude bots. They think through each question and act like real, careful people making the data look completely legitimate." Sean Westwood, Dartmouth, Proceedings of the National Academy of Sciences (source) [figure: Bar chart showing raw survey response fraud rates ranging from 15 percent on typical platforms to 45 percent on worst-case platforms, per NORC.] Vendors like CloudResearch, Prolific, and Cint compete directly on layered screening against numbers like these. That's a real and necessary fight. It's also a fight that, won outright, still leaves the buyer with nothing more than a clean pipe carrying an unverified answer. ## A clean panel can still hand you the wrong number It can, because passing every ESOMAR-37 check says nothing about whether the choice a respondent made reflects real preference or agreeable guessing. This is the older, better-documented problem: the hypothetical bias literature, including Loomis's review of stated-preference valuation studies, finds that stated willingness to pay and stated purchase intent overstate real behavior by roughly 1.35x to 3x, and the gap widens for novel or complex products where respondents have the least grounded intuition about their own future behavior. Fraud screening was never designed to touch this. A verified, attentive, human respondent answering a hypothetical purchase-intent question is still answering a hypothetical question. Nothing about digital fingerprinting changes that. ## Authentication checks vs causal quality checks The two approaches to "quality" answer different questions, and a buyer who conflates them ends up paying for the wrong one. ## Where does the ROI actually come from? It comes from avoiding the decision you'd have gotten backwards, not from the fraud you filtered. A randomized experiment, where the manipulation (price, feature bundle, message) is assigned at random and the resulting choices are analyzed with discrete choice models such as McFadden's multinomial logit, Mixed Logit, or ICLV, identifies causal effect through the randomization itself. The estimator doesn't make the claim causal; the design does. That distinction matters for a buyer's contract with the research: "randomized experiments analyzed with discrete choice models" is a defensible claim, "causal methods like discrete choice modeling" is not. Our best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73 (the causal fidelity paper); results are tracked on a public leaderboard. Two caveats a senior buyer should hold onto: this is a validation-set result, not a guarantee that it holds for a new, unstudied market, and because some of the human studies used for validation are published, they can sit inside a model's training data, which the replication protocol is built to detect and control for rather than a problem that goes away by assertion. The failure mode gets named here so a buyer can check it before betting a launch on it. A confidence interval produced this way covers the estimated effect within the simulated population studied. It is not an unconditional bound on what the real market will do. One more precision worth naming: a plain multinomial logit assumes independence of irrelevant alternatives, so a preference-share or substitution question run on a flat logit should be checked against a Mixed Logit or ICLV specification, which relax that assumption, before a buyer treats the share numbers as final. ## Does paying more for a cleaner panel fix the problem? No, because cost and causal validity are separate axes and paying more only moves you along the first one. Cost per usable, high-quality respondent varies nearly 4x by platform: about $1.90 on Prolific and $2.00 on CloudResearch against $8.17 on Qualtrics panels, driven almost entirely by screening depth (NCBI/PMC). That's real money, and it buys a real thing: fewer bots, fewer inattentive respondents, a cleaner denominator. It does not buy a causal estimate. A buyer who pays $8.17 a head for the cleanest possible panel and then asks a stated-intent question has spent four times as much to get the same hypothetical-bias problem, just with fewer bots mixed in. ## What should a senior buyer do before greenlighting the next study? Separate the two budget lines. Keep fraud screening, at whatever tier the study's stakes justify, as the floor that keeps the input pool real. Then ask a second, harder question of the research design itself: does this study manipulate something at random and measure a choice, or does it just ask people what they'd do? If it's the latter, the "ROI" being reported is fraud avoided, not a decision protected, and that's worth saying out loud before the number gets treated as fact in a pricing meeting. Methods that make this distinction concrete, including how replication is scored against human baselines, are covered in more depth on the methods and validation hub. Before the next panel study goes out, pick one decision already on the table (a price point, a feature tradeoff) and run it two ways: a standard stated-intent question through your usual panel, and a randomized experiment on the same decision with a held-out slice of respondents used to check replication. Compare the two confidence intervals side by side before either one reaches a deck. If you want a second pair of eyes on that design, the team is easy to reach. --- # How to Get Participants For Your Study URL: https://subconscious.ai/blog/implementation-and-procurement/get-participants-study Published: 2026-08-28 --- A research director who needs three hundred completes by next Friday has a sourcing decision to make, but it is not "which panel has the lowest bot rate." The decision is whether the study design itself, panel aside, produces an answer about why people choose rather than just what they claim they'd choose. A number without its limits is just marketing. A perfectly screened sample still only reports stated preference, and stated preference has never been the same thing as the behavior that drives a purchase, a signup, or a churn. - Panel hygiene (attention checks, speeder removal, fraud screens) fixes who answers, not what a survey can tell you about behavior. - The cleanest panels available today still only capture stated preference, which diverges from real choice behavior in well-documented ways. - Synthetic and LLM-simulated respondents introduce a second, larger gap: coefficients that don't match real human survey data, sometimes with the effect's sign reversed. - The one decision that matters for procurement: does the design use a randomized manipulation and a discrete choice model, or does it just collect opinions from a clean-looking sample. - Any sourcing choice, panel or synthetic, should be checked against a human baseline before the numbers go into a pricing or positioning decision. ## Panel hygiene fixes fraud, not the stated-preference gap Panel providers have spent the last two years competing on fraud metrics because they had to. Click-farm and bot infiltration pushed usable response rates in some pipelines from roughly 75% down to about 10%, which is a real operational crisis and a real reason to screen harder (CloudResearch). That figure comes from a vendor blog with a fraud-detection product to sell, not a peer-reviewed source, so treat it as a directional signal of a real trend rather than a precise industry rate. Naming where a fix stops is what lets a buyer check it before spending on it. But fraud screening answers a narrower question than buyers assume. It tells you the person on the other end of the survey is a human who read the question. It does not tell you that the answer they gave predicts what they will actually do. Those are separate problems, and a checklist that stops at "screen for attention checks, pick a reputable panel, set fair incentives" treats the second problem as solved by solving the first. ## Which panel should a buyer choose for a clean sample? Pick the panel whose attention-check pass rate and cost per quality respondent fit the budget, then treat the resulting data as stated preference, not causal proof. A cleaner panel produces more attentive answers, not more predictive ones. In the largest recent comparison of major panels, only 59% of MTurk respondents passed both attention checks, versus 87% on Prolific and 85% on CloudResearch, and cost per quality respondent ran $1.90 on Prolific against $4.36 on MTurk and $8.17 on Qualtrics (PLOS One, Douglas, Ewell, Brauer, 2023). That is a meaningful difference in data hygiene. It says nothing about whether the underlying survey question was structured to isolate a causal driver of choice versus a stated opinion shaped by social desirability or professional-panelist fatigue. A panel with a 95% attention-check pass rate can still return a stated-preference number that has no relationship to what happens when the product actually ships. "We found that compared to MTurk, Qualtrics, or an undergraduate student sample (i.e., SONA), participants on Prolific and CloudResearch were more likely to pass various attention checks, provide meaningful answers, follow instructions, remember previously presented information, have a unique IP address and geolocation, and work slowly enough to be able to read all the items." Douglas, Ewell, and Brauer, PLOS One (2023) (source) (All figures from PLOS One, 2023.) This table answers a procurement question. It does not answer the validity question, which sits one level up. ## Can synthetic respondents replace a human panel entirely? The misses go on the record next to the hits. No, not as a like-for-like substitute, and the evidence on this is specific. A peer-reviewed comparison found that 48% of regression coefficients estimated from ChatGPT-simulated survey responses differed significantly from real ANES human data, and the sign of the effect flipped in 32% of those mismatched cases (Political Analysis, Cambridge). That comparison covers one dataset (ANES), one model, and one ChatGPT vintage, not a general claim about every LLM or every survey domain, but the direction of the finding is the part that matters for a sourcing decision: coefficients moved, and some flipped sign. A flipped sign is not noise. It means a synthetic-respondent study can point a pricing or messaging decision in the wrong direction while still returning a clean, complete, well-formatted dataset. The underlying caution holds regardless of how many practitioners voice it: an LLM trained on internet text is not a randomized experiment, and asking it to role-play a respondent does not create one. ## Where fraud filtering and causal design solve different problems The reason buyers conflate these two fixes is that both happen inside the same "collect responses" step of a study, which makes it easy to assume that fixing one fixes the other. [figure: Two parallel tracks. The top track shows panel hygiene steps ending in a clean stated-preference dataset. The bottom track shows randomized design steps ending in a causal effect with a confidence interval.] A study can pass every fraud check on the top track and still fail the bottom one, because nothing in attention-check screening tests whether the survey isolates what actually drives the choice. ## What actually closes the say-do gap in a stated-preference study? A randomized manipulation, not a screening pass, closes the say-do gap, because it is the randomization that lets you attribute a change in choice to a specific intervention rather than to whoever happened to answer. Methods like McFadden discrete choice, Mixed Logit, and ICLV are estimators that fit a model to choice data; they are not themselves the source of causal identification. The precise description is a randomized experiment analyzed with a discrete choice model, not a "causal method like DCE." A standard multinomial logit also carries the IIA assumption, meaning it assumes adding or removing an option doesn't change the relative odds between the others, which is one reason Mixed Logit is preferred when substitution patterns matter. None of this depends on which panel supplied the respondents. It depends on whether the study randomized something and measured the resulting choice. ## How does a buyer validate a sourcing decision against real behavior? A buyer checks the design's output against a held-out human baseline before trusting it. The same discipline that applies to panel selection applies to any simulated or synthetic-respondent design. On causal fidelity testing, the best configuration reaches 87% of the measured human ceiling on one study: a 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959 correlation with each other. Across all 43 studies that passed design filters, the mean is 0.73, well below the single-study best case (causal fidelity paper). A correlation score is worth publishing only alongside where it stops holding. That number is a validation result on studies already run, not a guarantee for a market you haven't tested yet, and it carries the same caveat that applies to any comparison against a published human study: the published result may have been in a model's training data, which is exactly why a replication protocol and a held-out baseline matter more than a single correlation figure. A confidence interval built from a simulated experiment covers the effect within that simulated population; it is not a claim about the real market until it has been checked against holdout human data. Current comparisons across methods and providers are tracked on the leaderboard, and the underlying validation approach is described on the methods and validation hub. ## The one decision procurement checklists skip The checklist answers "did I get a clean sample." The decision that actually determines whether a study is worth running is "does this design produce a causal, replicable answer, regardless of which panel supplied the respondents." A senior buyer facing a participant-sourcing decision this week should pick the panel that fits the budget and screening bar, using the PLOS One figures above as a starting point, then check whether the design itself randomizes the variable under test, the one thing panel hygiene cannot fix. More panel comparisons are collected on /blog/comparisons, and current method-by-method fidelity results are on the leaderboard. When the design needs a held-out human baseline before it ships, /meet is the next step. --- # Diagnosis Over Vibes: The Science of Brand Ambassadors URL: https://subconscious.ai/blog/use-cases/diagnosis-over-vibes-science-brand-ambassadors Published: 2026-08-28 --- Diagnosis Over Vibes: The Science of Brand Ambassadors A brand marketing lead staring at a seven-figure ambassador contract isn't really choosing a face for the campaign. They're deciding whether to trust a room's collective read on "fit," or to test the actual attributes on the table against real trade-offs before signing. The second option is the only one that answers the question a signature is supposed to answer: which ambassador attributes move purchase choice, and by how much. That comes from a randomized experiment, not a focus group's verdict on whether the partnership feels credible. - Ambassador selection runs on the "match-up hypothesis" from 1990s advertising research, validated mostly by focus groups and stated-preference surveys that measure sentiment, not purchase behavior. - SocialLadder, a vendor of ambassador-program software, reports in its own white paper that just 14 percent of ambassadors drive 80 percent of program ROI. The paper doesn't disclose its sample or method, and a skewed return distribution doesn't by itself prove most selection decisions were wrong, but it does show how uneven ambassador performance is even after brands have signed. - Celebrities and influencers move purchase choice through different causal mechanisms: credibility for celebrities, parasocial bond for influencers, according to a 2025 study in the Asia Pacific Journal of Marketing and Logistics (Emerald). - Bud Light's Dylan Mulvaney partnership, a values-alignment call that looked defensible in a boardroom, cost Anheuser-Busch InBev an estimated $1 billion-plus in sales (CNN Business) and dropped the brand to third place more than a year later (Forbes). - On Subconscious's validation set, the best configuration reaches 87% of the measured human ceiling on one study: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959. Across all 43 studies that pass design filters the mean is 0.73, from the causal fidelity paper. It is a starting point for trust, not a guarantee for an untested market. ## What does "diagnosis over vibes" mean for ambassador selection? It means replacing a subjective fit judgment with a measured causal effect. A focus group or survey asks people to rate whether an ambassador "feels right" for the brand, which is a sentiment proxy collected after the fact. A randomized experiment does something structurally different: it puts real ambassador candidates into a choice task against real trade-offs: price, category risk, and a competing endorser, then randomizes which attributes each respondent sees. The resulting effect on choice is causal because the manipulation, not the respondent's stated opinion, is what varies. That's the diagnosis: not "does this feel credible," but "does this specific attribute move the purchase decision, and by how much." ## Why does most ambassador selection still run on vibes? Because the industry has grown faster than its evaluation methods have. Budgets for ambassador and influencer partnerships have been climbing for several years, and brands increasingly favor them over traditional paid ads for the trust they carry with audiences. That growth has outpaced the rigor of the underlying selection process, which still leans on the match-up hypothesis: pick whoever's image "fits," and confirm the choice with social listening or a stated-preference survey. Those tools were never built to isolate a causal driver. They tell a brand what a sample says about a pairing, not what would happen to purchase choice if that pairing competed against alternatives in the market. SocialLadder's white paper on ambassador program performance reports the same 14/80 split noted above: a distribution that shows most of the payoff is concentrated in a small share of partnerships, without telling a brand in advance which share that will be. ## The Bud Light lesson: what a vibes-based call actually costs Bud Light's partnership with Dylan Mulvaney is the industry's reference case for what happens when a values-fit judgment substitutes for a tested one. The backlash cost Anheuser-Busch InBev an estimated $1 billion or more in lost sales (CNN Business), and more than a year later the brand was still sitting in third place in the U.S. market, down from first (Forbes). No test could have predicted the exact scale or form of the backlash. What a randomized test before signing could have done is produce an effect estimate: how the ambassador's attributes were likely to move purchase choice with the brand's existing audience, a risk signal a boardroom read on "defensible" never produces, and something Anheuser-Busch could have weighed against the expected upside before signing rather than after the boycott began. ## Why do celebrities and influencers move purchase choice differently? Because they operate through different causal pathways, and treating them as interchangeable "ambassador" categories is where a lot of vetting processes go wrong. A Journal of Marketing Analytics study tested how endorser type interacts with ad medium and found celebrity ads outperform influencer ads in traditional media (TV, print), an advantage that disappears on social platforms (Springer). It's a single study, not yet replicated across other product categories, so treat the traditional-versus-social split as a hypothesis to test against your own audience rather than a fixed rule. The Asia Pacific Journal of Marketing and Logistics study cited above found that celebrities and influencers build brand equity through distinct mechanisms: credibility for celebrities, parasocial connection for influencers (Emerald). A brand that tests "does this person fit" without separating which mechanism it's buying is asking the wrong question for the channel it's about to run in. ## What does a randomized experiment for ambassador selection look like? It looks like a choice task, not a rating scale. Respondents in a simulated population see ambassador candidates paired with randomized attributes, category framing, price point, and a competing endorser, and they choose, rather than rate agreement with a fit statement. The estimator that recovers the effect matters less than the randomization that produces it: McFadden discrete choice, Mixed Logit, and ICLV (Integrated Choice and Latent Variable) models are estimators, not causal methods on their own. The causal identification comes from the randomized manipulation built into the experiment design. Mixed Logit is useful here because it allows preferences to vary across the simulated population instead of assuming everyone reacts to an ambassador the same way, which matters directly for the credibility-versus-parasocial split above. ICLV is useful when the attribute driving choice is something latent, like perceived authenticity, that can't be observed directly but can be linked statistically to the choice outcome. [figure: A flow diagram with six steps: identifying ambassador candidates, randomizing their attributes across choice sets, having respondents choose between randomized trade-offs rather than rate fit, comparing the outcome against a holdout group, reporting a causal effect with a confidence interval, and making the signing decision on that measured effect.] ## How reliable is a simulated test before you sign? It's reliable enough to be a filter before a contract, not a substitute for watching the real market respond. On Subconscious's validation set, the best configuration reaches 87% of the measured human ceiling (0.832 against a 0.959 human-to-human ceiling; mean 0.73 across the 43 studies passing design filters), from the causal fidelity paper. That number is a validation-set result: it describes how the method performed against a known set of prior human studies, not a guarantee for a brand-new market or an untested ambassador pairing. It also can't fully rule out that some of those published studies were part of the underlying model's training data, which is exactly why the replication protocol exists as an ongoing check rather than a one-time claim. A confidence interval reported from a simulated experiment covers the effect within that simulated population; it does not bound the real market on its own. And any preference-share question (if Ambassador A gains share, where does that share come from) carries the independence of irrelevant alternatives assumption under a flat logit, which is one reason Mixed Logit is the more defensible choice when ambassadors are competing for overlapping audience segments. The current model rankings and how they perform against held-out human studies are public on the leaderboard; the underlying protocol is documented on the methods and validation blog hub. ## What's the smallest test to run before the next ambassador contract? Take the shortlist you already have and turn the vetting meeting into a choice task instead of a discussion. List the two or three ambassadors under real consideration, the attributes actually in question (values alignment, category fit, price sensitivity of the audience, a plausible competing endorser), and run a randomized comparison before the term sheet goes out, not after. If the effect size on purchase choice is small or the confidence interval straddles zero, that's the answer a focus group was never built to give you. Start with the leaderboard to see how the method performs against known studies, and if you want a second read on a specific decision, a quick conversation with the team is the fastest way to scope it. --- # How CPG Teams Test a Price Increase Before a Shelf Reset URL: https://subconscious.ai/blog/industries/how-cpg-teams-test-price-increase-before-shelf-reset Published: 2026-08-30 A revenue growth management or brand lead facing a retailer's category reset in six to twelve weeks needs to know whether a price increase on an already-listed SKU will hold, and the honest answer is that scanner history alone cannot tell them, because scanner history only reports prices that were actually charged, not the higher price under consideration. Household panel and point-of-sale data describe a market that existed; the shelf reset question is about a price that has never existed on that shelf. A randomized choice experiment can answer it, because the price a shopper sees is assigned by the experiment rather than pulled from history that was mostly generated by promotions. - A price increase on a listed SKU should be tested with a randomized choice experiment, not scanner-derived elasticity, because scanner data reflects prices that were charged - largely promotional depths - not the new shelf price being proposed. - Scanner-based elasticity is confounded with the display, feature, and seasonality that traveled alongside historical promotions, and the same categories can flip between "elastic" and "inelastic" depending on which data source produced the estimate. - Gabor-Granger and Van Westendorp survey backstops ask a shopper to name a number with no budget and no tradeoff; stated willingness to pay runs an average of 21% above real willingness to pay (Schmidt and Bijmolt, Journal of the Academy of Marketing Science 48(3), 2020: https://research.rug.nl/en/publications/accurately-measuring-willingness-to-pay-for-consumer-goods-a-meta/). - A randomized experiment assigns the price against the real, upcoming competitive set on the shelf, producing an estimate of the effect of that specific increase, plus the cross-price effect on the rest of the category and category dollars rather than brand volume alone. - Every effect should ship with a confidence interval and a clear statement of which price points the design actually tested. ## What decision does a CPG team need to test before a shelf reset? The decision is narrow and consequential: whether to take a list price increase on a SKU that already has distribution, how large an increase, and whether to pair it with a pack-size change, all six to twelve weeks before the retailer's line review. The risk is not symmetric. Hold price and margin erodes quarter over quarter. Push the increase past what the category buyer will tolerate, and the SKU loses facings or gets delisted at the review - and winning that shelf space back typically takes a full planning cycle, not a quarter. That asymmetry is why the test has to happen before the reset, not after the retailer has already reacted. ## Why the scanner data everyone reaches for can't answer this question Retail scanner history and household panel data are records of what happened, not experiments about what would happen. The price variation embedded in that history is overwhelmingly promotional - temporary price cuts, feature and display activity - not permanent shelf-price movement, and the promotional periods carry their own lift from display and feature that gets folded into the elasticity estimate along with the price effect itself. An RGM team estimating elasticity from that history is really estimating the combined effect of a price cut plus a promotional event, and applying it to a permanent list price increase the SKU has never carried. The estimate answers a different question than the one the line review is going to ask. ## Why does historical elasticity break down right when you need it most? It breaks down because the estimate is fragile to the data source, not just the time period. Luke, Tonsor, and Schroeder's meat demand elasticity research (Agricultural and Resource Economics Review 55(1), 2026) finds the same meat categories flip between elastic and inelastic, and between luxury and necessity classification, depending on whether the analyst used public aggregate data or retail scanner data . A number that reverses sign depending on which dataset produced it is not a stable input for a decision with a year-long downside if it's wrong. The category buyer at the line review does not care which data source generated the RGM team's estimate; they care whether the shelf still sells at the new price. ## The survey backstop carries its own bias Most RGM teams pair scanner elasticity with a Gabor-Granger or Van Westendorp survey as a sanity check, and that backstop has a known, measured problem. Schmidt and Bijmolt's meta-analysis, spanning 77 studies in 47 papers with 24,347 hypothetical and 20,656 real willingness-to-pay observations, finds that stated willingness to pay overstates real willingness to pay by an average of 21% (Journal of the Academy of Marketing Science 48(3), 2020: https://research.rug.nl/en/publications/accurately-measuring-willingness-to-pay-for-consumer-goods-a-meta/). Asking a shopper to name a price with no budget constraint and no competing option in front of them produces a number that is systematically too generous. A team that anchors a price increase on that number is building in the hypothetical bias from the start, not correcting for it. ## What does a randomized shelf-reset price experiment look like? It looks like a discrete choice experiment where the price a respondent sees is randomly assigned rather than observed from history, run against the real, upcoming competitive set - the actual competitor SKUs, prices, and pack sizes expected to be on that shelf after the reset, not the current lineup. Because the assignment is random, the resulting effect is an estimate of what the price increase causes, not a correlation carried over from whatever price variation happened to occur historically. Randomized experiments analyzed with discrete choice models - McFadden discrete choice, Mixed Logit, and ICLV - are how the effect gets estimated once the manipulation is in place; the identification comes from the randomization in the design, not from the estimator itself. A flat multinomial logit model carries the IIA assumption, that a shopper's relative odds between two SKUs don't shift when a third option enters or leaves the set; Mixed Logit relaxes that assumption by letting preferences vary across the simulated population, which matters directly at a shelf reset because the competitive set itself is changing. ## Conventional approach and randomized experiment, side by side ## What does the retailer actually want to see at the line review? The category buyer wants category dollars and the cross-price effect on the rest of the set, not just the effect on the RGM team's own brand. A randomized experiment against the real competitive set delivers both directly, because every simulated shopper chooses among all the SKUs the design includes, not just the client's own line. That is the number a category buyer can act on: not "our brand's volume at the new price," but "what happens to the category's dollars, and which competitor picks up the share that moves." Report each effect with its confidence interval, understanding that the interval covers the effect within the simulated population and design tested, not the entire real market unconditionally, and state plainly which price points the design actually covered so the buyer knows the range the result speaks to. ## How much can you trust a randomized price experiment? Trust it to the extent the validation record supports, and no further, which is why the record should always ship with its denominator rather than a bare percentage. On the Hainmueller immigration conjoint study, our best configuration reaches 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959 - 87% of that measured human ceiling, on one study, at the best configuration. Across all 43 published randomized studies that pass design filters, the mean is 0.73 (https://fidelity.subconscious.ai/papers/causal-fidelity/causal_fidelity_paper.pdf). Published studies can sit in a model's training data, which is a real limitation the replication protocol behind that paper is built to address, not a problem that disappears by asserting validation. The full leaderboard, updated as new replications run, is public at leaderboard; the methodology behind the fidelity numbers and how they're built is covered on the methods and validation hub. [figure: Bar chart comparing three rank correlation values: human-to-human ceiling at 0.959, best configuration at 0.832, and mean across 43 studies at 0.73, all against the Hainmueller immigration conjoint benchmark.] Before the next line review, pull the actual competitive set expected on the reset shelf - competitor SKUs, prices, and pack sizes - and write down the specific price points under consideration for the increase; that set is the input a randomized experiment needs, and having it in hand is the first concrete step regardless of which testing method gets used. For a walkthrough of how a shelf-reset design gets built against a specific category, book time on /meet. --- # How to Test Patient Messaging in a Regulated Environment URL: https://subconscious.ai/blog/industries/how-to-test-patient-messaging-in-a-regulated-environment Published: 2026-08-30 Patient messaging under FDA oversight rarely gets tested where it matters. A brand team choosing which patient-directed message to build a campaign on needs the comparison to run before the message goes into legal and medical review, not after. Testing after review only ranks whichever executions already survived that process, so the result tells the team which survivor scored best - not which message, from the full set the team debated, deserves the media budget and can carry its required risk information. - Randomize the full candidate set before review, not the shortlist review already narrowed, so the evidence sets the option space instead of the review queue. - Keep the required risk information present in every version tested, since a message that can't carry its major statement can't ship regardless of how well it scores. - A monadic scale (appeal, believability, intent) measures how a message reads, not whether it changes what happens in a clinical encounter. - An in-market split arrives after the spend and after a prescriber has mediated the outcome, which is too late to inform the choice. - Check any validation number against its denominator: the strongest published result here is 0.832 against a measured human ceiling of 0.959, one study, best configuration, with a mean of 0.73 across the 43 studies that pass design filters. ## Why review order decides more than the test does The conventional workflow drafts several executions, sends them through legal and medical review, and only tests whatever comes out the other side. By the time a monadic test with appeal, believability, and intent scales runs, the option space has already been set by which executions cleared review, not by which ones would perform best. The test then answers a narrower question than the one the brand team actually faces: instead of "which message should we build the campaign on," it answers "which of the messages that already survived review scores highest." Those are different decisions, and only one of them is open when the test runs. ## Why doesn't a rating scale predict what a message does in a clinical encounter? A rating scale predicts how a message reads to the person taking the survey, not what a patient does with it in front of a prescriber. Appeal, believability, and intent are self-report constructs: a respondent tells you how convincing a message seemed, not whether it changed the question they asked their doctor or the request they made at the counter. Patient-directed messaging earns its media budget by moving behavior inside a clinical encounter the survey never observes. A message can score well on all three scales and still fail to move that behavior, because the scales were never built to measure it. ## Why does an in-market split arrive too late to guide the choice? An in-market split test only reports its result after the media has already been bought and aired. By then the budget decision the test exists to inform has already been made. The outcome it measures - a click, a recall lift, a site visit - is also mediated by a prescriber, since the patient's next step almost always runs through a clinical conversation the ad itself doesn't control. A click-through rate is several steps removed from the decision the campaign is trying to influence, and the delay means the evidence lands after the money is spent rather than before. ## What changes when the message is randomized before submission Randomizing the message before it goes to review changes what the test can answer. Instead of testing whichever executions survived review, the team varies the specific elements under debate - claim framing, tone, the placement and weight of risk information - and holds the required risk information present in every arm, so what gets tested is what could actually be submitted. Each element's effect is estimated with a confidence interval rather than a single score, and that interval describes the effect within the tested population, not a guarantee about the real market. [figure: A four-step flow showing draft, then review narrowing options, then testing only the narrowed set, then production, illustrating that the conventional test never sees the full candidate pool.] Two things follow from moving the test earlier. The choice happens before submission, so review time gets spent on the message the evidence already supports rather than on adjudicating between finalists nobody tested against the full field. And the process leaves a stated design and a set of effect estimates, which is the form of support a team can hand to its own legal and medical reviewers - not proof of compliance, but a documented basis for the choice they're being asked to approve. ## What the major statement rule requires of every arm you test FDA's rule on the major statement in broadcast advertisements requires that the statement be presented in a clear, conspicuous, and neutral manner - not buried in pacing, visuals, or competing audio that undercuts it (Federal Register vol. 88, no. 223, Nov. 21, 2023, p. 80958: govinfo.gov). The rule took effect May 20, 2024, with a compliance date of November 20, 2024. If a message tested for appeal or intent doesn't carry its risk information the same way it would need to on air, the test result describes a version of the message that can't legally run. Holding the risk information constant and present in every arm during the test, rather than adding it back in after a winner is chosen, is what keeps the test measuring something that can actually ship. Whether a specific execution meets the clear-conspicuous-neutral standard is a determination for the team's own legal and regulatory reviewers, not a claim any test output can make on its own. ## How reliable is a simulated randomized comparison? The strongest published result for this approach is a rank correlation of 0.832 against a measured human-to-human ceiling of 0.959 on the Hainmueller immigration conjoint study - one study, the best configuration reported, which works out to roughly 87% of that measured ceiling (causal fidelity paper, PDF). Across the full set of 43 published randomized studies that pass design filters, the mean rank correlation is 0.73, and that broader number is the more honest baseline to plan against, not the single-study best case. A limitation worth stating plainly: published human studies can sit inside a model's training data, which would let it pattern-match a known result rather than predict a new one; the replication protocol behind these figures screens for that risk, but the general problem of testing against material a model may have seen before doesn't disappear just because a protocol exists. The estimators behind the comparison - McFadden discrete choice, Mixed Logit, and ICLV - are statistical models for choice data, not causal methods on their own. Causal identification comes from randomizing the message elements in the experiment design; the estimator's job is to turn the resulting choices into effect sizes. Mixed Logit is used specifically because it relaxes the independence-of-irrelevant-alternatives assumption a flat multinomial logit imposes, which matters when patient messages are close substitutes for each other. Current results across studies and methods are tracked on the leaderboard, and the underlying design questions are covered in more depth on the methods and validation hub. ## Three ways to test a patient message ## What to do next Take the messages currently competing for the brand team's decision and list, for each one, whether the required risk information is present in the version being discussed - not a placeholder version, the one that would actually run. Any message where that's not true isn't ready to be tested yet, regardless of how well it might score. Once the candidate set carries its risk information consistently, that's the set worth comparing, before any of them goes to review. For a closer look at how that comparison is designed and validated, meet with the team. --- # Payer Perception Research When You Cannot A/B Test URL: https://subconscious.ai/blog/industries/payer-perception-research-when-you-cannot-a-b-test Published: 2026-08-30 A payer perception study replaces the A/B test it cannot run by randomizing the dossier elements a team controls before the meeting, then measuring which ones move the coverage decision. --- A market access lead locking a value story before the first payer meeting can still get a causal read on it, by randomizing the dossier instead of the plan. Each plan is approached once, its outcome negotiated rather than observed, and the pool of decision makers for a single product is too small for a randomized field trial at any budget. A randomized choice experiment moves the randomization onto the value story itself, estimating the effect of each dossier element on the coverage decision before that meeting happens. - A formulary decision cannot be A/B tested because each plan is approached once and negotiated, not observed repeatedly. - The workaround randomizes the dossier itself: endpoint emphasis, comparator, budget impact framing, rebate structure, restriction criteria. - Choice experiments analyzed with McFadden, Mixed Logit, or ICLV produce an estimated effect with a confidence interval; a themes summary does not. - On one published conjoint the best configuration reaches 87% of a measured human-to-human ceiling (0.832 against 0.959); across the 43 studies passing design filters the mean is 0.73 (Causal Fidelity paper). - Advisory boards keep a role after the experiment runs, on the questions it could not settle. ## Why can't a payer decision be A/B tested? A payer decision can't be A/B tested because there is one plan and one negotiation, and the outcome forms once, with no second arm to compare it against. A market access team does not get to submit two versions of a dossier to the same P&T committee and see which one clears review faster. The committee meets once, forms a position, and that position becomes the outcome. There is no untreated control group sitting next to it having reached a different conclusion under identical conditions. The population problem compounds this. The medical and pharmacy directors who will rule on a given product are few, against a marketing experiment that can split thousands of users into arms unnoticed, and no verified public figure exists for that population's size in the United States. A randomized trial on real committees at real formularies is not a designable study: the sample is too small, the stakes per observation too high, and the negotiation changes the moment a company instruments it. The causal question stays answerable once the object being randomized changes from the plan to the dossier. A randomized choice experiment assigns different value stories to different simulated or surveyed respondents and measures which elements move the coverage decision, with an interval around the estimate. ## What advisory boards get right, and where they stop Advisory boards surface objections a team did not anticipate. Ten or fifteen structured conversations with payer medical directors turn up concerns a slide deck missed, and no simulated experiment replaces the judgment of someone who sits on these committees. What they cannot produce is an effect: a summary of themes from twelve conversations carries no interval, so a team cannot say whether the gap between two value stories is large, small, or noise. A handful of interviews was never built to estimate an attribute-level effect. What a payer says in that room is also a stated position inside a bargaining context, with nothing at stake for the speaker, so a themes summary has no way to correct for a say-do gap. ## The decision in front of you: value story, price, contracting posture The senior buyer here is a market access lead who has to lock a value story, a price, and a contracting posture ahead of formulary and pathway discussions, with no second attempt in the room. That decision breaks into a handful of concrete calls: - Which clinical endpoint leads the narrative, and which becomes supporting evidence. - Whether the dossier argues budget impact or cost offset as the primary economic frame. - Which restriction criteria get conceded before the meeting and which get defended. - How the rebate and contracting structure gets positioned relative to list price. Every one of these choices gets made once, in front of the actual committee. The sequencing of concessions in particular cannot be learned by trying both orders, because the meeting happens once. ## Randomize the dossier, not the plan The plan cannot be randomized. The dossier can. A randomized choice experiment varies the clinical benefit size, the comparator, the budget impact framing, the rebate structure, and the restriction criteria across choice tasks presented to a payer-relevant sample, and estimates the effect of each element on the simulated coverage decision, with a confidence interval around each estimate. That interval covers the effect within the population sampled or simulated for the experiment; it does not, on its own, bound what a specific committee will do in a specific room. Rebate structure and price positioning deserve a separate caveat. A choice task is a stated-preference design, not an incentive-aligned one - nobody in the experiment is spending a real budget or signing a real contract. Estimated price and rebate sensitivity from this kind of design tends to run high relative to what a payer will actually concede once real money is on the table. Treat that estimate as a directional input for sequencing concessions, not as a negotiating floor. [figure: A chain showing five dossier elements - endpoint emphasis, comparator, budget impact framing, rebate structure, restriction criteria - feeding into a randomized choice task, which produces an estimated effect with a confidence interval for each element.] ## Which estimator: McFadden discrete choice, mixed logit, or ICLV? Each turns randomized choice data into an effect; the causal claim comes from the randomization in the design. A McFadden discrete choice model estimates attribute-level effects and assumes independence of irrelevant alternatives, which can flatten substitution between formulary tiers. Mixed Logit relaxes that assumption by letting preferences vary across respondents, which matters when a medical director and a pharmacy director weight the same restriction criterion differently. ICLV models a latent construct between the dossier and the choice, a belief about clinical credibility or budget risk, useful when the real driver is not observable in the attribute list. Which estimator fits is a design question to settle before data collection. ## How close does a simulated payer population get to real behavior? A simulated population's accuracy means something only next to the measured human ceiling it was validated against. On the Hainmueller immigration conjoint, Subconscious's best configuration reaches 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959 against each other, so 87% of that measured ceiling on one study. Across the 43 published studies passing design filters the mean is 0.73, the more representative number (Causal Fidelity paper). Two caveats belong next to this number. Published studies can sit inside a language model's training data, so a replication protocol has to be built to catch that; the methodology and per-study breakdown are on the leaderboard and in the methods and validation archive. This is also a validation result on studies already run, so a new therapeutic area or payer type is exactly where the mean across 43 studies is the honest number to plan against. None of it is validated for a reimbursement submission or accepted before a health technology assessment body; it supports the value story and the meeting prep. Numbers submitted into a reimbursement process do get checked against reality. Kossmeier, Themanns, Hatapoglu and colleagues compared company-submitted sales forecasts with actual post-launch sales for 102 products applying for reimbursement in Austria (Frontiers in Pharmacology, 2021, full study). That study measures forecast accuracy rather than payer perception method accuracy, and Subconscious publishes its misses against real human choice data on the same basis. ## What a defensible design documents A defensible dossier experiment documents its attributes and levels, choice-task construction, sample composition, and estimation model well enough for a health economics group to audit unaided. The field has a consensus checklist for this: Bridges et al., "Conjoint Analysis Applications in Health - a Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force," Value in Health, 2011 (DOI: 10.1016/j.jval.2010.11.013). A design that skips attribute-level justification, cannot state how choice tasks were generated, or reports a topline number without the underlying model is not auditable, however polished the output looks. ## Where the real advisory board still earns its budget The advisory board earns its budget on questions a randomized dossier experiment cannot settle: a committee's politics, a regulatory nuance the model never saw, or language too new to have comparable published data. Once the experiment has ranked which elements move the coverage decision, the advisory conversation becomes a focused check on what remains uncertain, which is a better use of a payer medical director's hour. Before the next meeting, take the last dossier this team built and mark which of the five elements above were ever varied and tested against each other, and which were asserted. The ones never varied are where a randomized choice experiment replaces a guess with an estimate. To walk through what that experiment looks like for this decision, talk to the team. --- # Research Methods for Hard to Reach Prescribers and Rare Disease Populations URL: https://subconscious.ai/blog/industries/research-methods-for-hard-to-reach-prescribers-and-rare-disease-populations Published: 2026-08-30 Research Methods for Hard to Reach Prescribers and Rare Disease Populations A commercial or medical affairs lead choosing a positioning, a service model, or a target profile for a rare disease or a narrow specialty can't wait for a discrete choice experiment sized the way the textbook requires: the frame is too small no matter what the honorarium reaches. Change what gets sampled: prove the substitute against published choice experiments already run in the same clinical area, using the replication record that shows where it holds and where it doesn't. - The binding constraint in a rare or narrow-specialty study is sample-size arithmetic: a discrete choice design sets a minimum respondent count from its number of choice tasks and attribute levels, and a small clinical frame can't supply it regardless of spend. - Raising honoraria or extending field time doesn't fix a frame that's too small: a large share of published health discrete choice experiments already run under recommended minimums, and 71% did not clearly report how they determined sample size (de Bekker-Grob et al., 2015). - Eight or twelve qualitative interviews can inform instrument wording, but they produce no effect size and no confidence interval, which is what a launch or positioning decision needs. - A simulated population, conditioned on the clinical and demographic profile and run through the same randomized choice design a fieldable study would use, extends coverage the frame can't reach, once validated against published discrete choice experiments in the same therapeutic area. - The current validation record for that approach is a mean rank correlation of 0.73 across 43 published randomized studies passing design filters, with a best configuration reaching 87% of a measured human ceiling (0.832 against 0.959) on one study: a validation result, and a new indication still has to clear that same bar on its own. ## The arithmetic problem in rare and narrow-specialty research A discrete choice experiment isn't sized by intuition. Its sample size follows from the number of choice tasks, attributes, and attribute levels in the design, and health economics has a stated rule of thumb (Johnson and Orme) for the minimum respondent count that design implies. When de Bekker-Grob, Donkers, Jonker, and Stolk reviewed published health discrete choice experiments against that rule, they found 32% used fewer than 100 respondents, 41% used 100 to 300, and 71% did not clearly report how they determined sample size (The Patient, 2015). That's the published literature before anyone tries to study a rare indication. In a specialty with a few hundred prescribing physicians nationally, or a disease with a diagnosed population in the low thousands, the recruitable frame sits below the design requirement before response rates are even considered. Response rates compound the problem. Physician surveys, including web-based ones, draw response rates that vary by specialty and by contact channel, and the physicians who do respond are not a random draw from the frame (Cunningham, Quan, Hemmelgarn, and colleagues, BMC Medical Research Methodology 15:32, 2015). Recruiting further through key opinion leaders and advocacy groups adds another layer of self-selection on top of that. ## Why do qualitative interviews fail as a substitute for a discrete choice experiment? Qualitative interviews fail as a substitute because they cannot produce an effect with a confidence interval, and a positioning or service-model decision needs exactly that. Eight or twelve interviews can surface themes, language, and objections. They cannot tell a commercial lead how much one attribute moves choice relative to another, or how tight that estimate is. A simulated arm can supply that effect and interval. The interval describes the effect within the simulated population, conditioned on the clinical profile; it doesn't extend automatically to the real, unmeasured population. When a qualitative deck is the only artifact behind a launch decision worth tens of millions, the decision rests on anecdote, not an estimate. ## What does the published evidence say about how DCE sample size is set? The published evidence says most health discrete choice experiments are already undersized or unjustified relative to the field's own rule of thumb, before anyone applies that design to a rare population. The three figures below come from the same review and describe different things: how many respondents a study reached, and whether it explained how it chose that number. Read them separately, not as slices of one pie. [figure: Bar chart showing that 32 percent of published health discrete choice experiments used fewer than 100 respondents, 41 percent used 100 to 300, and 71 percent did not clearly report how they determined sample size, per de Bekker-Grob et al. 2015.] ## How does a simulated arm change what's recruitable? A simulated arm replaces the constraint of who will answer a survey with a population built from the clinical and demographic profile the study needs, run through the same randomized choice design a fieldable study would use. The randomization does the causal work: a discrete choice model like McFadden's, a Mixed Logit, or an ICLV specification is an estimator applied after the fact, analyzing the randomized manipulation of attributes and levels inside the choice task. When the decision depends on substitution patterns as much as on which option wins - whether prescribers who reject option A shift to option B or to no treatment - a flat multinomial logit's independence-of-irrelevant-alternatives assumption can distort the answer; Mixed Logit and ICLV relax that assumption when substitution among a small set of options is part of the question. ## How do you validate a simulated arm before trusting it for a launch decision? You validate a simulated arm by replicating published randomized discrete choice experiments with it and reporting the correlation against their measured effects, including the misses. Across the 43 published randomized studies that pass design filters, the mean rank correlation against the original published result is 0.73. The best configuration reaches 87% of a measured human-to-human ceiling on one study: 0.832 rank correlation against the published result on the Hainmueller immigration conjoint, where two independent samples of real humans reach 0.959 with each other (Causal Fidelity paper). Across a wider set of roughly 300 replications, many built from studies with weaker designs or thinner reporting, the mean drops to 0.55 (Causal Fidelity paper). These are validation results on past studies. A new, untested indication has to clear that same bar on its own record before the simulated arm earns weight in the decision. One caveat this protocol has to carry rather than hide: published studies used for replication can sit inside a model's training data, so a strong correlation on a study that was public before training proves less than the same correlation on a study published afterward. The replication set and the current numbers by therapeutic area are public on the leaderboard; check whether a study close to your indication is in the filtered set before weighting the simulated arm heavily in a decision. ## What the small recruited sample does that the simulation can't ## Where willingness-to-pay and preference-share questions need extra care Willingness-to-pay questions need extra care because stated WTP from a discrete choice experiment runs high relative to what an incentive-aligned design would produce, unless the design ties choices to a real consequence. That bias runs in one direction, toward overstatement, so a positioning decision built on stated WTP from a rare-population study should treat the number as an upper bound on willingness to pay. Preference-share questions carry the IIA caveat noted above: a flat logit spreads share to a new or removed option in a fixed proportion, which rarely matches how prescribers or patients actually substitute in a narrow specialty with only a handful of real alternatives. ## What to do with this before the next planning cycle Start by checking whether a published randomized discrete choice experiment already exists in your therapeutic area on the leaderboard, since that's the study a simulated arm would need to replicate before it earns any weight in your decision. Use whatever recruitable sample you have, even eight or twelve prescribers or patients, to pressure-test the instrument's attributes and wording rather than to produce a topline number. Read the methods and validation coverage on how replication protocols are scored before deciding how much confidence a given correlation deserves. If you want to talk through how this applies to your specific indication or specialty, the team is a reasonable next stop once you've done the above. --- # Source of Volume Analysis: Category Growth vs Brand Switching URL: https://subconscious.ai/blog/industries/source-of-volume-analysis-category-growth-vs-brand-switching Published: 2026-08-30 Two respected decompositions of the same promotion data disagree by more than forty points on how much of the volume came from switching. An innovation or category lead facing a retailer listing decision - and a CFO asking whether the new item will eat the existing line - needs a pre-launch answer to that split. The answer comes from randomizing the shelf set the new item competes on, including the brand's own existing line, and reading the substitution shares directly from choices made under that assignment. - Source-of-volume splits into category growth, competitive switching, and self-cannibalization, and the split depends on which measure you use, not just which data you have. - The same promotion dataset gives roughly 74% switching in elasticity terms and roughly 33% in unit-sales terms, a documented forty-point gap (van Heerde, Gupta, and Wittink, 2003). - Panel gain-and-loss reports arrive after the listing is funded, which limits them to auditing a decision already made. - A flat choice specification forces proportional substitution and understates the loss to the closest sibling item, the one that matters most for a line extension. - Randomizing the shelf set to include the new item's own product line lets category expansion, competitive switching, and cannibalization fall out of the choice shares by arm, each with a confidence interval, before the money moves. ## Why do two respected decompositions disagree on the same promotion data? They disagree because source of volume is a substitution quantity, and substitution changes with the measure you choose to compute it. van Heerde, Gupta, and Wittink re-examined the standard elasticity-based decomposition used across the promotion literature and found it attributes about 74% of a sales bump to brand switching. Decomposing the identical data in unit-sales terms instead, they found only about 33% of the volume gain traced to losses by other brands in the category (van Heerde, Gupta, and Wittink, JMR 40(4), 2003). The two figures come from the same promotion and the same market; only the measurement convention changes, and the gap between them runs more than forty points. [figure: Bar chart comparing two decompositions of the same promotion dataset: elasticity-based switching estimate at 74%, unit-sales-based switching estimate at 33%.] ## Why are the retailer and the CFO asking the same underlying question? The retailer funds the listing if the item grows the category rather than reshuffling share within it. The CFO funds it if the incremental margin survives whatever the item takes from the brand's own line. Both questions resolve to the same substitution structure: where does each unit of volume come from, and how much of it was already the brand's before launch. A category lead who answers the retailer's question with intent-to-buy data and the CFO's question with a separate cannibalization study is running two analyses on one underlying causal quantity, and the two answers can contradict each other if they are not built from the same design. ## Why does a flat choice model get the cannibalization number wrong? A flat choice specification imposes proportional substitution: when a new item enters, it draws share from every existing option in proportion to that option's size, a consequence of the independence-of-irrelevant-alternatives (IIA) assumption baked into a simple multinomial logit. That assumption is convenient and wrong for a line extension. The sibling item on the same shelf is the closest substitute, and proportional draws understate exactly the loss the CFO is asking about. This is the proportional-substitution failure covered in the methods and validation library. The fix is a model specification, such as Mixed Logit or ICLV, that allows substitution patterns to differ by alternative instead of assuming they are uniform. ## How do you decompose source of volume before the listing is funded? You randomize the shelf set the new item competes on, run the choice task, and read the substitution shares directly from the assignment rather than inferring them from history. The design has to include the brand's own existing line as one of the alternatives on the shelf, not just competitor items, because that is the only way self-cannibalization shows up as a choice-share number instead of an assumption. Randomized experiments analyzed with discrete choice models produce the estimate. The method is McFadden discrete choice, Mixed Logit, or ICLV, chosen by how much substitution flexibility and latent preference structure the category needs. The choice shares by arm split into three parts: category expansion, the new buyers entering the category; competitive switching, the share taken from other brands; and self-cannibalization, the share taken from the brand's own line. Each part carries a confidence interval. That interval covers the effect within the simulated population under the tested design; it does not bound the real market outcome unconditionally, and a launch decision should treat it as a pre-launch estimate, not a guarantee. ## How much can you trust a pre-launch causal read? Our best configuration reaches a 0.832 rank correlation against the published human result on the Hainmueller immigration conjoint, against a measured human-to-human ceiling of 0.959 from two independent samples of real humans - 87% of that ceiling, stated with its denominator (Causal Fidelity paper). Across all 43 published randomized studies that pass design filters, the mean rank correlation is 0.73. These are validation results against studies that were already public, so some may sit inside a model's training data; the replication protocol is built to surface that risk rather than hide it. The Hainmueller study is an immigration conjoint, a political-preference task far from a retail shelf set, so its score validates the causal-inference machinery rather than performance on a category-growth-versus-switching decision specifically; a new category is a new test. The current standing of every published comparison is on the leaderboard, which is the place to check before treating any single number as settled. ## What should the category lead bring to the retailer meeting? Bring choice shares by arm. The retailer wants to see the category-expansion share; the CFO wants the cannibalization share against the existing line; both numbers come from the same randomized design and the same confidence interval, so there is nothing to reconcile between the two conversations. If a prior source-of-volume estimate on the table came from a flat choice model or a post-launch panel read, ask which measurement convention produced it before deciding what it means, because the gap between 74% and 33% on identical promotion data shows that the measurement convention can be doing most of the work (van Heerde, Gupta, and Wittink, 2003). The concrete next step: before the next listing pitch, write down which of the three source-of-volume buckets - category growth, competitive switching, self-cannibalization - the current forecast actually measures, and which it assumes. If any bucket is an assumption rather than a measured share, that is the gap a randomized shelf-set experiment closes. For a walkthrough of how that design gets set up for a specific launch, meet the team. --- # How to Run a Shelf Simulation Before a Retailer Meeting URL: https://subconscious.ai/blog/industries/how-to-run-a-shelf-simulation-before-a-retailer-meeting Published: 2026-08-30 A category or shopper insights lead with a line review in four to eight weeks needs a planogram recommendation backed by a number the retailer will act on: the effect of a specific shelf change on category dollars, stated with a confidence interval. Run a shelf simulation before that meeting when the shelf itself is randomized across simulated respondents inside one design. A realistic-looking virtual aisle answers whether shoppers behave plausibly in a simulated store, and the published validation work on virtual shelves already answers that question reasonably well (Waterlander, Jiang, Steenhuis and Ni Mhurchu, Journal of Medical Internet Research 17(4):e107, 2015, https://pmc.ncbi.nlm.nih.gov/articles/PMC4429224/). - Run the simulation when facings, adjacency, block position, price, and pack vary by assignment across respondents in one design built to cover the space of planograms. - Virtual shelf research validates well on category-level spend patterns; most retailer decks fail on design coverage. - A preference ranking across a handful of cells cannot report the effect of one change, with an interval, on category dollars. - Published virtual-versus-real comparisons show category-level agreement holds but is imperfect, so treat any absolute purchase-intent number with caution and lead with the relative effect (Waterlander et al. 2015). - Bring the retailer's outcome, category dollars, and name what the design covered. ## What decision does a shelf simulation need to answer before a retailer meeting? The decision is narrow: which items get facings and where the block sits. The simulation also has to state what that planogram change does to category value if the retailer accepts it. The retailer is asking an effect question. A simulation that outputs "shoppers preferred cell B over cell A and C" has not answered it, because it cannot say which element of cell B, held everywhere else constant, produced the gain. The retailer in the room is deciding on one specific shelf change and wants its size, with a stated confidence interval. ## What virtual shelf research already validates Virtual shelf research validates reasonably well on the question it can answer. Waterlander, Jiang, Steenhuis, and Ni Mhurchu ran 60 participants through three virtual shopping trips and matched the results against real till receipts (Journal of Medical Internet Research, 17(4):e107, 2015, https://pmc.ncbi.nlm.nih.gov/articles/PMC4429224/). The same four categories carried the highest spend in both settings: fresh produce at 14.3% of spend virtual versus 17.4% real, dairy at 19.1% versus 12.6%, meat and fish at 16.5% versus 16.8%, bread and bakery at 10.0% versus 8.2%. The internal order shifted - dairy led virtual spend, produce led real spend - and across those 60 participants and three trips, six of the study's categories showed statistically significant gaps between the two settings. A shelf simulation is a legitimate place to observe shopping behavior at the category level, with that gap in mind. ## Where a realistic shelf still misleads the room The problem shows up in what the fielding constraint forces you to test. A shopper panel and a hand-built virtual store cost real recruit time per cell, so most projects cap out at three or four configurations, and those three or four are usually the layouts a category team already favored going in. The simulation then reports which of those specific layouts shoppers liked best. It cannot report what happens if you add one facing to the number two item, because that configuration was never built. Waterlander's own comparison found significant gaps in six categories against real receipts (Waterlander et al. 2015), so a raw purchase-intent number pulled from that kind of study deserves the same caution: treat it as directional. ## How does randomizing the shelf change what you can claim? Randomizing facings, adjacency, block position, price, and pack across simulated respondents turns each variable into something estimable on its own, separate from the hand-picked cell it would otherwise sit inside. Because an added configuration does not require a new recruit, the design can cover the range of planograms worth knowing about. The output changes shape: the result names the changed attribute directly, for example adding one facing to an item, and reports the estimated shift in category dollars with a stated interval, attributable to that change because assignment put a shopper in front of it. The analysis method matters here. Randomized experiments analyzed with discrete choice models such as McFadden's model, Mixed Logit, and ICLV recover the effect because the shelf attributes were randomized in the design. A standard multinomial logit carries the independence of irrelevant alternatives assumption, which can misstate substitution between two similar items on the shelf; Mixed Logit relaxes that by letting preferences vary across simulated respondents, and ICLV adds a layer for latent constructs like health or convenience motivation behind the choice. A confidence interval from this design covers the effect within the simulated population tested; it does not extend automatically to the retailer's shopper base. Published human studies used to validate these models can sit inside a model's training data, which is why replication protocols test against newer, held-out studies rather than treating one match as proof. ## Conventional cell testing versus a randomized shelf design ## How much of the measured human ceiling does this reach? On one published benchmark study, our best configuration reaches 87% of the measured human ceiling: 0.832 rank correlation against the published human result, where two independent samples of real humans reach 0.959 against each other (causal fidelity paper). That is a best-case number on a single study, not a guarantee for a new retailer's category. Across all 43 published randomized studies that pass the paper's design filters, the mean rank correlation is 0.73. Bringing a category number into a retailer meeting means bringing it with that context and its denominator attached. [figure: Bar chart showing human-to-human rank correlation ceiling at 0.959, best single-study configuration at 0.832, and mean across 43 published studies at 0.73, all sourced to the causal fidelity paper.] Full study-by-study results sit on the public leaderboard, which is the place to check where a given method and market stand before treating any single number as settled. ## What the interval covers, and what it doesn't A confidence interval from this kind of design covers the estimated effect within the simulated population tested, given the randomized shelf attributes included in the design. It does not bound what happens in the retailer's actual store network, and it does not cover attribute combinations the design left out. If the design randomized facings, adjacency, block position, price, and pack, say so, and say which of those five the design held fixed, since a fixed attribute cannot produce an estimated effect. This is also where the IIA assumption matters: if the report leans on a flat multinomial logit for a preference-share or substitution claim between two similar items, name that assumption in the same sentence as the claim, because it is what shapes the substitution pattern the retailer will ask about directly. ## What to bring into the line review Bring the effect of each planogram change on category dollars, the interval around it, and a short statement of what the design covered. Skip a bare preference ranking across a small set of cells, and flag any purchase-intent or spend number pulled from a fixed-cell study as directional rather than exact. Methods pages on randomized experiments and validation walk through how the discrete choice models behind these estimates work, and the public leaderboard is the fastest way to check a method's validation record against a specific study before repeating its number in a room. Before the review: list every attribute the planogram recommendation depends on (facings, adjacency, block position, price, pack), check which ones the simulation varied by assignment, and cut any claim resting on an attribute the design held fixed. For a second look at the design before it runs, book time with the team.