Methods and validation

These articles explain discrete choice experiments, Mixed Logit, ICLV and checks against human data. They distinguish reported research from proposed validation, state the limits of a result and show how to question vendor claims, including ours.

• Why Observational Studies Cannot Prove Cause

  Observational data shows that two things move together. It cannot, on its own, show that one moves the other.

• Why a Favorite Still Loses Most of the Time: A Bracket Forecast Under Uncertainty

  A single-elimination bracket does not ask a team to be good once. It asks a team to be good repeatedly, against opponents decided by other matches it does not control.

• Bayesian Modeling for Marketing Attribution: Interpreting Uncertainty

  Missing tracking data can make channel attribution harder to trust.

• Pricing Research Methods: Choosing the Right Method Before You Field

  A pricing or product leader facing a launch, a tier change, or a repricing decision must choose a research method before fielding: willingness-to-pay, price sensitivity, conjoint…

• Do LLMs Understand Real-World Prices? A Pricing Benchmark for Synthetic Consumers

  An LLM-based synthetic panel can produce fluent, plausible-sounding survey answers without ever grounding those answers in real prices.

• Lock the Target Group and Stimulus Before Writing Survey Questions

  Before fieldwork, define the target population and measurement objective, pretest the questions and any stimulus, then lock the final instrument and analysis plan.

• Bayesian Spatial Modeling for Evaluating Hockey Goaltending Performance

  A goalie's save percentage answers a narrow question: what share of shots did they stop?

• When a Synthetic-Customer Read Is Enough, and When You Need a Causal Experiment

  A product or research leader has to decide, before engineering capacity or launch spend is committed, whether a synthetic-customer read is sufficient evidence for a feature…

• When a Fixed Forecasting Rule Should Become a Distribution

  A fixed shrinkage rule can be a useful forecasting baseline. A hierarchical alternative estimates pooling parameters and represents uncertainty, but it still needs to outperform…

• Aaru and EY: What a 0.90 Median Spearman Correlation Covers

  A synthetic-research vendor publishes a correlation against a Big Four partner's survey, and the number circulates as proof the category works.

• When Is a Synthetic Consumer Response Ready to Inform a Real Decision?

  A synthetic consumer response may support exploratory hypotheses when the decision is reversible and the limits are explicit.

• Simulating Data with PyMC

  This walkthrough draws on Ricardo Vieira and Tomás Capretto’s PyMC Labs simulation tutorial.

• Synthetic Consumer Research: 4 Steps to Check Before You Trust the Result

  A synthetic-consumer result is trustworthy only as far as its weakest step.

• MaxDiff vs. Conjoint vs. NPS: Which Instrument Matches the Decision

  An insights lead scoping a study has three common instruments to choose from, each answering a different question.

• A Research Operating Model for Matching Evidence to Decision Risk

  A research leader should route each question by the cost of being wrong. AI-assisted exploration can surface hypotheses.

• Prediction and Intervention Answer Different Business Questions

  A pricing team may forecast next month’s sales accurately and still lack an estimate of what changing the price would do.

• Between Brand Tracking Waves: How to Test a Brand Action Before the Next Report

  A brand or marketing leader who commissions a quarterly tracking wave knows its worst failure mode: a competitor moves, a message lands badly, or a public event shifts perception…

• Why a Bayesian Marketing Mix Model Still Needs Calibration Before You Reallocate Budget

  A Bayesian Marketing Mix Model (MMM) can tell you which channel looks most effective.

• Bayesian A/B Testing at Scale: Why Millions of Observations Slow MCMC Down

  A head of experimentation with a large A/B dataset needs posterior estimates that finish in time to inform the decision.

• AI Causal Graphs: 4 Checks Before You Act on One

  An AI model can read a short list of variable names, say TV spend, brand awareness, website visits, sales, and return a full cause-and-effect graph without seeing a single row of…

• Why a 20% Retention Rate Means Different Things for a 10-User Cohort and a Million-User Cohort

  A marketing analytics lead sees 20 percent retention. Two active users out of ten and 200,000 out of a million have the same observed ratio, but different sampling uncertainty…

• What Is a Simulated Buyer, and When Should You Trust One?

  A simulated buyer is a modeled stand-in that attempts to approximate responses from a defined audience.

• Likelihood Approximations Through Neural Networks: A Validation Checklist Before You Trust the Output

  A behavioral model that reports a choice probability or a causal effect is only as trustworthy as the likelihood behind it.

• Are My New Product's Sales Incremental or Cannibalistic?

  A new product launch into a saturated category can grow share, or it can just move units the company already had.

• Can You Trust a Vendor's Uncertainty Intervals? A Test Case in Honest Uncertainty Modeling

  An uncertainty interval that never changes shape, never widens near a data gap, and never reacts to a structural break in the underlying process deserves a second look.

• Marketing Mix Modeling: A Complete Guide

  Marketing mix modeling relates historical outcomes to transformed media inputs and other drivers. Adstock represents carryover; saturation represents diminishing returns.

• Bayesian Marketing Measurement When Individual Tracking Weakens

  When individual-level attribution loses coverage, marketing teams should not treat the remaining tracked journeys as the whole market.

• Pareto/NBD: Finding Silent Churn Before It Shows Up in Revenue

  A customer who buys on demand, not on a contract, never clicks "cancel." They just stop.

• The Model Was Right. The Decision Came First Anyway.

  Consider an illustrative planning scenario: an analyst receives budget what-ifs on Monday, the executive decision is due Thursday, and analysis takes a week.

• Predicting Swinging Strikes with Bayesian Additive Regression Trees

  Bayesian additive regression trees predict swinging strikes by summing many shallow trees fit through MCMC sampling over pitch-tracking features like velocity and spin rate…

• Feature Prioritization: Frameworks, Tools, and Evidence

  Feature prioritization helps a product manager decide which features deserve the next release by comparing their expected value and delivery effort.

• Bayesian Media Mix Modeling for Marketing Optimization

  A marketing analytics team allocating budget across TV, paid social, and direct mail usually starts from last-touch attribution or a spend-to-revenue heuristic.

• Can an LLM Stand In for a Human Survey Respondent? What One Benchmark Found

  In one GSS benchmark, several demographic-conditioned LLMs approached the random forest’s point error on political party identification.

• Can LLM-Generated Open-Ended Survey Responses Match Real Issue Distributions?

  A consumer insights leader deciding whether to trust LLM-generated open-ended answers before fielding qualitative research needs one question answered: do those answers reflect…

• How Partial Pooling Supports Decisions from Sparse Survey Data

  A sparse survey can support a segment decision when the model shares information across related groups and the result carries its uncertainty.

• What a World Cup Forecasting Model Teaches About Trusting a Model at All

  A model can hit its headline accuracy target and still be wrong on the exact numbers a decision depends on.

• Which Causal Method Fits Your Data When a Randomized Trial Isn't Possible

  A marketing or analytics leader wants to know whether a pricing change, a campaign, or a launch actually caused a shift in customer behavior.

• How to Pressure-Test a PRD Before the Engineering Kickoff

  A PRD flaw can consume engineering time before the team learns that the feature does not solve the target buyer’s problem.

• Should You Trust a Raw Tracking-Poll Average? A Hierarchical Model Answers That

  A tracking average can mix trend movement with source changes and noise.

• Survey Response Rates Are Falling. What Replaces the Survey for a Decision That Needs a Causal Answer?

  Declining response prompts a research leader to check whether the survey still measures its intended population and outcome.

• From Uncertainty to Insight: What Bayesian Reasoning Means for a Business Decision

  A marketing or analytics leader deciding how much stock to buy, how to price a product, or when to worry about churn usually has a forecast in hand.

• An Automated MMM Says Shift Budget. Should You Act on It?

  A marketing mix model (MMM) fits historical spend and outcome data to estimate each channel's contribution.

• When Is a Simulated Behavior Model Trustworthy Enough to Act On?

  A modeled behavioral effect becomes useful when relevant external evidence supports the estimate and its uncertainty, and the remaining risk is acceptable for the decision.

• Seven Survey Biases That Distort Market Research Numbers

  A consumer insights lead is about to greenlight a launch, a price, or a message on a survey number: does it describe what the market will actually do, or how the market wanted to…

• Is Your Marketing Model's Answer Data-Driven, or Just Your Priors Talking Back?

  A statistical model hands a marketing team a channel-attribution number, and the team has to decide: act on it now, or check it first.

• Quantify the Uncertainty Before You Pick a Risk Policy

  A planner committing capacity against a point forecast needs an explicit policy for forecast errors. A single expected value does not describe shortage probability or severity.

• Why Causal Effects Come With a Spread, Not a Single Number

  A vendor hands you a causal effect with a confidence interval attached.

• When a Flat Choice Model Misrepresents Cannibalization

  A consumer-goods pricing or revenue-growth leader planning a new product launch needs to know one thing before committing trade spend: will this product mostly take share from…

• LLM-Simulated Panels: When a Persona-Conditioned Survey Result Is Enough to Act On

  A persona-conditioned survey can support early theme exploration when a matched human reference shows useful agreement.

• What a Validation Gate Is, and Why It Should Decide What an AI Agent Ships

  An AI agent's output can look finished before its correctness has been checked.

• Sample Size Calculator and Guide to Survey Sample Size

  This guide links sample-size calculators and explains when their proportion formulas apply.

• Willingness to Pay: What It Is and How to Measure It

  Willingness to pay (WTP) is a reservation price: the maximum a buyer would pay for a specified quantity in a specified context.

• When Partial Pooling Helps a Thin Segment Estimate

  A segment-level causal estimate can look strong for a reason that has nothing to do with the segment.

• Why Synthetic Consumer Ratings Need Better Elicitation

  Synthetic consumers are easy to generate and hard to trust. Ask a language model for a score on a 1-5 purchase-intent scale and it may overuse the middle, avoid extreme answers…

• When an AI-Accelerated Marketing Mix Model Is Trustworthy

  AI can assist with configuring and diagnosing a marketing mix model. The PyMC Labs MMM-agent case describes that workflow.

• Why a Confounder Can Make a Marketing Channel Look Effective When It Isn't

  A marketing team sees sales rise whenever Google Ads run. Coordinated TV spending may explain that pattern.

• Why a Single-Number Forecast Hides the Decision You're Actually Making

  A point forecast can be useful, but an inventory decision also needs uncertainty and the cost of overstocking or missing demand.

• When a Marketing Mix Model Recommends a Budget Shift, Test the Claim Before You Move the Money

  A production Bayesian marketing mix model (MMM) can trace spend through a real funnel: upper-funnel spend shapes lower-funnel demand, demand runs into budget caps, caps shape…

• How Research Teams Check AI-Assisted Findings

  A research leader using AI to draft a survey or summarize responses still has to decide what the business question needs as evidence.

• Why a Media Mix Model Number Needs a Controlled Test Before It Moves Budget

  In an invented example, an MMM attributes 18 percent of last quarter's revenue to paid social.

• Why Per-Test Bayesian Model Loops Stop Scaling

  Many modest Bayesian A/B, ABC and ABCD datasets can spend substantial time in repeated compilation, warmup and adaptation.

• When a Fixed CAC Assumption Misleads a Budget Reallocation

  The decision this affects

• When a Forecasted-Control MMM Can (and Can't) Answer a Budget Question

  A forecasted-control MMM combines a fitted sales model with predictions of future external conditions.

• Survey scripting best practices in market research

  A research director needs both a valid study design and a correctly programmed instrument.

• Should I include adcepts in preference share simulations?

  Include creative in a preference-share study only when its exposure is part of an identified design.

• Understanding the margin of error in simulations

  A VP of consumer insights evaluating a synthetic pricing panel needs to separate uncertainty in the estimate from error in representing real buyers.

• What Should Transfer to a New Context? A PyMC Walkthrough for Evaluating Causal Simulation Predictions

  A causal simulation predicts a segment, product, or time period it never observed directly.

• Simulated Marketing Panels: What They Test Well, and Where Real Buyers Still Decide

  Marketing teams increasingly run early positioning, pricing, and messaging questions through a simulated panel before committing production or media budget.

• What a Rigorous Causal-Inference Pipeline Checks Before You Trust Its Output

  A causal-modeling vendor's output is only as trustworthy as the engineering discipline behind the pipeline that produced it.

• Synthetic Respondents: What They Are and When to Trust One

  A synthetic respondent is a model-generated stand-in for a participant in a survey or experiment.

• UX Survey Methods for Product Teams

  A product team about to ship a redesign usually reaches for a survey.

• What to Demand Before You Trust a Synthetic-Respondent Vendor's Accuracy Claim

  A vendor's self-reported accuracy number is a marketing claim until you can reproduce it.

• What is TURF Analysis and When to Use It?

  TURF, Total Unduplicated Reach and Frequency, finds candidate subsets with the greatest unique reach under a defined reach rule and budget or size constraint.

• What Are Synthetic Consumers? Knowing When to Trust the Answer

  Synthetic consumers are modeled personas that attempt to approximate a consumer audience's responses.

• Validating a Multi-Step Onboarding Flow Before Engineering Builds It

  A Head of Product can evaluate a redesigned sign-up-to-first-action sequence with storyboards and human prototypes before engineering builds it.

• Investigating an Unexplained Survey Result

  A four-week brand tracking wave lands on your desk and a key metric has moved. Stakeholders want an explanation by tomorrow.

• Models for calculating preference shares

  A preference-share forecast depends on both the estimated utilities and the rule converting utilities into choices.

• What reviewers should ask about synthetic market research data

  A reviewer needs a matched calibration study and the information needed to judge its scope: population, task, model version, prompt, exclusions, human reference, uncertainty, and…

• Mini-lecture: Conjointly's guide to sample selection

  A research director needs two checks: enough information for the intended estimate and a respondent pool appropriate to the population.

• How to perform smart sampling and data checking?

  A sampling protocol needs separate evidence for precision, respondent quality, identification, and transport.

• Mini-lecture: Conjointly's guide to inflation

  Conjointly’s inflation mini-lecture, dated December 13, 2021, discusses pricing research under rising costs.

• New data quality safeguards against fraudulent survey responses

  A research director needs to know what a fraud screen detects and what it misses.

• The ROI of quality checks: never run consumer panels without them

  Quality checks can reduce contamination and decision risk when their performance is measured on the intended study.

• A Regime-Aware Risk Model: What to Validate Before You Trust the Probability

  A single distribution can obscure changing return behavior. A regime-aware model is one possible representation, with probabilities conditional on its assumed states and…

• Getting the top and bottom levels from conjoints

  To identify top and bottom conjoint levels, rank their estimated part-worth utilities within the same attribute and report uncertainty for the contrast.

• How to compare simulated and human experimental results

  Compare simulated and human experiments only when the audience definition, stimuli, assignment and measured outcome are aligned.

• A Latent Timing Segment Is a Hypothesis, Not a Reason to Move Budget

  A model that groups customer activity by day and hour can hand you a clean story: "this segment is active Tuesday mornings." That pattern is already in the data.

• How To Interpret Marginal Willingness To Pay

  Marginal willingness to pay is a modeled tradeoff between an attribute change and money. A pricing lead should inspect its definition, uncertainty and population before using it.

• Hierarchical Bayesian Models for Customer Lifetime Value Across Cohorts

  A marketing analytics team comparing customer cohorts must decide whether to fit separate transaction models, one global model or a hierarchy that partially shares information.

• How Key Driver Analysis identifies what matters most

  A CX or insights leader looking at a Key Driver Analysis report has to decide whether to spend next quarter's roadmap budget on whatever attribute sits at the top of the ranking.

• What is implicit testing? (with examples)

  Implicit testing uses timed association tasks to estimate the strength of relative associations under a specific protocol.

• Hierarchical Bayesian Latent-Trait Estimation, Explained

  A number published without its limits reads as marketing. Trusting a model's output for a pricing or positioning decision means trusting how it handles uncertainty.

• Incrementality Testing: What It Proves, What It Cannot, and How to Run It Causally

  Incrementality testing estimates how a tested intervention changed an outcome relative to a control or credible counterfactual.

• How to manually calculate partworth utilities

  Buyers evaluating a conjoint vendor's output have one real decision to make: whether to trust the part-worth utilities enough to act on them, not whether the underlying regression…

• How to Tell If a Causal Vendor's Probability Model Is Auditable

  A data science leader vetting a causal behavioral platform for a paid pilot needs one question answered before budget moves: can this vendor show the exact path from its modeling…

• A Faster MMM Pipeline Doesn't Answer Whether the Numbers Are Causal

  A marketing mix modeling (MMM) team needs aligned spend and outcome data before it can fit a model.

• How to Develop Effective Likert Scale Questions?

  An effective Likert item states one idea, names a relevant context and provides interpretable response anchors.

• How to get quick feedback for survey testing?

  To get quick feedback before fielding a survey, first ask a few target respondents to explain each question in their own words, then watch them complete the programmed survey.

• Kano Model Templates: Uses and Limits for Feature Decisions

  A Kano Model Excel template helps a product team classify reactions to described features.

• Compiling Code Is Not Validating a Model

  A generated PyMC model can compile while using the wrong likelihood, an unjustified prior or an unsupported causal interpretation.

• Funnel-Aware MMM: A Bayesian Architecture for Full-Funnel Marketing Optimization

  An additive, funnel-blind marketing mix model (MMM) can omit a path that matters for budget decisions: upper-funnel activity creates demand that lower-funnel channels later…

• A Donor Value Forecast Does Not Identify Which Fundraising Action Works

  Annika Denninger and Abdulhamed Chribati describe an independently developed donor-value MVP for an anonymous children's rights NGO in their personal-capacity PyMC Labs community…

• How to get the most out of video interviewing in surveys?

  Video interviewing can capture the language, context, and follow-up detail a research team needs. The format also changes the measurement setting.

• Creating a correlation matrix for conjoint simulations

  A conjoint correlation matrix is useful only after its rows, columns and scenario variations are defined. Design correlations diagnose dependencies in encoded attributes.

• Counterfactual Causal Inference in PyMC

  Prediction forecasts the outcome under current conditions. Causal inference instead estimates the outcome that a different action would have produced.

• Estimating a Private-Market Benchmark When There Is No Public Price

  A public equity index is built from transaction prices that happen constantly: every trade updates the number.

• Criticisms and counter-criticisms of Kano model

  Kano surveys categorize reactions to a feature's presence and absence.

• Van Westendorp Templates: Uses and Limits for Pricing Decisions

  A Van Westendorp Price Sensitivity Meter (PSM) template turns four self-reported price thresholds into cumulative curves and intersections.

• Gaussian Process Geospatial Modeling: Beyond Hierarchical Models

  A geospatial model should use a justified relationship between locations.

• What is Conjoint Analysis? (with examples)

  Conjoint analysis studies preferences for profiles built from varying attributes and levels.

• When Conversational Surveys Help, and What They Leave Unvalidated

  Conversational surveys can help researchers collect detailed explanations.

• Conversational surveys explained in 6 minutes

  A conversational survey collects answers through a chat or voice interview, with follow-up questions that may adapt to earlier responses.

• How to get the most out of open-ended questions

  Open-ended questions help an insights team discover language, experiences, and explanations that a fixed response list would miss.

• How to compare results across segments with crosstabs

  A crosstab compares counts or rates across categories. It can summarize survey answers, observed purchases, administrative records or outcomes from a randomized experiment.

• Best tips and tricks for drafting surveys for response quality

  A research leader needs both response-quality controls and a design suited to the inference being claimed.

• Classifying Types of Conjoint Analysis

  Choosing a conjoint format means choosing a respondent task. Choice-based conjoint (CBC) asks people to select a profile; adaptive choice-based conjoint (ACBC) tailors parts of…

• Causal AI: What It Is and What It Buys a Business Decision

  Three price points sit on a CPG pricing lead's desk before a shelf reset; none of last year's sales data covers any of them.

• Choosing Between MaxDiff, Conjoint, and a Controlled Experiment

  A research team choosing a study design before fieldwork has to answer one question: does the business decision need a preference ranking, an importance score, a satisfaction…

• Calculation of volume, revenue, and profit in simulations

  A pricing lead reviewing a conjoint or discrete choice simulator needs one answer before signing off on a forecast built from it.

• Claims Test Methodology

  Claims test methodology decides which candidate message gets media spend, and for a brand leader or general counsel, that decision carries two separate risks.

• How to analyse time series results

  A brand or insights lead watching a tracker move after a launch, price change, or repositioning needs one thing: proof that the decision caused the move, not just proof that the…

• Busting Market Research Automation Misconceptions

  The biggest misconception in market research automation is treating the choice between human panels and synthetic respondents as the main question.

• Agency vs. in-house marketing measurement: where causal action testing fits

  A marketing analytics or data-science leader deciding how to measure effectiveness usually frames the choice as agency versus in-house.

• Classification of quality issues in survey sample

  A pricing manager reviewing a survey before setting a launch price needs four separate checks: respondent integrity, representation, measurement and predictive validity, and…

• Availability adjustment for conjoint preference shares

  A pricing lead deciding whether to raise price off a conjoint simulator's forecast is really asking one question: does matching the market make the preference share trustworthy?

• 5 Best Practices of Online Survey Design

  An online-survey buyer needs a usable instrument, credible respondents, and evidence that the measured endpoint answers the decision.