Which Push Notification Copy to Ship, Before the Live Send
A lifecycle or CRM marketing manager who owns push copy has one recurring decision: which variant goes to a defined cohort before the live send, and whether that cohort needs its own variant at all. Push is the highest-leverage channel most teams own and the one with the least copy review before it ships. A notification that reads as vague, pushy, or irrelevant does not just lose the tap. It costs the unsubscribe, and a user who turns off notifications is gone from that channel permanently. A wrong cohort-level call also burns send volume across the whole list on a variant that was never going to win.
Why guessing or sequential testing is not enough
Most teams resolve the copy decision one of three ways.
| Approach | What it optimizes for | What it misses |
|---|---|---|
| Ship the first acceptable variant | Speed | Never surfaces whether a stronger variant existed; no signal on unsubscribe risk before send |
| Sequential live A/B testing | Real, on-channel performance | Burns real send volume and calendar time testing weak variants one pair at a time |
| Run a controlled experiment before the live send | A causal read on which variant changes stated intent to open, before real users see it | Does not replace the live confirmation test on the real channel |
The first two approaches only tell you what happened after you already sent something. A controlled experiment answers the question before the send: which copy change moves a defined cohort's stated intent to open or engage, with a confidence interval around that effect.
What a push notification test actually needs to isolate
Push is short, fast, and emotional. A user sees it on a lockscreen, decides in under 2 seconds, and taps, swipes away, or turns off notifications entirely. That last option is the one most testing setups ignore. A copy test that only scores clarity, relevance, and curiosity will optimize for tap rate while missing the irritation signal that drives the unsubscribe. Subconscious treats irritation as a measured outcome alongside intent to open, because the two behaviors trade off against each other.
The workflow
- Define the send context. State the trigger event, the time of day the push fires, and what success looks like: opening the app, landing on a deep link, or finishing a flow already in progress. "User abandoned checkout six hours ago, evening send" is a different test than "user has not opened the app in two weeks, morning send, drive reactivation."
- Draft a spread of variants, not near-duplicates. Mix archetypes: direct utility, curiosity gap, social proof, urgency, personalized address, a question, and plain status. Two near-identical variants only tell you which guess you prefer.
- Define the cohort. Build the test audience around the real recipients of this send: active users, dormant users, a specific tenure band, or a job-title and team-size cut for B2B. The closer the test cohort matches the real send list, the more the result transfers.
- Run the controlled experiment. Present each variant with its send context to the defined cohort and measure stated intent to tap, intent to ignore, and intent to turn off notifications from the app, with reasoning attached to each response.
- Promote the top variants to a live confirmation test. Take the leading variants into your normal on-channel A/B test. As a planning example, teams commonly run this confirmation window over roughly 48 hours before committing to the full list. The controlled experiment tells you which variant should win; the live test still produces the open and tap-through number your forecast needs (Statsig).
When live testing still matters on its own
A pre-send experiment is not a substitute for live testing on high-stakes sends: the first push to a new user segment, a change in send volume, or an entirely new notification category the app has never sent before. For routine, established lifecycle sends, the experiment result is often enough to ship directly.
Proof and limitations
Subconscious runs push copy comparisons as controlled experiments: each variant is a treatment, the outcome is stated intent to open or engage against a defined cohort, and the result carries a confidence interval rather than a single point estimate. When a send is high-stakes or entering a new segment, the team can move from that simulated experiment to real-human validation without changing the underlying causal question: same treatment, same outcome, a real-participant sample instead of a simulated one.
What to do with the next send
Pick the next push queued to go out. Draft a spread of variants across the archetypes above, define the cohort that will actually receive it, and run the comparison before the send rather than after. See how the workflow runs end to end, review case studies from other teams testing before they ship, or book a walkthrough on your own send calendar.