Run cohort analysis for an intent program by starting with the decision and the counterfactual. Define who is eligible, what event assigns cohort entry, the analysis unit, the comparison group or baseline, the pre- and post-entry windows, the primary outcome, guardrails, exclusions, contamination rules, and the claim the design can support before looking at results.
A descriptive cohort can show what happened after accounts entered an intent audience. It cannot prove the program caused the change. Use a randomized holdout when feasible. If you use matched groups, a stable pre-period, or a counterfactual model, disclose the assumptions and downgrade the claim when comparability, volume, identity, or measurement is weak.
Intent, identity, and match signals are probabilistic inputs, not proof. A topic signal does not prove a purchase decision, a match does not guarantee person-level identity, and an attributed opportunity does not automatically establish incrementality. Consequential campaign, outreach, CRM, and budget actions need an accountable human reviewer.
Who is this for?
This framework is for CEOs, CFOs, CROs, VP Marketing, RevOps, analytics teams, and agencies deciding whether an intent program changes qualified pipeline, efficiency, velocity, or retention. It is useful when the team can define stable events, join exposure to outcomes, and maintain a credible comparison.
It is not decision-ready when cohort membership changes after seeing the result, exposure is unknown, CRM outcomes are inconsistent, groups are tiny or incomparable, or concurrent campaigns reach both groups. In those cases, report a descriptive pattern, improve the data, or design a stronger test rather than manufacturing certainty.
Nine steps for decision-grade cohort analysis
1. Write the business decision
State the action the analysis could change: continue the program, adjust topics, change activation, shift budget, narrow the ICP, or run a stronger experiment. Name the decision owner and the minimum evidence required.
Choose one primary question. “Does intent data work?” is not testable. “Does activating fresh, ICP-qualified account signals increase accepted opportunities relative to comparable eligible accounts not activated during the same window?” is much closer.
2. Define eligibility and the analysis unit
Specify the population before assigning cohorts. Include ICP rules, geography, company size, existing customer or opportunity state, data availability, minimum history, and exclusions. Choose account, person, opportunity, or campaign as the analysis unit and explain why.
Do not mix units in the denominator. If activation happens at the account level but outcomes are counted by contact, an account with many contacts can dominate the result.
3. Lock the cohort-entry event and index time
Define the event that assigns membership: first qualifying topic surge, first accepted site-visitor match, first activation, or another precise event. Record whether an entity can enter once or repeatedly and how ties, time zones, late-arriving events, and backfills are handled.
Google Analytics' cohort exploration uses an inclusion criterion, a return criterion, and a selected granularity. That structure is useful for thinking clearly about entry and later behavior, even if your program runs in a warehouse or CRM. See GA4's cohort-exploration documentation.
4. Choose the comparison or counterfactual
The comparison should represent what would likely have happened without the activation. A randomized eligible-account holdout is the strongest practical option when randomization, compliance, and volume are adequate.
If randomization is unavailable, consider a pre-period baseline, matched eligible accounts, geographic or team rollout, interrupted time series, or a model-based counterfactual. Record why the comparison is credible and what unobserved differences could remain.
5. Set observation and maturation windows
Define the lookback used for balance and baseline, the cohort-entry window, activation window, outcome window, and maturation delay. Account for the sales cycle without choosing a window after seeing which one produces a favorable result.
Specify how overlapping treatments, repeated signals, late CRM updates, reopened opportunities, and renewals are handled. Freeze these rules in a pre-analysis plan.
6. Choose one primary outcome and several guardrails
The primary outcome should match the decision: accepted opportunities, qualified pipeline, revenue when mature, cost per accepted opportunity, or time to a qualified stage. Define it using stable CRM fields and ownership.
Guardrails catch harmful tradeoffs. Track eligible reach, match rate, false or disputed matches, sales rejection, opt-outs, complaints, frequency, missing outcomes, and total cost. A program that creates more meetings but lower acceptance may not be an improvement.
7. Predefine exclusions and contamination rules
List customers, open opportunities, disqualified accounts, employees, test records, missing identifiers, concurrent campaigns, and accounts that cross from control to treatment. Decide whether each case remains, is excluded, or is analyzed separately.
Avoid post-treatment exclusions. Removing accounts because they did not engage after activation selects on the outcome and can make the treatment group look better.
8. Analyze uncertainty and sensitivity
Report cohort sizes, denominators, balance, missingness, effect estimates, absolute and relative differences, and uncertainty. A NIST explanation of confidence intervals is a useful reference: the interval expresses uncertainty around an estimated population parameter. It does not repair selection bias, contamination, identity error, or a missing counterfactual.
Run sensitivity checks across reasonable entry definitions, matching choices, outcome windows, and missing-data treatments. If the conclusion changes with every specification, that instability is part of the result.
9. Write the decision memo and retest plan
Separate facts, estimates, assumptions, and recommendations. State what the analysis supports, what it does not support, which action is approved, and which guardrails trigger rollback.
Keep the query or code version, data dictionary, cohort specification, results, reviewer comments, and approval together. Define the next test before operational teams change the program in ways that make replication impossible.
Build a cohort table the team can reproduce
At minimum, include:
- Stable analysis-unit ID and entity type.
- Eligibility state and the versioned rule that produced it.
- Cohort-entry event, source, index time, and repeated-entry rule.
- Intent topic, signal definition, specificity, recency, and expiry.
- Identity method, match confidence, conflicts, and correction state.
- Activation or exposure time, channel, campaign, message, and owner.
- Comparison assignment and crossover or contamination flag.
- Primary outcome, outcome time, CRM definition, and missingness state.
- Guardrails, exclusions, concurrent treatments, and data-quality flags.
- Transformation, query, model, and dataset version.
Separate account and person records. If several people map to one account, define deduplication and the outcome unit before analysis. Keep raw signals distinct from derived stage scores and activation decisions so the team can audit leakage and changes.
Five analysis methods and what each can claim
1. Descriptive cohort trend
Compare return behavior or outcomes for cohorts defined by entry period, topic, recency, or activation. This is fast and useful for operations.
Permitted interpretation: “This is what happened to these defined groups.”
Limitation: It does not isolate the program effect.
2. Matched observational cohort
Match activated and non-activated eligible accounts on pre-treatment fit, history, region, size, prior engagement, and other relevant observed variables. Check balance before comparing outcomes.
Permitted interpretation: “Comparable groups on measured variables had different outcomes, subject to residual confounding.”
Limitation: Unobserved differences can still drive the result.
3. Randomized holdout
Randomly assign eligible units to activation or holdout before treatment. Monitor assignment, compliance, crossover, sample size, and outcome collection.
Permitted interpretation: With a valid design and analysis, differences can support an incremental-effect estimate for the tested population and intervention.
Limitation: Small samples, noncompliance, spillover, or broken randomization can still invalidate the conclusion.
4. Interrupted time series or counterfactual model
Model what the outcome would have been after an intervention using stable pre-period behavior and appropriate control series. The Bayesian structural time-series research behind CausalImpact describes one such counterfactual approach.
Permitted interpretation: A model-based effect estimate under stated assumptions.
Limitation: Concurrent changes, unstable relationships, or a poor control series can produce a misleading counterfactual.
5. Attribution report
Allocate opportunity or revenue credit using a defined rule or model. Attribution is useful for operations, channel reporting, and reconciliation.
Permitted interpretation: “Under this attribution rule, these touches received this credit.”
Limitation: Attribution is not the same as incrementality. It does not show what would have happened without the program.
Variance-reduction techniques can improve experimental precision when appropriate pre-period covariates are available. Microsoft's CUPED and variance-reduction guidance explains the concept. Precision techniques do not create randomization or cure a biased cohort.
Which tools and artifacts are useful?
Choose tools after choosing the method:
- A spreadsheet or BI tool can support a small descriptive cohort with transparent formulas and careful QA.
- A warehouse and SQL or notebook workflow supports reproducible eligibility, joins, repeated analysis, and version control.
- A product or web analytics platform helps with event-based inclusion and return behavior when identity and event definitions fit the decision.
- An experiment platform manages assignment, exposure, guardrails, and repeated testing when randomization is available.
- A statistical environment supports matching, uncertainty, sensitivity, and counterfactual models under qualified analysis.
Regardless of tool, keep a cohort specification, data dictionary, pre-analysis plan, query or code version, balance report, outcome report, sensitivity log, decision memo, and approval record. The NIST/SEMATECH Engineering Statistics Handbook is a primary statistical reference for experimental design and analysis concepts, not an intent-program benchmark.
What does cohort analysis cost?
Budget for intent and identity data, warehouse or analytics tooling, experiment infrastructure, integrations, analyst or statistician time, CRM cleanup, quality assurance, privacy and security review, and decision-maker review. Include the opportunity cost of withholding activation from a holdout and the cost of delaying a decision while outcomes mature.
Separate one-time design and integration from recurring data, tooling, and reporting. Also model the cost of a wrong decision. A weak analysis that causes the team to scale ineffective outreach can cost more than a longer, well-designed test.
There is no universal sample-size, analyst-hour, or tool-cost benchmark. Use the expected outcome rate, minimum decision-relevant effect, acceptable uncertainty, design, contamination, and data readiness to create a scoped plan and obtain current quotes.
How should results be reported?
Every report should show eligibility, cohort assignment, exposure, sample size, denominators, pre-period balance, missingness, outcomes, absolute and relative differences, uncertainty, contamination, sensitivity checks, and the pre-agreed decision rule.
Label each sentence as descriptive, associative, model-based, or experimental in substance, even if those words do not appear every time. Avoid claims such as “intent generated this revenue” when the analysis only shows that high-intent accounts converted more often.
Objective public performance claims need adequate substantiation before dissemination. The Federal Trade Commission's advertising-substantiation policy statement is an authoritative reference for U.S. advertising claims. Legal and editorial reviewers should assess the actual claim, evidence, and disclosure.
Where BrandWell fits for agencies
BrandWell can fit an agency that wants to combine topic-level intent delivery with recurring cohort definitions, data QA, outcome joins, sensitivity checks, decision memos, and branded client reporting. It is a separate agency-reseller intent-data product, not the legacy BrandWell SEO writer. It does not replace the warehouse, experiment platform, statistician, CRM, lawful-use review, or human decision-maker.
BrandWell agency plans range from $2,500 to $5,000 per month, depending on topic count, term, and available contractually scoped topic exclusivity. The current written quote and Order Form control. This is not a universal public list price. Current scope, availability, a written quote, and product, pricing, privacy, and legal review control.
The agency model includes a complete white-label sales-and-delivery engine with branded portals, reports, modules, and automations, plus agency-controlled client billing and retail pricing. Conditional topic exclusivity applies only when it is available, scoped, purchased, and included in current written terms. Agencies can purchase BrandWell’s $70 seven-day reseller pilot. It includes agency-branded topic reports and the complete sales playbook under the current written pilot terms. Other product capabilities and any topic exclusivity remain subject to their separate current written scope.
BrandWell can provide agent-ready workflow instructions for Claude or ChatGPT, or optional direct browser execution through the separate Moxby product. An instruction can build the cohort input, flag missing or contaminated records, prepare descriptive charts, and draft a decision memo. A named analyst and client approver must review definitions, exclusions, causal language, and consequential actions.
Before operational use, complete product, pricing, privacy, security, compliance, legal, and platform-policy review.
Five failure modes that distort cohort analysis
1. Selecting on a future outcome
The cohort is defined using engagement or conversion that happened after activation, making the comparison circular.
2. Comparing high-intent accounts with the full database
The groups differ in fit and readiness before treatment, but the gap is called lift.
3. Ignoring exposure contamination
Holdout accounts receive the same ads, outreach, or sales attention through another route.
4. Leaking identity and outcomes across units
Person and account joins double count results, misattribute activity, or expose data beyond permitted use.
5. Publishing a causal claim from association
A descriptive pattern becomes a public revenue claim without a credible counterfactual or adequate substantiation.
Downgrade the claim or stop the analysis when the cohort is too small, groups are not comparable, exposure is unknown, outcome definitions changed, identity fails, or the conclusion is specification-dependent.
Cohort-analysis handoff checklist
Before a decision, confirm:
- Business decision, accountable owner, primary question, and minimum evidence.
- Eligibility population, analysis unit, index event, and repeated-entry rule.
- Comparison or counterfactual with documented assumptions.
- Pre-period, activation, maturation, and outcome windows.
- Primary outcome, guardrails, stable CRM definition, and outcome owner.
- Exclusions, concurrent treatments, crossover, and contamination rules.
- Identity, provenance, missingness, and data-quality controls.
- Effect estimate, denominators, uncertainty, balance, and sensitivity checks.
- Descriptive, associative, model-based, or experimental claim label.
- Decision, approval, rollback, reproducible query or code, and retest plan.
A useful cohort analysis does not force a positive answer. It makes the next business decision more defensible and shows exactly what evidence would change it.
What agencies receive in the $70 pilot
The BrandWell reseller pilot costs $70 and runs for seven days. During that window, BrandWell creates agency-branded topic reports and supplies the complete sales playbook the agency can use to present the offer and seek client commitments before choosing a full plan.
This gives the agency a practical way to test demand, compare expected commitments with its costs, and decide whether the service can operate as a profit center. No client commitment, cost coverage, or profit outcome is guaranteed. Review the $70 seven-day reseller pilot.
