How to Test Targeting in 2026: Location vs Cohort by 90-Day Retention Scale or Kill

This guide sets a single 90-day retention standard for comparing location and cohort targeting pilots. It also defines the kill rule for pilots that miss the baseline lift.

Location vs Cohort: How the Tests Work

Two targeting pilots can look identical in a planning document and behave nothing alike in the data. A location-targeted test assigns users by place — city, store radius, or district — and reads retention out per geography arm. A cohort-targeted test assigns users by pre-launch behavior segment — power users, lapsed users, channel source — and reads retention out per segment. In the first, the variable is where someone is. In the second, it is what someone already did.

Location tests are operationally simple: the geography is the arm, so every activated user inside the boundary is exposed, and you compare retention across districts. The catch is that districts differ for reasons unrelated to your targeting — demand density, competitor presence, seasonality. Treat a geographic lift as evidence only when each arm is compared against its own pre-pilot baseline rather than a whole-portfolio one, and when arms are matched before launch on activation volume.

Cohort tests trade that simplicity for fragility. Segments are built from pre-launch behavior, so the definitions must be frozen before exposure; re-cutting "power user" after the pilot makes the result uninterpretable. The readout grain is the segment, and the strongest comparison is segment-versus-segment inside the same market, which controls for geography but leaves you exposed to composition drift if the mix shifts mid-test.

Both designs must report into one currency: 90-day retention of activated users. Fix the denominator before launch — activated users, not installs and not signups — and fix the clock so day 90 counts from each user's activation date rather than from campaign start. Otherwise a slow-ramping pilot reads as a retention failure that it never was.

One pre-launch check decides whether either test is runnable: confirm your experiment platform logs the assignment dimension — geography ID or segment ID — at exposure time. Assignment cannot be reconstructed after the fact. If the platform cannot tell you which arm an activated user was in, the pilot produces no comparable retention readout at the day-90 checkpoint and should not launch.

The Evidence: What Grounding Supports

The grounding for this article references Lifetime Property Management's tenant turnover ROI guide but does not supply its specific cost figures, so do not quote dollar amounts or ROI percentages until they are confirmed on the published page. What the source does support is the direction of the economics: retention is framed as substantially cheaper than replacement, and that asymmetry is the reason a targeting pilot should be read on retention rather than on conversion at acquisition — losing an activated user is the expensive failure mode, so a cheap acquisition win does not establish economic value.

Dataintelo's portfolio retention analytics market research report (projected through 2033) is the grounding for institutional demand in this space, but any specific market size or CAGR figure should be verified directly in the report before being quoted, since the grounding summary does not state numbers. The supported claim is directional: financial institutions are investing in retention measurement and intervention as a portfolio discipline, not treating retention as a byproduct of acquisition targeting.

The same direction appears in lender-facing material: a LinkedIn post titled "Why lenders should prioritize portfolio retention" argues that portfolio retention will be one of the most important strategies over the next 24 months; Mortgage Innovators calls portfolio retention the key to good loan servicing; and Pennymac describes retention and loss-mitigation programs driven by portfolio modeling, targeted messaging, and warm transfers. None of these is a pilot result, but together they show retention economics being managed at portfolio level.

Once the Lifetime turnover and retention-program costs are verified on the source page, compute the replacement-to-retention cost ratio directly from its published figures. That ratio is the cost logic behind using 90-day retention of activated users as the common currency for comparing location-targeted and cohort-targeted tests: when replacement is the expensive failure mode, retention lift is the number that decides whether a pilot created value.

The evidence has limits. Lifetime's numbers are tenant-turnover economics rather than software-user retention; Dataintelo measures an analytics market rather than pilot outcomes; and neither source establishes a universal performance threshold. What they do establish is priority order: retention cost dominates replacement or acquisition cost, so a targeting pilot that cannot show retention lift has not shown economic value. That supports comparing pilots on one retention metric and treating failure at the checkpoint as a reason to kill rather than scale.

Kill, Iterate, or Scale: The Comparison

Set the three outcomes before launch, not after the readout. KILL means the pilot ends: 90-day retention of activated users is below the portfolio baseline, or arm purity has broken past 5% leakage. ITERATE means the pilot is promising but not proven: retention is positive yet below the 5-percentage-point bar, or the confidence interval still includes zero; re-run it with exactly one changed variable. SCALE means the pilot has earned expansion: retention is at or above the 5-percentage-point bar and the confidence interval excludes zero.

The default outcome at the checkpoint is kill: any pilot short of the 5-percentage-point retention lift is ended, regardless of how cheap or easy the targeting method was to run. Only arms that clear the bar with a confidence interval excluding zero are scaled. If your portfolio policy permits one rerun for sub-bar pilots, treat that as a pre-registered exception — a new charter with a new checkpoint and exactly one changed variable — not as a weakening of the kill rule.

Decide arm count by duration, not ambition. A 90-day readout is expensive, so cap each pilot at two location arms and two cohort arms; that keeps a full cycle inside one quarter. Pre-commit the single variable you will change if the pilot lands in iterate, so the second cycle is a comparison rather than a redesign.

Apply the same currency to every pilot: 90-day retention of activated users against the portfolio baseline. A cheap targeting method does not earn a lower bar; if it has not cleared the 5-percentage-point gate with a confidence interval excluding zero, it stays in iterate or ends.

Costs and Numbers That Matter

The cost of being wrong has two prices, and both should be written into the pilot charter before launch. One is the cash you burn by keeping a pilot that never clears the 90-day retention bar; the other is the value you forfeit by killing a pilot that would have cleared it. Neither price should be estimated after the fact.

Cost of a wrongly kept pilot = daily marginal run cost × 90 days. Build the daily figure from three inputs only: incremental platform spend, the staffing allocation fraction assigned to the pilot, and any media budget behind the location or cohort arms. Count only marginal run cost, not sunk setup cost, because the day-90 decision is whether to continue, not whether to have started. Record the result in the charter and use it as the price tag in the kill conversation.

Cost of a wrongly killed pilot = expected retention lift × downstream LTV per retained user × addressable population. This is the strategic cost of a false negative, not a cash cost. Use the same population definition for addressable population that the pilot used for activation, so lift and population are not mismatched. If the organization cannot state LTV per retained user, the formula cannot be completed; record that missing input as an unpriced exposure and require an estimate before any kill decision is finalized.

At the checkpoint, apply the kill rule directly rather than debating opinions. Compute both cost formulas in the charter before launch so the price of a wrong call is known in advance, but the decision itself is not cost-conditioned: any pilot that has not reached at least a 5-percentage-point retention lift over the portfolio baseline is killed. A pilot at 4.9 points still fails the floor, no matter how large its wrongly-killed cost would be.

Keep the math method-only: no invented rates, no assumed conversion values, and no hidden staffing percentages. The charter should name the owner of each input, the date it was estimated, and the date it will be refreshed. If an input changes materially, recompute both formulas and note the change in the decision log.

, Limits, and Kill Rules

Worked example: set the portfolio baseline at 30% day-90 retention, enroll 10,000 activated users, and split them 2,500 each across four arms—two city arms and two cohort arms. The scale bar is 35% day-90 retention in the winning arm with a confidence interval that excludes 30%. Check assignment purity at day 30 and require at least 95%; at day 60, check only that the leading arm shows directional lift; at day 90, read final retention and apply the edge-case table below before any kill or scale call.

Limits: the common currency assumes comparable activated users across arms. If day-30 purity is below 95%, stop the comparison, repair assignment, and restart the clock; reading day-90 retention from a contaminated arm would compare different populations rather than targeting methods.

Before launch, complete a pilot charter with the same fields used in the worked example above: pilot name, targeting method (location or cohort), assignment unit, arms and allocation, baseline metric and value (30% day-90 retention in the example), the three checkpoints (day-30 purity at 95%, day-60 directional lift, day-90 final retention against the 35% bar), the kill rule, owner, and start and end dates. Leave no field blank; the comparison currency must be defined before launch.

The 90-day rule is a kill rule, not a scale approval: a pilot that clears 35% with a confidence interval excluding 30% has earned comparison under the earlier decision table, while a pilot that misses it is killed even if the targeting method was inexpensive or easy to run. Keep the charter, checkpoints, and edge-case exceptions in the same document so the exception is visible before the pilot starts rather than after a miss.

What to do next

Frequently Asked Questions

What metric is the common currency for comparing location and cohort pilots?

The common currency is 90-day retention of activated users, not cost or ease of targeting.

What retention lift does the kill-rule checkpoint require?

The checkpoint requires at least a 5-percentage-point retention lift over the portfolio baseline at 90 days.

Which portfolios do the comparison and kill rule govern?

The comparison and kill rule govern 2026 corporate pilot portfolios.

What do financial institutions use portfolio retention analytics for?

Financial institutions use portfolio retention analytics to identify at-risk clients, predict churn, and implement targeted retention (2033 market research report).

How does a location-targeted test assign and read users?

A location-targeted test assigns users by place — city, store radius, or district — and reads retention out per geography arm.

How does a cohort-targeted test assign and read users?

A cohort-targeted test assigns users by pre-launch behavior segment — power users, lapsed users, channel source — and reads retention out per segment.

Quick answers

Also worth reading: 3 Pre-Launch Pricing Methods: Evidence and Anchor Selection: 3 Pre-Launch Pricing Methods: Evidence · Disney Is Buying Back $8 Billion in Stock as Employees Face Layoffs and Stricter Office Rules: Disney Is Buying Back $8