# How to scale innovation pilots: 12-week test, pre-set scale-or-stop criteria

Ivy Nakamura · October 20, 2026

> Learn how to scale innovation pilots with a 12-week test, one pre-set metric and threshold, and a firm week-12 scale, stop, or extend decision process.

| Takeaway | Detail |
| --- | --- |
| Run each innovation pilot for 12 weeks. | The fixed pilot period ends at week 12, when the scale-or-stop decision is due. |
| Register one metric and one threshold before week 1. | The pilot must have one primary metric and one predetermined threshold documented before the test begins. |
| Make the decision at week 12, without exceptions. | The pre-set number alone determines whether the pilot scales, stops, or receives one 4-week extension. |
| Design roughly two-thirds of pilots to exit. | A kill-or-scale system should intentionally stop about two-thirds of pilots rather than preserve an opinion-driven pipeline. |

A practical system for testing innovation pilots against pre-registered thresholds instead of subjective momentum. Each 12-week test converts evidence into a scale, stop, or one 4-week extension decision.

![Sunlit glass and steel innovation opening onto quiet courtyard with](https://static.mm-ais.com/article-images-ai/how-to-scale-innovation-pilots-12-week-t-ai-f212933e.jpg)
Sunlit glass and steel innovation opening onto quiet courtyard with

## Why Unstructured Pilots Pile Up

Initiative overload is usually described as a creativity problem, but the deeper failure is structural: organizations are good at generating ideas and launching pilots, yet weak at connecting measured results to leadership decisions. Dr. Lisa McAllister’s point on LinkedIn is that organizations struggle with innovation because they lack a testing system, not because they lack activity. More activity can make matters worse when every pilot begins but no formal mechanism exists to decide which ones should continue, stop, or expand. The result is a portfolio of “zombie projects”—initiatives that remain technically alive because nobody has authority and a scheduled occasion to terminate them.

A practical check is to trace one current pilot backward and forward: What idea prompted it? What was measured? Where was the result presented? Who decided what? If the answer ends with “we discussed it” rather than “leadership made a decision,” the pilot is not governed by a portfolio system. It is governed by momentum. McAllister frames the alternative as a sequence of Ideas → Pilot → Measure → Leadership decision. The final step must be a scheduled gate with a named decision owner, not an informal judgment about whether a project sounds promising.

That distinction matters because scale and stop decisions made without a prearranged evidence standard are vulnerable to sunk costs, executive enthusiasm, and the political weight of sponsors. The portfolio logic is therefore not a suggestion to document activity more diligently. It is a requirement to create a decision point at which evidence has consequences. A useful rule is that no pilot may remain in the portfolio simply because the team wants another chance. Leadership should be able to point to the gate when the pilot arrives and identify the decision that the gate requires.

Safe-to-fail design is the prerequisite for making that gate credible. Management Today emphasizes safe-to-fail experiments that are small-scale, low-cost, and capable of rapidly scaling up or shutting down based on observed outcomes. The purpose is not to avoid failure but to learn from it. Before launch, check whether each pilot has a bounded scope, an affordable test, and a genuine exit route. If the team cannot define what learning could justify stopping the initiative, the experiment lacks a credible way to fail safely.

The operating implication is straightforward: design the portfolio so that testing and deciding are inseparable. If pilots are routinely launched but rarely closed, the bottleneck is probably not idea generation or execution capacity. It is the missing Measure-to-Decision link. Adding more experimentation will multiply the backlog unless leadership establishes scheduled gates, gives pre-set evidence the authority to drive the outcome, and treats termination as a successful portfolio decision when the evidence warrants it.

![Misty coastal test site with narrow footpath through](https://static.mm-ais.com/article-images-ai/how-to-scale-innovation-pilots-12-week-t-ai-e4523f6e.jpg)
Misty coastal test site with narrow footpath through

## The 12-Week Pilot Architecture

A 12-week pilot should follow one fixed week-by-week split: weeks 1–3 build, weeks 4–10 measure, and weeks 11–12 decide. The schedule is not merely a delivery target; it is a governance system that prevents a pilot from remaining under evaluation indefinitely.

| Weeks | Phase | Required work |
| --- | --- | --- |
| 1–3 | Build | Create the minimum testable version and finalize the pre-registered decision contract. |
| 4–10 | Measure | Run the live test and record results at a fixed weekly cadence. |
| 11–12 | Decide | Review the evidence and issue a scale, stop, or extension decision. |

During weeks 1–3, the team builds only the minimum testable version needed to produce evidence, rather than expanding the pilot into a broader rollout. Before the pilot starts, the decision contract must identify one primary metric, one threshold, and the week 12 decision date, including the rule that maps the resulting number to each permitted outcome. Anything not written by the end of week 3 cannot become a criterion later. This protects the test from being retrofitted to fit its results.

During weeks 4–10, the live test runs on a fixed measurement cadence. The team records a weekly checkpoint reading, even when no product or operational change is planned. A final-only reading can conceal deterioration, stalled adoption, or an unstable result; weekly readings show the trajectory leading into the decision. The purpose of these checkpoints is observation, not repeated ad hoc decision-making. Measurement changes cannot be used to alter the primary metric, threshold, or decision date already registered.

Weeks 11–12 are a decision window, not an extension of the build phase. The team consolidates the evidence, checks the integrity of the live test, and applies the pre-registered rule. At week 12, the pre-set number alone determines whether the pilot scales, stops, or receives one 4-week extension—no exceptions. If an extension is granted, the team must re-register the threshold, define the new decision date, and specify what evidence will be required. A second extension is a stop by definition.

The check for every portfolio is simple: can the pilot’s owner point to the written contract, the weekly readings, and the recorded week 12 decision? If any element is missing, the pilot is not following the architecture. The result is a disciplined transition from testing to an explicit portfolio decision, with neither indefinite evaluation nor post-hoc reinterpretation treated as an acceptable endpoint.

## Setting Pre-Registered Thresholds

Every pilot should begin with one primary metric, not a dashboard of worthy outcomes. Choose adoption, cost per transaction, or cycle-time reduction, depending on the business mechanism the pilot is meant to change. A pilot with two primary metrics has no primary metric: after the results arrive, either measure can be declared the winner because each has a plausible constituency. Secondary measures may help diagnose what happened, but they cannot determine the decision. The operating rule is simple: if both measures appear equally important, select the one closest to break-even and designate the other as diagnostic.

Set the threshold from break-even economics rather than ambition. First, calculate the incremental cost of operating the proposed solution at scale, including implementation, integration, support, training, and any continuing labor or technology expense. Then work backward to the metric value required to cover that cost. For adoption, estimate the contribution generated by each additional active user and determine how many users are needed to offset the scaled operating cost. For cost per transaction, divide all-in scaled costs by expected transaction volume and identify the maximum cost per transaction that remains economically viable. For cycle-time reduction, translate the required time savings into labor capacity and compare the value of that capacity with the recurring cost of the solution. The threshold is the metric value at which scale begins paying for itself.

Document the metric, threshold, and decision date in a written contract, then obtain the sponsoring executive’s approval before week 1. Specify how the metric will be measured, the relevant user or transaction population, the baseline, and the source of the data. This three-part decision contract is the only valid admission ticket: without all three parts, the pilot is not ready to begin. An unsigned threshold lacks accountable sponsorship, and a threshold recorded only in a discussion, presentation, or working document is too easy to reinterpret once results become politically inconvenient.

At week 12, use the pre-set number to select the decision. The contract should map the result in advance to scale, stop, or one four-week extension, with no exceptions and no new negotiation over what counts as success. The extension is a bounded continuation, not a renewed evaluation period, and it ends with the same decision discipline. If the pilot fails to meet the break-even threshold, an extension should not become a way to search for a more favorable measure.

Before signature, have the sponsor ask three review questions: Does the metric reflect the economic mechanism? Did the threshold come from the cost of scaling? Can the result be calculated from an agreed data source on the decision date? If any answer is unclear, the pilot is not ready for admission. This check makes the threshold testable before operational excitement enters the room and gives the executive a written basis for acting on the number later.

## Kill Rates: What the Evidence Supports

The roughly two-thirds exit rate used throughout this article is a design target for portfolio governance, not a published cross-industry statistic, and no external source in this piece establishes one. Treat it as a benchmark to test against your own history: count how many pilots from the last 12 months were stopped, scaled, or left unresolved, and compare that base rate with the target before adopting it. A portfolio leader can check the rule by counting how many pilots exit rather than merely how many launch.

This approach also matches the logic described by Management Today. Safe-to-fail experiments are small-scale, low-cost tests designed to produce evidence that can trigger either rapid scaling or shutdown. If nearly every experiment continues, leadership has no meaningful signal about which ideas deserve broader investment. The practical check is therefore not whether the portfolio avoids failure, but whether unsuccessful pilots terminate cheaply enough to preserve capacity for the stronger candidates.

The portfolio framing on LinkedIn reinforces the same point: the sequence of idea, pilot, measurement, and leadership decision matters because the decision gate exists to make a disposition, not merely to document activity. In operational terms, each pilot should leave the portfolio in one of three states: scaled, stopped, or held for the single permitted extension. A review file that cannot identify the current state is not functioning as a decision gate.

Leaders should evaluate the kill rate at the portfolio level, not by pressuring every team to meet the target independently. Projects differ in evidence quality, cost, strategic relevance, and measurement maturity, so a uniform project-level quota could reward arbitrary decisions. The portfolio check is whether resources are flowing toward the pilots with the strongest pre-registered results while the remaining pilots are ending on schedule. This keeps “kill” from becoming a euphemism for relabeling delay or quietly extending a weak test.

The target also needs an interpretive safeguard: it is a governance benchmark, not a forecast of any one organization’s results. Reviewers should ask whether exited pilots were stopped because they missed the registered threshold, whether scaled pilots cleared it, and whether any extension was the only permitted disposition. If the portfolio reports many “successes,” the organization should inspect the decision records for goalpost changes, selective measurement, or pilots that were never actually tested against a standard. A credible kill rate requires the same evidentiary discipline as a credible scale rate.

## Decision Designs Compared

Consider three portfolio decision designs: pre-registered thresholds, post-hoc executive judgment, and keep-all. The head-to-head comparison favors pre-registration because it moves the decision from a negotiation over what counts as success to a test of whether a pilot met a standard selected before the results were known. This follows the experimentation logic described by Dr. Lisa McAllister—ideas proceed through pilots, measurement, and leadership decisions—but adds a binding decision rule to the sequence.

| Decision design | What happens at week 12 | Failure mode |
| --- | --- | --- |
| Pre-registered threshold | Winner: scale, stop, or grant one 4-week extension according to the pre-set result. | Decision cost is near zero because the rule is already settled, and politics are pre-empted: the metric was chosen when nobody knew the outcome. |
| Post-hoc executive judgment | Loses: leaders reconsider the evidence, redefine success, or negotiate another review period. | The decision date becomes negotiable, weak pilots survive through sponsor advocacy, and the portfolio drifts toward keep-all. |
| Keep-all | Loses worst: graduate every pilot that finishes, regardless of whether its results justify deployment. | Per the voice-agent analysis, too many pilots with low conversion to scale stall the graduation of genuinely good ones. |

The practical check for pre-registration is simple: an executive should be able to read the rule and calculate the week-12 disposition without asking the pilot team to reinterpret the metric. If explaining the decision requires a new definition of success, reconsidering the denominator, or assessing arguments that were not part of the standard, the portfolio has returned to post-hoc judgment. The review may still discuss execution and risk, but those considerations cannot replace the pre-set result.

Post-hoc review is especially vulnerable because sponsor advocacy can make a weak pilot look strategically valuable even when it misses the agreed test. A useful audit is to compare the originally scheduled decision date with the actual decision date and identify every pilot that continued despite missing its threshold. Keep-all fails a different audit: count the pilots that finished but did not meet the standard for scale. If that count is high, the apparent portfolio breadth is masking constrained deployment capacity.

As specified in the 12-week architecture above, the week-12 number alone determines the disposition. The test of the design is not whether executives like the outcome; it is whether they can apply the outcome they committed to before anyone knew which result would appear.

## What the Evidence Does Not Establish

The available sources does not establish a published cross-industry benchmark for the share of pilots that should be stopped. The roughly two-thirds exit rate is a voice-agent-specific recommendation for this site’s portfolio design, not a universal statistic. Before adopting it, compare the rule with your own base rate: review pilots launched in the last 12 months and calculate how many were stopped, scaled, or remained unresolved. If that history differs materially from the recommendation, use the history to test the design rather than treating the proposed rate as evidence.

Twelve weeks also cannot represent every adoption curve. Annual procurement cycles, seasonal demand, and regulated products may need more time before results can be interpreted. The rule still wins for these cases, but the four-week extension must be the deliberate lane rather than a routine stall. Before choosing it, document the specific dependency—such as procurement timing or a seasonal measurement window—and specify what new evidence the extension will produce. If no dated dependency or decision-relevant evidence can be identified, the extension is not justified.

The rule breaks when the metric can be gamed. A target such as calls handled, active users, or survey responses may rise without demonstrating the outcome the pilot was meant to produce. Check whether the metric can be improved by changing behavior that matters only to the measurement, for example redirecting low-value interactions, counting duplicate activity, or selecting unusually favorable users. When that risk exists, retain the pre-registered decision but validate the result against an independent, harder-to-game measure before scaling.

| Condition | Where the 12-week rule breaks | Where it still wins | Required check |
| --- | --- | --- | --- |
| Seasonal or slow adoption | Results appear only after the fixed decision point. | The pilot still receives a bounded, evidence-producing extension. | Name the dated dependency and the evidence due by the extension’s end. |
| Gameable primary metric | The threshold can be met without meaningful value. | The threshold remains the decision anchor, subject to result validation. | Test the metric against an independent outcome that resists superficial improvement. |
| Regulated product | Evidence is incomplete because required approval or observation is still pending. | The pilot avoids indefinite evaluation by using a specific, limited extension. | Identify the unresolved approval or observation and its next decision point. |

The practical boundary is therefore not innovation versus caution. It is whether the pilot has a credible metric, a fixed decision point, and a documented reason for any extension. If the rule is being used merely to avoid a difficult stop, the apparent discipline is only administrative theater.

 The sources consistently support rapid learning and explicit scale-or-shut-down decisions: Management Today describes safe-to-fail experiments that can rapidly scale or close based on observed outcomes, while the LinkedIn source frames innovation as a sequence from idea to pilot, measurement, and leadership decision. The limits table keeps those principles from being applied where the clock or the metric would distort the answer.

## Worked Example: Run the Numbers

This worked example is an illustration, not a forecast: a customer-service team tests an AI-assisted reply tool. The pilot begins on the first working day of Week 1, and the decision date is the final working day of Week 12. The named parties are the operations sponsor, who owns the business decision; the product lead, who runs the pilot; and the analytics lead, who verifies the calculation. Before Week 1, they record one primary metric: net benefit points, defined as one point for each successful customer interaction and minus one point for each unsuccessful interaction. The single threshold is zero points. The decision rule is fixed in advance: above zero means scale, exactly zero means one four-week extension, and below zero means stop.

| Input or calculation | Illustrative result | Check |
| --- | --- | --- |
| Decision window | Weeks 1–12 | 12 weeks × 7 days = 84 days |
| Interactions observed | 10 | All 10 meet the example’s eligibility rule |
| Successful interactions | 8 | 8 × 1 point = 8 |
| Unsuccessful interactions | 2 | 2 × −1 point = −2 |
| Net benefit points | +6 | 8 − 2 = 6 |
| Break-even trigger | 0 points | 5 successes and 5 failures: 5 − 5 = 0 |
| Maximum extension | 4 weeks | One extension only; review at Week 16 |

At the Week 12 review, the analytics lead confirms the denominator, verifies the classification of every interaction, and locks the calculation: 8 successful outcomes minus 2 unsuccessful outcomes equals 6 net benefit points. Because 6 is greater than the pre-registered threshold of 0, the sponsor scales the pilot. No post-meeting adjustment is permitted: the 8-to-2 split does not matter, only the pre-registered number does.

t become a new threshold, and a favorable anecdote does not replace the calculated result. This is the practical value of a safe-to-fail experiment, which *Management Today* describes as capable of rapidly scaling up or shutting down based on observed outcomes.

The extension branch also has a deterministic trigger. If the final result had been 5 successful and 5 unsuccessful interactions, the calculation would be 5 − 5 = 0, exactly at break-even. The only authorized response would be one four-week extension, ending at Week 16, with the same metric, threshold, and decision rule. The team would not be allowed to add a second metric, reset the baseline, or extend again. If the result had been 4 successful and 6 unsuccessful interactions, the score would be 4 − 6 = −2, and the decision would be stop.

The portfolio manager can now audit the example without reconstructing the meeting. For each pilot, the record contains the starting week, the Week 12 decision date, the raw success and failure counts, the calculation, and the resulting action. That auditability is what turns the pilot from a showcase into a decision system: one number determines whether the organization proceeds, pauses once for a defined test, or exits.

## And Decision Contract

A decision contract turns a pilot from an open-ended experiment into a commitment with a predetermined endpoint. For example, consider an AI-assisted invoice-processing pilot in one subsidiary. Before week 1, the sponsor completes a copy-usable one-page decision contract like this:

| Contract field | Completed entry |
| --- | --- |
| Pilot | AI-assisted invoice processing in one subsidiary |
| Primary metric | Cost per invoice processed |
| Scale threshold | ≤ €2.10 per invoice |
| Decision date | Week 12 |
| Extension | One 4-week extension; threshold unchanged |

The 2.10 threshold is derived, not negotiated after results arrive. The current manual cost is 3.40 per invoice, and the pilot adds 1.30 in tooling cost per invoice. Subtracting tooling cost from the current cost gives 3.40 − 1.30 = 2.10, an illustrative break-even point for the method. The check is simple: if the pilot’s final cost per invoice is at or below 2.10, scale it; if it is above 2.10, stop it. If the result is close enough to justify more evidence, grant the single 4-week extension with the same threshold and return to the same decision rule.

Interim readings provide operating context without replacing the week-12 decision. A week-4 reading of 2.95 is expected during the ramp effect. A week-7 reading of 2.45 shows movement toward the threshold, while a week-10 reading of 2.05 crosses the threshold and supports scaling. A flat week-10 reading at 3.00 does not support scaling and requires the contract’s stop or extension branch, depending on the pre-registered rule.

At week 12, the pre-set number alone determines the outcome. Leaders should not reinterpret the metric, lower the threshold, add a new condition, or extend beyond the one permitted 4-week period. This constraint matters because post-result negotiation makes the threshold easier to move than the pilot. The contract does not predict every business result; it makes the decision reproducible, and that is what allows a portfolio to learn from completed pilots rather than accumulate them indefinitely.

## What to do next

| Step | Action | Why it matters |
| --- | --- | --- |
| 1 | Choose one primary metric for each innovation pilot and document its pre-registered threshold before week 1. | Creates an objective test standard before momentum or opinion can influence the result. |
| 2 | Set the decision date for the end of the 12-week test, with no exceptions or additional review points. | Prevents evidence from being selectively extended or re-litigated. |
| 3 | Structure the innovation pipeline so roughly two-thirds of pilots are expected to exit rather than advance. | Challenges the default of preserving every pilot and makes stopping an intentional outcome. |
| 4 | At week 12, compare the primary metric with the pre-set threshold; let that number alone determine scale, stop, or one 4-week extension. | Makes the decision consistent, auditable, and free from subjective momentum. |
| 5 | If the result falls short, use the single 4-week extension only; do not add another metric, threshold, or extension option. | Keeps the original decision rule intact while allowing one limited test period. |
| 6 | Record the scale-or-stop outcome when the decision date arrives and close or advance the pilot accordingly. | Turns the 12-week test into a clear operating decision rather than an open-ended pipeline. |

## Frequently Asked Questions

**What must be documented before a pilot begins?**

The pilot must have one primary metric and one predetermined threshold documented before the test begins.

**What decision must be made when the fixed pilot period ends?**

The scale-or-stop decision is due, and it must be made without exceptions.

**What options are available at the end of the fixed pilot period?**

The pilot may scale, stop, or receive one 4-week extension.

**What determines whether a pilot scales, stops, or receives an extension?**

The pre-set number alone determines the outcome.

**What share of pilots should a kill-or-scale system intentionally stop?**

It should intentionally stop about two-thirds of pilots rather than preserve an opinion-driven pipeline.

**Why should organizations design roughly two-thirds of pilots to exit?**

A kill-or-scale system is intended to stop about two-thirds of pilots instead of allowing activity without formal decisions.

## Quick answers

| How long should each innovation pilot run? | Run each innovation pilot as a 12-week test. |
| --- | --- |
| When should the scale-or-stop decision be made? | Make the decision at the end of the 12-week test, without exceptions. |
| What must be documented before the pilot begins? | Document one primary metric and one predetermined threshold before the test begins. |
| What decisions can result from the pre-set number? | The pre-set number determines whether the pilot scales, stops, or receives one 4-week extension. |
| How many pilots should a kill-or-scale system intentionally stop? | A kill-or-scale system should intentionally stop roughly two-thirds of pilots. |

Also worth reading: **How to kill failing pilots: 21 days vs 84 days kill decision in 2026**: [How to kill failing pilots:](https://tlab.fun/blog/how-to-kill-failing-pilots-21-days-vs-84-days-kill-decision-in-2026.php) · **How to pick co-brand partners: 40 pilots show 12-month revenue lift criteria**: [How to pick co-brand partners:](https://tlab.fun/blog/how-to-pick-co-brand-partners-40-pilots-show-12-month-revenue-lift-criteria.php) · **3 Pre-Launch Pricing Methods: Evidence and Anchor Selection**: [3 Pre-Launch Pricing Methods: Evidence](https://tlab.fun/blog/3-pre-launch-pricing-methods-evidence-and-anchor-selection.php)

### Related reading

- [How to pick co-brand partners: 40 pilots show 12-month revenue lift criteria](https://tlab.fun/blog/how-to-pick-co-brand-partners-40-pilots-show-12-month-revenue-lift-criteria.php)
- [71% GenAI Pilots Never Reach Scale: 28-Day Lift vs Cost](https://tlab.fun/blog/71-genai-pilots-never-reach-scale-28-day-lift-vs-cost.php)
- [How to Test Targeting in 2026: Location vs Cohort by 90-Day Retention Scale or Kill](https://tlab.fun/blog/how-to-test-targeting-in-2026-location-vs-cohort-by-90-day-retention-scale-or-kill.php)
- [Cut Innovation Risk: 2 Metrics to Win 2026 Board Funding](https://tlab.fun/blog/cut-innovation-risk-2-metrics-to-win-2026-board-funding.php)
- [Corporate Pilot Kill Criteria: $15K Gate Kill vs Extend vs Venture Client](https://tlab.fun/blog/corporate-pilot-kill-criteria-15k-gate-kill-vs-extend-vs-venture-client.php)
- [Hardware + Subscription Models: A Verify-Before-You-Commit Guide](https://tlab.fun/blog/hardware-subscription-models-a-verify-before-you-commit-guide.php)

### Latest

- [How to Test Targeting in 2026: Location vs Cohort by 90-Day Retention Scale or...](https://tlab.fun/blog/how-to-test-targeting-in-2026-location-vs-cohort-by-90-day-retention-scale-or-kill.php)
- [Hardware + Subscription Models: A Verify-Before-You-Commit Guide](https://tlab.fun/blog/hardware-subscription-models-a-verify-before-you-commit-guide.php)
- [How to pick co-brand partners: 40 pilots show 12-month revenue lift criteria](https://tlab.fun/blog/how-to-pick-co-brand-partners-40-pilots-show-12-month-revenue-lift-criteria.php)

Canonical: https://tlab.fun/blog/how-to-scale-innovation-pilots-12-week-test-pre-set-scale-or-stop-criteria.php
Markdown: https://tlab.fun/blog/how-to-scale-innovation-pilots-12-week-test-pre-set-scale-or-stop-criteria.php/index.md