An A/B test stopping rule defines in advance when a team will ship, reject, pause, or declare no decision based on runtime, uncertainty, business effect, and guardrails.
Set a minimum runtime
Cover normal weekly shopping patterns and avoid deciding from an early campaign spike. The minimum period should reflect the store's buying cycle, not a generic number of days.
Require practical value
Specify the smallest improvement that justifies rollout. A result can look certain while the likely gain remains below the cost of maintaining the change.
Define safety exits
Pause early when the variant causes errors, breaks a commerce path, or breaches a guardrail by a material amount. Safety exits are different from stopping because a favorable trend appears.
Allow an inconclusive decision
Set a maximum runtime or traffic budget. If uncertainty remains too wide or the likely effect is too small, record no decision and use the learning to redesign or deprioritize the idea.
Run the test in Shopify
Sofcy supports URL, template, theme, and custom JavaScript API experiments.
Start free trial