Smoke testing in production determines whether a deployment introduced a critical failure before real users encounter it. A smoke test suite covers the minimum set of user journeys that, if broken, would make the application unusable: authentication, core workflow navigation, primary data operations, and payment or subscription flows in commerce applications. Running this suite against production after each deployment — or on a scheduled interval — provides early warning of outages, configuration errors, and regression failures that bypass staging because production data, secrets, or infrastructure differs from the test environment. This guide covers how to implement production smoke testing with TestInspector: scheduling options, live result streaming, deployment gating via the CI/CD trigger API, and self-healing behavior that reduces false positives from transient failures.

The audience for this guide is QA engineers and engineering managers responsible for production reliability. The framing is operational: specific TestInspector configuration options, the trade-offs in scheduling approaches, and the workflow decisions that determine whether a production smoke test suite is a reliable quality gate or a source of noise that teams learn to ignore. For teams evaluating whether to build production monitoring internally or rely on external expertise to set it up correctly, QA consulting services are most valuable when the production environment is complex — multiple subdomains, third-party integrations, environment-specific authentication — because the configuration effort is proportional to environment complexity, not test count. A correctly configured 20-test smoke suite that runs in under two minutes and produces zero false positives is more operationally valuable than a 200-test regression suite that runs in 45 minutes and pages on-call engineers 15% of the time for tests that fail due to test brittleness rather than application failure.

What Smoke Testing in Production Actually Covers (and What It Does Not)

Smoke testing in production is not regression testing. Regression testing validates that existing functionality continues to work across a broad surface area — it covers edge cases, secondary flows, negative scenarios, and integration points that represent medium-to-low risk per individual test case but collectively ensure the application behaves consistently. Running a full regression suite against production after every deployment is feasible for some teams but introduces risk: tests that modify data, create records, or trigger side effects (emails, webhooks, payment flows) can affect production data or create support load. Smoke testing is the subset of regression testing that is safe to run in production repeatedly — tests that are read-only or use isolated test accounts, cover only the critical paths, and complete fast enough that the total run time does not delay deployments meaningfully.

The practical scope of a production smoke suite is typically 15–30 test cases covering 4–8 critical user journeys. For a SaaS application, that means: login with a test account, core dashboard navigation, the primary feature workflow (creating, reading, updating the application's main entity), and logout. For an e-commerce application: browse, search, add to cart, and checkout through to order confirmation using a test payment method. For an API-first service: key endpoint health checks, authentication token validation, and the primary data read/write operations via HTTP request steps. The test count and runtime target should both be defined before building the suite — a smoke test suite that grows to 200 tests over time has drifted from smoke testing to full regression and will eventually be treated as such (not run in production, not used as a deployment gate). Test automation services that help establish production smoke suites typically define scope constraints as part of the implementation — keeping the suite small and stable is a design decision, not a consequence of minimal effort.

How TestInspector Scheduled Runs Work in Practice

TestInspector supports three scheduling modes for running test suites automatically: cron-based scheduling (standard cron syntax, down to per-minute granularity), interval scheduling (every N minutes, hours, or days), and one-time scheduling (run at a specific future timestamp). For production smoke testing, the appropriate choice depends on how often deployments occur and what the team's acceptable time-to-detection target is. Teams deploying multiple times per day typically run smoke tests on a 15-to-30-minute interval — frequent enough to catch a broken deployment before it affects the majority of users, infrequent enough that the test account usage and infrastructure calls do not generate meaningful noise in application logs. Teams with once-daily deployment windows often prefer a cron-based schedule tied to the deployment window — run immediately after the deployment completes, then again two hours later to catch delayed initialization issues in long-running services.

Scheduling in TestInspector is configured at the suite level, not the individual test level. A suite is a named grouping of tests that shares a variable scope (variables defined at the suite level are available to all tests in the suite), a schedule, and a run configuration (browser, viewport, environment). For production smoke testing, organizing by environment — a "Production Smoke" suite that runs against the production base URL and a "Staging Smoke" suite that runs against the staging URL — is cleaner than managing environment variables at the test level because the suite-level variable hierarchy (org defaults → suite → test) applies at run time. The production suite uses the production URL and production-environment credentials stored as encrypted variables; the staging suite uses staging values. The same test definitions run in both suites without modification, which keeps the test suite content in sync across environments without duplication.

Test data for production smoke runs requires a dedicated test account that is isolated from real user data. TestInspector's variable interpolation supports {{VAR}} substitution from encrypted storage for credentials, {{TOTP:secret}} for accounts protected by two-factor authentication, and {{TIMESTAMP}} and {{ALPHANUMERIC}} for generating unique values in tests that create records. For smoke tests that create data as part of the critical path (creating an order, submitting a form), the test account and test records created during the run should be excluded from production analytics and reporting — typically by flagging the test account in the application's data model and filtering it at the reporting layer. Teams that run smoke tests without this isolation discover the problem when production dashboards show anomalous activity driven by test runs rather than real user behavior.

Using WebSocket Streaming to Monitor Production Test Runs Live

TestInspector streams test run results in real time via WebSocket, which means a developer or QA engineer monitoring a deployment can watch individual test steps execute as the suite runs — not wait for the suite to complete and then review a static report. Live streaming is relevant for smoke testing specifically because deployments are a watched event: someone triggered the deployment, and that person (or an on-call engineer) wants to know whether it is safe to proceed or whether to roll back. A streaming view that shows individual test steps passing or failing as they execute gives immediate signal within 30–60 seconds for the first test in the suite rather than requiring the team to wait for all tests to complete before seeing any result.

For deployment watches, the combination of scheduled CI/CD trigger (described in the next section) and streaming output means an engineering team can instrument a deployment pipeline so that smoke tests are triggered automatically after a deployment completes, results stream to a monitoring channel, and a pass/fail signal is available before the team moves from the deployment phase to the monitoring phase of the release process. This is the operational pattern that production smoke testing is designed to enable — not a manual "someone goes and runs the tests" process, but an automated run that produces an early warning signal with no human trigger required.

Deployment Gating: Triggering Smoke Tests from CI/CD Pipelines

TestInspector's CI/CD trigger API allows external systems to initiate a suite run via an authenticated HTTP POST request and poll for results. This is the mechanism for deployment gating: the CI/CD pipeline triggers the smoke suite after a successful deployment, waits for the result, and fails the pipeline stage if the smoke tests fail. A failed pipeline stage can block promotion to the next environment (staging to production) or trigger a rollback workflow, depending on how the pipeline is configured. The implementation is straightforward: a pipeline step sends a POST to the TestInspector API with the suite ID and any run-time variable overrides (environment URL, build number), then polls the run status endpoint until the run reaches a terminal state (passed, failed, or timed out), and exits with the appropriate code.

The key configuration decisions for deployment-gated smoke testing are: which test failures block the deployment and which do not. Not all smoke test failures indicate a deployment-caused regression. A smoke test that fails because a third-party integration service is returning 503 errors is a real failure, but it is not caused by the deployment — blocking the deployment in response to an external service outage prevents a good deployment from reaching production because of an infrastructure problem outside the team's control. Teams that have experienced this problem implement a two-tier failure response: critical path failures (authentication, core workflow) that indicate the application itself is broken block the deployment; integration failures and optional-feature failures are reported but do not block. TestInspector's test results include per-test pass/fail status, which allows the pipeline script to evaluate results at the test level rather than treating the suite pass/fail as a binary gate. For teams building this integration, performance testing infrastructure follows the same gating pattern — a performance check step in the deployment pipeline that blocks if latency targets are exceeded.

Self-Healing Tests and Why They Matter in Production Monitoring

Self-healing test behavior is the difference between a production smoke suite that the team trusts and uses for deployment gating and one that generates enough false positives that on-call engineers stop treating failures as actionable signals. TestInspector's self-healing mechanism operates at the element selector level: when a test step fails to locate an element using its primary selector, TestInspector retries using AI-generated alternative selectors derived from the element's attributes, position, and surrounding DOM context. If an alternative selector locates the element, the test continues; the selector suggestion is logged for the test author to review and update the test if the DOM change is permanent rather than transient.

For production smoke testing, self-healing matters in two scenarios. First, when a deployment changes the DOM structure of a page — modifying a button's class name, adding a wrapper element, or restructuring a form — the smoke test would fail without self-healing even though the application is functioning correctly. The failure would block the deployment or trigger an alert for a working deployment, which is a false positive. Self-healing allows the test to continue in the current run; the test author receives a selector suggestion and can update the test before the next deployment without having experienced a false-positive block. Second, production environments occasionally have transient rendering delays — a slow third-party script, a brief CDN cache miss — that cause elements to be temporarily unavailable when the test step executes. Self-healing's auto-retry on selector failure handles these transient cases without requiring the test to define explicit wait logic for every element interaction.

The operational goal of self-healing in production smoke testing is maintaining a high true-positive rate: when the smoke suite fails, the failure should indicate a real application problem rather than a test brittleness issue. Teams that build smoke suites without attention to selector stability and retry behavior typically see false positive rates of 10–20% within three months of deployment, which is high enough that on-call engineers start ignoring alerts. A self-healing test infrastructure combined with a stable test account and a scoped suite (15–30 tests covering critical paths only) keeps the false positive rate low enough that every failure is investigated immediately — which is the operational requirement for production smoke testing to function as an early warning system rather than background noise. For teams looking to implement this end-to-end, hiring a QA engineer or engaging a specialist to configure the initial production smoke suite is often faster than building it through internal iteration, particularly when the production environment involves complex authentication or multi-step flows.

Frequently Asked Questions

How many tests should a production smoke suite contain?

The practical range for most applications is 15–30 test cases. Fewer than 15 typically means the suite is not covering all critical user journeys; more than 30 usually means scope has drifted from smoke testing into partial regression testing. The target runtime is under five minutes — if the suite takes longer, it is too large for the role it plays in production monitoring. Scope constraints are a design decision that should be revisited every quarter as the application grows and new critical paths are added.

How do you handle tests that create data in production as part of the smoke run?

Use a dedicated test account isolated from production analytics, and build cleanup into the test flow wherever possible. For tests that create records (orders, tickets, accounts), a teardown step in the test that deletes or marks the created record avoids accumulating test data in production over time. When cleanup is not feasible — payment flows, for example, where the record has downstream consequences — design the test to use a payment method and order flow that is recognizable as test activity and filters out cleanly in reporting.

What is the difference between production smoke testing and synthetic monitoring?

Synthetic monitoring runs scripted user flows against a production environment continuously to detect availability and performance issues, and it typically uses a dedicated synthetic monitoring service. Production smoke testing with a tool like TestInspector serves the same function but is configured and maintained by the QA team alongside the test suite — the same tool, the same test definitions, the same selector logic used for pre-production testing is extended to production monitoring. The practical difference is ownership: synthetic monitoring is often an ops or infrastructure function; production smoke testing using the QA team's existing tool is a QA function with a production targeting configuration.

How do you prevent a failing smoke test from blocking a critical deployment when the failure is caused by an external service?

Implement a two-tier failure policy in the CI/CD pipeline integration: tests that cover the core application paths block on failure; tests that cover third-party integrations report failure but do not block the deployment. Tag tests by tier in the test suite, and write the pipeline script to evaluate results by tag rather than treating suite pass/fail as a single binary. This requires identifying which failures are deployment-caused and which are infrastructure-caused before the smoke suite is deployed into the pipeline, which is a useful exercise for scoping the suite correctly in the first place.

How do you keep production smoke tests from becoming brittle over time?

Three practices maintain test stability: use stable selectors (IDs and data-test attributes rather than class names and DOM position), keep tests scoped to critical paths rather than expanding coverage incrementally, and review self-healing suggestions promptly when they fire. Self-healing suggestions indicate a selector that is drifting — the element still exists but its primary identifier has changed. Updating the test after the first self-healing event prevents the accumulating drift that makes tests brittle. Quarterly suite reviews that prune tests covering retired features and add tests for new critical paths keep the suite current without letting it grow beyond the intended scope.

Can TestInspector smoke tests run against multiple environments in the same run?

Not in a single run — a run targets the base URL configured for the suite or supplied at trigger time via the CI/CD API. Teams that need to validate both a staging promotion and a production state run separate suites: a staging smoke suite triggered by the pre-promotion pipeline stage and a production smoke suite triggered after the promotion completes. The same test definitions are used in both suites; only the base URL and credential variables differ. This arrangement validates the application at both stages of the promotion pipeline without requiring a single run to target multiple environments.