Quality
QA coverage
How Test Coverage Autopilot measures a baseline, proposes tests-only tasks for approval and reports only measured coverage numbers.
Test Coverage Autopilot analyses a repository, measures its real test coverage, proposes tests-only tasks for the weakest high-risk areas, and runs the ones you approve through the normal review and approval workflow. It lives in the QA Center in PWR Studio.
Its central rule: a coverage percentage appears only when a parsed coverage report produced it. When coverage cannot be measured, you see the reason instead of a number.
Status: Implemented and verified end to end on local fixtures with a real
pytest --covrun and a simulated GitHub. It has not run against a live GitHub repository. Only the Python runner has been executed; see the runner table below.
The stages
| Stage | What happens |
|---|---|
| 1. Analysis | A read-only scan: languages, test frameworks, existing tests, CI configuration, a module map and high-risk components. No coverage number is produced here. |
| 2. Baseline | A worker checks out the code fresh, runs the coverage command in the sandbox with no secrets, and parses the report. |
| 3. Strategy | Files are ranked by risk × (1 − coverage) × size, and one task is proposed per production file below target, each with a credit estimate. |
| 4. Implementation | You approve task keys. Approved tasks run as a goal with a budget, each producing its own pull request. |
| 5. Measurement | Each task branch is measured at its head and compared with the baseline. |
A run moves through analyzing → baseline → awaiting_approval → executing → measuring → completed, or ends failed or cancelled.
Starting a run
Open QA Center, pick a project and choose Start Coverage Autopilot. Developers and above can start runs. The project needs a workspace source; otherwise the start is refused with 409 source_unavailable.
Options include a target percentage (default 80), include and exclude paths, a unit or integration emphasis, branch coverage, a budget in USD (default 10), parallelism (1 to 5), a maximum number of tasks (default 8) and an audit-only mode that plans without proposing execution.
The run uses the agent you choose, or your organization's QA / SDET agent, which is created if missing.
Tests only, enforced
Every approved task carries a scope: test paths are allowed, the module's production files are forbidden. Before anything is pushed, a guard checks the actual change and blocks it when:
- a production file changed, or a file outside the allowed paths changed;
- a test file was deleted;
- a modified test file lost more assertion lines than it gained;
- coverage exclusion markers were added, such as
pragma: no cover,istanbul ignoreorExcludeFromCodeCoverage; - coverage configuration was edited, such as
.coveragercorcoverageThreshold.
A violation stops the workflow at push_failed with a scope violation message. Nothing leaves the worker.
The assertion check is a line-count heuristic. A rewrite that keeps the same number of assertion lines but loosens them passes the guard. The independent review still sees the full diff.
Measured, heuristic and estimated
| What you see | Nature |
|---|---|
| Baseline and per-branch line and branch coverage | Measured from a parsed report, or empty with a reason |
| High-risk components, import fan-in, ranking score | Heuristic and labelled as such |
| Credits per proposed task, remaining effort | Estimated as a range |
| Credits consumed | Measured from metered usage |
Per-branch results are never summed, because the union of coverage across branches cannot be computed honestly. The only combined number is a measurement of the default branch after merges, which you trigger explicitly.
Runner support
| Stack | State |
|---|---|
| Python with pytest and pytest-cov | Verified end to end with a real run |
| Node with Jest, Vitest or Mocha + c8 | Report parsers verified on sample reports; the runner itself has not been executed |
| .NET with xUnit, NUnit or MSTest + coverlet | Report parser verified on a sample; the runner itself has not been executed |
| Go | Detected; coverage profile is not parsed, so reported as unavailable |
| Unity, Godot | Detected as a limitation; needs an engine host and is never run |
The Python command is:
python -m pytest --cov=<pkgs> --cov-branch --cov-report=xml --cov-report=json --junitxml=junit.xml -q
The runner must be installed in the workspace. If it is not, the measurement reports runner unavailable with the reason rather than a number.
Continuous policy
Per project you can set a coverage threshold that is checked against each pull request head's measured coverage. On a breach PWR can notify you and draft a tests-only task, within a monthly credit budget you set. The policy never merges, approves or blocks anything.
Human gates stay in place
Approving a plan creates tasks and a goal. Every pull request still goes through validation, independent review, human approval and the merge gate described in GitHub integration.
Limitations
- The optional model-refined plan is fail-closed: if the model's reply is malformed, the deterministic plan stands. It has not been exercised against a live model.
- Creating Jira tickets for coverage gaps is not supported, because the Jira connector cannot create issues. PWR tasks are always created.
- Optional mutation testing uses
mutmutfor Python only and is off by default. Output is recorded as-is and no score is derived. - Risk ranking is a heuristic over file size, dependents, error handling and security-sensitive names.
Game projects
For game projects the QA Center also shows the engine profile, the capability status per engine and capture evidence. See Unity setup and Unreal setup.