Skip to content

Test Plan

Purpose

The project-level test plan defines the testing strategy for the full project: test levels and scope, framework choices, coverage targets, critical paths, test data strategy, infrastructure requirements, and sequencing. It selects meaningful new or existing verification and provides traceability from requirements to test execution.

Example

Show a worked example of this artifact
---
ddx:
  id: example.test-plan.depositmatch
  authoring:
    home: repo
  depends_on:
    - example.prd.depositmatch
    - example.feature-specification.depositmatch.csv-import
    - example.user-story.depositmatch.upload-csv
    - example.contract.depositmatch.import-session-api
---

# Test Plan

## Strategy

**Goal**: Before implementation is accepted, prove CSV import, traceability, security boundaries, and the critical reviewer paths. P0 requirements and accepted contracts block merge when failing.

**Out of scope**: Bank feeds, accounting API sync, payroll, inventory, tax workflows, and match-scoring optimization beyond FEAT-001.

## Framework Selection & Cost

| Layer | Framework | Justification | CI Time Impact | Maintenance Burden |
|-------|-----------|---------------|----------------|--------------------|
| Contract | HTTP tests against Fastify with API-001 fixtures | Checks API behavior and problem-details errors independently of the UI | ~1 min | Low |
| Integration | API test runner, PostgreSQL 16 container, S3-compatible fake | Exercises transaction, storage, and repository boundaries together | ~3 min | Med (container startup) |
| Unit | Vitest | Fast feedback on parsing, validation, and UI state | <1 min | Low |
| E2E | Playwright, desktop browser | Only layer that proves the reviewer workflow and accessible upload controls | ~4 min | High (flaky if overused) |

**Cost/benefit summary**: About 9 minutes of CI. E2E is the costliest and most fragile, so it covers P0 happy and rejection paths only; detailed file cases are checked at the contract and integration layers.

**Baseline**: CI time measured at `3f9c2ab` (main); re-measure rather than edit.

## Critical Paths (P0)

1. Upload valid bank and invoice CSV files for one client.
2. Reject missing, oversized, or non-CSV files before parsing.
3. Preserve source file metadata for accepted uploads.
4. Keep raw financial row values out of analytics and application logs.
5. Open mapping review only after a draft session exists.

## Acceptance Criteria Layer Allocation

Every P0 acceptance criterion maps to exactly one primary test layer. Per-criterion rows (`US-<n>-AC<m>` to named tests) live in each story test plan; here, allocate criterion classes.

| AC class / source | Story Test Plan | Primary Layer | Why this layer |
|-------------------|-----------------|---------------|----------------|
| Upload acceptance and draft-session creation (US-001-AC1) | [[STP-001-upload-csv-files]] | Integration | API-001, source-file storage, and draft-session persistence must hold together |
| File rejection before parsing (US-001-AC2) | [[STP-001-upload-csv-files]] | Contract | API-001 problem-details errors are the observable boundary |
| Mapping review opens once both files land (US-001-AC3) | [[STP-001-upload-csv-files]] | E2E | Reviewer-visible navigation outcome |

## Secondary Paths (P1-P2)

- **P1**: saved mapping reuse, row-level validation, import summary confirmation.
- **P2**: large fixture performance, abandoned draft-session cleanup.

## Test Data Strategy

Versioned CSV fixtures: valid, missing amount column, malformed amount, duplicate source identifier, non-CSV upload. Factories for client, import session, and authenticated firm user. S3-compatible fake for storage; no bank or accounting API mocks in v1. No production financial data in fixtures.

## Known Gaps

Match suggestion accuracy waits for FEAT-002. Accessibility coverage starts with upload controls and expands with mapping and review stories. Fixtures are synthetic until pilot exports can be anonymized.

## Build Commands

```bash
pnpm test
pnpm test:coverage
pnpm test:e2e
```

**Blocking gate**: All P0 contract, integration, security, and E2E tests pass, and no raw financial fixture values appear in logs.

## Diagnostic Test Strategy

The diagnostic adoption slice allocates integration tests to actual OTel
receiver/mapping and trace correlation, security tests to pre-sink privacy,
and failure tests to capture/export bounds and query rotation. A service and
CLI pilot will measure diagnosis accuracy, evidence citations, time/calls/context
and overhead against agreed budgets. The slice needs its own governing story,
Contract and Story Test Plan; this example does not claim those tests ran.

Reference

ActivityTest — Define how we know it works. Plans, suites, and procedures that bind specs to implementation.
Default locationdocs/helix/03-test/test-plan.md
RequiresNone
EnablesNone
InformsImplementation Plan
Referenced byPRD
HELIX documentsdocs/helix/03-test/test-plans/TP-014-helix-workflow-coverage.md
Generation prompt
Show the full generation prompt
# Test Plan Generation Prompt

Scope: project-level verification strategy — goal, framework choices with cost, critical paths, data strategy, known gaps, and build commands.

Related:
- [Story Test Plan](../story-test-plan/prompt.md) — per-story AC↔test traceability
- [Test Suites](../test-suites/prompt.md) — suite inventory and boundaries under `tests/`
- [Test Procedures](../test-procedures/prompt.md) — runner procedures, commands, evidence capture

## Reference Anchors

Use these local resource summaries as grounding:

- `docs/resources/google-test-sizes.md` grounds test levels by scope,
  isolation, dependencies, and CI enforcement.
- `docs/resources/fowler-practical-test-pyramid.md` grounds balanced coverage
  across fast focused tests and fewer broad end-to-end tests.

## Storage Location

`docs/helix/03-test/test-plan.md`

## What to Include

- the goal: what must be true to ship, and what is out of scope
- framework choice per layer, with why, CI time, and maintenance burden
- critical paths (P0)
- acceptance-criteria **layer allocation**: map each in-scope `US-<n>-AC<m>`
  criterion class to one primary layer and confirm every P0 is allocated (the
  STP owns the per-AC rows)
- test data strategy
- known gaps and accepted risks
- build commands and the blocking gate

Do not hand-maintain test counts or timings. If you cite a baseline, pin it to a commit.

## Keep In Mind

- tests encode governing behavior and may be consolidated as implementation improves
- every test should trace to a requirement or story
- put effort where failure costs most; reuse existing coverage and add tests for distinct gaps
- assess test maintenance cost, including setup, test code volume, runtime and assertion quality; use coverage percentages only with their measured scope and limitations
- do not add generic testing doctrine that the template already implies

## Boundary Test

| If you are writing... | Put it in... |
|---|---|
| Overall strategy, framework choices, critical paths, CI gates | Test Plan |
| One story's concrete test cases and fixtures | Story Test Plan |
| Feature behavior or acceptance criteria | Feature Specification or User Story |
| Implementation sequencing for code changes | Implementation Plan |

Use template at `workflows/activities/03-test/artifacts/test-plan/template.md`.

## Boundary Verification

For handwritten source, apply `modularity-and-encapsulation`. Name the actual
project-local dependency checker command and exercise permitted imports and a
forbidden import that must fail; cover cycles/private access where supported.
Verify invariant protection through public APIs, review public vendor-type leakage,
and test core isolation/substitution when promised by the selected style. Record
unsupported static checks as named semantic review obligations; a presence check
or green import graph does not prove encapsulation. Existing-debt baselines may
allow unchanged violations but must reject new violations.

## Diagnostic Test Strategy

When `o11y-otel` applies, allocate the template's diagnostic test layers and
project-owned budgets. Require real receiver evidence and privacy/failure proof;
put exact assertions and runnable tests in Story Test Plans. Use the pilot in
`workflows/references/agent-diagnostics.md`; distinguish planned proof from
executed measurements. Service-only expectations are conditional on actual
surfaces, not on whether an agent edits the project.
Template
Show the template structure
---
ddx:
  id: TP-XXX
  authoring:
    home: repo
---

# Test Plan

## Strategy

**Goal**: [What must be true for the feature to be production-ready?]

**Out of scope**: [Behaviors not covered by this plan]

## Framework Selection & Cost

| Layer | Framework | Justification | CI Time Impact | Maintenance Burden |
|-------|-----------|---------------|---------------|--------------------|
| Contract | [Framework] | [Why this framework for API contract tests] | [~] | [~] |
| Unit | [Framework] | [Why this framework for unit tests] | [2min / ~neutral / +30%] | [Low/Med/High] |
| Integration | [Framework] | [Why this framework for integration] | [~] | [~] |
| E2E | [Framework] | [Why this framework for e2e] | [~] | [~] |

**Cost/benefit summary**: [Total CI time impact, maintenance vs. confidence tradeoff, decision rationale]

**Baseline**: [Only if you cite a measurement: what was measured and the commit it was taken at. Never hand-maintain counts; re-measure.]

## Critical Paths (P0)

These must pass before shipping:

1. [Path description]
2. [Path description]
3. [Path description]

## Acceptance Criteria Layer Allocation

Every P0 acceptance criterion maps to exactly one primary test layer. Per-criterion rows (`US-<n>-AC<m>` to named tests) live in each story test plan; here, allocate criterion classes.

| AC class / source | Story Test Plan | Primary Layer | Why this layer |
|-------------------|-----------------|---------------|----------------|
| [e.g. upload/validation criteria] | [[STP-XXX]] | [Integration] | [what boundary it proves] |

## Secondary Paths (P1-P2)

- **P1**: [Feature coverage]
- **P2**: [Edge cases]

## Test Data Strategy

[How do you seed test data? Static fixtures, factories, production snapshots?]

## Known Gaps

[Accepted risks, deferred coverage, infrastructure constraints]

## Build Commands

```bash
[test command]
[coverage command]
```

**Blocking gate**: [What must pass before the build is considered complete]

<!-- For source projects, name the real boundary-check command, allowed/forbidden
import controls, supported cycle/private-access checks, invariant tests and
remaining semantic review obligations. Apply the modularity-and-encapsulation
baseline; do not equate populated document fields with proven boundaries. -->

## Diagnostic Test Strategy (when applicable)

For selected `o11y-otel` practices, allocate layers and budgets for real OTel
receiver/mapping/correlation proof, duplicate ingestion, pre-sink privacy,
bounded capture/export failures and authorized retrieval. Define service/CLI
agent-pilot ground truth and diagnosis/evidence/time/call/context measurements
against a baseline. Exact tests, commands and story AC rows belong in Story
Test Plans. JSONL parsing alone does not establish an OTel integration.
Innsigle seal: model-primary by HELIX

The signature covers the markdown source of this page, not these HTML bytes. This page quotes that seal; verify it against the source file.

Composition
model-primary
Issuer
HELIX helix
Signing key
ed25519:b0865d76d834a52c48506414d16f4e5a (build key)
Signed source
artifact-types/test/test-plan.md
Signed
2026-11-16T15:52:45Z
Content digest
sha256:4efc218d…17418d27

This build key is endorsed by the human key for build signing; the signature is not a detector and not a truth guarantee.

Raw attestation JSON
{
  "payload": {
    "innsigle": "1",
    "type": "https://innsigle.dev/claim/colophon/v1",
    "issued_at": "2026-11-16T15:52:45Z",
    "issuer": {
      "id": "helix",
      "name": "HELIX",
      "key_id": "ed25519:b0865d76d834a52c48506414d16f4e5a",
      "key_url": "https://documentdrivendx.github.io/helix/.well-known/innsigle/keys.json"
    },
    "subjects": [
      {
        "uri": "https://documentdrivendx.github.io/helix/artifact-types/test/test-plan/",
        "digest": {
          "alg": "sha256",
          "value": "4efc218d14b6b77e9e57741fa324891053d6019aa967a6eb1cb391ae17418d27"
        }
      }
    ],
    "colophon": {
      "schema_version": "1",
      "composition": "model-primary",
      "ingredients": [
        {
          "kind": "model",
          "name": "Claude",
          "role": "draft"
        },
        {
          "kind": "tool",
          "name": "sloptimizer",
          "role": "rewrite"
        },
        {
          "kind": "human",
          "name": "operator",
          "role": "structure-edit"
        }
      ],
      "notes": null
    }
  },
  "payload_encoding": "json",
  "signatures": [
    {
      "key_id": "ed25519:b0865d76d834a52c48506414d16f4e5a",
      "alg": "ed25519",
      "sig": "LVXwCef0RAKZ1jmiKkYEszHe8PK67whTbiERhEOWdQmr_Rvr71cv2mi_lxdrVU9E9AdTruA4Kd1oCu9nASz4Aw",
      "signed_at": "2026-11-16T15:52:45Z"
    }
  ]
}