Testing Strategy

Spyrl's riskiest failures are wrong money (payroll, statutory deductions), wrong data boundaries (one tenant seeing another's data) and lost presence data (punches). The strategy spends most effort on those three. It implements the CI stages in §11 of the architecture and the pipeline in deployment.

1. Test pyramid by layer

Layer	Tooling	What it covers	Runs	Share of tests
Pure unit	Vitest	packages/payroll-engine, packages/policy evaluators, date and rounding utilities, formula language	Every PR, under 2 min	~60%
Property-based	Vitest + fast-check	Leave and attendance engines, payroll invariants	Every PR (100 runs per property); nightly (10,000 runs)	~5%
Golden files	Vitest snapshot of structured outputs	Payroll and statutory file generation for fixed scenarios	Every PR	~5%
Module integration	Vitest + Testcontainers (Postgres 16, Redis)	Repositories, RLS, migrations, outbox, BullMQ jobs, module public interfaces	Every PR	~20%
Contract	zod ↔ OpenAPI diff, provider fakes	Web/mobile/API contracts, public API compatibility, connector adapters	Every PR	~5%
E2E web	Playwright	Critical journeys across roles	PR smoke on preview; full suite on staging	~3%
E2E mobile	Maestro (primary), Detox for flows needing native control	Punch, offline queue, approvals, kiosk	Staging nightly and per release	~1%
Non-functional	k6, ZAP, axe-core, Lighthouse	Load, security, accessibility, performance	Staging per release; load tests monthly and before payroll-sensitive releases	—

Rules: a bug fix ships with a test at the lowest layer that would have caught it; E2E tests cover journeys, not branches; flaky tests are quarantined within a day and fixed or deleted within a week.

2. Payroll golden-file tests

The payroll engine is a pure, deterministic library (architecture §2.1). Each golden scenario is a YAML input (employee, compensation, attendance, statutory pack version, declarations) and an expected output (every component, deduction and employer contribution, with rounding). Payroll SMEs author and review scenarios; engineers never "update the snapshot" without SME approval in the PR.

The statutory values below are examples to verify against the statutory pack in force; they are not the source of truth.

# packages/payroll-engine/golden/epf-ceiling-basic-above-15000.yaml
scenario: EPF on wage ceiling, basic above ceiling, contribution on ceiling
statutory_pack: IN-2026.04
employee: { state: KA, epf_member: true, eps_eligible: true, esi_applicable: false }
month: 2026-06
payable_days: 30
days_in_month: 30
earnings: { basic: 25000, hra: 10000, special: 8000 }       # gross 43,000
expect:
  epf_wages: 15000
  employee_epf: 1800          # 12% of 15,000
  employer_eps: 1250          # 8.33% of 15,000 = 1,249.50 → rounded 1,250
  employer_epf: 550           # 1,800 − 1,250
  edli: 75                    # 0.5% of 15,000
  epf_admin: 75               # 0.5% of 15,000 (verify current admin charge)
  pt: 200                     # Karnataka, gross ≥ 25,000 (verify)

# packages/payroll-engine/golden/esi-below-ceiling.yaml
scenario: ESI applicable, gross below ceiling
statutory_pack: IN-2026.04
employee: { state: MH, esi_applicable: true, gender: M }
month: 2027-02
earnings: { basic: 10000, hra: 4000, special: 4000 }        # gross 18,000
expect:
  esi_wages: 18000
  employee_esi: 135           # 0.75%, rounded up to the next rupee per ESIC rule (verify)
  employer_esi: 585           # 3.25%
  pt: 300                     # Maharashtra February instalment (verify)

# packages/payroll-engine/golden/tds-new-regime-15l.yaml
scenario: TDS under the new regime, annual taxable salary 15,00,000, no other income
statutory_pack: IN-2026.04       # FY 2026-27 slabs assumed unchanged from FY 2025-26 (verify)
regime: new
annual_gross_salary: 1500000
expect:
  standard_deduction: 75000
  taxable_income: 1425000
  tax_before_cess: 93750      # 4–8L 5% = 20,000; 8–12L 10% = 40,000; 12–14.25L 15% = 33,750
  rebate_87a: 0               # income above 12,00,000
  cess: 3750                  # 4%
  annual_tax: 97500
  monthly_tds_flat: 8125      # 97,500 / 12 when projected evenly from April

Scenario catalogue (minimum for R1): EPF below and above ceiling, voluntary higher contribution, EPS ineligible (joined after 58 or above ceiling at joining), ESI entry and exit by contribution period, PT for every supported state including February and half-yearly regimes, LWF per state and frequency, LOP days and mid-month joiners/leavers, arrears from a back-dated revision, TDS old vs new regime with declarations and a mid-year regime switch, previous-employer income, rounding at every boundary, and a statutory pack change mid-year. Statutory file generators (ECR, ESIC, PT) have golden files byte-compared with SME-approved fixtures.

When a statutory pack is published, its golden scenarios are part of the pack review (upgrades).

3. Property-based tests (leave and attendance)

Example-based tests miss the edge combinations these engines hit in production (night shifts, sandwich rules, mid-period policy changes). Properties use fast-check generators for policies, calendars, punches and requests.

Engine	Property
Leave	Balance always equals the sum of ledger entries (no hidden state)
Leave	Applying and then cancelling a request returns every balance to its previous value
Leave	Accrual for a period is the same whether computed in one run or in daily increments
Leave	A balance never goes below zero unless the policy allows negative balance, and never below the allowed limit
Leave	Sandwich rule is idempotent: recomputing a request's chargeable days gives the same result
Leave	An effective-dated policy change only affects days on or after its effective date
Attendance	Day computation is deterministic: same inputs give the same day status and hours
Attendance	Punch ingestion order does not matter (shuffled punches give the same result)
Attendance	Ingesting the same punch twice changes nothing (deduplication)
Attendance	Worked hours for a shift date are between 0 and 24, and a shift crossing midnight is attributed to its shift date
Attendance	Payable days in a month = present + paid leave + paid holidays + paid week-offs, and never exceed days in the month
Payroll	Net = gross − deductions exactly (decimal arithmetic, no floats); recomputing a locked run's snapshot gives identical lines

Failing seeds found nightly are saved as regular example tests.

4. Tenant-isolation and RLS tests

Specified in the security overview; this is how they run.

Test	Layer	Gate
RLS coverage (every table with tenant_id has forced RLS and a policy)	Integration	Every PR; blocks merge
App DB role is NOBYPASSRLS and not a table owner	Integration	Every PR
Repository fuzz: tenant B context with tenant A IDs returns nothing, for every repository method (generated from the module registry)	Integration	Every PR
Missing context fails closed	Integration	Every PR
API cross-tenant tests: every route with A's IDs and B's token gives 404	E2E (API)	Staging per release
Authorisation matrix tests: for each permission key and seeded role, allow inside scope and deny outside, generated from the RBAC matrix source	Integration	Every PR
Cache, queue and storage key prefix checks	Integration	Every PR
Production canary tenant check	Synthetic	Every 15 min per pooled cell

5. Contract tests

• Shared contracts: packages/contracts (zod) is the source for API request and response types. OpenAPI 3.1 is generated from it; CI diffs the generated spec against main and fails on breaking changes to /v1 (removed fields, narrowed types, new required inputs) unless the PR is labelled for a versioned change (API overview).
• Web and mobile compile against the same contracts; mobile additionally runs a compatibility test against the API from the oldest supported app version's contract snapshot.
• Webhooks: payload schemas per event type and version, with example payloads tested against the schema.
• Provider adapters: each connector has a fake that replays recorded sandbox responses (SES, MSG91, Rekognition, Razorpay, Zoho) and a nightly "live sandbox" test where the provider offers one (integrations).

6. End-to-end tests

Web (Playwright), against preview (smoke) and staging (full), with seeded tenants and one browser context per role:

Journey	Roles
Owner activation → MFA → setup wizard → employee import (dry-run, fix, commit)	org_owner
Apply leave → manager approves → balance updates → calendar shows it	employee, line_manager
Regularisation request and approval; attendance period lock	employee, line_manager, hr_manager
Payroll run: create → calculate → review variance → approve (different user) → lock → payslip visible → ECR and bank file generated	payroll_manager, finance_admin, employee
Segregation of duties: preparer cannot approve	payroll_manager
Support access request → owner consent → audited session → tenant sees access log	Platform support_engineer, org_owner
Claims submit → approve → finance verify → payout via payroll	employee, line_manager, finance_admin

Mobile (Maestro first, Detox where needed): login by company code, punch in with selfie inside and outside the geo-fence, offline punch queued and synced after reconnect (airplane mode), approval from a push notification, kiosk enrolment and punch, minimum-version 426 handling, face consent and enrolment (R2), tracking start/stop with punch (R3). Runs on an Android emulator in CI and on a small real-device farm before each store release (low-end Android devices common among field staff are included).

7. Load tests (k6) for month-end peaks

Scenario	Model	Pass thresholds
Morning punch peak	30k employees per cell punching within 30 min; peak 5× average (about 85 punches/s), mixed mobile and device ingest	p95 punch acceptance under 2 s; error rate under 0.1%
Payroll month-end	50 tenants start runs in the same hour (sizes from 20 to 1,000 employees) while normal traffic continues	1,000-employee run under 5 min; API p95 within SLO for other tenants (noisy-neighbour check)
Payslip fan-out	30k payslips published and notified within 1 h	Notification p95 under 60 s from outbox; no DLQ growth
Report exports	100 concurrent large exports	Workers do not starve payroll queues (group concurrency)
GPS ingest (R3)	5,000 tracked users, one point per 60 s, with bursts after offline sync	Ingest lag under 5 min; DB write rate within trigger

Load tests run against staging sized like a production cell, with synthetic data at production volume, monthly and before any payroll-sensitive release. Results are kept to track regressions; capacity triggers are cross-checked (observability).

8. Security testing

SAST, dependency, secret and container scanning on every PR; ZAP baseline on staging per release and an authenticated scan monthly; the authorisation matrix and isolation suites above; abuse-case tests (rate limits, lockout, invite token reuse, refresh-token reuse detection, CSV formula injection, upload type spoofing, webhook SSRF targets such as 169.254.169.254); and external pentests on the cadence in the security overview.

9. Accessibility testing

Target WCAG 2.1 AA for web and mobile.

• axe-core checks inside Playwright journeys (fail on serious and critical violations).
• Storybook components tested with the axe add-on; colour tokens checked for contrast in light and dark themes.
• Manual checks per release for new screens: keyboard-only navigation, NVDA (web) and TalkBack/VoiceOver (mobile), zoom to 200%, and touch targets of at least 44×44 pt.
• Kiosk and punch screens are tested for one-handed use, glare and low-literacy use (icons plus text, local language labels as they ship).

10. Test data management

• Synthetic only. Real client data is never copied into any non-production environment, including UAT. Debugging a client issue uses support access in production or a synthetic reproduction.
• Generators (Faker with the Indian locale plus our own) create tenants, org structures, employees, shifts, punches and payroll history at chosen volumes.
• Identifiers are valid-format but unissued: PAN-format values with a reserved ZZZZZ prefix, Aadhaar-format values starting with 0 or 1 (UIDAI does not issue these; verify) with a valid Verhoeff checksum, bank accounts with an invalid test IFSC range, and mobile numbers in a reserved test range. Production validators reject these patterns unless the tenant is flagged is_test.
• Golden scenarios are hand-authored and versioned with the engine.
• Seeds are deterministic (fixed random seed) so failures reproduce.

11. Environments

Environment	Used for
Local / CI containers	Unit, property, integration, contract tests
Preview (per PR)	Playwright smoke, product review
Dev	Exploratory testing on main
Staging	Full E2E, load, ZAP, migration rehearsal on production-like volume
UAT	Pilot-client acceptance, QA sign-off with client configuration and synthetic people data

See deployment for how each is hosted.

12. Entry and exit criteria per release

Entry to staging (release candidate cut)

• All PR checks green on the candidate digest; no quarantined test in payroll, isolation or auth suites.
• Migrations follow the expand/contract checklist.
• New permissions are in the RBAC matrix; new data fields are classified.

Exit to production (wave 0)

• Full Playwright and Maestro suites green on staging.
• Golden and property suites green (nightly run on the candidate).
• No open critical or high security findings without risk acceptance.
• k6 smoke within thresholds; full load test passed if the release is payroll-sensitive or touches ingest.
• axe: no new serious/critical violations.
• Payroll SME sign-off for payroll-sensitive changes; release notes reviewed.
• Rollback note present for each migration.

13. UAT with pilot clients

Item	Approach
Pilots	3–5 design-partner tenants per release theme (for example a 50-person office, a 300-person factory with biometrics, a field sales team for R3)
Where	UAT environment with the pilot's real configuration (policies, structures) and synthetic people; or, after go-live, feature flags enabled for the pilot in production
Scripts	Role-based scenario scripts derived from the user stories and acceptance criteria in the PRD
Payroll	A parallel run against the pilot's legacy payroll (client onboarding) is the payroll UAT
Feedback	In-app feedback widget tagged with the release; weekly triage
Sign-off	Pilot sign-off per theme: no open severity 1–2 defects, agreed workarounds for severity 3