testing-strategy.cspec
1 # Testing Strategy 2 # purpose: Test types, coverage, environments, property testing 3 # status: draft 4 # updated: 2026-01-05 5 6 # === TESTING PYRAMID === 7 pyramid: 8 unit_tests: 9 percentage: 70% 10 scope: Individual functions, modules 11 speed: Fast (<1ms per test) 12 isolation: No external dependencies 13 14 integration_tests: 15 percentage: 20% 16 scope: Module interactions, API contracts 17 speed: Medium (<1s per test) 18 dependencies: Database, mock services 19 20 e2e_tests: 21 percentage: 10% 22 scope: Full system behavior 23 speed: Slow (seconds to minutes) 24 dependencies: Full environment 25 26 # === TEST TYPES === 27 test_types: 28 unit: 29 location: src/**/tests.rs, src/**/*_test.rs 30 runner: cargo test 31 coverage_target: 80% 32 33 integration: 34 location: tests/integration/ 35 runner: cargo test --test '*' 36 setup: Test fixtures, mock services 37 38 property: 39 location: tests/property/ 40 framework: proptest 41 use_cases: 42 - Consensus invariants 43 - Token arithmetic 44 - Serialization round-trips 45 46 fuzz: 47 location: fuzz/ 48 framework: cargo-fuzz 49 targets: 50 - Transaction parsing 51 - Block validation 52 - RPC input handling 53 54 benchmark: 55 location: benches/ 56 framework: criterion 57 metrics: [throughput, latency, memory] 58 59 contract: 60 location: tests/contracts/ 61 purpose: API compatibility 62 approach: Consumer-driven contracts 63 64 # === COVERAGE REQUIREMENTS === 65 coverage: 66 minimum: 67 overall: 70% 68 consensus: 90% 69 cryptography: 95% 70 api: 80% 71 72 tools: 73 rust: cargo-tarpaulin, llvm-cov 74 report: HTML, LCOV 75 76 exceptions: 77 - Generated code 78 - FFI bindings 79 - Debug/display implementations 80 81 # === TEST ENVIRONMENTS === 82 environments: 83 local: 84 purpose: Developer testing 85 setup: cargo test 86 data: In-memory, fixtures 87 88 ci: 89 purpose: Automated verification 90 runner: Forgejo Actions 91 parallel: Yes (8 concurrent) 92 artifacts: Test reports, coverage 93 94 testnet: 95 purpose: Integration, validator testing 96 persistence: Long-running 97 data: Testnet tokens (faucet) 98 99 staging: 100 purpose: Pre-production validation 101 data: Production-like 102 isolation: Separate network 103 104 # === CI TEST PIPELINE === 105 ci_pipeline: 106 stages: 107 1_lint: 108 - cargo fmt --check 109 - cargo clippy -- -D warnings 110 duration: ~30s 111 112 2_unit: 113 - cargo test --lib 114 - cargo test --bins 115 duration: ~2min (warm) 116 117 3_integration: 118 - cargo test --test '*' 119 duration: ~3min (warm) 120 121 4_property: 122 - cargo test --features property-tests 123 iterations: 100 per property 124 duration: ~2min 125 126 5_coverage: 127 - cargo tarpaulin 128 threshold: 70% 129 duration: ~5min 130 131 total_duration: 132 cold: ~15min 133 warm: ~6min (sccache) 134 135 # === CONSENSUS TESTING === 136 consensus_tests: 137 invariants: 138 - Total supply constant (no inflation/deflation bugs) 139 - State transitions deterministic 140 - Fork choice consistent 141 - Finality irreversible 142 143 scenarios: 144 normal_operation: 145 - Block production 146 - Transaction inclusion 147 - State updates 148 149 byzantine: 150 - Double voting 151 - Equivocation 152 - Network partition 153 154 edge_cases: 155 - Empty blocks 156 - Maximum block size 157 - Epoch boundaries 158 159 # === CROSS-CHAIN TESTING === 160 cross_chain_tests: 161 ipc_message: 162 - Lock attestation flow 163 - Unlock attestation flow 164 - Governance outcome propagation 165 166 invariants: 167 - sAX supply == Locked AX 168 - Attestations verified before execution 169 - No double-spend across chains 170 171 failure_modes: 172 - Message delayed 173 - Message lost (retry) 174 - Invalid attestation 175 176 # === PROPERTY-BASED TESTING === 177 property_tests: 178 framework: proptest 179 180 examples: 181 token_arithmetic: 182 property: "Adding then subtracting equals original" 183 generators: [Amount, Account] 184 185 serialization: 186 property: "serialize(deserialize(x)) == x" 187 generators: [Transaction, Block, Message] 188 189 consensus: 190 property: "All honest nodes agree on finalized blocks" 191 simulation: Network with byzantine actors 192 193 configuration: 194 cases: 100 (CI), 1000 (nightly) 195 shrinking: enabled 196 seed: reproducible 197 198 # === FUZZING === 199 fuzzing: 200 framework: cargo-fuzz (libFuzzer) 201 202 targets: 203 transaction_parser: 204 input: Raw bytes 205 crashes: Transaction parsing panics 206 207 block_validator: 208 input: Block bytes 209 crashes: Validation panics 210 211 rpc_handler: 212 input: JSON-RPC requests 213 crashes: Handler panics 214 215 corpus: 216 location: fuzz/corpus/ 217 seeding: Valid examples from tests 218 219 running: 220 ci: 10min per target 221 nightly: 1h per target 222 223 # === TEST DATA MANAGEMENT === 224 test_data: 225 fixtures: 226 location: tests/fixtures/ 227 format: JSON, binary 228 generation: From mainnet samples (anonymized) 229 230 factories: 231 purpose: Generate test objects 232 pattern: Builder pattern 233 randomization: Seeded for reproducibility 234 235 cleanup: 236 strategy: Each test cleans up after itself 237 isolation: Separate database per test (if needed) 238 239 # === FLAKY TEST POLICY === 240 flaky_tests: 241 definition: Test that fails intermittently 242 action: 243 1: Add to quarantine 244 2: Investigate root cause 245 3: Fix or delete 246 tolerance: 0% (flaky tests block merge) 247 tracking: Label in issue tracker 248 249 # === REGRESSION TESTING === 250 regression: 251 trigger: Bug fix PR 252 requirement: Test case covering the bug 253 location: tests/regression/ 254 naming: test_regression_{issue_number}