dependency-architecture-refactor.md
1 # Dependency Architecture Refactor 2 3 **Status**: Complete 4 **Scope**: Workspace-wide restructuring of crate dependencies, test utilities, and publish pipeline 5 **Breaking changes**: Yes (pre-launch, acceptable) 6 7 --- 8 9 ## Problem Statement 10 11 Publishing any crate to crates.io requires manually removing dev-dependencies, inlining test helpers, and publishing with `--no-verify` or `--allow-dirty`. This is because: 12 13 1. **`auths-test-utils` is a monolith** that depends on 7 workspace crates (`auths-core`, `auths-crypto`, `auths-id`, `auths-storage`, `auths-sdk`, `auths-telemetry`, `auths-verifier`). Any crate that dev-depends on it cannot publish until it's on crates.io — but it can't be published until all its dependencies are. 14 15 2. **`auths-id` ↔ `auths-storage` circular dev-dependency**: `auths-storage` depends on `auths-id` (for traits), `auths-id` dev-depends on `auths-storage` (for testing with real Git backend). Neither can publish first. 16 17 3. **`auths-id` has a `git-storage` feature** that pulls in `git2`, `dirs`, `tempfile`, `tokio` — mixing domain logic with infrastructure concerns. Storage implementation code is split between `auths-id` and `auths-storage`. 18 19 4. **No automated publish ordering** — manual `sleep 60` between publishes, fragile and error-prone. 20 21 --- 22 23 ## Principles 24 25 1. **Dependency flow is strictly downward.** Foundation → Domain → Infrastructure → Orchestration → Presentation. No reverse dependencies, not even dev-deps pointing upward. 26 2. **Each crate owns its own test helpers.** Feature-gated `test-utils` modules replace the monolithic test-utils crate. This is the pattern used by reth (150+ crates), alloy, and tokio. 27 3. **Traits live with their domain, implementations live in infrastructure.** `auths-id` defines what storage looks like; `auths-storage` provides the implementations. Tests in `auths-id` use in-memory fakes, not real backends. 28 4. **Contract tests live with the trait they verify.** Exported as macros so implementations can pull them in. 29 30 --- 31 32 ## Target Architecture 33 34 ``` 35 Layer 0 — Foundation (no workspace deps) 36 ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ ┌─────────────┐ 37 │ auths-crypto│ │ auths-policy │ │ auths-telemetry │ │ auths-index │ 38 └─────────────┘ └──────────────┘ └─────────────────┘ └─────────────┘ 39 40 Layer 1 — Domain (depends only on Layer 0) 41 ┌──────────────────┐ ┌──────────┐ 42 │ auths-verifier │ │ auths-id │ 43 │ (crypto) │ │ (crypto, policy, verifier) 44 └──────────────────┘ └──────────┘ 45 46 Layer 2 — Infrastructure (depends on Layer 0 + 1) 47 ┌────────────────┐ ┌────────────────┐ ┌──────────────────┐ 48 │ auths-storage │ │ auths-infra-git│ │ auths-infra-http │ 49 │ (id, core, │ │ (core, sdk, │ │ (core, verifier) │ 50 │ verifier) │ │ verifier) │ │ │ 51 └────────────────┘ └────────────────┘ └──────────────────┘ 52 53 Layer 3 — Orchestration (depends on all above) 54 ┌───────────┐ 55 │ auths-sdk │ 56 │ (core, id, policy, crypto, verifier) 57 └───────────┘ 58 59 Layer 4 — Presentation (depends on all above) 60 ┌───────────┐ 61 │ auths-cli │ 62 └───────────┘ 63 ``` 64 65 **Key change**: No arrows point upward. No dev-dependencies cross layer boundaries upward. 66 67 --- 68 69 ## Phase 1: Distribute test utilities into per-crate `test-utils` features 70 71 This is the highest-impact change. It eliminates the `auths-test-utils` monolith and all circular dev-dependency issues. 72 73 ### 1a. `auths-crypto` — add `test-utils` feature 74 75 Move from `auths-test-utils/src/crypto.rs`: 76 - `create_test_keypair(seed: &[u8; 32]) -> (Ed25519KeyPair, [u8; 32])` 77 - `get_shared_keypair() -> &'static [u8]` 78 - `gen_keypair() -> Ed25519KeyPair` 79 80 ```toml 81 # auths-crypto/Cargo.toml 82 [features] 83 test-utils = ["dep:ring"] # ring is already an optional dep 84 ``` 85 86 ```rust 87 // auths-crypto/src/testing.rs 88 #[cfg(feature = "test-utils")] 89 pub mod testing { 90 use ring::signature::{Ed25519KeyPair, KeyPair}; 91 use std::sync::OnceLock; 92 93 pub fn create_test_keypair(seed: &[u8; 32]) -> (Ed25519KeyPair, [u8; 32]) { 94 let keypair = Ed25519KeyPair::from_seed_unchecked(seed).unwrap(); 95 let public_key: [u8; 32] = keypair.public_key().as_ref().try_into().unwrap(); 96 (keypair, public_key) 97 } 98 99 pub fn get_shared_keypair() -> &'static [u8] { /* OnceLock pattern */ } 100 pub fn gen_keypair() -> Ed25519KeyPair { /* random seed */ } 101 } 102 ``` 103 104 **Consumers**: Every crate that currently imports `auths_test_utils::crypto::*` switches to: 105 ```toml 106 [dev-dependencies] 107 auths-crypto = { workspace = true, features = ["test-utils"] } 108 ``` 109 110 ### 1b. `auths-id` — add `test-utils` feature 111 112 Move from `auths-test-utils/src/fakes/`, `contracts/`, `fixtures/`, `mocks/`, `storage_fakes.rs`: 113 114 **Fakes** (implement traits defined in `auths-id` itself — no cross-crate dependency needed): 115 - `FakeRegistryBackend` (implements `RegistryBackend`) 116 - `FakeAttestationSink` / `FakeAttestationSource` (implements `AttestationSink` / `AttestationSource`) 117 - `FakeIdentityStorage` (implements `IdentityStorage`) 118 - `InMemoryStorage` (implements `BlobReader`, `BlobWriter`, `RefReader`, `RefWriter`, `EventLogReader`, `EventLogWriter`) 119 - `MockClock` (implements `ClockProvider`) 120 - `MockCryptoProvider` (implements `CryptoProvider`) 121 - `DeterministicUuidProvider` (implements `UuidProvider`) 122 - `FakeGitDiagnosticProvider`, `FakeCryptoDiagnosticProvider` 123 - `FakeGitLogProvider` (implements `GitLogProvider`) 124 125 **Contract test macros**: 126 - `registry_backend_contract_tests!` 127 - `git_log_provider_contract_tests!` 128 - `session_store_contract_tests!` 129 - `event_sink_contract_tests!` 130 131 **Fixtures**: 132 - `test_inception_event(key_seed: &str) -> Event` 133 - `test_attestation(device_did, issuer) -> Attestation` 134 135 **Mockall mocks**: 136 - `MockIdentityStorage` 137 - `MockAttestationSource` 138 139 ```toml 140 # auths-id/Cargo.toml 141 [features] 142 test-utils = [ 143 "auths-crypto/test-utils", # chain the feature 144 "dep:mockall", 145 "dep:rand", 146 "dep:tempfile", 147 ] 148 ``` 149 150 ```rust 151 // auths-id/src/testing/mod.rs 152 #[cfg(feature = "test-utils")] 153 pub mod testing { 154 pub mod fakes; // FakeRegistryBackend, FakeAttestationSource, etc. 155 pub mod contracts; // contract test macros 156 pub mod fixtures; // test_inception_event, test_attestation 157 pub mod mocks; // MockIdentityStorage, MockAttestationSource 158 } 159 ``` 160 161 **Why this works**: All the fakes implement traits defined in `auths-id` itself. The mock implementations use only types from `auths-id` and its dependencies (Layer 0). No upward dependency on `auths-storage` or `auths-sdk`. 162 163 ### 1c. `auths-telemetry` — add `test-utils` feature 164 165 Move from `auths-test-utils/src/fakes/telemetry.rs`: 166 - `MemoryEventSink` (implements `EventSink`) 167 168 ```toml 169 # auths-telemetry/Cargo.toml 170 [features] 171 test-utils = [] 172 ``` 173 174 ### 1d. `auths-core` — expand existing `test-utils` feature 175 176 `auths-core` already declares `test-utils = []` as a feature. Populate it with any test helpers specific to core (if any exist beyond what's in `auths-crypto`). 177 178 ### 1e. Git test helpers — move to `auths-infra-git` 179 180 Move from `auths-test-utils/src/git.rs`: 181 - `init_test_repo() -> (TempDir, git2::Repository)` 182 - `get_cloned_test_repo() -> TempDir` 183 - `copy_directory(src, dst)` 184 185 ```toml 186 # auths-infra-git/Cargo.toml 187 [features] 188 test-utils = ["dep:tempfile"] 189 ``` 190 191 These are only needed by crates that test against real Git repositories. 192 193 ### 1f. Delete `auths-test-utils` 194 195 After all helpers are distributed, remove `crates/auths-test-utils/` entirely: 196 - Remove from workspace `members` in root `Cargo.toml` 197 - Remove from `[workspace.dependencies]` 198 - Remove all `auths-test-utils.workspace = true` lines from every crate 199 200 --- 201 202 ## Phase 2: Clean up `auths-id` — remove infrastructure dependencies 203 204 Currently `auths-id` has a `git-storage` feature that brings in `git2`, `dirs`, `tempfile`, `tokio`. This mixes domain logic with infrastructure. 205 206 ### 2a. Audit what `git-storage` feature provides in `auths-id` 207 208 Identify all code gated behind `#[cfg(feature = "git-storage")]` in `auths-id/src/`. This likely includes: 209 - Git-based `IdentityStorage` implementation 210 - Local `~/.auths` directory management 211 - Git ref reading/writing for identity data 212 213 ### 2b. Move git-storage code from `auths-id` to `auths-storage` 214 215 All Git-based storage implementations should live in `auths-storage`: 216 - Move the git-gated code to `auths-storage/src/git/` 217 - `auths-storage` already depends on `auths-id` — it can implement the traits 218 - Remove `git-storage` feature from `auths-id` 219 - Remove `git2`, `dirs`, `tempfile` from `auths-id`'s dependencies 220 221 ### 2c. Remove `auths-storage` dev-dependency from `auths-id` 222 223 After Phase 1, `auths-id` tests use in-memory fakes (from its own `test-utils` feature) instead of `GitRegistryBackend`. The real Git backend is tested in `auths-storage` using the contract test macros exported by `auths-id/test-utils`. 224 225 ```rust 226 // auths-storage/tests/cases/registry_contract.rs 227 // Import the contract test macro from auths-id 228 auths_id::testing::contracts::registry_backend_contract_tests!( 229 git_backend, 230 { /* construct GitRegistryBackend */ }, 231 ); 232 ``` 233 234 This is how reth does it: the trait crate exports contract tests, the implementation crate runs them. 235 236 ### 2d. Result — `auths-id` becomes a pure domain crate 237 238 After this phase, `auths-id`'s dependencies are: 239 ```toml 240 [dependencies] 241 auths-core.workspace = true 242 auths-crypto.workspace = true 243 auths-policy.workspace = true 244 auths-verifier.workspace = true 245 # ... plus pure Rust deps (chrono, serde, etc.) 246 # NO git2, NO dirs, NO tempfile, NO tokio 247 ``` 248 249 No dev-dependencies on infrastructure crates. Clean Layer 1 crate. 250 251 --- 252 253 ## Phase 3: Consolidate `auths-core` role 254 255 `auths-core` currently depends on `auths-crypto` and `auths-verifier`. It provides: 256 - Platform keychains (macOS, Linux, Windows) 257 - Agent/passphrase management 258 - Encryption primitives 259 - Config management 260 261 ### 3a. Evaluate whether `auths-core` should depend on `auths-verifier` 262 263 `auths-verifier` is designed as a minimal, embeddable crate. If `auths-core` pulls it in as a dependency, that adds `auths-core` to `auths-verifier`'s reverse dependency tree, which complicates the layer model. 264 265 If the dependency is only used in a few places, consider: 266 - Making it optional: `auths-verifier = { workspace = true, optional = true }` 267 - Or duplicating the minimal verification logic needed 268 269 ### 3b. Ensure `auths-core` stays at Layer 0 270 271 `auths-core` should only depend on `auths-crypto` (Layer 0). If it needs types from `auths-id`, that's a sign those types should be in a lower layer. 272 273 --- 274 275 ## Phase 4: Automate publishing 276 277 ### 4a. Adopt `cargo publish --workspace` (Rust 1.90+) 278 279 Since Rust 1.90 (September 2025), Cargo natively supports workspace publishing: 280 ```bash 281 cargo publish --workspace 282 ``` 283 284 This topologically sorts crates and publishes in dependency order. After Phases 1-3 eliminate all circular dev-deps, this works out of the box. 285 286 ### 4b. Consider `release-plz` for CI 287 288 For automated releases via GitHub PRs: 289 - Auto-generates changelogs from conventional commits 290 - Integrates `cargo-semver-checks` for breaking change detection 291 - Opens a Release PR, publishes on merge 292 - Handles `sleep` between publishes automatically 293 294 ### 4c. Define publish order explicitly 295 296 After the refactor, the publish order is deterministic: 297 ``` 298 Tier 0 (parallel): auths-crypto, auths-policy, auths-telemetry, auths-index 299 Tier 1 (parallel): auths-verifier, auths-core 300 Tier 2 (sequential): auths-id (after verifier, core) 301 Tier 3 (parallel): auths-storage, auths-infra-git, auths-infra-http 302 Tier 4: auths-sdk 303 Tier 5: auths-cli 304 ``` 305 306 No tier depends on a crate in the same or later tier. No circular dependencies. 307 308 --- 309 310 ## Phase 5: Cleanup and verification 311 312 ### 5a. Remove all temporary inlined helpers 313 314 Remove the `create_test_keypair` functions that were inlined in: 315 - `auths-crypto/tests/cases/provider.rs` 316 - `auths-verifier/src/verify.rs` 317 - `auths-verifier/src/witness.rs` 318 - `auths-verifier/tests/cases/expiration_skew.rs` 319 - `auths-verifier/tests/cases/revocation_adversarial.rs` 320 321 Replace with: 322 ```rust 323 use auths_crypto::testing::create_test_keypair; 324 ``` 325 326 ### 5b. Re-add dev-dependencies that were removed for publishing 327 328 Restore any dev-deps that were stripped purely for the initial publish (e.g., `auths-storage` in `auths-id` — though after Phase 2, this should no longer be needed). 329 330 ### 5c. Full workspace verification 331 332 ```bash 333 cargo fmt --check --all 334 cargo clippy --all-targets --all-features -- -D warnings 335 cargo nextest run --workspace 336 cargo test --all --doc 337 cargo publish --workspace --dry-run 338 ``` 339 340 ### 5d. WASM verification 341 342 ```bash 343 cd crates/auths-verifier && cargo check --target wasm32-unknown-unknown --no-default-features --features wasm 344 ``` 345 346 --- 347 348 ## Migration Map 349 350 | Current location | Target location | What | 351 |---|---|---| 352 | `auths-test-utils/src/crypto.rs` | `auths-crypto/src/testing.rs` | `create_test_keypair`, `get_shared_keypair`, `gen_keypair` | 353 | `auths-test-utils/src/git.rs` | `auths-infra-git/src/testing.rs` | `init_test_repo`, `get_cloned_test_repo` | 354 | `auths-test-utils/src/fakes/*.rs` | `auths-id/src/testing/fakes/*.rs` | All fake trait implementations | 355 | `auths-test-utils/src/contracts/*.rs` | `auths-id/src/testing/contracts/*.rs` | All contract test macros | 356 | `auths-test-utils/src/fixtures/*.rs` | `auths-id/src/testing/fixtures/*.rs` | `test_inception_event`, `test_attestation` | 357 | `auths-test-utils/src/mocks/*.rs` | `auths-id/src/testing/mocks/*.rs` | `MockIdentityStorage`, `MockAttestationSource` | 358 | `auths-test-utils/src/storage_fakes.rs` | `auths-id/src/testing/fakes/storage.rs` | `InMemoryStorage` | 359 | `auths-test-utils/src/fakes/telemetry.rs` | `auths-telemetry/src/testing.rs` | `MemoryEventSink` | 360 | `auths-id` git-storage code | `auths-storage/src/git/` | Git-based identity storage | 361 | `crates/auths-test-utils/` | **deleted** | — | 362 363 --- 364 365 ## Consumer Migration 366 367 Every crate that currently has `auths-test-utils.workspace = true` in dev-dependencies gets replaced: 368 369 ```toml 370 # Before 371 [dev-dependencies] 372 auths-test-utils.workspace = true 373 374 # After — only enable the features you actually use 375 [dev-dependencies] 376 auths-crypto = { workspace = true, features = ["test-utils"] } 377 auths-id = { workspace = true, features = ["test-utils"] } 378 ``` 379 380 The `test-utils` features chain transitively — `auths-id/test-utils` enables `auths-crypto/test-utils` automatically. 381 382 --- 383 384 ## Risks and Mitigations 385 386 | Risk | Mitigation | 387 |---|---| 388 | Large diff touching many files | Execute in phases; each phase is independently shippable | 389 | Contract test macros may have complex dependencies | Audit macro expansions before moving; may need to simplify | 390 | `auths-id` git-storage removal may break `auths-cli` | `auths-cli` already depends on `auths-storage`; rewire imports | 391 | Feature flag proliferation | Only two feature flags per crate max (`test-utils` + one domain feature) | 392 | `mockall` and `rand` become regular deps (optional) of published crates | Gated behind `test-utils` feature; not compiled by default consumers | 393 394 --- 395 396 ## Success Criteria 397 398 1. `cargo publish --workspace --dry-run` passes with zero manual intervention 399 2. `auths-test-utils` crate no longer exists 400 3. No crate has dev-dependencies on crates in the same or higher layer 401 4. All 1395+ tests pass 402 5. WASM build passes 403 6. Each crate's dependency list fits its architectural layer