README.md
1 # Documentation Index 2 3 Welcome to the Radicle Infrastructure documentation! This comprehensive guide covers everything from initial setup to production operations. 4 5 --- 6 7 ## ๐ Quick Navigation 8 9 | Category | Description | Documents | 10 |----------|-------------|-----------| 11 | [๐ Getting Started](#-getting-started) | Quick start and installation guides | 5 docs | 12 | [๐๏ธ Architecture](#๏ธ-architecture) | System design and components | 8 docs | 13 | [โ๏ธ Setup](#๏ธ-setup) | Installation and configuration | 20+ docs | 14 | [๐ง Operations](#-operations) | Day-to-day operations and maintenance | 15+ docs | 15 | [๐ Security](#-security) | Security policies and scanning | 10+ docs | 16 | [๐ฆ Package Registries](#-package-registries) | npm, PyPI, Docker registry guides | 8 docs | 17 | [๐ CI/CD](#-cicd) | Continuous integration and deployment | 10+ docs | 18 | [๐ Monitoring](#-monitoring) | Prometheus, Grafana, metrics | 8 docs | 19 | [๐งช Testing](#-testing) | Testing strategies and validation | 5 docs | 20 | [๐ ๏ธ Developer Tools](#๏ธ-developer-tools) | Tools and utilities | 6 docs | 21 | [๐ Guides](#-guides) | How-to guides and tutorials | 10+ docs | 22 | [๐ Reference](#-reference) | API and command reference | 5 docs | 23 24 **Total Documentation**: 101 files organized across 14 categories 25 26 --- 27 28 ## ๐ Getting Started 29 30 ### Essential First Steps 31 32 Start here if you're new to the infrastructure: 33 34 1. **[Quick Start Guide](QUICK_START.md)** โญ 35 - Get up and running in 15 minutes 36 - Installation, configuration, first project 37 38 2. **[Using the Infrastructure](using-the-infrastructure.md)** โญโญโญ 39 - Comprehensive usage guide (650+ lines) 40 - 20+ detailed use cases 41 - Daily development workflows 42 - Package registry integration 43 44 3. **[Quick Start Plan](QUICK-START-PLAN.md)** 45 - 15-minute, 1-hour, and 1-week timelines 46 - Actionable steps for immediate use 47 48 ### Installation Guides 49 50 - **[Installation Guide](setup/installation-guide.md)** - Complete installation procedure 51 - **[Quick Start Guide](setup/quickstart.md)** - Fast setup for experienced users 52 53 ### Project Overview 54 55 - **[Implementation Summary](IMPLEMENTATION_SUMMARY.md)** - What we built and why 56 - **[Infrastructure Complete](INFRASTRUCTURE-COMPLETE.md)** - Final status and achievements 57 - **[Roadmap Progress Review](roadmap-progress-review.md)** - All 13 phases reviewed 58 59 --- 60 61 ## ๐๏ธ Architecture 62 63 ### System Design 64 65 Understanding how everything fits together: 66 67 - **[Architecture Overview](architecture/)** - High-level system design 68 - **System Components**: 69 - Radicle peer-to-peer network 70 - Package registries (npm, PyPI, Docker) 71 - CI/CD pipeline 72 - Monitoring stack (Prometheus + Grafana) 73 - Security scanning (Trivy + Semgrep) 74 75 ### Network Architecture 76 77 - **[Tailscale Network Topology](setup/tailscale-network-topology.md)** - Mesh network design 78 - **[Seed Network Design](setup/auxo-private-seed-network.md)** - Multi-device replication 79 - **[Network Mobility Guide](setup/network-mobility-guide.md)** - Working across networks 80 81 --- 82 83 ## โ๏ธ Setup 84 85 ### Initial Setup (New Installations) 86 87 **Single Machine Setup:** 88 1. [Installation Guide](setup/installation-guide.md) - Base installation 89 2. [Quick Start](setup/quickstart.md) - Get running fast 90 3. [Organizational Identity](setup/organizational-identity.md) - Set up your identity 91 92 **Multi-Machine Setup (Tailscale Mesh):** 93 1. [Tailscale Radicle Setup](setup/tailscale-radicle-setup.md) - Mesh networking 94 2. [Tailscale Quick Config](setup/tailscale-quick-config.md) - Fast configuration 95 3. [Network Topology](setup/tailscale-network-topology.md) - Understanding the mesh 96 97 ### Seed Node Setup 98 99 Setting up additional seed nodes: 100 101 - **[MacBook Seed Setup](setup/macbook-seed-setup.md)** - Full MacBook seed node guide 102 - **[MacBook Seed Quick Setup](setup/macbook-seed-quick-setup.md)** - Fast setup 103 - **[MacBook Seed Quick Fix](setup/macbook-seed-quick-fix.md)** - Troubleshooting 104 - **[MacBook Seed Network Summary](setup/macbook-seed-network-summary.md)** - Overview 105 106 ### MacBook2 Specific Setup 107 108 Complete setup documentation for macbook2: 109 110 - **[MacBook2 Complete Setup](setup/macbook2-complete-setup.md)** โญ - Full guide 111 - **[MacBook2 Final Solution](setup/macbook2-final-solution.md)** - Tested solution 112 - **[MacBook2 Correct Clone](setup/macbook2-correct-clone.md)** - Cloning repos 113 - **[MacBook2 Alternative Clone](setup/macbook2-alternative-clone.md)** - Alternative methods 114 - **[MacBook2 Connection Fix](setup/macbook2-connection-fix.md)** - Connection issues 115 - **[MacBook2 Tailscale Config](setup/macbook2-tailscale-config.md)** - Tailscale setup 116 117 ### Tailscale Configuration 118 119 Mesh networking documentation: 120 121 - **[Tailscale Radicle Setup](setup/tailscale-radicle-setup.md)** - Integration guide 122 - **[Tailscale Quick Config](setup/tailscale-quick-config.md)** - Quick setup 123 - **[Tailscale Verification Checklist](setup/tailscale-verification-checklist.md)** - Validation 124 - **[Tailscale Radicle Findings](setup/tailscale-radicle-findings.md)** - Lessons learned 125 126 ### Mobility and Portability 127 128 - **[Network Mobility Guide](setup/network-mobility-guide.md)** - Working across networks 129 - **[Auxo Mobility Example](setup/auxo-mobility-example.md)** - Real-world example 130 131 --- 132 133 ## ๐ง Operations 134 135 ### Daily Operations 136 137 Running and maintaining the infrastructure: 138 139 - **[Operations Overview](operations/)** - Overview of operational procedures 140 - **[Troubleshooting Guide](operations/troubleshooting.md)** โญ - Common issues and solutions 141 - **[FAQ](operations/faq.md)** - Frequently asked questions 142 - **[Mesh Maintenance](operations/mesh-maintenance.md)** - Maintaining the mesh network 143 144 ### Deployment 145 146 Initial deployment and updates: 147 148 - **[Phase 2 Deployment Guide](operations/PHASE2_DEPLOYMENT_GUIDE.md)** - Phase 2 deployment 149 - **[Phase 3 CI Deployment](operations/PHASE3_CI_DEPLOYMENT.md)** - CI/CD deployment 150 - **[Deployment Documentation](deployment/)** - General deployment guides 151 152 ### Disaster Recovery 153 154 Backup and restoration: 155 156 - **[Disaster Recovery](operations/disaster-recovery.md)** โญโญโญ - Complete DR procedures 157 - **[Disaster Recovery Test](operations/disaster-recovery-test.md)** - Testing DR 158 - **Backup Scripts**: 159 - `~/radicle-registry/backup-registries.sh` - Daily backups (2:00 AM) 160 - `~/radicle-registry/restore-registries.sh` - Restore from backup 161 - `~/radicle-registry/validate-dr.sh` - Validate DR readiness 162 163 ### Status and Monitoring 164 165 - **[Status](operations/STATUS.md)** - Current infrastructure status 166 - **[Radicle CI Status](operations/RADICLE_CI_STATUS.md)** - CI/CD status 167 - **[Mission Complete](operations/MISSION_COMPLETE.md)** - Achievement summary 168 169 ### Documentation Archives 170 171 Historical documentation: 172 173 - **[CI Improvements](operations/CI_IMPROVEMENTS.md)** - CI enhancement history 174 - **[Engineering Handoff](operations/ENGINEERING_HANDOFF.md)** - Handoff documentation 175 176 --- 177 178 ## ๐ Security 179 180 ### Security Overview 181 182 Comprehensive security documentation: 183 184 - **[Security Policy](../SECURITY.md)** โญโญโญ - Main security policy (root directory) 185 - **[Security Overview](security/security-overview.md)** - Security architecture 186 - **[Security Scanning Guide](security-scanning.md)** - Running scans 187 188 ### Current Security Posture 189 190 - **Security Health Score**: 100/100 191 - **Critical Vulnerabilities**: 0 192 - **High Vulnerabilities**: 0 193 - **Daily Automated Scanning**: Trivy + Semgrep 194 195 ### Security Procedures 196 197 - **[Vulnerability Response Procedures](security/vulnerability-response-procedures.md)** - Handling vulnerabilities 198 - **[Phase 12 Software Security](security/phase-12-software-security.md)** - Security implementation 199 - **[Mesh Security Checklist](security/mesh-security-checklist.md)** - Network security 200 201 ### Security Scanning 202 203 Running security scans: 204 205 ```bash 206 # Trivy vulnerability scan 207 ~/Projects/radicle/scripts/security/run-trivy.sh filesystem . 208 209 # Semgrep SAST scan 210 ~/Projects/radicle/scripts/security/run-semgrep.sh --severity ERROR 211 212 # Full security report 213 ~/Projects/radicle/scripts/security/scheduled-scan.sh 214 215 # Mesh security check 216 ~/Projects/radicle/scripts/security/mesh-security-check.sh --verbose 217 ``` 218 219 ### Security Metrics 220 221 - **[Security Metrics Script](../scripts/monitoring/security-metrics.sh)** - Check security posture 222 - View in Grafana: http://localhost:3000 (Security Posture dashboard) 223 224 --- 225 226 ## ๐ฆ Package Registries 227 228 ### Registry Overview 229 230 Complete package sovereignty across three ecosystems: 231 232 | Registry | Type | Port | Web UI | Status | 233 |----------|------|------|--------|--------| 234 | **Verdaccio** | npm | 4873 | Yes | โ Running | 235 | **devpi** | PyPI | 3141 | Yes | โ Running | 236 | **Docker Registry v2** | Docker | 5000 | No | โ Running | 237 238 ### Registry Documentation 239 240 - **[Registry Documentation](registry/)** - Complete registry guides 241 - **[Package Registry Usage](using-the-infrastructure.md#package-registries)** - How to use registries 242 243 ### Using Registries 244 245 **npm (Verdaccio):** 246 ```bash 247 # Configure 248 npm config set registry http://localhost:4873 249 250 # Publish private package 251 npm publish 252 253 # Install from registry 254 npm install your-package 255 ``` 256 257 **Python (devpi):** 258 ```bash 259 # Configure (project-level) 260 pip config set global.index-url http://localhost:3141/root/pypi/+simple/ 261 262 # Upload package 263 devpi upload 264 265 # Install from registry 266 pip install your-package 267 ``` 268 269 **Docker (Registry v2):** 270 ```bash 271 # Tag image 272 docker tag my-image localhost:5000/my-image 273 274 # Push to registry 275 docker push localhost:5000/my-image 276 277 # Pull from registry 278 docker pull localhost:5000/my-image 279 ``` 280 281 ### Registry Management 282 283 - **Registry Manager**: `~/radicle-registry/registry-manager.sh` 284 - Start: `registry-manager.sh start` 285 - Stop: `registry-manager.sh stop` 286 - Status: `registry-manager.sh status` 287 - Restart: `registry-manager.sh restart` 288 289 --- 290 291 ## ๐ CI/CD 292 293 ### CI/CD Pipeline 294 295 Automated testing and deployment: 296 297 - **[CI/CD Documentation](ci-cd/)** - Pipeline documentation 298 - **[Phase 3 CI Deployment](operations/PHASE3_CI_DEPLOYMENT.md)** - CI setup 299 - **[CI Improvements](operations/CI_IMPROVEMENTS.md)** - Enhancement history 300 301 ### CI/CD Features 302 303 - **Automated Testing**: Every patch triggers CI 304 - **Shellcheck Validation**: All scripts validated 305 - **Security Scanning**: Integrated Trivy + Semgrep 306 - **Notifications**: Telegram alerts for job status 307 - **Success Rate**: 68.75% 308 309 ### Common CI Issues 310 311 - **[Common CI Pitfalls](ci/common-ci-pitfalls.md)** - Avoiding common mistakes 312 - **[Phase 10 Improvements](ci/phase-10-improvements.md)** - CI enhancements 313 314 ### Running CI Locally 315 316 ```bash 317 # Run CI job for current commit 318 ./scripts/ci-cd/run-ci-job.sh $(git rev-parse HEAD) 319 320 # Run all tests 321 ./scripts/ci-cd/run-all-tests.sh 322 323 # View CI logs 324 tail -f ~/radicle-ci/logs/ci-job-*.log 325 ``` 326 327 ### Notifications 328 329 - **[Notifications Guide](notifications.md)** - Setting up Telegram notifications 330 - **Notification Server**: Auto-starts on boot (port 8080) 331 332 --- 333 334 ## ๐ Monitoring 335 336 ### Monitoring Stack 337 338 Comprehensive monitoring with Prometheus + Grafana: 339 340 - **[Monitoring Documentation](monitoring/)** - Complete monitoring guides 341 - **[Monitoring Architecture](monitoring/MONITORING-ARCHITECTURE.md)** - System design 342 - **[Monitoring Quick Start](monitoring/MONITORING-QUICK-START.md)** - Getting started 343 - **[Monitoring Implementation Summary](monitoring/MONITORING-IMPLEMENTATION-SUMMARY.md)** - What's implemented 344 345 ### Accessing Dashboards 346 347 ```bash 348 # Grafana (main dashboards) 349 open http://localhost:3000 350 # Login: admin / admin 351 352 # Prometheus (raw metrics) 353 open http://localhost:9090 354 ``` 355 356 ### Available Dashboards 357 358 1. **DORA Metrics Dashboard** 359 - Deployment frequency 360 - Lead time for changes 361 - Time to restore service 362 - Change failure rate 363 - Overall DORA score: **4.0/4.0 (Elite)** 364 365 2. **Security Posture Dashboard** 366 - Vulnerability counts 367 - Security health score: **100/100** 368 - Scan results 369 - Security trends 370 371 3. **Infrastructure Health Dashboard** 372 - Service status 373 - Resource usage 374 - Uptime tracking: **99.9%+** 375 376 4. **CI/CD Metrics Dashboard** 377 - Job success rate: **68.75%** 378 - Execution time 379 - Queue depth 380 381 ### Monitoring Services 382 383 All auto-start via launchd: 384 385 - **Prometheus**: Port 9090 (metrics collection) 386 - **Grafana**: Port 3000 (visualization) 387 - **Prometheus Exporter**: Port 8000 (custom metrics) 388 389 ### Onboarding Automation 390 391 - **[Onboarding Automation](monitoring/ONBOARDING_AUTOMATION.md)** - Automated setup 392 393 --- 394 395 ## ๐งช Testing 396 397 ### Testing Strategy 398 399 Comprehensive testing across all layers: 400 401 - **[Testing Documentation](testing/)** - Testing guides 402 - **[Infrastructure Stress Test](infrastructure-stress-test.md)** - Load testing 403 - **[Phase 5 Validation](phase5-validation.md)** - Phase 5 testing 404 405 ### Running Tests 406 407 ```bash 408 # Run all tests 409 ./scripts/ci-cd/run-all-tests.sh 410 411 # Run integration tests 412 ./tests/run-integration-tests.sh 413 414 # Run CI validation 415 ./scripts/ci-cd/run-ci-job.sh $(git rev-parse HEAD) 416 ``` 417 418 ### Validation 419 420 - **[Disaster Recovery Validation](~/radicle-registry/validate-dr.sh)** - DR validation 421 - **[Tailscale Verification Checklist](setup/tailscale-verification-checklist.md)** - Network validation 422 423 --- 424 425 ## ๐ ๏ธ Developer Tools 426 427 ### Development Tools 428 429 Tools and utilities for developers: 430 431 - **[Developer Tools Documentation](developer-tools/)** - Tool guides 432 - **[GitLab Features Comprehensive](gitlab-features-comprehensive.md)** - GitLab comparison 433 434 ### Radicle Workflow Scripts 435 436 Located in `scripts/workflow/`: 437 438 - `sync-status.sh` - Check sync status 439 - `create-patch.sh` - Create a new patch 440 - `update-patch.sh` - Update existing patch 441 - `merge-patch.sh` - Merge approved patch 442 - `list-patches.sh` - List all patches 443 444 ### Package Management Scripts 445 446 Located in `scripts/registry/`: 447 448 - Package upload/download automation 449 - Registry health checks 450 - Backup and restore 451 452 --- 453 454 ## ๐ Guides 455 456 ### How-To Guides 457 458 Step-by-step guides for common tasks: 459 460 - **[Guides Directory](guides/)** - All how-to guides 461 - **[Troubleshooting Guide](operations/troubleshooting.md)** โญ - Problem solving 462 - **[Network Mobility Guide](setup/network-mobility-guide.md)** - Working across networks 463 464 ### Best Practices 465 466 - **Code Standards**: See [CONTRIBUTING.md](../CONTRIBUTING.md) 467 - **Security Best Practices**: See [SECURITY.md](../SECURITY.md) 468 - **Commit Guidelines**: See [CONTRIBUTING.md](../CONTRIBUTING.md#commit-guidelines) 469 470 --- 471 472 ## ๐ Reference 473 474 ### API and Command Reference 475 476 Technical reference documentation: 477 478 - **[Reference Documentation](reference/)** - API and command reference 479 - **[Scripts Reference](../scripts/README.md)** - All automation scripts 480 - **Radicle CLI**: `rad --help` 481 - **Registry Manager**: `~/radicle-registry/registry-manager.sh --help` 482 483 ### Key Scripts Location 484 485 ``` 486 scripts/ 487 โโโ workflow/ # Radicle workflow automation 488 โโโ ci-cd/ # CI/CD pipeline scripts 489 โโโ security/ # Security scanning scripts 490 โโโ monitoring/ # Monitoring and metrics scripts 491 โโโ deployment/ # Service deployment scripts 492 โโโ registry/ # Package registry management 493 ``` 494 495 --- 496 497 ## ๐ฏ Documentation by Use Case 498 499 ### I want to... 500 501 **Get started quickly:** 502 โ [Quick Start Guide](QUICK_START.md) 503 โ [Quick Start Plan](QUICK-START-PLAN.md) 504 505 **Use the infrastructure for development:** 506 โ [Using the Infrastructure](using-the-infrastructure.md) โญโญโญ 507 508 **Set up a new seed node:** 509 โ [MacBook Seed Setup](setup/macbook-seed-setup.md) 510 โ [Tailscale Radicle Setup](setup/tailscale-radicle-setup.md) 511 512 **Understand the architecture:** 513 โ [Architecture Overview](architecture/) 514 โ [Implementation Summary](IMPLEMENTATION_SUMMARY.md) 515 516 **Solve a problem:** 517 โ [Troubleshooting Guide](operations/troubleshooting.md) 518 โ [FAQ](operations/faq.md) 519 520 **Use package registries:** 521 โ [Using the Infrastructure - Package Registries](using-the-infrastructure.md#package-registries) 522 โ [Registry Documentation](registry/) 523 524 **Set up monitoring:** 525 โ [Monitoring Quick Start](monitoring/MONITORING-QUICK-START.md) 526 โ [Monitoring Architecture](monitoring/MONITORING-ARCHITECTURE.md) 527 528 **Contribute code:** 529 โ [CONTRIBUTING.md](../CONTRIBUTING.md) 530 โ [Radicle Workflow Scripts](../scripts/workflow/) 531 532 **Report a security issue:** 533 โ [SECURITY.md](../SECURITY.md) 534 โ [Vulnerability Response Procedures](security/vulnerability-response-procedures.md) 535 536 **Perform disaster recovery:** 537 โ [Disaster Recovery Guide](operations/disaster-recovery.md) 538 โ [Disaster Recovery Test](operations/disaster-recovery-test.md) 539 540 --- 541 542 ## ๐ Documentation Statistics 543 544 - **Total Files**: 101 markdown documents 545 - **Total Categories**: 14 organized directories 546 - **Lines of Documentation**: 15,000+ lines 547 - **Setup Guides**: 20+ documents 548 - **Operational Guides**: 15+ documents 549 - **Security Guides**: 10+ documents 550 551 --- 552 553 ## ๐ Search Tips 554 555 ### Finding Documentation 556 557 1. **Use this index** - Start here for organized navigation 558 2. **Search by keyword**: Use your editor's search across files 559 3. **Check the category**: Documents are organized by topic 560 4. **Look for โญ markers**: These indicate essential/comprehensive guides 561 562 ### Most Referenced Documents 563 564 **Essential (Read First):** 565 - โญโญโญ [Using the Infrastructure](using-the-infrastructure.md) 566 - โญโญโญ [Disaster Recovery](operations/disaster-recovery.md) 567 - โญโญโญ [Security Policy](../SECURITY.md) 568 569 **Important (Read Early):** 570 - โญโญ [Quick Start Guide](QUICK_START.md) 571 - โญโญ [Troubleshooting Guide](operations/troubleshooting.md) 572 - โญโญ [Monitoring Quick Start](monitoring/MONITORING-QUICK-START.md) 573 574 **Useful (Read As Needed):** 575 - โญ [MacBook2 Complete Setup](setup/macbook2-complete-setup.md) 576 - โญ [Installation Guide](setup/installation-guide.md) 577 - โญ [Common CI Pitfalls](ci/common-ci-pitfalls.md) 578 579 --- 580 581 ## ๐ Need Help? 582 583 ### Getting Support 584 585 1. **Check this index** - Find the relevant documentation 586 2. **Read the guide** - Most questions are answered in docs 587 3. **Check troubleshooting** - [Troubleshooting Guide](operations/troubleshooting.md) 588 4. **Check FAQ** - [FAQ](operations/faq.md) 589 5. **Ask the team** - Contact your team lead 590 591 ### Contributing to Documentation 592 593 Found an issue or want to improve documentation? 594 595 1. Follow [CONTRIBUTING.md](../CONTRIBUTING.md) guidelines 596 2. Create a patch with your changes 597 3. Submit for review 598 599 --- 600 601 ## ๐ Recent Updates 602 603 - **November 12, 2025**: Complete documentation reorganization (Phase 3) 604 - **November 12, 2025**: Repository polish - Phase 1 & 2 complete 605 - **November 12, 2025**: Phase 13 complete - Production hardening 606 - **November 10, 2025**: Phase 12 complete - Tailscale mesh network 607 - **November 6, 2025**: Phase 10 complete - DORA metrics implementation 608 609 See [CHANGELOG.md](../CHANGELOG.md) for complete version history. 610 611 --- 612 613 ## ๐ Achievement Summary 614 615 **Infrastructure Status**: โ Production Ready (Version 2.0.0) 616 617 - โ All 13 phases complete 618 - โ Elite DORA performance (4.0/4.0) 619 - โ Perfect security (100/100) 620 - โ 101 comprehensive documentation files 621 - โ 35+ production-ready scripts 622 - โ 99.9%+ uptime 623 - โ Complete package sovereignty 624 625 --- 626 627 **Last Updated**: November 12, 2025 628 **Version**: 2.0.0 629 **Status**: Production Ready