1  # Radicle Secure - Optimization & Security Implementation Guide
   2  
   3  **Project:** Radicle Heartwood Fork with Security Enhancements
   4  **Goal:** Add secret scanning, vulnerability detection, and reduce hosting costs by 70-85%
   5  **Target Cost:** $3-8/month (from $30/month baseline)
   6  
   7  ---
   8  
   9  ## Table of Contents
  10  
  11  1. [Project Overview](#project-overview)
  12  2. [Adding Security Features](#adding-security-features)
  13  3. [Hosting Cost Breakdown Analysis](#hosting-cost-breakdown-analysis)
  14  4. [Optimization Strategies](#optimization-strategies)
  15  5. [Implementation Roadmap](#implementation-roadmap)
  16  6. [Deployment Guide](#deployment-guide)
  17  7. [Monitoring & Maintenance](#monitoring--maintenance)
  18  
  19  ---
  20  
  21  ## Project Overview
  22  
  23  ### Repository Structure
  24  
  25  ```
  26  radicle-secure/
  27  ├── crates/
  28  │   ├── radicle/              # Core library
  29  │   ├── radicle-cli/          # Command-line interface
  30  │   ├── radicle-node/         # P2P network daemon
  31  │   ├── radicle-remote-helper/# Git remote helper
  32  │   └── ...                   # Other supporting crates
  33  ├── Cargo.toml                # Workspace configuration
  34  └── systemd/                  # Systemd unit files
  35  ```
  36  
  37  ### Key Components
  38  
  39  - **radicle-cli**: User-facing CLI tool
  40  - **radicle-node**: Always-on P2P daemon (optimization target)
  41  - **radicle-httpd**: HTTP API gateway (if added)
  42  - **rad-web**: Web interface (deploy separately to reduce costs)
  43  
  44  ### License
  45  
  46  MIT OR Apache-2.0 (fully permissive - fork-friendly)
  47  
  48  ---
  49  
  50  ## Adding Security Features
  51  
  52  ### 1. Secret Scanning Integration
  53  
  54  **Available Rust Libraries:**
  55  
  56  | Library    | Speed                        | Accuracy                | Integration Complexity |
  57  |------------|------------------------------|-------------------------|------------------------|
  58  | ripsecrets | 95x faster than alternatives | High                    | Low - simple CLI       |
  59  | secretscan | 51,020 files/sec             | 99% detection           | Low - library API      |
  60  | guardy     | Multi-threaded               | High (entropy analysis) | Medium - git hooks     |
  61  
  62  #### Implementation Approach
  63  
  64  ```rust
  65  // Create: crates/radicle/src/security/mod.rs
  66  pub mod secrets;
  67  pub mod vulnerabilities;
  68  pub mod compression;
  69  
  70  // Create: crates/radicle/src/security/secrets.rs
  71  use secretscan::{Scanner, SecretMatch};
  72  use std::path::Path;
  73  
  74  pub struct SecretScanner {
  75      scanner: Scanner,
  76  }
  77  
  78  impl SecretScanner {
  79      pub fn new() -> Self {
  80          Self {
  81              scanner: Scanner::default(),
  82          }
  83      }
  84  
  85      /// Scan a git diff for secrets
  86      pub fn scan_diff(&self, diff: &str) -> Result<Vec<SecretMatch>, Error> {
  87          self.scanner.scan_text(diff)
  88      }
  89  
  90      /// Scan a file for secrets
  91      pub fn scan_file(&self, path: &Path) -> Result<Vec<SecretMatch>, Error> {
  92          self.scanner.scan_file(path)
  93      }
  94  
  95      /// Scan an entire repository
  96      pub fn scan_repository(&self, repo_path: &Path) -> Result<Vec<SecretMatch>, Error> {
  97          self.scanner.scan_directory(repo_path)
  98      }
  99  }
 100  ```
 101  
 102  #### Integration Points
 103  
 104  **A. Pre-commit hooks in radicle-cli**
 105  
 106  ```rust
 107  // Modify: crates/radicle-cli/src/commands/push.rs
 108  use radicle::security::secrets::SecretScanner;
 109  
 110  pub fn run(options: Options, ctx: impl term::Context) -> anyhow::Result<()> {
 111      // Existing code...
 112  
 113      // Add secret scanning before push
 114      let scanner = SecretScanner::new();
 115      let commits = get_new_commits(&repo)?;
 116  
 117      for commit in commits {
 118          let diff = commit.diff()?;
 119          if let Ok(secrets) = scanner.scan_diff(&diff) {
 120              if !secrets.is_empty() {
 121                  term::error(format!("Found {} potential secrets in commit {}",
 122                      secrets.len(), commit.id()));
 123                  for secret in secrets {
 124                      term::warning(format!("  {} at line {}", secret.kind, secret.line));
 125                  }
 126                  return Err(anyhow!("Commit contains potential secrets. Aborting."));
 127              }
 128          }
 129      }
 130  
 131      // Continue with push...
 132  }
 133  ```
 134  
 135  **B. Real-time scanning in radicle-node during replication**
 136  
 137  ```rust
 138  // Modify: crates/radicle-node/src/service/sync.rs
 139  use radicle::security::secrets::SecretScanner;
 140  
 141  async fn sync_repository(&self, repo: &Repository) -> Result<(), Error> {
 142      let scanner = SecretScanner::new();
 143  
 144      // Scan incoming commits during replication
 145      for commit in repo.new_commits() {
 146          let diff = commit.diff()?;
 147          if let Ok(secrets) = scanner.scan_diff(&diff) {
 148              if !secrets.is_empty() {
 149                  log::warn!("Found {} potential secrets in replicated commit {}",
 150                      secrets.len(), commit.id());
 151                  // Optionally: quarantine, reject, or alert
 152              }
 153          }
 154      }
 155  
 156      // Continue sync...
 157      Ok(())
 158  }
 159  ```
 160  
 161  **C. API endpoint for on-demand scans**
 162  
 163  ```rust
 164  // Create: crates/radicle-node/src/api/security.rs
 165  use radicle::security::secrets::SecretScanner;
 166  use axum::{Json, extract::Path};
 167  
 168  #[derive(Deserialize)]
 169  pub struct ScanRequest {
 170      repo_id: String,
 171      commit_id: Option<String>,
 172  }
 173  
 174  pub async fn scan_repository(
 175      Path(repo_id): Path<String>,
 176  ) -> Result<Json<ScanResult>, Error> {
 177      let scanner = SecretScanner::new();
 178      let repo = Repository::open(&repo_id)?;
 179  
 180      let secrets = scanner.scan_repository(repo.path())?;
 181  
 182      Ok(Json(ScanResult {
 183          total_secrets: secrets.len(),
 184          secrets: secrets.into_iter().map(|s| s.into()).collect(),
 185      }))
 186  }
 187  ```
 188  
 189  ---
 190  
 191  ### 2. Vulnerability Detection Integration
 192  
 193  #### RustSec Integration
 194  
 195  ```rust
 196  // Create: crates/radicle/src/security/vulnerabilities.rs
 197  use rustsec::{Database, Lockfile, Warning};
 198  use std::path::Path;
 199  
 200  pub struct VulnerabilityScanner {
 201      db: Database,
 202  }
 203  
 204  impl VulnerabilityScanner {
 205      pub fn new() -> Result<Self, Error> {
 206          let db = Database::fetch()?;
 207          Ok(Self { db })
 208      }
 209  
 210      /// Scan a Cargo.lock file for known vulnerabilities
 211      pub fn scan_cargo_lock(&self, lockfile_path: &Path) -> Result<ScanReport, Error> {
 212          let lockfile = Lockfile::load(lockfile_path)?;
 213          let warnings = rustsec::Warning::load_from_lockfile(&lockfile, &self.db)?;
 214  
 215          Ok(ScanReport {
 216              vulnerabilities: warnings.vulnerabilities.len(),
 217              warnings: warnings.warnings.len(),
 218              details: warnings,
 219          })
 220      }
 221  
 222      /// Scan all Cargo.lock files in a repository
 223      pub fn scan_repository(&self, repo_path: &Path) -> Result<Vec<ScanReport>, Error> {
 224          let mut reports = Vec::new();
 225  
 226          for entry in walkdir::WalkDir::new(repo_path)
 227              .follow_links(true)
 228              .into_iter()
 229              .filter_map(|e| e.ok())
 230          {
 231              if entry.file_name() == "Cargo.lock" {
 232                  if let Ok(report) = self.scan_cargo_lock(entry.path()) {
 233                      reports.push(report);
 234                  }
 235              }
 236          }
 237  
 238          Ok(reports)
 239      }
 240  }
 241  
 242  #[derive(Debug, Serialize)]
 243  pub struct ScanReport {
 244      pub vulnerabilities: usize,
 245      pub warnings: usize,
 246      pub details: Warning,
 247  }
 248  ```
 249  
 250  #### Multi-Language Support (Trivy Integration)
 251  
 252  ```rust
 253  // Create: crates/radicle/src/security/trivy.rs
 254  use std::process::Command;
 255  use serde_json::Value;
 256  
 257  pub struct TrivyScanner {
 258      trivy_path: PathBuf,
 259  }
 260  
 261  impl TrivyScanner {
 262      pub fn new() -> Result<Self, Error> {
 263          // Check if trivy is installed
 264          let trivy_path = which::which("trivy")?;
 265          Ok(Self { trivy_path })
 266      }
 267  
 268      /// Scan a filesystem for vulnerabilities
 269      pub fn scan_filesystem(&self, path: &Path) -> Result<TrivyReport, Error> {
 270          let output = Command::new(&self.trivy_path)
 271              .args(&["fs", "--format", "json", path.to_str().unwrap()])
 272              .output()?;
 273  
 274          let report: Value = serde_json::from_slice(&output.stdout)?;
 275          Ok(TrivyReport::from_json(report))
 276      }
 277  }
 278  ```
 279  
 280  ---
 281  
 282  ## Hosting Cost Breakdown Analysis
 283  
 284  ### Where Costs Come From
 285  
 286  | Component          | Cost Driver               | % of Total | Optimization Potential |
 287  |--------------------|---------------------------|------------|------------------------|
 288  | **Compute (VM)**   | vCPU + RAM always running | 40-50%     | ⭐⭐⭐⭐⭐ High             |
 289  | **Storage**        | SSD persistent disks      | 15-20%     | ⭐⭐⭐ Medium             |
 290  | **Bandwidth (Egress)** | Data transfer out     | 25-35%     | ⭐⭐⭐⭐ High              |
 291  | **Static IP**      | Reserved address          | 5-10%      | ⭐ Low                  |
 292  | **Snapshots/Backups** | Optional redundancy    | 5-10%      | ⭐⭐⭐ Medium             |
 293  
 294  ### Detailed Cost Breakdown (e2-small example)
 295  
 296  ```
 297  Monthly Costs:
 298  ├─ Compute: $12.23 (730 hrs × $0.01675/hr)
 299  ├─ Storage: $3.20 (20 GB × $0.16/GB)
 300  ├─ Egress: $12.00 (100 GB × $0.12/GB)
 301  ├─ Static IP: $2.88 (when not attached to running instance)
 302  └─ Total: ~$30.31/month
 303  ```
 304  
 305  **Key Insight:** Bandwidth egress is surprisingly expensive and scales with usage!
 306  
 307  ---
 308  
 309  ## Optimization Strategies
 310  
 311  ### 1. Compute Optimization (Reduce VM costs by 60-80%)
 312  
 313  #### A. On-Demand Architecture
 314  
 315  **Current:** radicle-node runs 24/7
 316  **Optimized:** Socket activation + smart sleep
 317  
 318  ```rust
 319  // Modify: crates/radicle-node/src/main.rs
 320  use std::time::{Duration, Instant};
 321  use tokio::sync::RwLock;
 322  use std::sync::Arc;
 323  
 324  struct IdleMonitor {
 325      last_activity: Arc<RwLock<Instant>>,
 326      idle_timeout: Duration,
 327  }
 328  
 329  impl IdleMonitor {
 330      fn new(idle_timeout: Duration) -> Self {
 331          Self {
 332              last_activity: Arc::new(RwLock::new(Instant::now())),
 333              idle_timeout,
 334          }
 335      }
 336  
 337      async fn record_activity(&self) {
 338          let mut last = self.last_activity.write().await;
 339          *last = Instant::now();
 340      }
 341  
 342      async fn check_idle(&self) -> bool {
 343          let last = self.last_activity.read().await;
 344          last.elapsed() > self.idle_timeout
 345      }
 346  }
 347  
 348  #[tokio::main]
 349  async fn main() -> anyhow::Result<()> {
 350      let node = Node::new()?;
 351      let monitor = IdleMonitor::new(Duration::from_secs(600)); // 10 min timeout
 352  
 353      // Clone for idle monitoring task
 354      let monitor_clone = Arc::new(monitor);
 355      let node_clone = node.clone();
 356  
 357      // Spawn idle monitor
 358      tokio::spawn(async move {
 359          loop {
 360              tokio::time::sleep(Duration::from_secs(60)).await;
 361  
 362              if monitor_clone.check_idle().await {
 363                  log::info!("Node idle for 10 minutes, initiating shutdown...");
 364  
 365                  // Persist state to disk
 366                  if let Err(e) = node_clone.persist_state().await {
 367                      log::error!("Failed to persist state: {}", e);
 368                  }
 369  
 370                  // Graceful shutdown
 371                  node_clone.shutdown().await;
 372                  std::process::exit(0);
 373              }
 374          }
 375      });
 376  
 377      // Start node with activity tracking
 378      node.serve_with_monitor(monitor_clone).await?;
 379  
 380      Ok(())
 381  }
 382  ```
 383  
 384  **Cost Impact:**
 385  - From 24/7 uptime → ~4-8 hrs/day active
 386  - **Savings: ~70%** ($12/mo → $4/mo)
 387  
 388  #### B. Systemd Socket Activation
 389  
 390  ```ini
 391  # Create: systemd/radicle-node.socket
 392  [Unit]
 393  Description=Radicle Node Socket
 394  PartOf=radicle-node.service
 395  
 396  [Socket]
 397  ListenStream=8776
 398  Accept=false
 399  
 400  [Install]
 401  WantedBy=sockets.target
 402  ```
 403  
 404  ```ini
 405  # Modify: systemd/radicle-node.service
 406  [Unit]
 407  Description=Radicle Node
 408  Requires=radicle-node.socket
 409  After=network.target radicle-node.socket
 410  
 411  [Service]
 412  Type=notify
 413  ExecStart=/usr/local/bin/radicle-node
 414  Restart=on-failure
 415  
 416  # Auto-shutdown after idle
 417  TimeoutStopSec=10
 418  KillMode=mixed
 419  
 420  [Install]
 421  WantedBy=multi-user.target
 422  ```
 423  
 424  ---
 425  
 426  ### 2. Storage Optimization (Reduce by 40-50%)
 427  
 428  #### A. Compression Middleware
 429  
 430  ```rust
 431  // Create: crates/radicle/src/security/compression.rs
 432  use zstd::stream::{encode_all, decode_all};
 433  use std::io::{Read, Write};
 434  
 435  pub struct CompressionLayer {
 436      level: i32,
 437  }
 438  
 439  impl CompressionLayer {
 440      pub fn new(level: i32) -> Self {
 441          Self { level }
 442      }
 443  
 444      /// Compress data using zstd
 445      pub fn compress(&self, data: &[u8]) -> Result<Vec<u8>, Error> {
 446          Ok(encode_all(data, self.level)?)
 447      }
 448  
 449      /// Decompress zstd data
 450      pub fn decompress(&self, compressed: &[u8]) -> Result<Vec<u8>, Error> {
 451          Ok(decode_all(compressed)?)
 452      }
 453  }
 454  
 455  // Usage in storage layer
 456  // Modify: crates/radicle/src/storage/git.rs
 457  use crate::security::compression::CompressionLayer;
 458  
 459  impl Storage {
 460      pub fn store_object_compressed(&self, oid: Oid, data: &[u8]) -> Result<(), Error> {
 461          let compressor = CompressionLayer::new(3); // Level 3 for speed/ratio balance
 462          let compressed = compressor.compress(data)?;
 463  
 464          // Store compressed version
 465          self.write_object(oid, &compressed)?;
 466          Ok(())
 467      }
 468  
 469      pub fn read_object_compressed(&self, oid: Oid) -> Result<Vec<u8>, Error> {
 470          let compressed = self.read_object(oid)?;
 471          let compressor = CompressionLayer::new(3);
 472          Ok(compressor.decompress(&compressed)?)
 473      }
 474  }
 475  ```
 476  
 477  **Impact:** 30-50% storage reduction for text-heavy repos
 478  
 479  #### B. Tiered Storage
 480  
 481  ```rust
 482  // Create: crates/radicle-node/src/storage/tiered.rs
 483  use chrono::{DateTime, Utc, Duration};
 484  use std::path::PathBuf;
 485  
 486  pub struct TieredStorage {
 487      hot_storage: PathBuf,
 488      cold_storage: String, // GCS bucket URI
 489      archive_threshold: Duration,
 490  }
 491  
 492  impl TieredStorage {
 493      pub fn new(hot: PathBuf, cold: String) -> Self {
 494          Self {
 495              hot_storage: hot,
 496              cold_storage: cold,
 497              archive_threshold: Duration::days(90),
 498          }
 499      }
 500  
 501      /// Archive old commits to cold storage
 502      pub async fn archive_cold_data(&self, repo: &Repository) -> Result<(), Error> {
 503          let cutoff = Utc::now() - self.archive_threshold;
 504          let old_commits = repo.commits_older_than(cutoff)?;
 505  
 506          for commit in old_commits {
 507              // Upload to GCS cold storage
 508              self.move_to_cold_storage(&commit).await?;
 509  
 510              // Keep metadata locally, remove blob data
 511              self.create_cold_reference(&commit)?;
 512          }
 513  
 514          Ok(())
 515      }
 516  
 517      async fn move_to_cold_storage(&self, commit: &Commit) -> Result<(), Error> {
 518          // Implementation using google-cloud-storage crate
 519          // Store in coldline storage class
 520          todo!()
 521      }
 522  }
 523  ```
 524  
 525  **Cost Comparison:**
 526  - SSD persistent disk: $0.16/GB/month
 527  - HDD persistent disk: $0.04/GB/month (75% cheaper)
 528  - Coldline Storage: $0.004/GB/month (96% cheaper!)
 529  
 530  **Savings:** For 50GB data, $8/mo → $2/mo (75%)
 531  
 532  ---
 533  
 534  ### 3. Bandwidth Optimization (Reduce by 60-70%)
 535  
 536  #### A. Delta Compression Enhancement
 537  
 538  ```rust
 539  // Modify: crates/radicle-fetch/src/transport.rs
 540  use git2::{DeltaType, Odb};
 541  
 542  pub struct OptimizedTransport {
 543      use_delta: bool,
 544  }
 545  
 546  impl OptimizedTransport {
 547      /// Send repository updates using delta compression
 548      pub async fn replicate_with_delta(
 549          &self,
 550          peer: &Peer,
 551          refs: &[Ref]
 552      ) -> Result<(), Error> {
 553          // Compute deltas instead of full objects
 554          let deltas = self.compute_minimal_deltas(refs)?;
 555  
 556          // Send packed delta stream
 557          peer.send_pack(deltas).await?;
 558  
 559          Ok(())
 560      }
 561  
 562      fn compute_minimal_deltas(&self, refs: &[Ref]) -> Result<Vec<Delta>, Error> {
 563          // Use git pack protocol with aggressive delta compression
 564          // 70-90% size reduction vs full objects
 565          todo!()
 566      }
 567  }
 568  ```
 569  
 570  **Impact:** 70-90% bandwidth reduction for incremental syncs
 571  
 572  #### B. HTTP Compression Middleware
 573  
 574  ```rust
 575  // Create: crates/radicle-node/src/api/compression.rs
 576  use async_compression::tokio::bufread::BrotliEncoder;
 577  use axum::{
 578      body::Body,
 579      http::{Request, Response, header},
 580      middleware::Next,
 581  };
 582  use tokio::io::AsyncReadExt;
 583  
 584  pub async fn compression_middleware(
 585      req: Request<Body>,
 586      next: Next,
 587  ) -> Result<Response<Body>, Error> {
 588      let response = next.run(req).await;
 589  
 590      // Check if client accepts brotli
 591      if !accepts_encoding(&req, "br") {
 592          return Ok(response);
 593      }
 594  
 595      // Compress response body
 596      let (parts, body) = response.into_parts();
 597      let body_bytes = body.collect().await?.to_bytes();
 598  
 599      let mut encoder = BrotliEncoder::new(&body_bytes[..]);
 600      let mut compressed = Vec::new();
 601      encoder.read_to_end(&mut compressed).await?;
 602  
 603      let mut response = Response::from_parts(parts, Body::from(compressed));
 604      response.headers_mut().insert(
 605          header::CONTENT_ENCODING,
 606          "br".parse().unwrap()
 607      );
 608  
 609      Ok(response)
 610  }
 611  
 612  fn accepts_encoding(req: &Request<Body>, encoding: &str) -> bool {
 613      req.headers()
 614          .get(header::ACCEPT_ENCODING)
 615          .and_then(|v| v.to_str().ok())
 616          .map(|v| v.contains(encoding))
 617          .unwrap_or(false)
 618  }
 619  ```
 620  
 621  **Impact:** 50-70% bandwidth reduction for API calls
 622  
 623  ---
 624  
 625  ### 4. Component Optimization
 626  
 627  #### Remove/Externalize Web UI
 628  
 629  **Current Architecture:**
 630  ```
 631  radicle-node (running 24/7)
 632  ├── P2P networking
 633  ├── HTTP API server
 634  └── Static web UI serving  ← Wastes resources
 635  ```
 636  
 637  **Optimized Architecture:**
 638  ```
 639  radicle-node (idle-aware)
 640  └── P2P networking + HTTP API
 641  
 642  rad-web (external)
 643  └── Deploy to Vercel/Netlify (FREE)
 644  ```
 645  
 646  **Implementation:**
 647  
 648  ```bash
 649  # Deploy web UI to Vercel
 650  cd rad-web  # If it exists in the repo
 651  vercel deploy --prod
 652  
 653  # Update environment configuration
 654  export RADICLE_API_URL=https://your-node-ip:8776
 655  ```
 656  
 657  **Savings:** ~500MB RAM, ~0.5 vCPU → ~$6-8/month
 658  
 659  ---
 660  
 661  ## Implementation Roadmap
 662  
 663  ### Phase 1: Low-Hanging Fruit (Week 1-2)
 664  
 665  **Estimated Savings: 30-40%**
 666  
 667  ```bash
 668  cd radicle-secure
 669  
 670  # 1. Add security dependencies
 671  cat >> Cargo.toml << 'EOF'
 672  
 673  # Security enhancements
 674  secretscan = "0.2"
 675  rustsec = "0.28"
 676  walkdir = "2"
 677  
 678  # Compression
 679  zstd = "0.13"
 680  async-compression = { version = "0.4", features = ["tokio", "brotli"] }
 681  EOF
 682  
 683  # 2. Create security module structure
 684  mkdir -p crates/radicle/src/security
 685  cat > crates/radicle/src/security/mod.rs << 'EOF'
 686  pub mod secrets;
 687  pub mod vulnerabilities;
 688  pub mod compression;
 689  EOF
 690  
 691  # 3. Update radicle lib.rs to include security module
 692  echo "pub mod security;" >> crates/radicle/src/lib.rs
 693  
 694  # 4. Implement secret scanning (copy code from above)
 695  # Create crates/radicle/src/security/secrets.rs
 696  
 697  # 5. Implement vulnerability scanning
 698  # Create crates/radicle/src/security/vulnerabilities.rs
 699  
 700  # 6. Implement compression
 701  # Create crates/radicle/src/security/compression.rs
 702  
 703  # 7. Test compilation
 704  cargo build --all
 705  
 706  # 8. Run tests
 707  cargo test --all
 708  ```
 709  
 710  **Deliverables:**
 711  - ✅ Secret scanning functionality
 712  - ✅ Vulnerability detection
 713  - ✅ Compression utilities
 714  - ✅ All tests passing
 715  
 716  ---
 717  
 718  ### Phase 2: Architecture Changes (Week 3-4)
 719  
 720  **Estimated Savings: 50-60% total**
 721  
 722  ```bash
 723  # 1. Implement idle timeout in radicle-node
 724  # Modify crates/radicle-node/src/main.rs (see code above)
 725  
 726  # 2. Update systemd units for socket activation
 727  cp systemd/radicle-node.service systemd/radicle-node.service.bak
 728  # Edit systemd files (see code above)
 729  
 730  # 3. Add compression middleware to HTTP API
 731  # Modify crates/radicle-node/src/api/mod.rs
 732  
 733  # 4. Test idle behavior
 734  cargo build --release
 735  ./target/release/radicle-node --idle-timeout 600
 736  
 737  # Monitor for 10+ minutes and verify shutdown
 738  ```
 739  
 740  **Deliverables:**
 741  - ✅ Idle-aware node daemon
 742  - ✅ Socket activation support
 743  - ✅ HTTP response compression
 744  - ✅ Reduced runtime hours
 745  
 746  ---
 747  
 748  ### Phase 3: Advanced Optimization (Week 5-8)
 749  
 750  **Estimated Savings: 70-80% total**
 751  
 752  ```bash
 753  # 1. Implement tiered storage
 754  # Create crates/radicle-node/src/storage/tiered.rs
 755  
 756  # 2. Add GCS cold storage support
 757  cargo add google-cloud-storage
 758  
 759  # 3. Optimize delta compression
 760  # Modify crates/radicle-fetch/src/transport.rs
 761  
 762  # 4. Deploy web UI externally
 763  cd rad-web  # If available
 764  vercel deploy --prod
 765  
 766  # 5. Configure CDN caching
 767  # Setup Cloud CDN for static content
 768  ```
 769  
 770  **Deliverables:**
 771  - ✅ Tiered hot/cold storage
 772  - ✅ Enhanced delta compression
 773  - ✅ External web UI deployment
 774  - ✅ CDN integration
 775  
 776  ---
 777  
 778  ## Deployment Guide
 779  
 780  ### Local Development Setup
 781  
 782  ```bash
 783  # 1. Install Rust toolchain
 784  curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
 785  
 786  # 2. Clone and build
 787  cd radicle-secure
 788  cargo build --release
 789  
 790  # 3. Install locally
 791  cargo install --path crates/radicle-cli --force --locked
 792  cargo install --path crates/radicle-node --force --locked
 793  cargo install --path crates/radicle-remote-helper --force --locked
 794  
 795  # 4. Initialize node
 796  rad auth init
 797  rad node start
 798  ```
 799  
 800  ---
 801  
 802  ### GCP Deployment (e2-micro - Optimized)
 803  
 804  #### 1. Create VM Instance
 805  
 806  ```bash
 807  # Create instance with minimal specs
 808  gcloud compute instances create radicle-node \
 809    --machine-type=e2-micro \
 810    --zone=us-central1-a \
 811    --image-family=ubuntu-2204-lts \
 812    --image-project=ubuntu-os-cloud \
 813    --boot-disk-size=20GB \
 814    --boot-disk-type=pd-standard \
 815    --tags=radicle-node
 816  
 817  # Reserve static IP
 818  gcloud compute addresses create radicle-ip \
 819    --region=us-central1
 820  
 821  # Attach static IP
 822  gcloud compute instances add-access-config radicle-node \
 823    --zone=us-central1-a \
 824    --address=$(gcloud compute addresses describe radicle-ip \
 825      --region=us-central1 --format='value(address)')
 826  ```
 827  
 828  #### 2. Configure Firewall
 829  
 830  ```bash
 831  # Allow P2P traffic
 832  gcloud compute firewall-rules create radicle-p2p \
 833    --allow=tcp:8776 \
 834    --target-tags=radicle-node \
 835    --description="Radicle P2P port"
 836  
 837  # Optional: Allow HTTP API (restrict to your IP)
 838  gcloud compute firewall-rules create radicle-api \
 839    --allow=tcp:8777 \
 840    --source-ranges=YOUR_IP/32 \
 841    --target-tags=radicle-node
 842  ```
 843  
 844  #### 3. Deploy Application
 845  
 846  ```bash
 847  # SSH into instance
 848  gcloud compute ssh radicle-node --zone=us-central1-a
 849  
 850  # Install Rust
 851  curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
 852  source $HOME/.cargo/env
 853  
 854  # Clone and build
 855  git clone https://github.com/YOUR_USERNAME/radicle-secure.git
 856  cd radicle-secure
 857  cargo build --release
 858  
 859  # Install binaries
 860  sudo cp target/release/radicle-node /usr/local/bin/
 861  sudo cp target/release/rad /usr/local/bin/
 862  sudo cp target/release/git-remote-rad /usr/local/bin/
 863  
 864  # Setup systemd
 865  sudo cp systemd/radicle-node.service /etc/systemd/system/
 866  sudo cp systemd/radicle-node.socket /etc/systemd/system/
 867  sudo systemctl daemon-reload
 868  sudo systemctl enable radicle-node.socket
 869  sudo systemctl start radicle-node.socket
 870  ```
 871  
 872  #### 4. Initialize Node
 873  
 874  ```bash
 875  # Create radicle user
 876  sudo useradd -r -s /bin/false radicle
 877  sudo mkdir -p /var/lib/radicle
 878  sudo chown radicle:radicle /var/lib/radicle
 879  
 880  # Initialize as radicle user
 881  sudo -u radicle rad auth init
 882  sudo -u radicle rad node start
 883  ```
 884  
 885  ---
 886  
 887  ### Cost Monitoring
 888  
 889  ```bash
 890  # Create monitoring script
 891  cat > /usr/local/bin/radicle-cost-monitor.sh << 'EOF'
 892  #!/bin/bash
 893  
 894  # Monitor resource usage
 895  echo "=== Radicle Node Resource Usage ==="
 896  echo "Uptime: $(uptime -p)"
 897  echo "CPU: $(top -bn1 | grep "radicle-node" | awk '{print $9"%"}')"
 898  echo "Memory: $(ps aux | grep radicle-node | awk '{print $4"%"}')"
 899  echo "Disk: $(df -h /var/lib/radicle | tail -1 | awk '{print $3 "/" $2 " (" $5 ")"}')"
 900  
 901  # Estimate monthly costs
 902  HOURS_UP=$(awk '{print int($1/3600)}' /proc/uptime)
 903  MONTHLY_HOURS=$((HOURS_UP * 30 / $(date +%d)))
 904  COMPUTE_COST=$(echo "$MONTHLY_HOURS * 0.00508" | bc -l)
 905  STORAGE_COST=$(echo "$(df -BG /var/lib/radicle | tail -1 | awk '{print $3}' | tr -d 'G') * 0.04" | bc -l)
 906  
 907  echo ""
 908  echo "=== Estimated Monthly Costs ==="
 909  printf "Compute (e2-micro): \$%.2f\n" $COMPUTE_COST
 910  printf "Storage (HDD): \$%.2f\n" $STORAGE_COST
 911  printf "Total: \$%.2f\n" $(echo "$COMPUTE_COST + $STORAGE_COST" | bc -l)
 912  EOF
 913  
 914  chmod +x /usr/local/bin/radicle-cost-monitor.sh
 915  
 916  # Add to crontab for daily reports
 917  (crontab -l 2>/dev/null; echo "0 9 * * * /usr/local/bin/radicle-cost-monitor.sh | mail -s 'Radicle Daily Report' you@example.com") | crontab -
 918  ```
 919  
 920  ---
 921  
 922  ## Monitoring & Maintenance
 923  
 924  ### Health Check Endpoints
 925  
 926  ```rust
 927  // Add to crates/radicle-node/src/api/health.rs
 928  use axum::{Json, response::IntoResponse};
 929  use serde_json::json;
 930  
 931  pub async fn health_check() -> impl IntoResponse {
 932      Json(json!({
 933          "status": "healthy",
 934          "version": env!("CARGO_PKG_VERSION"),
 935          "uptime": get_uptime(),
 936      }))
 937  }
 938  
 939  pub async fn metrics() -> impl IntoResponse {
 940      Json(json!({
 941          "peers": get_peer_count(),
 942          "repos": get_repo_count(),
 943          "disk_usage": get_disk_usage(),
 944          "memory_usage": get_memory_usage(),
 945      }))
 946  }
 947  ```
 948  
 949  ### Logging Configuration
 950  
 951  ```bash
 952  # Configure log rotation
 953  sudo cat > /etc/logrotate.d/radicle << EOF
 954  /var/log/radicle/*.log {
 955      daily
 956      rotate 7
 957      compress
 958      delaycompress
 959      missingok
 960      notifempty
 961      create 0640 radicle radicle
 962  }
 963  EOF
 964  ```
 965  
 966  ### Backup Strategy
 967  
 968  ```bash
 969  # Automated backup script
 970  cat > /usr/local/bin/radicle-backup.sh << 'EOF'
 971  #!/bin/bash
 972  
 973  BACKUP_DATE=$(date +%Y%m%d)
 974  BACKUP_DIR="/var/backups/radicle"
 975  DATA_DIR="/var/lib/radicle"
 976  
 977  mkdir -p $BACKUP_DIR
 978  
 979  # Backup repositories
 980  tar -czf $BACKUP_DIR/repos-$BACKUP_DATE.tar.gz $DATA_DIR/repos
 981  
 982  # Backup configuration
 983  tar -czf $BACKUP_DIR/config-$BACKUP_DATE.tar.gz $DATA_DIR/config
 984  
 985  # Upload to GCS (optional)
 986  gsutil cp $BACKUP_DIR/repos-$BACKUP_DATE.tar.gz gs://your-backup-bucket/
 987  gsutil cp $BACKUP_DIR/config-$BACKUP_DATE.tar.gz gs://your-backup-bucket/
 988  
 989  # Clean old backups (keep 7 days)
 990  find $BACKUP_DIR -name "*.tar.gz" -mtime +7 -delete
 991  
 992  echo "Backup completed: $BACKUP_DATE"
 993  EOF
 994  
 995  chmod +x /usr/local/bin/radicle-backup.sh
 996  
 997  # Schedule daily backups
 998  (crontab -l 2>/dev/null; echo "0 2 * * * /usr/local/bin/radicle-backup.sh") | crontab -
 999  ```
1000  
1001  ---
1002  
1003  ## Final Cost Comparison
1004  
1005  | Configuration                             | Monthly Cost | Savings |
1006  |-------------------------------------------|--------------|---------|
1007  | **Stock Radicle (e2-small)**              | $30          | Baseline |
1008  | **+ Secret Scanning**                     | $32          | -$2 (minimal overhead) |
1009  | **+ Phase 1 Optimizations**               | $20          | 33% |
1010  | **+ Phase 2 Optimizations**               | $12          | 60% |
1011  | **+ Phase 3 Optimizations**               | $8           | 73% |
1012  | **Full Optimization (e2-micro + aggressive)** | $3-5     | 83-90% |
1013  
1014  ---
1015  
1016  ## Security Best Practices
1017  
1018  ### 1. Secret Scanning Pre-Commit Hook
1019  
1020  ```bash
1021  # Install pre-commit hook
1022  cat > .git/hooks/pre-commit << 'EOF'
1023  #!/bin/bash
1024  
1025  # Run secret scanner on staged files
1026  rad scan --staged
1027  
1028  if [ $? -ne 0 ]; then
1029      echo "❌ Secret scanning failed. Commit aborted."
1030      exit 1
1031  fi
1032  
1033  echo "✅ Secret scanning passed."
1034  exit 0
1035  EOF
1036  
1037  chmod +x .git/hooks/pre-commit
1038  ```
1039  
1040  ### 2. Regular Vulnerability Audits
1041  
1042  ```bash
1043  # Weekly vulnerability scan
1044  cat > /etc/cron.weekly/radicle-audit << 'EOF'
1045  #!/bin/bash
1046  
1047  cd /opt/radicle-secure
1048  cargo audit >> /var/log/radicle/audit.log 2>&1
1049  
1050  # Alert if vulnerabilities found
1051  if grep -q "Vulnerabilities found" /var/log/radicle/audit.log; then
1052      echo "Vulnerabilities detected in Radicle dependencies" | \
1053          mail -s "Radicle Security Alert" admin@example.com
1054  fi
1055  EOF
1056  
1057  chmod +x /etc/cron.weekly/radicle-audit
1058  ```
1059  
1060  ### 3. Access Control
1061  
1062  ```bash
1063  # Restrict node access to specific peers
1064  rad config set node.policy restricted
1065  rad config set node.allowed-peers "peer1,peer2,peer3"
1066  
1067  # Enable signature verification
1068  rad config set node.verify-signatures true
1069  ```
1070  
1071  ---
1072  
1073  ## Troubleshooting
1074  
1075  ### Node Won't Start
1076  
1077  ```bash
1078  # Check systemd status
1079  sudo systemctl status radicle-node
1080  
1081  # View logs
1082  sudo journalctl -u radicle-node -f
1083  
1084  # Check socket activation
1085  sudo systemctl status radicle-node.socket
1086  
1087  # Verify permissions
1088  sudo ls -la /var/lib/radicle
1089  ```
1090  
1091  ### High Bandwidth Usage
1092  
1093  ```bash
1094  # Monitor network traffic
1095  sudo iftop -i eth0
1096  
1097  # Check peer connections
1098  rad node peers
1099  
1100  # Limit replication bandwidth
1101  rad config set node.max-bandwidth 1MB
1102  ```
1103  
1104  ### Storage Issues
1105  
1106  ```bash
1107  # Check disk usage
1108  du -sh /var/lib/radicle/*
1109  
1110  # Run garbage collection
1111  rad node gc
1112  
1113  # Archive old data
1114  /usr/local/bin/radicle-archive-cold-data.sh
1115  ```
1116  
1117  ---
1118  
1119  ## Next Steps
1120  
1121  1. **Complete Phase 1** implementation (security features)
1122  2. **Test** thoroughly in local environment
1123  3. **Deploy** to GCP e2-micro instance
1124  4. **Monitor** costs and performance for 1 week
1125  5. **Iterate** on Phase 2 & 3 optimizations
1126  6. **Document** team workflows and best practices
1127  
1128  ---
1129  
1130  ## References
1131  
1132  - [Radicle Documentation](https://radicle.xyz/guides)
1133  - [RustSec Advisory Database](https://rustsec.org/)
1134  - [Secretscan Crate](https://crates.io/crates/secretscan)
1135  - [GCP Pricing Calculator](https://cloud.google.com/products/calculator)
1136  - [Zstd Compression](https://facebook.github.io/zstd/)
1137  
1138  ---
1139  
1140  **Last Updated:** 2025-11-02
1141  **Version:** 1.0
1142  **Author:** Radicle Secure Project