1  # Field Notes: Production AI — Edition 4
  2  
  3  ## We Open-Sourced the Scoring Engine. We Kept the Self-Healing Loop.
  4  
  5  Today I am releasing argus-ai to the world.
  6  
  7  `pip install argus-ai`
  8  
  9  Three lines of code. Every LLM call in your application now has a quality score.
 10  
 11  ```python
 12  import argus_ai
 13  argus = argus_ai.init()
 14  result = argus.evaluate(prompt=prompt, response=response, context=context)
 15  ```
 16  
 17  Here is what that score tells you.
 18  
 19  ---
 20  
 21  ### The Problem Nobody Is Measuring
 22  
 23  Your LLM application is degrading right now. You cannot see it because you are not measuring it.
 24  
 25  Traditional observability catches latency spikes, error rates, and throughput drops. It does not catch a model that starts hallucinating 12% more after a provider update. It does not catch a prompt that silently loses grounding when context exceeds 80K tokens. It does not catch cost creep from token bloat that accumulates over weeks.
 26  
 27  I have watched this happen across every production LLM deployment I have worked on. At Duke Energy. At UnitedHealth Group. At R1 RCM. The pattern is always the same: the app works great at launch, then quietly degrades while traditional metrics show green across the board.
 28  
 29  ---
 30  
 31  ### G-ARVIS: Six Dimensions of LLM Quality
 32  
 33  The G-ARVIS framework evaluates every LLM response across six orthogonal quality dimensions:
 34  
 35  **G** Groundedness. Is the response anchored in provided context or is it fabricating?
 36  
 37  **A** Accuracy. Does it match ground truth? Is it internally consistent? Are numeric claims valid?
 38  
 39  **R** Reliability. Is the format consistent? Is it complete or truncated? Does latency meet SLA?
 40  
 41  **V** Variance. How deterministic is the output? How confident? How stable across similar inputs?
 42  
 43  **I** Inference Cost. Are tokens being used efficiently? Is cost proportionate to value delivered?
 44  
 45  **S** Safety. PII leakage? Toxicity? Prompt injection? Harmful content?
 46  
 47  Each dimension produces a 0-to-1 score. The weighted composite tells you, in a single number, whether your LLM is performing at production grade.
 48  
 49  ---
 50  
 51  ### What Is New: Agentic Evaluation Metrics
 52  
 53  In Edition 3 of this newsletter I introduced three metrics that address the evaluation gap for autonomous agent workflows. They are now part of argus-ai.
 54  
 55  **ASF** Agent Stability Factor. Completion rate multiplied by failure resilience multiplied by retry consistency. Measures whether your agent reliably finishes what it starts.
 56  
 57  **ERR** Error Recovery Rate. Recovered steps divided by failed steps. Measures whether your agent self-corrects or cascades failures.
 58  
 59  **CPCS** Cost Per Completed Step. Total spend normalized against successfully completed workflow steps. Measures economic efficiency of autonomous execution.
 60  
 61  These are the metrics that traditional LLM evaluation frameworks do not have. BLEU, ROUGE, perplexity were designed for static text generation. They tell you nothing about whether a 10-step agent workflow will survive its third tool call failure and still deliver the result.
 62  
 63  ---
 64  
 65  ### Why Open Source
 66  
 67  I built ARGUS as a full platform: scoring, monitoring, autonomous correction, and self-healing. The question was always which layers to open.
 68  
 69  The answer is the layer that creates dependency.
 70  
 71  argus-ai gives you the G-ARVIS scoring engine, threshold monitoring with sliding window breach detection, Prometheus and OpenTelemetry export, and drop-in wrappers for Anthropic and OpenAI. Install it, plug it into your LLM pipeline, and suddenly you can see the degradation you could not see before.
 72  
 73  What it does not give you is the fix. Detection without correction is a dashboard you stare at while your app degrades. The autonomous correction loop, the prompt optimizer, the closed-loop self-healing pipeline: that is ARGUS Platform. That is where the road leads.
 74  
 75  ---
 76  
 77  ### The Numbers
 78  
 79  Sub-5ms evaluation latency. 84 unit tests. 93% code coverage on the scoring core. Three runtime dependencies. Five pre-built weight profiles for enterprise, healthcare, finance, consumer, and agentic workloads. Prometheus and OTEL export out of the box. Python 3.9 through 3.13.
 80  
 81  The package is on PyPI today. The repo is at github.com/anilatambharii/argus-ai.
 82  
 83  ---
 84  
 85  ### Try It
 86  
 87  ```bash
 88  pip install argus-ai
 89  ```
 90  
 91  ```python
 92  import argus_ai
 93  
 94  argus = argus_ai.init(profile="enterprise")
 95  
 96  result = argus.evaluate(
 97      prompt="What are the Q3 revenue trends?",
 98      response="Revenue increased 12% year-over-year to $4.2B...",
 99      context="Q3 2024 financial report data...",
100      model_name="claude-sonnet-4",
101      latency_ms=1200.0,
102  )
103  
104  print(result.garvis_composite)   # 0.847
105  print(result.passing)            # True
106  ```
107  
108  If you are running LLMs in production, you need this. Not because I built it. Because your app is degrading and you cannot see it yet.
109  
110  Star the repo. File issues. Contribute scorers and exporters. This is the foundation.
111  
112  The self-healing loop comes next.
113  
114  Anil Prasad
115  Head of Engineering and Product
116  Ambharii Labs
117  
118  github.com/anilatambharii/argus-ai
119  anilsprasad.com | ambharii.com
120  
121  #HumanWritten #ProductionAI #LLMOps #OpenSource #GARVIS #ARGUS