github-pr-review-agent.md
1 --- 2 sidebar_position: 10 3 title: "Tutorial: GitHub PR Review Agent" 4 description: "Build an automated AI code reviewer that monitors your repos, reviews pull requests, and delivers feedback — hands-free" 5 --- 6 7 # Tutorial: Build a GitHub PR Review Agent 8 9 **The problem:** Your team opens PRs faster than you can review them. PRs sit for days waiting for eyeballs. Junior devs merge bugs because nobody had time to check. You spend your mornings catching up on diffs instead of building. 10 11 **The solution:** An AI agent that watches your repos around the clock, reviews every new PR for bugs, security issues, and code quality, and sends you a summary — so you only spend time on PRs that actually need human judgment. 12 13 **What you'll build:** 14 15 ``` 16 ┌───────────────────────────────────────────────────────────────────┐ 17 │ │ 18 │ Cron Timer ──▶ Hermes Agent ──▶ GitHub API ──▶ Review │ 19 │ (every 2h) + gh CLI (PR diffs) delivery │ 20 │ + skill (Telegram, │ 21 │ + memory Discord, │ 22 │ local) │ 23 │ │ 24 └───────────────────────────────────────────────────────────────────┘ 25 ``` 26 27 This guide uses **cron jobs** to poll for PRs on a schedule — no server or public endpoint needed. Works behind NAT and firewalls. 28 29 :::tip Want real-time reviews instead? 30 If you have a public endpoint available, check out [Automated GitHub PR Comments with Webhooks](./webhook-github-pr-review.md) — GitHub pushes events to Hermes instantly when PRs are opened or updated. 31 ::: 32 33 --- 34 35 ## Prerequisites 36 37 - **Hermes Agent installed** — see the [Installation guide](/docs/getting-started/installation) 38 - **Gateway running** for cron jobs: 39 ```bash 40 hermes gateway install # Install as a service 41 # or 42 hermes gateway # Run in foreground 43 ``` 44 - **GitHub CLI (`gh`) installed and authenticated**: 45 ```bash 46 # Install 47 brew install gh # macOS 48 sudo apt install gh # Ubuntu/Debian 49 50 # Authenticate 51 gh auth login 52 ``` 53 - **Messaging configured** (optional) — [Telegram](/docs/user-guide/messaging/telegram) or [Discord](/docs/user-guide/messaging/discord) 54 55 :::tip No messaging? No problem 56 Use `deliver: "local"` to save reviews to `~/.hermes/cron/output/`. Great for testing before wiring up notifications. 57 ::: 58 59 --- 60 61 ## Step 1: Verify the Setup 62 63 Make sure Hermes can access GitHub. Start a chat: 64 65 ```bash 66 hermes 67 ``` 68 69 Test with a simple command: 70 71 ``` 72 Run: gh pr list --repo NousResearch/hermes-agent --state open --limit 3 73 ``` 74 75 You should see a list of open PRs. If this works, you're ready. 76 77 --- 78 79 ## Step 2: Try a Manual Review 80 81 Still in the chat, ask Hermes to review a real PR: 82 83 ``` 84 Review this pull request. Read the diff, check for bugs, security issues, 85 and code quality. Be specific about line numbers and quote problematic code. 86 87 Run: gh pr diff 3888 --repo NousResearch/hermes-agent 88 ``` 89 90 Hermes will: 91 1. Execute `gh pr diff` to fetch the code changes 92 2. Read through the entire diff 93 3. Produce a structured review with specific findings 94 95 If you're happy with the quality, time to automate it. 96 97 --- 98 99 ## Step 3: Create a Review Skill 100 101 A skill gives Hermes consistent review guidelines that persist across sessions and cron runs. Without one, review quality varies. 102 103 ```bash 104 mkdir -p ~/.hermes/skills/code-review 105 ``` 106 107 Create `~/.hermes/skills/code-review/SKILL.md`: 108 109 ```markdown 110 --- 111 name: code-review 112 description: Review pull requests for bugs, security issues, and code quality 113 --- 114 115 # Code Review Guidelines 116 117 When reviewing a pull request: 118 119 ## What to Check 120 1. **Bugs** — Logic errors, off-by-one, null/undefined handling 121 2. **Security** — Injection, auth bypass, secrets in code, SSRF 122 3. **Performance** — N+1 queries, unbounded loops, memory leaks 123 4. **Style** — Naming conventions, dead code, missing error handling 124 5. **Tests** — Are changes tested? Do tests cover edge cases? 125 126 ## Output Format 127 For each finding: 128 - **File:Line** — exact location 129 - **Severity** — Critical / Warning / Suggestion 130 - **What's wrong** — one sentence 131 - **Fix** — how to fix it 132 133 ## Rules 134 - Be specific. Quote the problematic code. 135 - Don't flag style nitpicks unless they affect readability. 136 - If the PR looks good, say so. Don't invent problems. 137 - End with: APPROVE / REQUEST_CHANGES / COMMENT 138 ``` 139 140 Verify it loaded — start `hermes` and you should see `code-review` in the skills list at startup. 141 142 --- 143 144 ## Step 4: Teach It Your Conventions 145 146 This is what makes the reviewer actually useful. Start a session and teach Hermes your team's standards: 147 148 ``` 149 Remember: In our backend repo, we use Python with FastAPI. 150 All endpoints must have type annotations and Pydantic models. 151 We don't allow raw SQL — only SQLAlchemy ORM. 152 Test files go in tests/ and must use pytest fixtures. 153 ``` 154 155 ``` 156 Remember: In our frontend repo, we use TypeScript with React. 157 No `any` types allowed. All components must have props interfaces. 158 We use React Query for data fetching, never useEffect for API calls. 159 ``` 160 161 These memories persist forever — the reviewer will enforce your conventions without being told each time. 162 163 --- 164 165 ## Step 5: Create the Automated Cron Job 166 167 Now wire it all together. Create a cron job that runs every 2 hours: 168 169 ```bash 170 hermes cron create "0 */2 * * *" \ 171 "Check for new open PRs and review them. 172 173 Repos to monitor: 174 - myorg/backend-api 175 - myorg/frontend-app 176 177 Steps: 178 1. Run: gh pr list --repo REPO --state open --limit 5 --json number,title,author,createdAt 179 2. For each PR created or updated in the last 4 hours: 180 - Run: gh pr diff NUMBER --repo REPO 181 - Review the diff using the code-review guidelines 182 3. Format output as: 183 184 ## PR Reviews — today 185 186 ### [repo] #[number]: [title] 187 **Author:** [name] | **Verdict:** APPROVE/REQUEST_CHANGES/COMMENT 188 [findings] 189 190 If no new PRs found, say: No new PRs to review." \ 191 --name "pr-review" \ 192 --deliver telegram \ 193 --skill code-review 194 ``` 195 196 Verify it's scheduled: 197 198 ```bash 199 hermes cron list 200 ``` 201 202 ### Other useful schedules 203 204 | Schedule | When | 205 |----------|------| 206 | `0 */2 * * *` | Every 2 hours | 207 | `0 9,13,17 * * 1-5` | Three times a day, weekdays only | 208 | `0 9 * * 1` | Weekly Monday morning roundup | 209 | `30m` | Every 30 minutes (high-traffic repos) | 210 211 --- 212 213 ## Step 6: Run It On Demand 214 215 Don't want to wait for the schedule? Trigger it manually: 216 217 ```bash 218 hermes cron run pr-review 219 ``` 220 221 Or from within a chat session: 222 223 ``` 224 /cron run pr-review 225 ``` 226 227 --- 228 229 ## Going Further 230 231 ### Post Reviews Directly to GitHub 232 233 Instead of delivering to Telegram, have the agent comment on the PR itself: 234 235 Add this to your cron prompt: 236 237 ``` 238 After reviewing, post your review: 239 - For issues: gh pr review NUMBER --repo REPO --comment --body "YOUR_REVIEW" 240 - For critical issues: gh pr review NUMBER --repo REPO --request-changes --body "YOUR_REVIEW" 241 - For clean PRs: gh pr review NUMBER --repo REPO --approve --body "Looks good" 242 ``` 243 244 :::caution 245 Make sure `gh` has a token with `repo` scope. Reviews are posted as whoever `gh` is authenticated as. 246 ::: 247 248 ### Weekly PR Dashboard 249 250 Create a Monday morning overview of all your repos: 251 252 ```bash 253 hermes cron create "0 9 * * 1" \ 254 "Generate a weekly PR dashboard: 255 - myorg/backend-api 256 - myorg/frontend-app 257 - myorg/infra 258 259 For each repo show: 260 1. Open PR count and oldest PR age 261 2. PRs merged this week 262 3. Stale PRs (older than 5 days) 263 4. PRs with no reviewer assigned 264 265 Format as a clean summary." \ 266 --name "weekly-dashboard" \ 267 --deliver telegram 268 ``` 269 270 ### Multi-Repo Monitoring 271 272 Scale up by adding more repos to the prompt. The agent processes them sequentially — no extra setup needed. 273 274 --- 275 276 ## Troubleshooting 277 278 ### "gh: command not found" 279 The gateway runs in a minimal environment. Ensure `gh` is in the system PATH and restart the gateway. 280 281 ### Reviews are too generic 282 1. Add the `code-review` skill (Step 3) 283 2. Teach Hermes your conventions via memory (Step 4) 284 3. The more context it has about your stack, the better the reviews 285 286 ### Cron job doesn't run 287 ```bash 288 hermes gateway status # Is the gateway running? 289 hermes cron list # Is the job enabled? 290 ``` 291 292 ### Rate limits 293 GitHub allows 5,000 API requests/hour for authenticated users. Each PR review uses ~3-5 requests (list + diff + optional comments). Even reviewing 100 PRs/day stays well within limits. 294 295 --- 296 297 ## What's Next? 298 299 - **[Webhook-Based PR Reviews](./webhook-github-pr-review.md)** — get instant reviews when PRs are opened (requires a public endpoint) 300 - **[Daily Briefing Bot](/docs/guides/daily-briefing-bot)** — combine PR reviews with your morning news digest 301 - **[Build a Plugin](/docs/guides/build-a-hermes-plugin)** — wrap the review logic into a shareable plugin 302 - **[Profiles](/docs/user-guide/profiles)** — run a dedicated reviewer profile with its own memory and config 303 - **[Fallback Providers](/docs/user-guide/features/fallback-providers)** — ensure reviews run even when one provider is down