scraper.log
1 2025-11-29 00:22:44 | INFO | ============================================================ 2 2025-11-29 00:22:44 | INFO | News Scraper Daemon Starting 3 2025-11-29 00:22:44 | INFO | Fork: Hilo-Hilo/Hong-Kong-Fire-Documentary 4 2025-11-29 00:22:44 | INFO | Upstream: Hong-Kong-Emergency-Coordination-Hub/Hong-Kong-Fire-Documentary 5 2025-11-29 00:22:44 | INFO | Sync interval: 10 minutes 6 2025-11-29 00:22:44 | INFO | PR interval: 60 minutes 7 2025-11-29 00:22:44 | INFO | ============================================================ 8 2025-11-29 00:22:44 | INFO | Setting up git remotes... 9 2025-11-29 00:22:44 | INFO | Configured origin remote with authentication 10 2025-11-29 00:22:44 | INFO | ---------------------------------------- 11 2025-11-29 00:22:44 | INFO | Starting sync cycle... 12 2025-11-29 00:22:44 | INFO | Syncing with upstream... 13 2025-11-29 00:22:45 | INFO | Already up to date with upstream 14 2025-11-29 00:22:45 | INFO | Running scraper... 15 2025-11-29 00:22:45 | INFO | Found 492 new URLs to scrape 16 2025-11-29 01:02:49 | INFO | ============================================================ 17 2025-11-29 01:02:49 | INFO | News Scraper Daemon Starting 18 2025-11-29 01:02:49 | INFO | Fork: TestUser/TestRepo 19 2025-11-29 01:02:49 | INFO | Upstream: Hong-Kong-Emergency-Coordination-Hub/Hong-Kong-Fire-Documentary 20 2025-11-29 01:02:49 | INFO | Sync interval: 10 minutes 21 2025-11-29 01:02:49 | INFO | PR interval: 60 minutes 22 2025-11-29 01:02:49 | INFO | ============================================================ 23 2025-11-29 01:02:49 | INFO | Setting up git remotes... 24 2025-11-29 01:02:49 | INFO | Configured origin remote with authentication 25 2025-11-29 01:02:49 | INFO | ---------------------------------------- 26 2025-11-29 01:02:49 | INFO | Starting sync cycle... 27 2025-11-29 01:02:49 | INFO | Syncing with upstream... 28 2025-11-29 01:02:52 | INFO | Behind upstream by 8 commits, merging... 29 2025-11-29 01:02:54 | ERROR | Future exception was never retrieved 30 future: <Future finished exception=TargetClosedError('Target page, context or browser has been closed')> 31 playwright._impl._errors.TargetClosedError: Target page, context or browser has been closed 32 2025-11-29 01:02:54 | ERROR | Future exception was never retrieved 33 future: <Future finished exception=TargetClosedError('Target page, context or browser has been closed')> 34 playwright._impl._errors.TargetClosedError: Target page, context or browser has been closed 35 2025-11-29 01:02:54 | ERROR | Future exception was never retrieved 36 future: <Future finished exception=TargetClosedError('Target page, context or browser has been closed')> 37 playwright._impl._errors.TargetClosedError: Target page, context or browser has been closed 38 2025-11-29 01:02:54 | ERROR | Future exception was never retrieved 39 future: <Future finished exception=TargetClosedError('Target page, context or browser has been closed')> 40 playwright._impl._errors.TargetClosedError: Target page, context or browser has been closed 41 2025-11-29 01:02:54 | ERROR | Future exception was never retrieved 42 future: <Future finished exception=TargetClosedError('Target page, context or browser has been closed')> 43 playwright._impl._errors.TargetClosedError: Target page, context or browser has been closed 44 2025-11-29 01:40:08 | INFO | ============================================================ 45 2025-11-29 01:40:08 | INFO | News Scraper Daemon Starting 46 2025-11-29 01:40:08 | ERROR | FORK_REPO environment variable not set! 47 2025-11-29 01:40:08 | ERROR | Please set it: export FORK_REPO='username/repo-name' 48 2025-11-29 01:44:20 | INFO | ============================================================ 49 2025-11-29 01:44:20 | INFO | News Scraper Daemon Starting 50 2025-11-29 01:44:20 | INFO | Fork: Hilo-Hilo/Hong-Kong-Fire-Documentary 51 2025-11-29 01:44:20 | INFO | Upstream: Hong-Kong-Emergency-Coordination-Hub/Hong-Kong-Fire-Documentary 52 2025-11-29 01:44:20 | INFO | Sync interval: 10 minutes 53 2025-11-29 01:44:20 | INFO | PR interval: 60 minutes 54 2025-11-29 01:44:20 | INFO | ============================================================ 55 2025-11-29 01:44:20 | INFO | Setting up git remotes... 56 2025-11-29 01:44:20 | INFO | Added upstream remote: https://github.com/Hong-Kong-Emergency-Coordination-Hub/Hong-Kong-Fire-Documentary.git 57 2025-11-29 01:44:20 | INFO | Configured origin remote with authentication 58 2025-11-29 01:44:20 | INFO | ---------------------------------------- 59 2025-11-29 01:44:20 | INFO | Starting sync cycle... 60 2025-11-29 01:44:20 | INFO | Syncing with upstream... 61 2025-11-29 01:44:25 | INFO | Already up to date with upstream 62 2025-11-29 01:44:25 | INFO | Running scraper... 63 2025-11-29 01:44:25 | INFO | Found 402 new URLs to scrape 64 2025-11-29 02:05:03 | INFO | Scraper results: 402 success, 0 failed 65 2025-11-29 02:05:03 | INFO | Committing changes... 66 2025-11-29 02:05:05 | INFO | Committed: chore(scraper): auto-scrape 2025-11-29 02:05 67 2025-11-29 02:05:05 | INFO | Committed log updates 68 2025-11-29 02:05:05 | INFO | Sync cycle complete 69 2025-11-29 02:05:05 | INFO | ---------------------------------------- 70 2025-11-29 02:05:05 | INFO | Starting PR cycle... 71 2025-11-29 02:05:20 | ERROR | Command failed: git push origin main 72 2025-11-29 02:05:20 | ERROR | stderr: fatal: unable to access 'https://github.com/Hilo-Hilo/Hong-Kong-Fire-Documentary.git/': Could not resolve host: github.com 73 74 2025-11-29 02:05:20 | INFO | Managing PR... 75 2025-11-29 02:05:30 | ERROR | Failed to check for open PRs: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/Hong-Kong-Emergency-Coordination-Hub/Hong-Kong-Fire-Documentary/pulls?head=Hilo-Hilo%3Ascraper-updates&state=open (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0xe8a37cc35400>: Failed to resolve 'api.github.com' ([Errno -3] Temporary failure in name resolution)")) 76 2025-11-29 02:05:30 | INFO | Pushing to branch 'scraper-updates'... 77 2025-11-29 02:05:55 | ERROR | Command failed: git push origin scraper-updates --force 78 2025-11-29 02:05:55 | ERROR | stderr: fatal: unable to access 'https://github.com/Hilo-Hilo/Hong-Kong-Fire-Documentary.git/': Could not resolve host: github.com 79 80 2025-11-29 02:05:55 | ERROR | Failed to push: Command '['git', 'push', 'origin', 'scraper-updates', '--force']' returned non-zero exit status 128. 81 2025-11-29 02:05:55 | ERROR | Failed to push to PR branch 82 2025-11-29 02:05:55 | INFO | PR cycle complete 83 2025-11-29 02:05:55 | INFO | Run once mode, exiting... 84 2025-11-29 02:10:46 | INFO | ============================================================ 85 2025-11-29 02:10:46 | INFO | News Scraper Daemon Starting 86 2025-11-29 02:10:46 | INFO | Fork: Hilo-Hilo/Hong-Kong-Fire-Documentary 87 2025-11-29 02:10:46 | INFO | Upstream: Hong-Kong-Emergency-Coordination-Hub/Hong-Kong-Fire-Documentary 88 2025-11-29 02:10:46 | INFO | Sync interval: 10 minutes 89 2025-11-29 02:10:46 | INFO | PR interval: 60 minutes 90 2025-11-29 02:10:46 | INFO | ============================================================ 91 2025-11-29 02:10:46 | INFO | Setting up git remotes... 92 2025-11-29 02:10:46 | INFO | Configured origin remote with authentication 93 2025-11-29 02:10:46 | INFO | ---------------------------------------- 94 2025-11-29 02:10:46 | INFO | Starting sync cycle... 95 2025-11-29 02:10:46 | INFO | Syncing with upstream... 96 2025-11-29 02:10:52 | INFO | Already up to date with upstream 97 2025-11-29 02:10:52 | INFO | Running scraper... 98 2025-11-29 02:10:52 | INFO | Found 53 new URLs to scrape