ralph-circuit-breaker.md (10500B)
1 # Ralph Circuit Breaker — Model Rate Limit Fallback 2 3 > Classic circuit breaker pattern (Hystrix / Polly / Resilience4j) applied to Copilot model selection. 4 > When the preferred model hits rate limits, Ralph automatically degrades to free-tier models, then self-heals. 5 6 ## Problem 7 8 When running multiple Ralph instances across repos, Copilot model rate limits cause cascading failures. 9 All Ralphs fail simultaneously when the preferred model (e.g., `claude-sonnet-4.6`) hits quota. 10 11 Premium models burn quota fast: 12 | Model | Multiplier | Risk | 13 |-------|-----------|------| 14 | `claude-sonnet-4.6` | 1x | Moderate with many Ralphs | 15 | `claude-opus-4.6` | 10x | High | 16 | `gpt-5.4` | 50x | Very high | 17 | `gpt-5.4-mini` | **0x** | **Free — unlimited** | 18 | `gpt-5-mini` | **0x** | **Free — unlimited** | 19 | `gpt-4.1` | **0x** | **Free — unlimited** | 20 21 ## Circuit Breaker States 22 23 ``` 24 ┌─────────┐ rate limit error ┌────────┐ 25 │ CLOSED │ ───────────────────► │ OPEN │ 26 │ (normal)│ │(fallback)│ 27 └────┬────┘ ◄──────────────── └────┬────┘ 28 │ 2 consecutive │ 29 │ successes │ cooldown expires 30 │ ▼ 31 │ ┌──────────┐ 32 └───── success ◄──────── │HALF-OPEN │ 33 (close) │ (testing) │ 34 └──────────┘ 35 ``` 36 37 ### CLOSED (normal operation) 38 - Use preferred model from config 39 - Every successful response confirms circuit stays closed 40 - On rate limit error → transition to OPEN 41 42 ### OPEN (rate limited — fallback active) 43 - Fall back through the free-tier model chain: 44 1. `gpt-5.4-mini` 45 2. `gpt-5-mini` 46 3. `gpt-4.1` 47 - Start cooldown timer (default: 10 minutes) 48 - When cooldown expires → transition to HALF-OPEN 49 50 ### HALF-OPEN (testing recovery) 51 - Try preferred model again 52 - If 2 consecutive successes → transition to CLOSED 53 - If rate limit error → back to OPEN, reset cooldown 54 55 ## State File: `.squad/ralph-circuit-breaker.json` 56 57 ```json 58 { 59 "state": "closed", 60 "preferredModel": "claude-sonnet-4.6", 61 "fallbackChain": ["gpt-5.4-mini", "gpt-5-mini", "gpt-4.1"], 62 "currentFallbackIndex": 0, 63 "cooldownMinutes": 10, 64 "openedAt": null, 65 "halfOpenSuccesses": 0, 66 "consecutiveFailures": 0, 67 "metrics": { 68 "totalFallbacks": 0, 69 "totalRecoveries": 0, 70 "lastFallbackAt": null, 71 "lastRecoveryAt": null 72 } 73 } 74 ``` 75 76 ## PowerShell Functions 77 78 Paste these into your `ralph-watch.ps1` or source them from a shared module. 79 80 ### `Get-CircuitBreakerState` 81 82 ```powershell 83 function Get-CircuitBreakerState { 84 param([string]$StateFile = ".squad/ralph-circuit-breaker.json") 85 86 if (-not (Test-Path $StateFile)) { 87 $default = @{ 88 state = "closed" 89 preferredModel = "claude-sonnet-4.6" 90 fallbackChain = @("gpt-5.4-mini", "gpt-5-mini", "gpt-4.1") 91 currentFallbackIndex = 0 92 cooldownMinutes = 10 93 openedAt = $null 94 halfOpenSuccesses = 0 95 consecutiveFailures = 0 96 metrics = @{ 97 totalFallbacks = 0 98 totalRecoveries = 0 99 lastFallbackAt = $null 100 lastRecoveryAt = $null 101 } 102 } 103 $default | ConvertTo-Json -Depth 3 | Set-Content $StateFile 104 return $default 105 } 106 107 return (Get-Content $StateFile -Raw | ConvertFrom-Json) 108 } 109 ``` 110 111 ### `Save-CircuitBreakerState` 112 113 ```powershell 114 function Save-CircuitBreakerState { 115 param( 116 [object]$State, 117 [string]$StateFile = ".squad/ralph-circuit-breaker.json" 118 ) 119 120 $State | ConvertTo-Json -Depth 3 | Set-Content $StateFile 121 } 122 ``` 123 124 ### `Get-CurrentModel` 125 126 Returns the model Ralph should use right now, based on circuit state. 127 128 ```powershell 129 function Get-CurrentModel { 130 param([string]$StateFile = ".squad/ralph-circuit-breaker.json") 131 132 $cb = Get-CircuitBreakerState -StateFile $StateFile 133 134 switch ($cb.state) { 135 "closed" { 136 return $cb.preferredModel 137 } 138 "open" { 139 # Check if cooldown has expired 140 if ($cb.openedAt) { 141 $opened = [DateTime]::Parse($cb.openedAt) 142 $elapsed = (Get-Date) - $opened 143 if ($elapsed.TotalMinutes -ge $cb.cooldownMinutes) { 144 # Transition to half-open 145 $cb.state = "half-open" 146 $cb.halfOpenSuccesses = 0 147 Save-CircuitBreakerState -State $cb -StateFile $StateFile 148 Write-Host " [circuit-breaker] Cooldown expired. Testing preferred model..." -ForegroundColor Yellow 149 return $cb.preferredModel 150 } 151 } 152 # Still in cooldown — use fallback 153 $idx = [Math]::Min($cb.currentFallbackIndex, $cb.fallbackChain.Count - 1) 154 return $cb.fallbackChain[$idx] 155 } 156 "half-open" { 157 return $cb.preferredModel 158 } 159 default { 160 return $cb.preferredModel 161 } 162 } 163 } 164 ``` 165 166 ### `Update-CircuitBreakerOnSuccess` 167 168 Call after every successful model response. 169 170 ```powershell 171 function Update-CircuitBreakerOnSuccess { 172 param([string]$StateFile = ".squad/ralph-circuit-breaker.json") 173 174 $cb = Get-CircuitBreakerState -StateFile $StateFile 175 $cb.consecutiveFailures = 0 176 177 if ($cb.state -eq "half-open") { 178 $cb.halfOpenSuccesses++ 179 if ($cb.halfOpenSuccesses -ge 2) { 180 # Recovery! Close the circuit 181 $cb.state = "closed" 182 $cb.openedAt = $null 183 $cb.halfOpenSuccesses = 0 184 $cb.currentFallbackIndex = 0 185 $cb.metrics.totalRecoveries++ 186 $cb.metrics.lastRecoveryAt = (Get-Date).ToString("o") 187 Save-CircuitBreakerState -State $cb -StateFile $StateFile 188 Write-Host " [circuit-breaker] RECOVERED — back to preferred model ($($cb.preferredModel))" -ForegroundColor Green 189 return 190 } 191 Save-CircuitBreakerState -State $cb -StateFile $StateFile 192 Write-Host " [circuit-breaker] Half-open success $($cb.halfOpenSuccesses)/2" -ForegroundColor Yellow 193 return 194 } 195 196 # closed state — nothing to do 197 } 198 ``` 199 200 ### `Update-CircuitBreakerOnRateLimit` 201 202 Call when a model response indicates rate limiting (HTTP 429 or error message containing "rate limit"). 203 204 ```powershell 205 function Update-CircuitBreakerOnRateLimit { 206 param([string]$StateFile = ".squad/ralph-circuit-breaker.json") 207 208 $cb = Get-CircuitBreakerState -StateFile $StateFile 209 $cb.consecutiveFailures++ 210 211 if ($cb.state -eq "closed" -or $cb.state -eq "half-open") { 212 # Open the circuit 213 $cb.state = "open" 214 $cb.openedAt = (Get-Date).ToString("o") 215 $cb.halfOpenSuccesses = 0 216 $cb.currentFallbackIndex = 0 217 $cb.metrics.totalFallbacks++ 218 $cb.metrics.lastFallbackAt = (Get-Date).ToString("o") 219 Save-CircuitBreakerState -State $cb -StateFile $StateFile 220 221 $fallbackModel = $cb.fallbackChain[0] 222 Write-Host " [circuit-breaker] RATE LIMITED — falling back to $fallbackModel (cooldown: $($cb.cooldownMinutes)m)" -ForegroundColor Red 223 return 224 } 225 226 if ($cb.state -eq "open") { 227 # Already open — try next fallback in chain if current one also fails 228 if ($cb.currentFallbackIndex -lt ($cb.fallbackChain.Count - 1)) { 229 $cb.currentFallbackIndex++ 230 $nextModel = $cb.fallbackChain[$cb.currentFallbackIndex] 231 Write-Host " [circuit-breaker] Fallback also limited — trying $nextModel" -ForegroundColor Red 232 } 233 # Reset cooldown timer 234 $cb.openedAt = (Get-Date).ToString("o") 235 Save-CircuitBreakerState -State $cb -StateFile $StateFile 236 } 237 } 238 ``` 239 240 ## Integration with ralph-watch.ps1 241 242 In your Ralph polling loop, wrap the model selection: 243 244 ```powershell 245 # At the top of your polling loop 246 $model = Get-CurrentModel 247 248 # When invoking copilot CLI 249 $result = copilot-cli --model $model ... 250 251 # After the call 252 if ($result -match "rate.?limit" -or $LASTEXITCODE -eq 429) { 253 Update-CircuitBreakerOnRateLimit 254 } else { 255 Update-CircuitBreakerOnSuccess 256 } 257 ``` 258 259 ### Full integration example 260 261 ```powershell 262 # Source the circuit breaker functions 263 . .squad-templates/ralph-circuit-breaker-functions.ps1 264 265 while ($true) { 266 $model = Get-CurrentModel 267 Write-Host "Polling with model: $model" 268 269 try { 270 # Your existing Ralph logic here, but pass $model 271 $response = Invoke-RalphCycle -Model $model 272 273 # Success path 274 Update-CircuitBreakerOnSuccess 275 } 276 catch { 277 if ($_.Exception.Message -match "rate.?limit|429|quota|Too Many Requests") { 278 Update-CircuitBreakerOnRateLimit 279 # Retry immediately with fallback model 280 continue 281 } 282 # Other errors — handle normally 283 throw 284 } 285 286 Start-Sleep -Seconds $pollInterval 287 } 288 ``` 289 290 ## Configuration 291 292 Override defaults by editing `.squad/ralph-circuit-breaker.json`: 293 294 | Field | Default | Description | 295 |-------|---------|-------------| 296 | `preferredModel` | `claude-sonnet-4.6` | Model to use when circuit is closed | 297 | `fallbackChain` | `["gpt-5.4-mini", "gpt-5-mini", "gpt-4.1"]` | Ordered fallback models (all free-tier) | 298 | `cooldownMinutes` | `10` | How long to wait before testing recovery | 299 300 ## Metrics 301 302 The state file tracks operational metrics: 303 304 - **totalFallbacks** — How many times the circuit opened 305 - **totalRecoveries** — How many times it recovered to preferred model 306 - **lastFallbackAt** — ISO timestamp of last rate limit event 307 - **lastRecoveryAt** — ISO timestamp of last successful recovery 308 309 Query metrics with: 310 ```powershell 311 $cb = Get-Content .squad/ralph-circuit-breaker.json | ConvertFrom-Json 312 Write-Host "Fallbacks: $($cb.metrics.totalFallbacks) | Recoveries: $($cb.metrics.totalRecoveries)" 313 ```