Skip to content

Fix/18 auto continue on usage reset #32

Open
irmorteza wants to merge 3 commits into
Neokil:mainfrom
irmorteza:fix/18-auto-continue-on-usage-reset
Open

Fix/18 auto continue on usage reset #32
irmorteza wants to merge 3 commits into
Neokil:mainfrom
irmorteza:fix/18-auto-continue-on-usage-reset

Conversation

@irmorteza

Copy link
Copy Markdown
Contributor

Summary

  • add quota_monitor.go: background daemon tracks quotaReached flag and probes the provider every 15 minutes to detect when quota has reset
  • block workers via waitIfQuotaReached() when quota is hit and resume automatically
  • re-queue jobs on ErrTokensExhausted instead of marking them failed
  • add ProbeProvider() to Orchestrator for lightweight LLM ping
  • add FlowStatusRescheduled status for tickets waiting on quota reset
  • add rescheduled to OpenAPI schema for future WebUI support

Testing

  • go test ./...
  • set provider to bash -c 'echo "usage limit reached" >&2; exit 1' and run a ticket
  • observe ticket status changes to rescheduled and job re-queues automatically
  • after 20 minutes observe quota monitor probes and resumes workers

~/.auto-pr/config.yaml

provider: fake-exhausted

providers:
  fake-exhausted:
    command: bash
    args:
      - -c
      - 'echo "usage limit reached" >&2; exit 1'

Notes

  • system jobs (cleanup) blocked during quota pause will be addressed in a follow-up issue

Known Limitations

when a ticket is waiting for quota reset, all workers are paused including system jobs (cleanup, cleanup-all). this will be fixed in a follow-up issue by splitting the job queue into llmJobs and systemJobs channels with dedicated worker pools.

Comment thread internal/server/jobs.go
persistErr := s.persistTicketFailure(repoID, repoRoot, ticket, repoRt, job, err)
if persistErr != nil {
return fmt.Errorf("%w (also failed to persist ticket failure: %w)", err, persistErr)
if errors.Is(err, providers.ErrTokensExhausted) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is already handled inside of the StartFlow -> runState or MoveToState -> transitionTo -> runState so the ticket is updated twice

@Neokil Neokil left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one comment on duplicate state saves and also please execute the golangi-lint run and make sure the issues (like empty lines) are resolved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants