Skip to content

SUPPORT-17719 Wait for app response headers up to the upstream timeout - #2665

Merged
pepamartinec merged 1 commit into
mainfrom
pepa/SUPPORT-17719_response-header-timeout
Sep 30, 2026
Merged

pepamartinec merged 1 commit into
mainfrom
pepa/SUPPORT-17719_response-header-timeout

Conversation

@pepamartinec

Copy link
Copy Markdown
Contributor

Release Notes

https://keboola.atlassian.net/browse/SUPPORT-17719

Apps-proxy gave up on an app after 15 seconds if the app had not sent its response headers yet, and returned 502 Request to application failed.. This happened even though the whole request may take up to Upstream.HTTPTimeout (4m30s). Apps that do work before they answer (for example an MCP tool call that returns once the tool finishes) failed at exactly 15.00 s with net/http: timeout awaiting response headers. In the last 7 days, more than 25 apps on many stacks hit this error.

The upstream transport now takes its ResponseHeaderTimeout from Upstream.HTTPTimeout instead of the hard-coded 15 s constant. I kept an explicit header timeout instead of dropping it, because the E2B webhook reverse proxy uses the same transport and has no request deadline of its own. So HTTPTimeout stays the single upper bound, and the existing rule that it must stay below the LB timeout now also covers the header wait.

Plans for customer communication

Reply on SUPPORT-17719 after the rollout.

Impact analysis

Apps that send headers later than 15 s now work, up to Upstream.HTTPTimeout. An app that never answers now fails after 4m30s instead of 15 s. The same applies to the E2B webhook proxy.

Change type

Fix

Justification

Requests with a slow first byte failed with a 502 after 15 s, well below the upstream request timeout (SUPPORT-17719).

Deployment

Merge, release a production-apps-proxy-v* tag, automatic rollout to stacks.

Rollback plan

Revert of this PR.

Post release support plan

Check that apps-proxy stops logging timeout awaiting response headers after the rollout.

@pepamartinec
pepamartinec force-pushed the pepa/SUPPORT-17719_response-header-timeout branch from 5c1d319 to 5059eb2 Compare September 30, 2026 09:55
@pepamartinec
pepamartinec marked this pull request as ready for review September 30, 2026 10:13
Copilot AI balanced review requested due to automatic review settings September 30, 2026 10:13

@keboola-pr-reviewer-bot keboola-pr-reviewer-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: needs_human (risk 3/5) · profile psgo

Load-bearing apps-proxy request-path timeout change; small and well-tested, but policy sends it to a human.

Impact flags: possible rollback re-introduction — see Check Run summary.

Concerns:

  • internal/pkg/service/appsproxy/proxy/transport/transport.go: Never-responding apps now hang ~4m30s instead of 15s on request path.
  • internal/pkg/service/appsproxy/proxy/server.go: E2B webhook proxy has no request deadline; header wait rises to 4m30s.
  • internal/pkg/service/appsproxy/proxy/transport/transport_test.go: Test uses time.Sleep in handler; flake amplifier (flag, not block).

Suggested reviewers: @, this needs a human because it changes timeout behavior on the load-bearing apps-proxy request path (POLICY: load-bearing areas deserve a human even when small)., Please assign a PSGO teammate via the Reviewers panel who owns apps-proxy.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation consistently applies the configured timeout and includes focused regression coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Aligns upstream response-header waiting with the configured HTTP timeout, preventing premature 502 responses for slow-starting apps.

Changes:

  • Passes Upstream.HTTPTimeout into the shared transport.
  • Adds coverage for successful and timed-out response headers.
File Description
internal/​pkg/​service/​appsproxy/​proxy/​transport/​transport.go Makes the response-header timeout configurable.
internal/​pkg/​service/​appsproxy/​proxy/​transport/​transport_test.go Tests both timeout outcomes.
internal/​pkg/​service/​appsproxy/​dependencies/​dependencies.go Supplies the upstream HTTP timeout to the transport.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@pepamartinec
pepamartinec merged commit fb3a293 into main Sep 30, 2026
19 checks passed
@pepamartinec
pepamartinec deleted the pepa/SUPPORT-17719_response-header-timeout branch September 30, 2026 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants