Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/getting-started/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,14 @@ docker compose -f "$HOME/.oac/core/compose.yaml" logs --tail 200 core

Don't paste `docker compose config`, `docker inspect` or raw logs into public issue reports.

### Environment file rejection logs

The Environment file creation handler emits one `api_request_rejected` event when it writes a response with HTTP status 400 or higher. Match its `request_id` to the response's `X-Request-Id`; Core generates this ID, so it need not match a caller's request ID. The existing log context also supplies `trace_id` and `span_id`, matching the response's `traceparent` whether the request carries a valid incoming trace or Core generates a new trace. This event covers `operation=environment_file_create` and the fixed route template `/v1/agents/environments/{environment_id}/files`. Requests rejected before this handler are outside its coverage.

The event contains `operation`, `route`, `status`, an allowlisted `code` (unrecognized codes become `unknown`), and a static `reason`. Reasons are `unknown`, `invalid_payload`, `unknown_field`, `invalid_path`, `invalid_base64`, `inline_too_large`, `hosted_environment_provisioning`, `destination_directory`, and `destination_unsafe`. `unknown` means the rejection has no classified reason; a 400 alone does not establish that an Environment is still provisioning.

The event does not record request or response bodies, query strings, raw URL paths, resource IDs, user file paths, field values, arbitrary error messages or credentials. It does not change the HTTP response or execution behavior. Check that the deployed Core version includes this diagnostic before relying on it; missing events do not establish success, and diagnostics alone do not show that file uploads work.

## Stop and restart

Let active work settle before a planned restart:
Expand Down
10 changes: 9 additions & 1 deletion docs/zh/getting-started/operations.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "管理你的安装"
source: docs/getting-started/operations.md
source_hash: e60a6e96cd61692b6adc7664874ba0ed2ce489982e7da2fe2347631ee2a94c0a
source_hash: 9c4b5a1bdcdecb5dd3c0b73886d1befb58c6725df0f35873559eab0fecd3e7cd
---

安装运维人员负责 Core 主机、存储和可用性。节点主机运行各自的服务;参阅[节点](nodes.md)。设置见[配置参考](../configuration.md)。
Expand Down Expand Up @@ -47,6 +47,14 @@ docker compose -f "$HOME/.oac/core/compose.yaml" logs --tail 200 core

不要将 `docker compose config`、`docker inspect` 或原始日志粘贴到公开问题报告。

### Environment 文件拒绝日志 {#environment-file-rejection-logs}

Environment 文件创建 handler 写出HTTP状态码为400或以上的响应时,会发出一条 `api_request_rejected` 事件。用其中的 `request_id` 关联响应的 `X-Request-Id`;该ID由Core生成,不一定与调用方的请求ID相同。既有日志上下文还会注入 `trace_id` 和 `span_id`;无论请求携带有效的入站trace,还是由Core生成新trace,这两个字段均与响应的 `traceparent` 一致。事件覆盖 `operation=environment_file_create`,路由固定为模板 `/v1/agents/environments/{environment_id}/files`。进入该handler之前被拒绝的请求不在其覆盖范围内。

事件包含 `operation`、`route`、`status`、白名单内的 `code`(未识别的code记为 `unknown`)以及静态 `reason`。reason取值为 `unknown`、`invalid_payload`、`unknown_field`、`invalid_path`、`invalid_base64`、`inline_too_large`、`hosted_environment_provisioning`、`destination_directory` 和 `destination_unsafe`。`unknown` 表示拒绝原因尚未分类;仅凭400不能判定Environment仍在准备中。

事件不记录请求或响应正文、查询字符串、原始URL路径、资源ID、用户文件路径、字段值、任意错误文案或凭据,也不改变HTTP响应和执行行为。使用前应核实已部署的Core版本包含此诊断;缺少事件不能证明成功,诊断信息本身也不能证明文件上传可用。

## 停止与重启 {#stop-and-restart}

计划重启前,先等待活动工作结束:
Expand Down
92 changes: 92 additions & 0 deletions services/core/internal/api/api_rejection.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
package api

import (
"context"
"net/http"

"github.com/MiniMax-AI/OpenAgentCore/internal/obs/log"
)

// Reasons are assigned only at explicit rejection branches, never from error
// messages or request data. The zero value leaves an unclassified failure unknown.
type apiRejectionReason uint8

const (
rejectionUnknown apiRejectionReason = iota
rejectionInvalidPayload
rejectionUnknownField
rejectionInvalidPath
rejectionInvalidBase64
rejectionInlineTooLarge
rejectionHostedEnvironmentProvisioning
rejectionDestinationDirectory
rejectionDestinationUnsafe
)

func (reason apiRejectionReason) String() string {
switch reason {
case rejectionInvalidPayload:
return "invalid_payload"
case rejectionUnknownField:
return "unknown_field"
case rejectionInvalidPath:
return "invalid_path"
case rejectionInvalidBase64:
return "invalid_base64"
case rejectionInlineTooLarge:
return "inline_too_large"
case rejectionHostedEnvironmentProvisioning:
return "hosted_environment_provisioning"
case rejectionDestinationDirectory:
return "destination_directory"
case rejectionDestinationUnsafe:
return "destination_unsafe"
default:
return "unknown"
}
}

// The existing response writer owns diagnostics as well as response timing.
// Unwrapping preserves the controller's deadline/flush path and adds no wrapper.
// A non-nil context enables the diagnostic using the handler context after
// tracing middleware has run. Later rejection branches pass nil to only mark a reason.
func environmentFileDiagnostic(w http.ResponseWriter, ctx context.Context, reason apiRejectionReason) {
for w != nil {
if writer, ok := w.(*processingTimeWriter); ok {
if !writer.stamped && (ctx != nil || writer.environmentFile) {
if ctx != nil {
writer.ctx = ctx
}
writer.environmentFile = true
writer.rejectionReason = reason
}
return
}
unwrapper, ok := w.(interface{ Unwrap() http.ResponseWriter })
if !ok {
return
}
w = unwrapper.Unwrap()
}
}

func safeAPIErrorCode(code string) string {
switch code {
case "invalid_request", "invalid_request_error", "not_found_error", "request_too_large", "conflict_error", "server_error", "execution_unavailable":
return code
default:
return "unknown"
}
}

func (w *processingTimeWriter) logRejection(status int) {
if !w.environmentFile || status < 400 || w.stamped {
return
}
log.Warn(w.ctx, "api_request_rejected",
"operation", "environment_file_create",
"route", "/v1/agents/environments/{environment_id}/files",
"status", status,
"code", safeAPIErrorCode(w.errorCode),
"reason", w.rejectionReason.String())
}
221 changes: 221 additions & 0 deletions services/core/internal/api/api_rejection_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
package api

import (
"bytes"
"encoding/json"
"fmt"
"log/slog"
"net/http"
"net/http/httptest"
"reflect"
"strings"
"testing"
"time"

obslog "github.com/MiniMax-AI/OpenAgentCore/internal/obs/log"
"github.com/MiniMax-AI/OpenAgentCore/services/core/internal/execution"
)

func rejectionLogBuffer(t *testing.T) *bytes.Buffer {
t.Helper()
var output bytes.Buffer
previous := slog.Default()
slog.SetDefault(slog.New(obslog.NewContextHandler(slog.NewJSONHandler(&output, nil))))
t.Cleanup(func() { slog.SetDefault(previous) })
return &output
}

func rejectionRecords(t *testing.T, output *bytes.Buffer) []map[string]any {
t.Helper()
var records []map[string]any
for _, line := range strings.Split(strings.TrimSpace(output.String()), "\n") {
if line == "" {
continue
}
var record map[string]any
if err := json.Unmarshal([]byte(line), &record); err != nil {
t.Fatal(err)
}
if record["msg"] == "api_request_rejected" {
records = append(records, record)
}
}
return records
}

func TestEnvironmentFileRejectionDiagnostics(t *testing.T) {
output := rejectionLogBuffer(t)
const canary = "private-canary-value"
const valid = `{"type":"inline","data":"YWJj","path":"/workspace/private-canary-value"}`
for _, tc := range []struct {
name, body, reason, code, message string
status int
err error
pending bool
param string
}{
{name: "payload", body: `{"type":"inline","path":"/workspace/private-canary-value"}`, reason: "invalid_payload", code: "invalid_request", message: invalidInputMessage, status: 400},
{name: "unknown field", body: `{"type":"inline","data":"YWJj","path":"/workspace/a","private-canary-value":"secret"}`, reason: "unknown_field", code: "invalid_request_error", message: "Unknown parameter: 'private-canary-value'.", status: 400, param: canary},
{name: "path", body: `{"type":"inline","data":"YWJj","path":"/private-canary-value"}`, reason: "invalid_path", code: "invalid_request_error", message: errEnvironmentFileCreatePath.message, status: 400},
{name: "base64", body: `{"type":"inline","data":"private-canary-value","path":"/workspace/a"}`, reason: "invalid_base64", code: "invalid_request", message: invalidInputMessage, status: 400},
{name: "inline size", body: inlineCreateBody((5 << 20) + 1), reason: "inline_too_large", code: "invalid_request_error", message: errEnvironmentFileInlineTooLarge.message, status: 400},
{name: "provisioning", body: `{"type":"file_id","file_id":"private-canary-value","path":"/workspace/private-canary-value"}`, reason: "hosted_environment_provisioning", code: "invalid_request_error", message: errHostedEnvironmentProvisioning.message, status: 400, pending: true},
{name: "directory", body: valid, reason: "destination_directory", code: "invalid_request_error", message: errEnvironmentFileConflict.message, status: 400, err: fmt.Errorf("%s: %w", canary, execution.ErrEnvironmentFileDirectory)},
{name: "unsafe", body: valid, reason: "destination_unsafe", code: "invalid_request_error", message: errEnvironmentFileUnsafe.message, status: 400, err: fmt.Errorf("%s: %w", canary, execution.ErrEnvironmentFileUnsafe)},
{name: "unknown", body: valid, reason: "unknown", code: "execution_unavailable", message: "Execution is not available on this service.", status: 503, err: fmt.Errorf("%s: %w", canary, execution.ErrExecutionUnavailable)},
} {
t.Run(tc.name, func(t *testing.T) {
output.Reset()
unavailable := 0
handler, f := environmentFileCreateHandler(t, countEnvironmentFilesUnavailable(&unavailable))
f.err = tc.err
if tc.pending {
f.environment.Status = "pending"
}
request := httptest.NewRequest(http.MethodPost, "/v1/agents/environments/"+f.environment.ID+"/files?secret="+canary, strings.NewReader(tc.body))
request.Header.Set("Authorization", "Bearer files-key")
request.Header.Set("OpenAI-Beta", "agents=v1")
request.Header.Set("Content-Type", "application/json")
request.Header.Set("X-Request-Id", canary)
response := httptest.NewRecorder()
handler.ServeHTTP(environmentFilesRecorder{response}, request)
expected := httptest.NewRecorder()
var param []string
if tc.param != "" {
param = []string{tc.param}
}
writeError(expected, tc.status, tc.code, tc.message, param...)
if response.Code != expected.Code || response.Body.String() != expected.Body.String() {
t.Fatalf("response changed: %d %s; expected %d %s", response.Code, response.Body, expected.Code, expected.Body)
}
records := rejectionRecords(t, output)
if len(records) != 1 {
t.Fatalf("expected one diagnostic, got %d", len(records))
}
record := records[0]
want := map[string]any{"operation": "environment_file_create", "route": "/v1/agents/environments/{environment_id}/files", "status": float64(tc.status), "code": tc.code, "reason": tc.reason, "request_id": response.Header().Get("X-Request-Id")}
carrier, err := obslog.ParseTraceparent(response.Header().Get(obslog.HeaderName))
if err != nil {
t.Fatal(err)
}
want["trace_id"], want["span_id"] = carrier.Trace.String(), carrier.Span.String()
for key, value := range want {
if record[key] != value {
t.Fatalf("%s=%v want %v", key, record[key], value)
}
}
if !requestIDPattern.MatchString(want["request_id"].(string)) {
t.Fatal("missing server request ID")
}
for key := range record {
if key != "time" && key != "level" && key != "msg" {
if _, ok := want[key]; !ok {
t.Fatalf("unapproved diagnostic field: %s", key)
}
}
}
for _, secret := range []string{canary, "files-key", f.environment.ID, f.environment.TenantID} {
if strings.Contains(output.String(), secret) {
t.Fatalf("sensitive data leaked: %s", secret)
}
}
})
}
}

func TestRejectionDiagnosticsPreserveResponseWriter(t *testing.T) {
output := rejectionLogBuffer(t)
for _, stream := range []bool{false, true} {
output.Reset()
out := &detailDeadlineWriter{ResponseRecorder: httptest.NewRecorder()}
var observed []string
handler := responseHeadersWithErrors(coreErrorResponses(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
environmentFileDiagnostic(w, r.Context(), rejectionUnknown)
deadline := time.Unix(1234, 0)
if err := http.NewResponseController(w).SetWriteDeadline(deadline); err != nil {
t.Fatal(err)
}
if stream {
w.Header().Set("Content-Type", "text/event-stream")
if err := http.NewResponseController(w).Flush(); err != nil {
t.Fatal(err)
}
_, _ = w.Write([]byte("data: safe\n\n"))
environmentFileDiagnostic(w, nil, rejectionInvalidPath)
reportAPIError(w, "invalid_request_error")
} else {
writeJSON(w, 201, map[string]string{"result": "safe"})
}
})), func(code string) { observed = append(observed, code) })
handler.ServeHTTP(out, httptest.NewRequest("POST", "/fixture", nil))
if !out.deadline.Equal(time.Unix(1234, 0)) || out.Header().Get("Openai-Processing-Ms") == "" || len(observed) != 0 || len(rejectionRecords(t, output)) != 0 {
t.Fatal("success/stream behavior changed")
}
if stream && (!out.Flushed || out.Body.String() != "data: safe\n\n") {
t.Fatal("SSE changed")
}
}
output.Reset()
var observed []string
handler := responseHeadersWithErrors(coreErrorResponses(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
environmentFileDiagnostic(w, r.Context(), apiRejectionReason(255))
writeError(w, 400, "secret-code-canary", "secret-message-canary", "secret-param-canary")
reportAPIError(w, "late-code")
w.WriteHeader(500)
})), func(code string) { observed = append(observed, code) })
response := httptest.NewRecorder()
handler.ServeHTTP(response, httptest.NewRequest("POST", "/secret-path-canary", nil))
records := rejectionRecords(t, output)
if len(records) != 1 || records[0]["code"] != "unknown" || records[0]["reason"] != "unknown" || strings.Contains(output.String(), "canary") || !reflect.DeepEqual(observed, []string{"secret-code-canary"}) {
t.Fatal("observer/allowlist changed")
}
}

func TestEnvironmentFileDiagnosticsStayWithinCreateHandler(t *testing.T) {
output := rejectionLogBuffer(t)
h, f := environmentFileCreateHandler(t)
f.environment.Status = "pending"
if response := requestEnvironmentFiles(h, f.environment.ID, "", "files-key"); response.Code != 400 {
t.Fatal("expected provisioning rejection")
}
if response := requestCreateEnvironmentFile(h, f.environment.ID, `{"type":"inline","data":"","path":"/workspace/a"}`, ""); response.Code != 401 {
t.Fatal("expected authentication rejection")
}
if len(rejectionRecords(t, output)) != 0 {
t.Fatal("diagnostic escaped create handler scope")
}
f.environment.Status = "connected"
if response := requestCreateEnvironmentFile(h, f.environment.ID, `{"type":"inline","data":"","path":"/workspace/a"}`, "files-key"); response.Code != 201 || f.writes != 1 {
t.Fatal("success changed")
}
if len(rejectionRecords(t, output)) != 0 {
t.Fatal("success logged as rejection")
}
}

func TestEnvironmentFileDiagnosticUsesHandlerTraceContext(t *testing.T) {
output := rejectionLogBuffer(t)
const inbound = "00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01"
for _, parent := range []string{"", inbound} {
output.Reset()
h, f := environmentFileCreateHandler(t)
f.environment.Status = "pending"
r := httptest.NewRequest("POST", "/v1/agents/environments/"+f.environment.ID+"/files", strings.NewReader(`{"type":"inline","data":"","path":"/workspace/a"}`))
r.Header.Set("Authorization", "Bearer files-key")
r.Header.Set("OpenAI-Beta", "agents=v1")
r.Header.Set(obslog.HeaderName, parent)
out := httptest.NewRecorder()
h.ServeHTTP(environmentFilesRecorder{out}, r)
records := rejectionRecords(t, output)
carrier, err := obslog.ParseTraceparent(out.Header().Get(obslog.HeaderName))
if err != nil || out.Code != 400 || len(records) != 1 {
t.Fatalf("missing response/diagnostic trace: %v", err)
}
if parent != "" && out.Header().Get(obslog.HeaderName) != parent {
t.Fatal("inbound carrier changed")
}
if records[0]["request_id"] != out.Header().Get("X-Request-Id") || records[0]["trace_id"] != carrier.Trace.String() || records[0]["span_id"] != carrier.Span.String() {
t.Fatal("diagnostic did not use the traced handler context")
}
}
}
1 change: 1 addition & 0 deletions services/core/internal/api/environment_files.go
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ func environmentFilesAccessible(w http.ResponseWriter, environment sessions.Envi
Type string `json:"type"`
}
if environment.Status == "pending" && json.Unmarshal(environment.Configuration, &configuration) == nil && configuration.Type == "openai_hosted" {
environmentFileDiagnostic(w, nil, rejectionHostedEnvironmentProvisioning)
writeFieldError(w, errHostedEnvironmentProvisioning)
return false
}
Expand Down
Loading