Skip to content

Commit 34cb957

Browse files
committed
fix: increase title generation token budget for reasoning models
The previous budget of 20 max_output_tokens was sufficient for non-reasoning models, but reasoning models (o-series, gpt-5) include hidden reasoning tokens in that same budget. Even with low effort, 20 tokens is too tight -- the model can exhaust them on reasoning and produce no visible title text. Raise the budget to 256, which gives reasoning models enough room for low reasoning overhead plus a short title. Refs #2318
1 parent c9f14c7 commit 34cb957

1 file changed

Lines changed: 9 additions & 3 deletions

File tree

pkg/sessiontitle/generator.go

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,15 @@ const (
2121
systemPrompt = "You are a helpful AI assistant that generates concise, descriptive titles for conversations. You will be given up to 2 recent user messages and asked to create a single-line title that captures the main topic. Never use newlines or line breaks in your response."
2222
userPromptFormat = "Based on the following recent user messages from a conversation with an AI assistant, generate a short, descriptive title (maximum 50 characters) that captures the main topic or purpose of the conversation. Return ONLY the title text on a single line, nothing else. Do not include any newlines, explanations, or formatting.\n\nRecent user messages:\n%s\n\n"
2323

24+
// titleMaxTokens is the max output token budget for title generation.
25+
// This must be large enough for reasoning models (o-series, gpt-5) where
26+
// max_output_tokens includes hidden reasoning tokens. With minimal
27+
// reasoning effort a short title needs ~200-250 tokens total.
28+
titleMaxTokens = 256
29+
2430
// titleGenerationTimeout is the maximum time to wait for title generation.
25-
// Title generation should be quick since we disable thinking and use low max_tokens.
26-
// If the API is slow or hanging (e.g., due to server-side thinking), we should timeout.
31+
// Title generation should be quick since we use minimal thinking and a
32+
// small token budget. If the API is slow or hanging, we should timeout.
2733
titleGenerationTimeout = 30 * time.Second
2834
)
2935

@@ -103,7 +109,7 @@ func (g *Generator) Generate(ctx context.Context, sessionID string, userMessages
103109
ctx,
104110
baseModel,
105111
options.WithStructuredOutput(nil),
106-
options.WithMaxTokens(20),
112+
options.WithMaxTokens(titleMaxTokens),
107113
options.WithNoThinking(),
108114
options.WithGeneratingTitle(),
109115
)

0 commit comments

Comments
 (0)