[Feature] Reduce Response Verbosity by Improving System Prompt Precision #117

Problem

As more data sources are integrated into the retrieval pipeline, the chatbot responses are becoming increasingly long and noisy. This is a known concern raised in the project and is caused by two compounding factors:

1. Retrieval side - multiple data sources return large numbers of documents, many of which are tangentially relevant (partially addressed by the FlashRank reranking PR Add FlashRank reranker to HybridRetriever to improve retrieval quality #116)
2. Generation side - the current system prompts in both reactome/prompt.py and uniprot/prompt.py explicitly instruct the LLM to be comprehensive and capture all mechanistically relevant details, which causes the LLM to include everything it finds regardless of how directly relevant it is to the question

Root Cause in Current Prompts

reactome/prompt.py:

"comprehensively, mechanistically, and with precision"
"Capture ALL mechanistically relevant details"
"information-rich narrative"
"presenting background, mechanisms, and significance"

uniprot/prompt.py:

"comprehensively, accurately, and in an engaging manner"
"Answer the question comprehensively and accurately"
"Provide any useful background information"

These instructions tell the LLM to maximise coverage - which worked fine with one data source but causes response bloat as more sources are added.

Proposed Fix

Modify both system prompts to shift from exhaustive coverage to relevance-first precision:

• Replace "comprehensively" with "accurately and precisely"
• Replace "Capture ALL details" with "Include only the most directly relevant details"
• Add explicit instructions: "Answer ONLY what was asked", "Do NOT repeat information", "Do NOT add background unless essential"
• Add stop condition: "stops when the question is fully answered"
• Retain all citation formatting, source discipline, and tone guidelines unchanged

Files to Change

src/retrievers/reactome/prompt.py
src/retrievers/uniprot/prompt.py

Status

Implementation is ready. Currently testing the changes locally to gather before/after response comparisons as proof before raising a PR.

Relation to Other Work

This complements PR #116 (FlashRank reranking) which addresses the retrieval side of the same problem. Together they form a complete solution:

• PR Add FlashRank reranker to HybridRetriever to improve retrieval quality #116 ensures better documents go to the LLM.
• This PR ensures LLM generates most relevant content.