feat: integrate memsearch for agent memory system by HeartLinked · Pull Request #22 · syswonder/robonix

HeartLinked · 2026-03-27T08:49:43Z

Core Features:

MCP Provider: Added a new Python package memsearch_service using FastMCP and memsearch[onnx], providing search_memory, save_memory, and compact_memory tools.
Silent RAG: Enhanced robonix-agent to silently query the memory context via MCP before invoking the VLM, injecting relevant past context into the system prompt.

Edge-case Handling:

Properly serialize query to prevent JSON parsing panics (milvus segcore error).
Implement UTF-8 safe char boundary checking when truncating long Chinese logs.
Removed top_k parameter from MCP schema to prevent LLM hallucinations, hardcoding it to 2 internally for tighter context.

Log Reduction:

Suppressed verbose gRPC and absl heartbeat warnings from milvus-lite.
Redirected FastMCP internal HTTP session logs to a local file (memsearch.log).
Lowered robonix_agent logging level to warn in run.sh to provide a clean and immersive CLI chat experience.

Documentation:

Added comprehensive documentation for the memory service under the system-services.md interface catalog.

Core Features: - MCP Provider: Added a new Python package memsearch_service using FastMCP and memsearch[onnx], providing search_memory, save_memory, and compact_memory tools. - Silent RAG: Enhanced robonix-agent to silently query the memory context via MCP before invoking the VLM, injecting relevant past context into the system prompt. Edge-case Handling: - Properly serialize query to prevent JSON parsing panics (milvus segcore error). - Implement UTF-8 safe char boundary checking when truncating long Chinese logs. - Removed top_k parameter from MCP schema to prevent LLM hallucinations, hardcoding it to 2 internally for tighter context. Log Reduction: - Suppressed verbose gRPC and absl heartbeat warnings from milvus-lite. - Redirected FastMCP internal HTTP session logs to a local file (memsearch.log). - Lowered robonix_agent logging level to warn in run.sh to provide a clean and immersive CLI chat experience. Documentation: - Added comprehensive documentation for the memory service under the system-services.md interface catalog.

Copilot

Pull request overview

Adds an MCP-based “memsearch” memory provider and integrates silent memory recall/compaction into the Rust agent, along with example-script wiring and dependency updates.

Changes:

Introduce memsearch_service Python package (FastMCP + MemSearch) and add it to the e2e example runner.
Update robonix-agent ReAct loop to query memory via MCP before VLM calls and to compact memory on session exit.
Update example requirements/manifests and repo ignore rules to support the new memory service.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
rust/examples/run.sh	Starts memsearch service and adjusts Python/grpc checks/logging defaults.
rust/examples/requirements.txt	Updates Python dependency set to include memsearch service dependencies.
rust/examples/packages/vlm_service/robonix_manifest.yaml	Adjusts Python start command/PYTHONPATH.
rust/examples/packages/memsearch_service/scripts/build.sh	Adds build script to set up PYTHONPATH for the package and proto stubs.
rust/examples/packages/memsearch_service/robonix_manifest.yaml	Defines memsearch MCP service node startup and logging redirection.
rust/examples/packages/memsearch_service/requirements.txt	Adds package-level Python dependencies for memsearch service.
rust/examples/packages/memsearch_service/memsearch_service/service.py	Implements MCP tools (search/save/compact) and control-plane registration + HTTP server.
rust/examples/packages/memsearch_service/memsearch_service/init.py	Declares Python package.
rust/crates/robonix-agent/src/react.rs	Adds silent memory recall into system prompt and compaction on quit; UTF-8 safe log truncation.
.gitignore	Ignores local memory DB/files and Python/robonix build artifacts.

Comments suppressed due to low confidence (1)

rust/examples/run.sh:58

The script switched from python3 to python, but other example scripts/containers in this repo explicitly require python3 (and many systems don’t provide a python shim). Consider using python3 here (or a configurable PYTHON_BIN defaulting to python3) to avoid failing to run in common environments.

check_python_dep() {
  local module="$1"
  python - <<PY >/dev/null 2>&1
import importlib.util
import sys
sys.exit(0 if importlib.util.find_spec("$module") else 1)
PY

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-30T09:36:48Z

+    start: |
+      export PYTHONPATH="$(pwd):$(pwd)/../../proto_gen:${PYTHONPATH:-}"
+      # FastMCP internal logs can be verbose, redirect them to a file or suppress
+      exec python -m memsearch_service.service > memsearch.log 2>&1


Using exec python ... can fail on systems where python isn’t available (only python3 is). Prefer python3 (or a configurable python executable) for portability. Also consider whether redirecting all stdout/stderr to memsearch.log is desirable, since it can hide startup failures from rbnx logs.

Suggested change

exec python -m memsearch_service.service > memsearch.log 2>&1

exec "${PYTHON_EXEC:-python3}" -m memsearch_service.service > memsearch.log 2>&1

Copilot · 2026-03-30T09:36:49Z

+            });
+            let args_str = args.to_string();
+            if let Ok(result) = execute_mcp_tool(sdk, &mcp_tools, "search_memory", &args_str).await {
+                if !result.contains("No relevant memories found") {


The agent decides whether to use memory recall by checking whether the tool output contains the literal string "No relevant memories found". This tightly couples Rust logic to the memsearch service’s human-readable message and will break if that text changes. Prefer returning structured data (e.g., JSON with a boolean/array) from the MCP tool and parsing it here.

Suggested change

if !result.contains("No relevant memories found") {

// Prefer structured JSON output from the MCP tool to decide whether

// there are relevant memories, falling back to the legacy string check.

let trimmed = result.trim();

let mut use_memory = false;

if let Ok(json) = serde_json::from_str::<Value>(trimmed) {

if let Some(obj) = json.as_object() {

// 1) Explicit boolean flag, if provided by the service.

if let Some(flag) = obj.get("has_relevant").and_then(|v| v.as_bool()) {

use_memory = flag;

// 2) Non-empty "memories" array indicates relevant memories.

} else if let Some(arr) = obj.get("memories").and_then(|v| v.as_array()) {

use_memory = !arr.is_empty();

// 3) Non-empty "results" array as another common schema.

} else if let Some(arr) = obj.get("results").and_then(|v| v.as_array()) {

use_memory = !arr.is_empty();

}

}

}

// Backward-compatible fallback: if we couldn't infer from JSON,

// use the legacy "No relevant memories found" string check.

if !use_memory && !result.contains("No relevant memories found") {

use_memory = true;

}

if use_memory {

Copilot · 2026-03-30T09:36:50Z

-    start: exec python3 -m vlm_service.service
+    start: |
+      export PYTHONPATH="$(pwd):$(pwd)/../../proto_gen:${PYTHONPATH:-}"
+      exec python -m vlm_service.service


Using exec python ... can fail on systems where python isn’t available (only python3 is). For consistency with other example scripts and better portability, prefer python3 (or a configurable python executable) in the start command.

Suggested change

exec python -m vlm_service.service

exec "${PYTHON:-python3}" -m vlm_service.service

Copilot · 2026-03-30T09:36:50Z

+    from datetime import date
+    p = Path(MEMORY_DIR) / f"{date.today()}_notes.md"
+    p.parent.mkdir(parents=True, exist_ok=True)
+    with open(p, "a") as f:


save_memory appends to a markdown file without specifying an encoding. On systems with non-UTF-8 locale defaults this can corrupt or fail when saving non-ASCII (e.g., Chinese) content. Open the file with an explicit UTF-8 encoding.

Suggested change

with open(p, "a") as f:

with open(p, "a", encoding="utf-8") as f:

Copilot · 2026-03-30T09:36:50Z

+    # Note: requires llm_provider setup for compacting, defaulting to openai if not set
+    try:
+        summary_path = await mem.compact(llm_provider="openai")
+        return f"Memory compacted successfully to {summary_path}."
+    except Exception as e:
+        return f"Failed to compact memory: {e}"


compact_memory hardcodes llm_provider="openai"; if openai isn’t installed/configured this tool will always fail (currently only returning an exception string). Consider either adding an explicit dependency/config requirement or detecting missing configuration and returning a clearer actionable message.

enkerewpo · 2026-03-30T11:53:34Z

Since VLM service and this mem service (and later, the map service) are robonix "core" services, they should be available in every robonix deployment (like system_server, surfaceflinger... in Android). I'm considering providing a folder for "core service implementation" and maintain the packages there.

HeartLinked added 2 commits March 27, 2026 16:41

chore: update docs submodule to dcd86db (memory module)

1462bc0

HeartLinked force-pushed the dev-memory-1 branch from 43c86c0 to 1462bc0 Compare March 27, 2026 09:08

enkerewpo requested a review from Copilot March 30, 2026 09:32

Copilot started reviewing on behalf of enkerewpo March 30, 2026 09:33

View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

fix: resolve merge conflicts in react.rs

4d6393b

enkerewpo approved these changes Mar 31, 2026

View reviewed changes

enkerewpo merged commit d797099 into syswonder:dev Mar 31, 2026
1 check passed

Chenyantt pushed a commit to Chenyantt/robonix that referenced this pull request Aug 8, 2026

Merge pull request syswonder#22 from HeartLinked/dev-memory-1

951950a

Navigation Menu

Repository navigation

feat: integrate memsearch for agent memory system - #22

feat: integrate memsearch for agent memory system#22
enkerewpo merged 3 commits into
syswonder:devfrom
HeartLinked:dev-memory-1

HeartLinked commented Mar 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

enkerewpo commented Mar 30, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	exec python -m memsearch_service.service > memsearch.log 2>&1
	exec "${PYTHON_EXEC:-python3}" -m memsearch_service.service > memsearch.log 2>&1

-                if !result.contains("No relevant memories found") {
+                // Prefer structured JSON output from the MCP tool to decide whether
+                // there are relevant memories, falling back to the legacy string check.
+                let trimmed = result.trim();
+                let mut use_memory = false;
+                if let Ok(json) = serde_json::from_str::<Value>(trimmed) {
+                    if let Some(obj) = json.as_object() {
+                        // 1) Explicit boolean flag, if provided by the service.
+                        if let Some(flag) = obj.get("has_relevant").and_then(|v| v.as_bool()) {
+                            use_memory = flag;
+                        // 2) Non-empty "memories" array indicates relevant memories.
+                        } else if let Some(arr) = obj.get("memories").and_then(|v| v.as_array()) {
+                            use_memory = !arr.is_empty();
+                        // 3) Non-empty "results" array as another common schema.
+                        } else if let Some(arr) = obj.get("results").and_then(|v| v.as_array()) {
+                            use_memory = !arr.is_empty();
+                        }
+                    }
+                }
+                // Backward-compatible fallback: if we couldn't infer from JSON,
+                // use the legacy "No relevant memories found" string check.
+                if !use_memory && !result.contains("No relevant memories found") {
+                    use_memory = true;
+                }
+                if use_memory {

	exec python -m vlm_service.service
	exec "${PYTHON:-python3}" -m vlm_service.service

	with open(p, "a") as f:
	with open(p, "a", encoding="utf-8") as f:

Conversation

HeartLinked commented Mar 27, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

enkerewpo commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

enkerewpo commented Mar 30, 2026 •

edited

Loading