diff --git a/.env.example b/.env.example index eb8508c..62f748f 100644 --- a/.env.example +++ b/.env.example @@ -52,6 +52,7 @@ SANDBOX_EMULATOR_BASE_URL=http://host.docker.internal:8000 # Exposed port (nginx) SICO_PORT=8080 +SICO_APP_NAME=sico # BACKEND_ROOT overrides the auto-detected backend source root (used to # locate configs/migrations). The compiled Docker image already lays out the diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 46d2fac..15cf290 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -10,6 +10,12 @@ Backend ↔ Core communicate via gRPC (`:50053`). Core calls back to the backend See `CLAUDE.md` for the full architecture and coding-style reference. +## Source file conventions + +- Every new source/config file (`.go`, `.py`, `.proto`, `.yaml`, shell scripts, etc.) must start with the standard 19-line MIT header from `LICENSE` — `# Copyright (c) 2026 Sico Authors` for `#`-comment languages, `// Copyright (c) 2026 Sico Authors` for `//`-comment languages. +- The `addlicense` pre-commit hook (`.pre-commit-config.yaml`) inserts/refreshes the header automatically; run `pre-commit run addlicense --all-files` (or `pre-commit run --files `) before committing new files. +- Do not strip the header when editing existing files. + ## Backend (Go) guidelines - **Do not edit generated files:** diff --git a/.gitignore b/.gitignore index 4225c8e..af1778c 100644 --- a/.gitignore +++ b/.gitignore @@ -1,429 +1,354 @@ -## Ignore Visual Studio temporary files, build results, and -## files generated by popular Visual Studio add-ons. -## -## Get latest from https://github.com/github/gitignore/blob/main/VisualStudio.gitignore - -# User-specific files -*.rsuser -*.suo -*.user -*.userosscache -*.sln.docstates -*.env - -# User-specific files (MonoDevelop/Xamarin Studio) -*.userprefs - -# Mono auto generated files -mono_crash.* - -# Build results -[Dd]ebug/ -[Dd]ebugPublic/ -[Rr]elease/ -[Rr]eleases/ -x64/ -x86/ -[Ww][Ii][Nn]32/ -[Aa][Rr][Mm]/ -[Aa][Rr][Mm]64/ -[Aa][Rr][Mm]64[Ee][Cc]/ -bld/ -[Oo]bj/ -[Oo]ut/ -[Ll]og/ -[Ll]ogs/ - -# Build results on 'Bin' directories -**/[Bb]in/* -# Uncomment if you have tasks that rely on *.refresh files to move binaries -# (https://github.com/github/gitignore/pull/3736) -#!**/[Bb]in/*.refresh - -# Visual Studio 2015/2017 cache/options directory -.vs/ -# Uncomment if you have tasks that create the project's static files in wwwroot -#wwwroot/ - -# Visual Studio 2017 auto generated files -Generated\ Files/ - -# MSTest test Results -[Tt]est[Rr]esult*/ -[Bb]uild[Ll]og.* -*.trx - -# NUnit -*.VisualState.xml -TestResult.xml -nunit-*.xml - -# Approval Tests result files -*.received.* - -# Build Results of an ATL Project -[Dd]ebugPS/ -[Rr]eleasePS/ -dlldata.c - -# Benchmark Results -BenchmarkDotNet.Artifacts/ - -# .NET Core -project.lock.json -project.fragment.lock.json -artifacts/ - -# ASP.NET Scaffolding -ScaffoldingReadMe.txt - -# StyleCop -StyleCopReport.xml - -# Files built by Visual Studio -*_i.c -*_p.c -*_h.h -*.ilk -*.meta -*.obj -*.idb -*.iobj -*.pch -*.pdb -*.ipdb -*.pgc -*.pgd -*.rsp -# but not Directory.Build.rsp, as it configures directory-level build defaults -!Directory.Build.rsp -*.sbr -*.tlb -*.tli -*.tlh +### Backup ### +*.bak +*.gho +*.ori +*.orig *.tmp -*.tmp_proj -*_wpftmp.csproj -*.log -*.tlog -*.vspscc -*.vssscc -.builds -*.pidb -*.svclog -*.scc - -# Chutzpah Test files -_Chutzpah* - -# Visual C++ cache files -ipch/ -*.aps -*.ncb -*.opendb -*.opensdf -*.sdf -*.cachefile -*.VC.db -*.VC.VC.opendb - -# Visual Studio profiler -*.psess -*.vsp -*.vspx -*.sap - -# Visual Studio Trace Files -*.e2e - -# TFS 2012 Local Workspace -$tf/ - -# Guidance Automation Toolkit -*.gpState - -# ReSharper is a .NET coding add-in -_ReSharper*/ -*.[Rr]e[Ss]harper -*.DotSettings.user - -# TeamCity is a build add-in -_TeamCity* - -# DotCover is a Code Coverage Tool -*.dotCover - -# AxoCover is a Code Coverage Tool -.axoCover/* -!.axoCover/settings.json - -# Coverlet is a free, cross platform Code Coverage Tool -coverage*.json -coverage*.xml -coverage*.info - -# Visual Studio code coverage results -*.coverage -*.coveragexml - -# NCrunch -_NCrunch_* -.NCrunch_* -.*crunch*.local.xml -nCrunchTemp_* - -# MightyMoose -*.mm.* -AutoTest.Net/ - -# Web workbench (sass) -.sass-cache/ - -# Installshield output folder -[Ee]xpress/ - -# DocProject is a documentation generator add-in -DocProject/buildhelp/ -DocProject/Help/*.HxT -DocProject/Help/*.HxC -DocProject/Help/*.hhc -DocProject/Help/*.hhk -DocProject/Help/*.hhp -DocProject/Help/Html2 -DocProject/Help/html - -# Click-Once directory -publish/ - -# Publish Web Output -*.[Pp]ublish.xml -*.azurePubxml -# Note: Comment the next line if you want to checkin your web deploy settings, -# but database connection strings (with potential passwords) will be unencrypted -*.pubxml -*.publishproj - -# Microsoft Azure Web App publish settings. Comment the next line if you want to -# checkin your Azure Web App publish settings, but sensitive information contained -# in these scripts will be unencrypted -PublishScripts/ - -# NuGet Packages -*.nupkg -# NuGet Symbol Packages -*.snupkg -# The packages folder can be ignored because of Package Restore -**/[Pp]ackages/* -# except build/, which is used as an MSBuild target. -!**/[Pp]ackages/build/ -# Uncomment if necessary however generally it will be regenerated when needed -#!**/[Pp]ackages/repositories.config -# NuGet v3's project.json files produces more ignorable files -*.nuget.props -*.nuget.targets - -# Microsoft Azure Build Output -csx/ -*.build.csdef - -# Microsoft Azure Emulator -ecf/ -rcf/ - -# Windows Store app package directories and files -AppPackages/ -BundleArtifacts/ -Package.StoreAssociation.xml -_pkginfo.txt -*.appx -*.appxbundle -*.appxupload - -# Visual Studio cache files -# files ending in .cache can be ignored -*.[Cc]ache -# but keep track of directories ending in .cache -!?*.[Cc]ache/ - -# Others -ClientBin/ -~$* -*~ -*.dbmdl -*.dbproj.schemaview -*.jfm -*.pfx -*.publishsettings -orleans.codegen.cs - -# Including strong name files can present a security risk -# (https://github.com/github/gitignore/pull/2483#issue-259490424) -#*.snk - -# Since there are multiple workflows, uncomment next line to ignore bower_components -# (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) -#bower_components/ - -# RIA/Silverlight projects -Generated_Code/ - -# Backup & report files from converting an old project file -# to a newer Visual Studio version. Backup files are not needed, -# because we have git ;-) -_UpgradeReport_Files/ -Backup*/ -UpgradeLog*.XML -UpgradeLog*.htm -ServiceFabricBackup/ -*.rptproj.bak - -# SQL Server files -*.mdf -*.ldf -*.ndf - -# Business Intelligence projects -*.rdl.data -*.bim.layout -*.bim_*.settings -*.rptproj.rsuser -*- [Bb]ackup.rdl -*- [Bb]ackup ([0-9]).rdl -*- [Bb]ackup ([0-9][0-9]).rdl - -# Microsoft Fakes -FakesAssemblies/ - -# GhostDoc plugin setting file -*.GhostDoc.xml - -# Node.js Tools for Visual Studio -.ntvs_analysis.dat -node_modules/ - -# Visual Studio 6 build log -*.plg -# Visual Studio 6 workspace options file -*.opt - -# Visual Studio 6 auto-generated workspace file (contains which files were open etc.) -*.vbw - -# Visual Studio 6 auto-generated project file (contains which files were open etc.) -*.vbp - -# Visual Studio 6 workspace and project file (working project files containing files to include in project) -*.dsw -*.dsp - -# Visual Studio 6 technical files -*.ncb -*.aps +### Local validation artifacts ### +scripts/acceptance-*.json + +### Git ### +# Created by git for backups. To disable backups in Git: +# $ git config --global mergetool.keepBackup false + +# Created by git when using merge tools for conflicts +*.BACKUP.* +*.BASE.* +*.LOCAL.* +*.REMOTE.* +*_BACKUP_*.txt +*_BASE_*.txt +*_LOCAL_*.txt +*_REMOTE_*.txt + +### Go ### +# Binaries for programs and plugins +*.exe +*.exe~ +*.dll +*.so +*.dylib + +# Test binary, built with `go test -c` +*.test + +# Output of the go coverage tool, specifically when used with LiteIDE +*.out + +# Dependency directories (remove the comment below to include it) +# vendor/ + +### Go Patch ### +/vendor/ +/Godeps/ + +### JetBrains ### +# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio, WebStorm and Rider +# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839 +.idea + +# User-specific stuff +.idea/**/workspace.xml +.idea/**/tasks.xml +.idea/**/usage.statistics.xml +.idea/**/dictionaries +.idea/**/shelf + +# Generated files +.idea/**/contentModel.xml + +# Sensitive or high-churn files +.idea/**/dataSources/ +.idea/**/dataSources.ids +.idea/**/dataSources.local.xml +.idea/**/sqlDataSources.xml +.idea/**/dynamic.xml +.idea/**/uiDesigner.xml +.idea/**/dbnavigator.xml + +# Gradle +.idea/**/gradle.xml +.idea/**/libraries + +# Gradle and Maven with auto-import +# When using Gradle or Maven with auto-import, you should exclude module files, +# since they will be recreated, and may cause churn. Uncomment if using +# auto-import. +# .idea/artifacts +# .idea/compiler.xml +# .idea/jarRepositories.xml +# .idea/modules.xml +# .idea/*.iml +# .idea/modules +# *.iml +# *.ipr + +# CMake +cmake-build-*/ + +# Mongo Explorer plugin +.idea/**/mongoSettings.xml + +# File-based project format +*.iws -# Visual Studio LightSwitch build output -**/*.HTMLClient/GeneratedArtifacts -**/*.DesktopClient/GeneratedArtifacts -**/*.DesktopClient/ModelManifest.xml -**/*.Server/GeneratedArtifacts -**/*.Server/ModelManifest.xml -_Pvt_Extensions +# IntelliJ +out/ + +# mpeltonen/sbt-idea plugin +.idea_modules/ + +# JIRA plugin +atlassian-ide-plugin.xml + +# Cursive Clojure plugin +.idea/replstate.xml + +# Crashlytics plugin (for Android Studio and IntelliJ) +com_crashlytics_export_strings.xml +crashlytics.properties +crashlytics-build.properties +fabric.properties + +# Editor-based Rest Client +.idea/httpRequests + +# Android studio 3.1+ serialized cache file +.idea/caches/build_file_checksums.ser + +### JetBrains Patch ### +# Comment Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-215987721 + +# *.iml +# modules.xml +# .idea/misc.xml +# *.ipr + +# Sonarlint plugin +# https://plugins.jetbrains.com/plugin/7973-sonarlint +.idea/**/sonarlint/ + +# SonarQube Plugin +# https://plugins.jetbrains.com/plugin/7238-sonarqube-community-plugin +.idea/**/sonarIssues.xml + +# Markdown Navigator plugin +# https://plugins.jetbrains.com/plugin/7896-markdown-navigator-enhanced +.idea/**/markdown-navigator.xml +.idea/**/markdown-navigator-enh.xml +.idea/**/markdown-navigator/ + +# Cache file creation bug +# See https://youtrack.jetbrains.com/issue/JBR-2257 +.idea/$CACHE_FILE$ + +# CodeStream plugin +# https://plugins.jetbrains.com/plugin/12206-codestream +.idea/codestream.xml + +### Tags ### +# Ignore tags created by etags, ctags, gtags (GNU global) and cscope +TAGS +.TAGS +!TAGS/ +tags +.tags +!tags/ +gtags.files +GTAGS +GRTAGS +GPATH +GSYMS +cscope.files +cscope.out +cscope.in.out +cscope.po.out + + +### Test ### +### Ignore all files that could be used to test your code and +### you wouldn't want to push + +# Reference https://en.wikipedia.org/wiki/Metasyntactic_variable + +# Most common +*foo +*fubar +*foobar +*baz + +# Less common +*qux +*quux +*bongo +*bazola +*ztesch + +# UK, Australia +*wibble +*wobble +*wubble +*flob +*blep +*blah +*boop +*beep + +# Japanese +*hoge +*piyo +*fuga +*hogera +*hogehoge + +# Portugal, Spain +*fulano +*sicrano +*beltrano +*mengano +*perengano +*zutano + +# France, Italy, the Netherlands +*toto +*titi +*tata +*tutu +*pipppo +*pluto +*paperino +*aap +*noot +*mies + +# Other names that would make sense +*temp +*tempdir +*tempfile +*tempfiles +*tmp +*tmpdir +*tmpfile +*tmpfiles +*lol + +### Vim ### +# Swap +[._]*.s[a-v][a-z] +!*.svg # comment out if you don't need vector files +[._]*.sw[a-p] +[._]s[a-rt-v][a-z] +[._]ss[a-gi-z] +[._]sw[a-p] + +# Session +Session.vim +Sessionx.vim + +# Temporary +.netrwhist +*~ +# Auto-generated tag files +# Persistent undo +[._]*.un~ -# Paket dependency manager -**/.paket/paket.exe -paket-files/ +### Emacs ### +# -*- mode: gitignore; -*- +*~ +\#*\# +/.emacs.desktop +/.emacs.desktop.lock +*.elc +auto-save-list +tramp +.\#* -# FAKE - F# Make -**/.fake/ +# Org-mode +.org-id-locations +*_archive +ltximg/** -# CodeRush personal settings -**/.cr/personal +# flymake-mode +*_flymake.* -# Python Tools for Visual Studio (PTVS) -**/__pycache__/ -*.pyc +# eshell files +/eshell/history +/eshell/lastdir -# Cake - Uncomment if you are using it -#tools/** -#!tools/packages.config +# elpa packages +/elpa/ -# Tabs Studio -*.tss +# reftex files +*.rel -# Telerik's JustMock configuration file -*.jmconfig +# AUCTeX auto folder +/auto/ -# BizTalk build output -*.btp.cs -*.btm.cs -*.odx.cs -*.xsd.cs +# cask packages +.cask/ +dist/ -# OpenCover UI analysis results -OpenCover/ +# Flycheck +flycheck_*.el -# Azure Stream Analytics local run output -ASALocalRun/ +# server auth directory +/server/ -# MSBuild Binary and Structured Log -*.binlog -MSBuild_Logs/ +# projectiles files +.projectile -# AWS SAM Build and Temporary Artifacts folder -.aws-sam +# directory configuration +.dir-locals.el -# NVidia Nsight GPU debugger configuration file -*.nvuser +# network security +/network-security.data -# MFractors (Xamarin productivity tool) working folder -**/.mfractor/ +### vscode ### +.vscode/* +!.vscode/settings.json +!.vscode/tasks.json +!.vscode/launch.json +!.vscode/extensions.json +*.code-workspace -# Local History for Visual Studio -**/.localhistory/ +# End of https://www.toptal.com/developers/gitignore/api/vim,jetbrains,vscode,git,go,tags,backup,test -# Visual Studio History (VSHistory) files -.vshistory/ +# Start by iam -# BeatPulse healthcheck temp database -healthchecksdb +# log +*.log -# Backup folder for Package Reference Convert tool in Visual Studio 2017 -MigrationBackup/ +# Output of backend and frontend +/_output +/_debug -# Ionide (cross platform F# VS Code tools) working folder -**/.ionide/ +# Misc +.DS_Store +*.env +.env.* +!.env.example +dist -# Fody - auto-generated XML schema -FodyWeavers.xsd +# files used by the developer +.idea.md +.todo.md +.note.md -# VS Code files for those working on multiple tools -.vscode/* -!.vscode/settings.json -!.vscode/tasks.json -!.vscode/launch.json -!.vscode/extensions.json -!.vscode/*.code-snippets +# config files, may contain sensitive information -# Local History for Visual Studio Code -.history/ +# VSCode +.vscode +backend/scripts/playground/ -# Built Visual Studio Code Extensions -*.vsix +__pycache__/ -# Windows Installer files from build outputs -*.cab -*.msi -*.msix -*.msm -*.msp +# Helm chart packages +*.tgz -# macOS Finder metadata -.DS_Store -.AppleDouble -.LSOverride +# Temporary files +tmp/ +.local/ -# JetBrains IDEs (IntelliJ IDEA, GoLand, PyCharm, WebStorm, ...) -.idea/ -*.iml -*.ipr -*.iws +# Frontend build artifacts (unzipped from frontend-dist.zip) +frontend-dist/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 943a114..97ff77c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,37 @@ Guidelines for editors: add the compare link at the bottom. --> +## [0.2.0] - 2026-06-11 + +### Added + +- **Android Tester skill enhancements:** precondition record-replay system, clipboard tools, file tools, swipe speed levels, force-stop app action, ternary status codes, batch execution trace grouping, structured recorder data, and multi-turn conversation history for operator. +- **Skill resolver API:** request explicit skill versions from core, improved retry diagnostics, and skill version detail endpoints. +- **Task runtime improvements:** batch-level liveness with orphan pod defense, bucket batch concurrency by sandbox type, state machine and event bus architecture. +- **Chat routing fast path:** intent-check model for direct responses on simple queries, short-cutting the full planning pipeline. +- **Conversation re-connect:** allow resuming interrupted conversations. +- **Adapter tool:** workbook resolver adapter with general adapter support for delegating tasks. +- **Documentation:** technical report (survey paper), updated architecture diagrams, quickstart with Android emulator instructions, DW-type creation guide, roadmap revision, and Digital Tester troubleshooting guide. + +### Changed + +- Refactored core task runtime: split manager, added dispatch union, prepared batch, view renderers, narrowed `submit_prepared` surface with `TurnContext`, and removed credentials/redaction code. +- Plan structure now uses sub tool calls and message updates for delegated tasks. +- Updated system prompts and extraction prompts. + +### Fixed + +- Sandbox pod stuck in `ContainerCreating` when mounts share one PVC. +- Kafka OOM resolved via resource limits. +- Android Tester: device offline detection, connection error retries, atomic file writes, empty text input handling, malformed script handling, precondition step reporting format. +- Backend: auth context key fix, user retrieval from context, migration fixes. +- Skill: return full download URL, historical skill version details. + +### Security + +- Bumped vulnerable dependencies. +- Tightened Content Security Policy (CSP) in frontend. + ## [Unreleased] ### Added diff --git a/CLAUDE.md b/CLAUDE.md index 7343e6b..11b9628 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -104,7 +104,7 @@ Key patterns: - **`app/biz/chat/`** — Chat orchestration: multi-turn conversation, tool calling, prompt construction, plan execution. - **`app/biz/llm/`** — LLM service layer. - **`app/biz/reverse_grpc/`** — Reverse gRPC client stubs that call back to the backend. -- **`app/tools/`** — Agent tool implementations (read, write, grep, web_search, run_python, sandbox_tools, etc.). +- **`app/tools/`** — Agent tool implementations (read, write, grep, web_search, run_python, etc.). - **`app/llmhubs/`** — LLM provider abstraction layer with adapter pattern for multiple providers. - **`app/schemas/`** — Pydantic models for chat, conversation. - **`app/pb/`** — Auto-generated protobuf/betterproto2 stubs (do not edit manually). @@ -120,6 +120,11 @@ Proto definitions are organized by domain in `proto/`: agent, chat, common, conv ## Code Style +### Source file headers + +- Every source/config file (`.go`, `.py`, `.proto`, `.yaml`, shell scripts, etc.) must start with the standard 19-line MIT header from `LICENSE` (`# Copyright (c) 2026 Sico Authors` for `#`-style languages, `// Copyright (c) 2026 Sico Authors` for `//`-style languages). +- The `addlicense` pre-commit hook (`.pre-commit-config.yaml`) inserts/refreshes the header automatically — run `pre-commit run addlicense --all-files` (or `pre-commit run --files `) before committing new files. + ### Go function signatures Wrapping rules (apply mechanically): diff --git a/Makefile b/Makefile index 53b2ac9..5993503 100644 --- a/Makefile +++ b/Makefile @@ -37,6 +37,7 @@ help: @echo " make compose-up Build and start full stack" @echo " make compose-up SERVICE=core Rebuild/recreate one service image" @echo " make compose-down Stop and remove containers" + @echo " make compose-down-volumes Stop and remove containers + volumes (data loss)" @echo " make compose-logs Tail logs" @echo "" @echo "Kind (local Kubernetes):" @@ -135,6 +136,9 @@ compose-up: compose-down: $(COMPOSE) down --remove-orphans +compose-down-volumes: + $(COMPOSE) down --volumes --remove-orphans + compose-logs: $(COMPOSE) logs -f diff --git a/README.md b/README.md index 97a5390..5a2d957 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,8 @@ **Sico: an infrastructure for symbiotic intelligence, where humans and Digital Workers co-evolve.** +[![Discord](https://img.shields.io/badge/Discord-Community-5865F2?logo=discord&logoColor=white)](https://discord.gg/F3tVCBHmE6) +[![Microsoft Teams](https://img.shields.io/badge/Teams-Community-6264A7)](https://teams.microsoft.com/l/team/19%3AtgoS3iJHg6-s2vhO9W5PhUtsnhqNog7yiTdcEi6sNZQ1%40thread.tacv2/conversations?groupId=720e2630-e290-40c7-9c3e-55f5ab643f32&tenantId=72f988bf-86f1-41af-91ab-2d7cd011db47) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) [![Go](https://img.shields.io/badge/Go-1.25+-00ADD8.svg)](backend/go.mod) [![Python](https://img.shields.io/badge/Python-3.13+-3776AB.svg)](core/pyproject.toml) @@ -36,11 +38,13 @@ Its anatomy consists of: - **Action**: execution via domain skills, workflows, and sandboxed tools - **Memory & Sense**: accumulated knowledge, execution experience and contextual awareness for grounding and continuous improvement +Human operators supervise execution quality, intervene when necessary, and guide capability improvement. + This creates a practical **Co-Evolution** loop where humans and Digital Workers continuously improve together through real work. For a comprehensive survey of this direction, refer to [**Agentic Evolution: From Self-Improving Agents to Co-Evolving Human–AI Systems** ](docs/agentic-evolution.pdf) -> Learn more: [What is Sico](docs/overview.md) +> Learn more: [What is Sico](docs/overview.md). ## Who is Sico for? diff --git a/backend/api/openapi/docs.go b/backend/api/openapi/docs.go index 6977717..1d39379 100644 --- a/backend/api/openapi/docs.go +++ b/backend/api/openapi/docs.go @@ -641,6 +641,57 @@ const docTemplate = `{ } } }, + "/api/sico/conversation/batch_summaries": { + "get": { + "security": [ + { + "BearerAuth": [] + } + ], + "produces": [ + "application/json" + ], + "tags": [ + "Conversation" + ], + "parameters": [ + { + "type": "integer", + "name": "conversationId", + "in": "query", + "required": true + }, + { + "minimum": 1, + "type": "integer", + "default": 1, + "name": "page", + "in": "query" + }, + { + "maximum": 100, + "minimum": 1, + "type": "integer", + "default": 20, + "name": "pageSize", + "in": "query" + }, + { + "type": "integer", + "name": "turnId", + "in": "query" + } + ], + "responses": { + "200": { + "description": "OK", + "schema": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesResponse" + } + } + } + } + }, "/api/sico/conversation/chat": { "post": { "security": [ @@ -872,6 +923,11 @@ const docTemplate = `{ "name": "agentInstanceId", "in": "query" }, + { + "type": "integer", + "name": "conversationId", + "in": "query" + }, { "type": "integer", "name": "turnId", @@ -3067,7 +3123,7 @@ const docTemplate = `{ }, "/api/sico/sandbox/apply": { "post": { - "description": "Apply for a sandbox of the specified type. One instanceID can have multiple sandboxes.", + "description": "Apply for a sandbox supplying the requested OS. One instanceID can have multiple sandboxes.", "consumes": [ "application/json" ], @@ -3115,7 +3171,7 @@ const docTemplate = `{ "required": true }, { - "description": "Sandbox type to apply", + "description": "Sandbox OS to apply", "name": "request", "in": "body", "required": true, @@ -3654,37 +3710,6 @@ const docTemplate = `{ } } }, - "/api/sico/skills/details": { - "get": { - "security": [ - { - "BearerAuth": [] - } - ], - "produces": [ - "application/json" - ], - "tags": [ - "skills" - ], - "parameters": [ - { - "type": "integer", - "name": "id", - "in": "query", - "required": true - } - ], - "responses": { - "200": { - "description": "OK", - "schema": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.GetSkillDetailsResponse" - } - } - } - } - }, "/api/sico/skills/list": { "get": { "security": [ @@ -4224,6 +4249,9 @@ const docTemplate = `{ "type": "string" } }, + "creatorUsername": { + "type": "string" + }, "name": { "type": "string" }, @@ -4428,12 +4456,50 @@ const docTemplate = `{ } } }, + "sico-backend_internal_transport_http_dto_conversation.BatchSummaryItem": { + "type": "object", + "properties": { + "batchId": { + "type": "string" + }, + "createdAt": { + "type": "integer" + }, + "endedAt": { + "type": "integer" + }, + "parentConversationId": { + "type": "integer" + }, + "parentTurnId": { + "type": "integer" + }, + "reason": { + "type": "string" + }, + "status": { + "type": "string" + }, + "summaryUri": { + "type": "string" + }, + "totalCount": { + "type": "integer" + }, + "updatedAt": { + "type": "integer" + } + } + }, "sico-backend_internal_transport_http_dto_conversation.CancelPlanRequest": { "type": "object", "properties": { "agentInstanceId": { "type": "integer" }, + "conversationId": { + "type": "integer" + }, "turnId": { "type": "integer" } @@ -4500,6 +4566,9 @@ const docTemplate = `{ "content": { "type": "string" }, + "conversationId": { + "type": "integer" + }, "functionContext": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.FunctionContext" }, @@ -4703,6 +4772,34 @@ const docTemplate = `{ } } }, + "sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesData": { + "type": "object", + "properties": { + "hasMore": { + "type": "boolean" + }, + "items": { + "type": "array", + "items": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.BatchSummaryItem" + } + } + } + }, + "sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesResponse": { + "type": "object", + "properties": { + "code": { + "type": "integer" + }, + "data": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesData" + }, + "msg": { + "type": "string" + } + } + }, "sico-backend_internal_transport_http_dto_conversation.ListConversationData": { "type": "object", "properties": { @@ -4796,6 +4893,9 @@ const docTemplate = `{ "content": { "type": "string" }, + "conversationId": { + "type": "integer" + }, "createdAt": { "type": "integer" }, @@ -4901,6 +5001,9 @@ const docTemplate = `{ "items": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCall" } + }, + "updatedAt": { + "type": "integer" } } }, @@ -4968,24 +5071,49 @@ const docTemplate = `{ } } }, - "sico-backend_internal_transport_http_dto_conversation.ToolCall": { + "sico-backend_internal_transport_http_dto_conversation.TaskRuntimeExecutionInfo": { "type": "object", "properties": { - "batchCalls": { - "type": "array", - "items": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCall" - } + "attempt": { + "type": "integer" }, - "batchItemIndex": { + "currentStage": { + "description": "Lifecycle stage of a single task run: plan | workspace | sandbox | execute | upload | release.", + "type": "string" + }, + "latestProgressMessage": { + "type": "string" + }, + "maxAttempts": { "type": "integer" }, + "sandboxEndpoint": { + "type": "string" + }, + "sandboxId": { + "type": "string" + }, + "sandboxType": { + "type": "string" + } + } + }, + "sico-backend_internal_transport_http_dto_conversation.ToolCall": { + "type": "object", + "properties": { "deliverables": { "type": "array", "items": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolDeliverable" } }, + "display": { + "description": "Structured display labels (task_label, sandbox_label, sandbox_label_plural,\nsandbox_ready_label, sandbox_releasing_label, sandbox_release_label,\nenvironment_label, runner_label, batch_subject_singular,\nbatch_subject_plural, ...) sourced from the capability card. The frontend\nshould read these directly instead of pattern-matching ` + "`" + `message` + "`" + `.", + "type": "object", + "additionalProperties": { + "type": "string" + } + }, "executionInfo": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolExecutionInfo" }, @@ -4993,10 +5121,13 @@ const docTemplate = `{ "description": "message is for frontend display", "type": "string" }, - "runningList": { + "subCallIndex": { + "type": "integer" + }, + "subCalls": { "type": "array", "items": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItem" + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCall" } }, "toolCallId": { @@ -5007,40 +5138,12 @@ const docTemplate = `{ }, "toolName": { "type": "string" - } - } - }, - "sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItem": { - "type": "object", - "properties": { - "name": { - "type": "string" }, - "status": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItemStatus" + "updatedAt": { + "type": "integer" } } }, - "sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItemStatus": { - "type": "integer", - "format": "int32", - "enum": [ - 0, - 1, - 2, - 3, - 4, - 5 - ], - "x-enum-varnames": [ - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_UNKNOWN", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_PENDING", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_RUNNING", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_DONE", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_FAILED", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_CANCELLED" - ] - }, "sico-backend_internal_transport_http_dto_conversation.ToolCallStatus": { "type": "integer", "format": "int32", @@ -5053,7 +5156,8 @@ const docTemplate = `{ 5, 6, 7, - 8 + 8, + 9 ], "x-enum-varnames": [ "ToolCallStatus_TOOL_CALL_STATUS_UNKNOWN", @@ -5064,7 +5168,8 @@ const docTemplate = `{ "ToolCallStatus_TOOL_CALL_STATUS_FAILED_ANALYZED", "ToolCallStatus_TOOL_CALL_STATUS_RETRY_RUNNING", "ToolCallStatus_TOOL_CALL_STATUS_RETRY_SUCCESSFUL", - "ToolCallStatus_TOOL_CALL_STATUS_RETRY_FAILED" + "ToolCallStatus_TOOL_CALL_STATUS_RETRY_FAILED", + "ToolCallStatus_TOOL_CALL_STATUS_PENDING" ] }, "sico-backend_internal_transport_http_dto_conversation.ToolDeliverable": { @@ -5153,6 +5258,9 @@ const docTemplate = `{ "builtinToolName": { "type": "string" }, + "taskRuntime": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.TaskRuntimeExecutionInfo" + }, "toolType": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolType" } @@ -7761,34 +7869,15 @@ const docTemplate = `{ "properties": { "skill": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.Skill" - } - } - }, - "sico-backend_internal_transport_http_dto_skill.GetSkillDetailsData": { - "type": "object", - "properties": { - "files": { + }, + "versions": { "type": "array", "items": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillFile" + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillVersion" } } } }, - "sico-backend_internal_transport_http_dto_skill.GetSkillDetailsResponse": { - "type": "object", - "properties": { - "code": { - "type": "integer" - }, - "data": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.GetSkillDetailsData" - }, - "msg": { - "type": "string" - } - } - }, "sico-backend_internal_transport_http_dto_skill.GetSkillResponse": { "type": "object", "properties": { @@ -7840,21 +7929,12 @@ const docTemplate = `{ "agentId": { "type": "string" }, - "assetId": { - "type": "integer" - }, "createdAt": { "type": "integer" }, - "creatorUsername": { - "type": "string" - }, "description": { "type": "string" }, - "failReason": { - "type": "string" - }, "id": { "type": "integer" }, @@ -7864,11 +7944,25 @@ const docTemplate = `{ "projectId": { "type": "integer" }, - "status": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillStatus" - }, "updatedAt": { "type": "integer" + }, + "version": { + "type": "string" + } + } + }, + "sico-backend_internal_transport_http_dto_skill.SkillAction": { + "type": "object", + "properties": { + "advancedSettings": { + "type": "string" + }, + "description": { + "type": "string" + }, + "name": { + "type": "string" } } }, @@ -7911,24 +8005,95 @@ const docTemplate = `{ "SkillStatus_SKILL_STATUS_FAILED" ] }, + "sico-backend_internal_transport_http_dto_skill.SkillVersion": { + "type": "object", + "properties": { + "actions": { + "type": "array", + "items": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillAction" + } + }, + "createdAt": { + "type": "integer" + }, + "creatorUsername": { + "type": "string" + }, + "description": { + "type": "string" + }, + "failReason": { + "type": "string" + }, + "id": { + "type": "integer" + }, + "name": { + "type": "string" + }, + "skillId": { + "type": "integer" + }, + "status": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillStatus" + }, + "updatedAt": { + "type": "integer" + }, + "url": { + "type": "string" + }, + "version": { + "type": "string" + } + } + }, "sico-backend_internal_transport_http_dto_skill.UpdateSkillData": { "type": "object", "properties": { - "skill": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.Skill" + "assetId": { + "type": "integer" + }, + "description": { + "type": "string" + }, + "name": { + "type": "string" + }, + "skillId": { + "type": "integer" + }, + "version": { + "type": "string" } } }, "sico-backend_internal_transport_http_dto_skill.UpdateSkillRequest": { "type": "object", "required": [ - "assetId", + "currentVersion", "id" ], "properties": { + "actions": { + "type": "array", + "items": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillAction" + } + }, "assetId": { "type": "integer" }, + "currentVersion": { + "type": "string" + }, + "files": { + "type": "array", + "items": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillFile" + } + }, "id": { "type": "integer" } diff --git a/backend/api/openapi/swagger.json b/backend/api/openapi/swagger.json index 835ed68..1a1895e 100644 --- a/backend/api/openapi/swagger.json +++ b/backend/api/openapi/swagger.json @@ -633,6 +633,57 @@ } } }, + "/api/sico/conversation/batch_summaries": { + "get": { + "security": [ + { + "BearerAuth": [] + } + ], + "produces": [ + "application/json" + ], + "tags": [ + "Conversation" + ], + "parameters": [ + { + "type": "integer", + "name": "conversationId", + "in": "query", + "required": true + }, + { + "minimum": 1, + "type": "integer", + "default": 1, + "name": "page", + "in": "query" + }, + { + "maximum": 100, + "minimum": 1, + "type": "integer", + "default": 20, + "name": "pageSize", + "in": "query" + }, + { + "type": "integer", + "name": "turnId", + "in": "query" + } + ], + "responses": { + "200": { + "description": "OK", + "schema": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesResponse" + } + } + } + } + }, "/api/sico/conversation/chat": { "post": { "security": [ @@ -864,6 +915,11 @@ "name": "agentInstanceId", "in": "query" }, + { + "type": "integer", + "name": "conversationId", + "in": "query" + }, { "type": "integer", "name": "turnId", @@ -3059,7 +3115,7 @@ }, "/api/sico/sandbox/apply": { "post": { - "description": "Apply for a sandbox of the specified type. One instanceID can have multiple sandboxes.", + "description": "Apply for a sandbox supplying the requested OS. One instanceID can have multiple sandboxes.", "consumes": [ "application/json" ], @@ -3107,7 +3163,7 @@ "required": true }, { - "description": "Sandbox type to apply", + "description": "Sandbox OS to apply", "name": "request", "in": "body", "required": true, @@ -3646,37 +3702,6 @@ } } }, - "/api/sico/skills/details": { - "get": { - "security": [ - { - "BearerAuth": [] - } - ], - "produces": [ - "application/json" - ], - "tags": [ - "skills" - ], - "parameters": [ - { - "type": "integer", - "name": "id", - "in": "query", - "required": true - } - ], - "responses": { - "200": { - "description": "OK", - "schema": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.GetSkillDetailsResponse" - } - } - } - } - }, "/api/sico/skills/list": { "get": { "security": [ @@ -4216,6 +4241,9 @@ "type": "string" } }, + "creatorUsername": { + "type": "string" + }, "name": { "type": "string" }, @@ -4420,12 +4448,50 @@ } } }, + "sico-backend_internal_transport_http_dto_conversation.BatchSummaryItem": { + "type": "object", + "properties": { + "batchId": { + "type": "string" + }, + "createdAt": { + "type": "integer" + }, + "endedAt": { + "type": "integer" + }, + "parentConversationId": { + "type": "integer" + }, + "parentTurnId": { + "type": "integer" + }, + "reason": { + "type": "string" + }, + "status": { + "type": "string" + }, + "summaryUri": { + "type": "string" + }, + "totalCount": { + "type": "integer" + }, + "updatedAt": { + "type": "integer" + } + } + }, "sico-backend_internal_transport_http_dto_conversation.CancelPlanRequest": { "type": "object", "properties": { "agentInstanceId": { "type": "integer" }, + "conversationId": { + "type": "integer" + }, "turnId": { "type": "integer" } @@ -4492,6 +4558,9 @@ "content": { "type": "string" }, + "conversationId": { + "type": "integer" + }, "functionContext": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.FunctionContext" }, @@ -4695,6 +4764,34 @@ } } }, + "sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesData": { + "type": "object", + "properties": { + "hasMore": { + "type": "boolean" + }, + "items": { + "type": "array", + "items": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.BatchSummaryItem" + } + } + } + }, + "sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesResponse": { + "type": "object", + "properties": { + "code": { + "type": "integer" + }, + "data": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesData" + }, + "msg": { + "type": "string" + } + } + }, "sico-backend_internal_transport_http_dto_conversation.ListConversationData": { "type": "object", "properties": { @@ -4788,6 +4885,9 @@ "content": { "type": "string" }, + "conversationId": { + "type": "integer" + }, "createdAt": { "type": "integer" }, @@ -4893,6 +4993,9 @@ "items": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCall" } + }, + "updatedAt": { + "type": "integer" } } }, @@ -4960,24 +5063,49 @@ } } }, - "sico-backend_internal_transport_http_dto_conversation.ToolCall": { + "sico-backend_internal_transport_http_dto_conversation.TaskRuntimeExecutionInfo": { "type": "object", "properties": { - "batchCalls": { - "type": "array", - "items": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCall" - } + "attempt": { + "type": "integer" }, - "batchItemIndex": { + "currentStage": { + "description": "Lifecycle stage of a single task run: plan | workspace | sandbox | execute | upload | release.", + "type": "string" + }, + "latestProgressMessage": { + "type": "string" + }, + "maxAttempts": { "type": "integer" }, + "sandboxEndpoint": { + "type": "string" + }, + "sandboxId": { + "type": "string" + }, + "sandboxType": { + "type": "string" + } + } + }, + "sico-backend_internal_transport_http_dto_conversation.ToolCall": { + "type": "object", + "properties": { "deliverables": { "type": "array", "items": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolDeliverable" } }, + "display": { + "description": "Structured display labels (task_label, sandbox_label, sandbox_label_plural,\nsandbox_ready_label, sandbox_releasing_label, sandbox_release_label,\nenvironment_label, runner_label, batch_subject_singular,\nbatch_subject_plural, ...) sourced from the capability card. The frontend\nshould read these directly instead of pattern-matching `message`.", + "type": "object", + "additionalProperties": { + "type": "string" + } + }, "executionInfo": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolExecutionInfo" }, @@ -4985,10 +5113,13 @@ "description": "message is for frontend display", "type": "string" }, - "runningList": { + "subCallIndex": { + "type": "integer" + }, + "subCalls": { "type": "array", "items": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItem" + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCall" } }, "toolCallId": { @@ -4999,40 +5130,12 @@ }, "toolName": { "type": "string" - } - } - }, - "sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItem": { - "type": "object", - "properties": { - "name": { - "type": "string" }, - "status": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItemStatus" + "updatedAt": { + "type": "integer" } } }, - "sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItemStatus": { - "type": "integer", - "format": "int32", - "enum": [ - 0, - 1, - 2, - 3, - 4, - 5 - ], - "x-enum-varnames": [ - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_UNKNOWN", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_PENDING", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_RUNNING", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_DONE", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_FAILED", - "ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_CANCELLED" - ] - }, "sico-backend_internal_transport_http_dto_conversation.ToolCallStatus": { "type": "integer", "format": "int32", @@ -5045,7 +5148,8 @@ 5, 6, 7, - 8 + 8, + 9 ], "x-enum-varnames": [ "ToolCallStatus_TOOL_CALL_STATUS_UNKNOWN", @@ -5056,7 +5160,8 @@ "ToolCallStatus_TOOL_CALL_STATUS_FAILED_ANALYZED", "ToolCallStatus_TOOL_CALL_STATUS_RETRY_RUNNING", "ToolCallStatus_TOOL_CALL_STATUS_RETRY_SUCCESSFUL", - "ToolCallStatus_TOOL_CALL_STATUS_RETRY_FAILED" + "ToolCallStatus_TOOL_CALL_STATUS_RETRY_FAILED", + "ToolCallStatus_TOOL_CALL_STATUS_PENDING" ] }, "sico-backend_internal_transport_http_dto_conversation.ToolDeliverable": { @@ -5145,6 +5250,9 @@ "builtinToolName": { "type": "string" }, + "taskRuntime": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.TaskRuntimeExecutionInfo" + }, "toolType": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolType" } @@ -7753,34 +7861,15 @@ "properties": { "skill": { "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.Skill" - } - } - }, - "sico-backend_internal_transport_http_dto_skill.GetSkillDetailsData": { - "type": "object", - "properties": { - "files": { + }, + "versions": { "type": "array", "items": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillFile" + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillVersion" } } } }, - "sico-backend_internal_transport_http_dto_skill.GetSkillDetailsResponse": { - "type": "object", - "properties": { - "code": { - "type": "integer" - }, - "data": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.GetSkillDetailsData" - }, - "msg": { - "type": "string" - } - } - }, "sico-backend_internal_transport_http_dto_skill.GetSkillResponse": { "type": "object", "properties": { @@ -7832,21 +7921,12 @@ "agentId": { "type": "string" }, - "assetId": { - "type": "integer" - }, "createdAt": { "type": "integer" }, - "creatorUsername": { - "type": "string" - }, "description": { "type": "string" }, - "failReason": { - "type": "string" - }, "id": { "type": "integer" }, @@ -7856,11 +7936,25 @@ "projectId": { "type": "integer" }, - "status": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillStatus" - }, "updatedAt": { "type": "integer" + }, + "version": { + "type": "string" + } + } + }, + "sico-backend_internal_transport_http_dto_skill.SkillAction": { + "type": "object", + "properties": { + "advancedSettings": { + "type": "string" + }, + "description": { + "type": "string" + }, + "name": { + "type": "string" } } }, @@ -7903,24 +7997,95 @@ "SkillStatus_SKILL_STATUS_FAILED" ] }, + "sico-backend_internal_transport_http_dto_skill.SkillVersion": { + "type": "object", + "properties": { + "actions": { + "type": "array", + "items": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillAction" + } + }, + "createdAt": { + "type": "integer" + }, + "creatorUsername": { + "type": "string" + }, + "description": { + "type": "string" + }, + "failReason": { + "type": "string" + }, + "id": { + "type": "integer" + }, + "name": { + "type": "string" + }, + "skillId": { + "type": "integer" + }, + "status": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillStatus" + }, + "updatedAt": { + "type": "integer" + }, + "url": { + "type": "string" + }, + "version": { + "type": "string" + } + } + }, "sico-backend_internal_transport_http_dto_skill.UpdateSkillData": { "type": "object", "properties": { - "skill": { - "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.Skill" + "assetId": { + "type": "integer" + }, + "description": { + "type": "string" + }, + "name": { + "type": "string" + }, + "skillId": { + "type": "integer" + }, + "version": { + "type": "string" } } }, "sico-backend_internal_transport_http_dto_skill.UpdateSkillRequest": { "type": "object", "required": [ - "assetId", + "currentVersion", "id" ], "properties": { + "actions": { + "type": "array", + "items": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillAction" + } + }, "assetId": { "type": "integer" }, + "currentVersion": { + "type": "string" + }, + "files": { + "type": "array", + "items": { + "$ref": "#/definitions/sico-backend_internal_transport_http_dto_skill.SkillFile" + } + }, "id": { "type": "integer" } diff --git a/backend/api/openapi/swagger.yaml b/backend/api/openapi/swagger.yaml index b4e88b5..8d3e118 100644 --- a/backend/api/openapi/swagger.yaml +++ b/backend/api/openapi/swagger.yaml @@ -296,6 +296,8 @@ definitions: items: type: string type: array + creatorUsername: + type: string name: type: string role: @@ -430,10 +432,35 @@ definitions: role: type: string type: object + sico-backend_internal_transport_http_dto_conversation.BatchSummaryItem: + properties: + batchId: + type: string + createdAt: + type: integer + endedAt: + type: integer + parentConversationId: + type: integer + parentTurnId: + type: integer + reason: + type: string + status: + type: string + summaryUri: + type: string + totalCount: + type: integer + updatedAt: + type: integer + type: object sico-backend_internal_transport_http_dto_conversation.CancelPlanRequest: properties: agentInstanceId: type: integer + conversationId: + type: integer turnId: type: integer type: object @@ -477,6 +504,8 @@ definitions: properties: content: type: string + conversationId: + type: integer functionContext: $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.FunctionContext' isFinal: @@ -608,6 +637,24 @@ definitions: msg: type: string type: object + sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesData: + properties: + hasMore: + type: boolean + items: + items: + $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.BatchSummaryItem' + type: array + type: object + sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesResponse: + properties: + code: + type: integer + data: + $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesData' + msg: + type: string + type: object sico-backend_internal_transport_http_dto_conversation.ListConversationData: properties: conversations: @@ -673,6 +720,8 @@ definitions: type: array content: type: string + conversationId: + type: integer createdAt: type: integer functionContext: @@ -747,6 +796,8 @@ definitions: items: $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCall' type: array + updatedAt: + type: integer type: object sico-backend_internal_transport_http_dto_conversation.PlanStepStatus: enum: @@ -798,26 +849,51 @@ definitions: required: - agentInstanceId type: object - sico-backend_internal_transport_http_dto_conversation.ToolCall: + sico-backend_internal_transport_http_dto_conversation.TaskRuntimeExecutionInfo: properties: - batchCalls: - items: - $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCall' - type: array - batchItemIndex: + attempt: type: integer + currentStage: + description: 'Lifecycle stage of a single task run: plan | workspace | sandbox + | execute | upload | release.' + type: string + latestProgressMessage: + type: string + maxAttempts: + type: integer + sandboxEndpoint: + type: string + sandboxId: + type: string + sandboxType: + type: string + type: object + sico-backend_internal_transport_http_dto_conversation.ToolCall: + properties: deliverables: items: $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolDeliverable' type: array + display: + additionalProperties: + type: string + description: |- + Structured display labels (task_label, sandbox_label, sandbox_label_plural, + sandbox_ready_label, sandbox_releasing_label, sandbox_release_label, + environment_label, runner_label, batch_subject_singular, + batch_subject_plural, ...) sourced from the capability card. The frontend + should read these directly instead of pattern-matching `message`. + type: object executionInfo: $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolExecutionInfo' message: description: message is for frontend display type: string - runningList: + subCallIndex: + type: integer + subCalls: items: - $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItem' + $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCall' type: array toolCallId: type: integer @@ -825,31 +901,9 @@ definitions: $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCallStatus' toolName: type: string + updatedAt: + type: integer type: object - sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItem: - properties: - name: - type: string - status: - $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItemStatus' - type: object - sico-backend_internal_transport_http_dto_conversation.ToolCallRunningListItemStatus: - enum: - - 0 - - 1 - - 2 - - 3 - - 4 - - 5 - format: int32 - type: integer - x-enum-varnames: - - ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_UNKNOWN - - ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_PENDING - - ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_RUNNING - - ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_DONE - - ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_FAILED - - ToolCallRunningListItemStatus_TOOL_CALL_RUNNING_LIST_ITEM_STATUS_CANCELLED sico-backend_internal_transport_http_dto_conversation.ToolCallStatus: enum: - 0 @@ -861,6 +915,7 @@ definitions: - 6 - 7 - 8 + - 9 format: int32 type: integer x-enum-varnames: @@ -873,6 +928,7 @@ definitions: - ToolCallStatus_TOOL_CALL_STATUS_RETRY_RUNNING - ToolCallStatus_TOOL_CALL_STATUS_RETRY_SUCCESSFUL - ToolCallStatus_TOOL_CALL_STATUS_RETRY_FAILED + - ToolCallStatus_TOOL_CALL_STATUS_PENDING sico-backend_internal_transport_http_dto_conversation.ToolDeliverable: properties: acquiredSandbox: @@ -934,6 +990,8 @@ definitions: properties: builtinToolName: type: string + taskRuntime: + $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.TaskRuntimeExecutionInfo' toolType: $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ToolType' type: object @@ -2663,23 +2721,11 @@ definitions: properties: skill: $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.Skill' - type: object - sico-backend_internal_transport_http_dto_skill.GetSkillDetailsData: - properties: - files: + versions: items: - $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.SkillFile' + $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.SkillVersion' type: array type: object - sico-backend_internal_transport_http_dto_skill.GetSkillDetailsResponse: - properties: - code: - type: integer - data: - $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.GetSkillDetailsData' - msg: - type: string - type: object sico-backend_internal_transport_http_dto_skill.GetSkillResponse: properties: code: @@ -2713,26 +2759,29 @@ definitions: properties: agentId: type: string - assetId: - type: integer createdAt: type: integer - creatorUsername: - type: string description: type: string - failReason: - type: string id: type: integer name: type: string projectId: type: integer - status: - $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.SkillStatus' updatedAt: type: integer + version: + type: string + type: object + sico-backend_internal_transport_http_dto_skill.SkillAction: + properties: + advancedSettings: + type: string + description: + type: string + name: + type: string type: object sico-backend_internal_transport_http_dto_skill.SkillFile: properties: @@ -2764,19 +2813,66 @@ definitions: - SkillStatus_SKILL_STATUS_UPLOADING - SkillStatus_SKILL_STATUS_UPLOADED - SkillStatus_SKILL_STATUS_FAILED + sico-backend_internal_transport_http_dto_skill.SkillVersion: + properties: + actions: + items: + $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.SkillAction' + type: array + createdAt: + type: integer + creatorUsername: + type: string + description: + type: string + failReason: + type: string + id: + type: integer + name: + type: string + skillId: + type: integer + status: + $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.SkillStatus' + updatedAt: + type: integer + url: + type: string + version: + type: string + type: object sico-backend_internal_transport_http_dto_skill.UpdateSkillData: properties: - skill: - $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.Skill' + assetId: + type: integer + description: + type: string + name: + type: string + skillId: + type: integer + version: + type: string type: object sico-backend_internal_transport_http_dto_skill.UpdateSkillRequest: properties: + actions: + items: + $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.SkillAction' + type: array assetId: type: integer + currentVersion: + type: string + files: + items: + $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.SkillFile' + type: array id: type: integer required: - - assetId + - currentVersion - id type: object sico-backend_internal_transport_http_dto_skill.UpdateSkillResponse: @@ -3183,6 +3279,38 @@ paths: - BearerAuth: [] tags: - Conversation + /api/sico/conversation/batch_summaries: + get: + parameters: + - in: query + name: conversationId + required: true + type: integer + - default: 1 + in: query + minimum: 1 + name: page + type: integer + - default: 20 + in: query + maximum: 100 + minimum: 1 + name: pageSize + type: integer + - in: query + name: turnId + type: integer + produces: + - application/json + responses: + "200": + description: OK + schema: + $ref: '#/definitions/sico-backend_internal_transport_http_dto_conversation.ListBatchSummariesResponse' + security: + - BearerAuth: [] + tags: + - Conversation /api/sico/conversation/chat: post: consumes: @@ -3318,6 +3446,9 @@ paths: - in: query name: agentInstanceId type: integer + - in: query + name: conversationId + type: integer - in: query name: turnId type: integer @@ -4701,8 +4832,8 @@ paths: post: consumes: - application/json - description: Apply for a sandbox of the specified type. One instanceID can have - multiple sandboxes. + description: Apply for a sandbox supplying the requested OS. One instanceID + can have multiple sandboxes. parameters: - description: Instance context (JSON with agentInstanceId) in: header @@ -4729,7 +4860,7 @@ paths: name: X-Sico-Signature required: true type: string - - description: Sandbox type to apply + - description: Sandbox OS to apply in: body name: request required: true @@ -5053,24 +5184,6 @@ paths: - BearerAuth: [] tags: - skills - /api/sico/skills/details: - get: - parameters: - - in: query - name: id - required: true - type: integer - produces: - - application/json - responses: - "200": - description: OK - schema: - $ref: '#/definitions/sico-backend_internal_transport_http_dto_skill.GetSkillDetailsResponse' - security: - - BearerAuth: [] - tags: - - skills /api/sico/skills/list: get: parameters: diff --git a/backend/cmd/dbgen/dbgen.go b/backend/cmd/dbgen/dbgen.go index 55648fb..84c341d 100644 --- a/backend/cmd/dbgen/dbgen.go +++ b/backend/cmd/dbgen/dbgen.go @@ -75,6 +75,12 @@ type jsonColumn struct { type tableSpec struct { name string jsonColumns []jsonColumn + // datatypesJSONColumns lists JSON columns whose Go type must be the + // self-serializing gorm datatypes.JSON ([]byte). Unlike jsonColumns these + // must NOT receive a `serializer:json` tag: datatypes.JSON already + // implements driver.Valuer / sql.Scanner, so layering the JSON serializer on + // top would re-encode the raw bytes (base64) and corrupt the column. + datatypesJSONColumns []string } // storeSpec groups the tables that belong to one generated query package. @@ -153,6 +159,21 @@ var stores = []storeSpec{ outDir: "internal/store/skill/internal/dal/query", tables: []tableSpec{ {name: "t_skill"}, + {name: "t_skill_version"}, + }, + }, + { + outDir: "internal/store/taskruntime/internal/dal/query", + fieldNullable: true, + tables: []tableSpec{ + { + name: "t_task_runtime_batch", + datatypesJSONColumns: []string{"counts_json", "batch_json"}, + }, + { + name: "t_task_runtime_run", + datatypesJSONColumns: []string{"run_json", "result_json"}, + }, }, }, } @@ -191,7 +212,9 @@ func applyMigrations() error { if err != nil { return err } + log.Printf("migrations applied, version=%d", version) + return nil } @@ -201,6 +224,7 @@ func closeDB(db *gorm.DB) { log.Printf("resolve underlying sql.DB failed: %v", err) return } + if err := sqlDB.Close(); err != nil { log.Printf("close sql.DB failed: %v", err) } @@ -215,6 +239,11 @@ func generateStore(db *gorm.DB, root string, s storeSpec) error { }) g.UseDB(db) g.WithOpts(gen.FieldType("deleted_at", "gorm.DeletedAt")) + if storeUsesDatatypesJSON(s) { + // Ensure the generated model package imports gorm.io/datatypes for the + // columns retyped as datatypes.JSON below. + g.WithImportPkgPath("gorm.io/datatypes") + } // Derive the import path of the models package that the generator will // emit. Types declared inside that package must be rendered unqualified @@ -243,22 +272,65 @@ func generateStore(db *gorm.DB, root string, s storeSpec) error { missing = append(missing, fmt.Sprintf("%s.%s", t.name, c.name)) } } + for _, c := range t.datatypesJSONColumns { + if !applied[t.name][c] { + missing = append(missing, fmt.Sprintf("%s.%s", t.name, c)) + } + } } + if len(missing) > 0 { return fmt.Errorf("json column overrides did not match any generated field: %s", strings.Join(missing, ", ")) } return nil } +// storeUsesDatatypesJSON reports whether any table in s declares a column that +// must be retyped as datatypes.JSON. +func storeUsesDatatypesJSON(s storeSpec) bool { + for _, t := range s.tables { + if len(t.datatypesJSONColumns) > 0 { + return true + } + } + + return false +} + func modelOptions(t tableSpec, modelPkgPath string, hits map[string]bool) []gen.ModelOpt { - opts := make([]gen.ModelOpt, 0, len(t.jsonColumns)+1) + opts := make([]gen.ModelOpt, 0, len(t.jsonColumns)+2) for _, c := range t.jsonColumns { opts = append(opts, gen.FieldModify(jsonColumnModifier(c, modelPkgPath, hits))) } + + if len(t.datatypesJSONColumns) > 0 { + opts = append(opts, gen.FieldModify(datatypesJSONModifier(t.datatypesJSONColumns, hits))) + } opts = append(opts, gen.FieldModify(timestampModifier)) + return opts } +// datatypesJSONModifier retypes the named columns to the self-serializing +// gorm datatypes.JSON. It records each applied column in hits so the caller can +// detect stale configuration, and deliberately leaves the gorm tag untouched +// (no `serializer:json`) so datatypes.JSON handles its own Scan/Value. +func datatypesJSONModifier(columns []string, hits map[string]bool) func(gen.Field) gen.Field { + want := make(map[string]bool, len(columns)) + for _, c := range columns { + want[c] = true + } + + return func(f gen.Field) gen.Field { + if !want[f.ColumnName] { + return f + } + hits[f.ColumnName] = true + f.Type = "datatypes.JSON" + return f + } +} + // jsonColumnModifier retypes a column to match the Go type of c.sample and // tags it with the gorm `serializer:json` tag. When the modifier matches, it // records the hit in hits so the caller can detect stale configuration. @@ -285,6 +357,7 @@ func timestampModifier(f gen.Field) gen.Field { case "updated_at": f.GORMTag.Set("autoUpdateTime", "milli") } + return f } @@ -310,5 +383,6 @@ func renderGoType(t reflect.Type, byValue bool, modelPkgPath string) string { if byValue { return name } + return "*" + name } diff --git a/backend/cmd/sico-server/seeds/seeder.go b/backend/cmd/sico-server/seeds/seeder.go index ed78f34..e815de6 100644 --- a/backend/cmd/sico-server/seeds/seeder.go +++ b/backend/cmd/sico-server/seeds/seeder.go @@ -29,6 +29,7 @@ import ( "errors" "fmt" "mime/multipart" + "sort" "strconv" "time" @@ -44,6 +45,7 @@ import ( agentrepo "sico-backend/internal/store/agent/singleagent/repository" projectrepo "sico-backend/internal/store/project/repository" userrepo "sico-backend/internal/store/rbac/repository" + skillrepo "sico-backend/internal/store/skill/repository" agentdto "sico-backend/internal/transport/http/dto/agent/single_agent" projectdto "sico-backend/internal/transport/http/dto/project" rbacCommon "sico-backend/internal/transport/http/dto/rbac/common" @@ -61,10 +63,21 @@ const ( iconExt = "svg" iconFileType = "image" - androidTesterSkillName = "android-tester.zip" - defaultSkillContentType = "application/zip" - defaultSkillExt = "zip" - defaultSkillFileType = "archive" + androidTesterSkillName = "android-tester.zip" + testCasesRewriteSkillName = "test-cases-rewrite.zip" + threeDArtistSkillName = "ai-3d-model.zip" + pmCompetitiveSkillName = "pm-competitive-analysis.zip" + pmSlidesSkillName = "pm-frontend-slides.zip" + pmOKRsSkillName = "pm-setting-okrs-goals.zip" + pmPRDsSkillName = "pm-writing-prds.zip" + marketingBrandSkillName = "marketing-brand-storytelling.zip" + marketingContentSkillName = "marketing-content-marketing.zip" + marketingImageSkillName = "marketing-image-generation.zip" + marketingLaunchSkillName = "marketing-launch-marketing.zip" + marketingPositioningSkillName = "marketing-positioning-messaging.zip" + defaultSkillContentType = "application/zip" + defaultSkillExt = "zip" + defaultSkillFileType = "archive" defaultProjectId = 1 @@ -83,6 +96,11 @@ type embeddedFile struct { func (embeddedFile) Close() error { return nil } +type seedSkillFile struct { + FileName string + Content []byte +} + // ---------- Required Data ------------ func getDefaultProject(iconURI string) *projectrepo.ProjectModel { @@ -699,12 +717,19 @@ func ensureAgentAndroidTester(ctx context.Context, injector *di.Injector) error return err } - // Register the default android-tester skill against the tester agent. + // Register the default Android tester skills against the tester agent. androidTesterSkillBytes, err := embeddata.AndroidTesterSkillZip() if err != nil { return err } - skills := [][]byte{androidTesterSkillBytes} + testCasesRewriteSkillBytes, err := embeddata.TestCasesRewriteSkillZip() + if err != nil { + return err + } + skills := []seedSkillFile{ + {FileName: androidTesterSkillName, Content: androidTesterSkillBytes}, + {FileName: testCasesRewriteSkillName, Content: testCasesRewriteSkillBytes}, + } if err := ensureSkill(ctx, injector, skills, testerAgent.AgentId); err != nil { return err } @@ -742,8 +767,7 @@ func ensureAgent3DArtist(ctx context.Context, injector *di.Injector) error { } // Register the default 3D artist skill against the 3D artist agent. - artistSkillBytes := embeddata.ThreeDArtistSkillZip - skills := [][]byte{artistSkillBytes} + skills := []seedSkillFile{{FileName: threeDArtistSkillName, Content: embeddata.ThreeDArtistSkillZip}} if err := ensureSkill(ctx, injector, skills, artistAgent.AgentId); err != nil { return err } @@ -770,11 +794,11 @@ func ensureAgentProductManager(ctx context.Context, injector *di.Injector) error return err } - skills := [][]byte{ - embeddata.ProductManagerSkillCompetitiveAnalysisZip, - embeddata.ProductManagerSkillFrontendSlidesZip, - embeddata.ProductManagerSkillSettingOKRsGoalsZip, - embeddata.ProductManagerSkillWritingPRDsZip, + skills := []seedSkillFile{ + {FileName: pmCompetitiveSkillName, Content: embeddata.ProductManagerSkillCompetitiveAnalysisZip}, + {FileName: pmSlidesSkillName, Content: embeddata.ProductManagerSkillFrontendSlidesZip}, + {FileName: pmOKRsSkillName, Content: embeddata.ProductManagerSkillSettingOKRsGoalsZip}, + {FileName: pmPRDsSkillName, Content: embeddata.ProductManagerSkillWritingPRDsZip}, } if err := ensureSkill(ctx, injector, skills, pmAgent.AgentId); err != nil { return err @@ -801,12 +825,12 @@ func ensureAgentMarketing(ctx context.Context, injector *di.Injector) error { return err } - skills := [][]byte{ - embeddata.MarketingSkillBrandStorytellingZip, - embeddata.MarketingSkillContentMarketingZip, - embeddata.MarketingSkillImageGenerationZip, - embeddata.MarketingSkillLaunchMarketingZip, - embeddata.MarketingSkillPositioningMessagingZip, + skills := []seedSkillFile{ + {FileName: marketingBrandSkillName, Content: embeddata.MarketingSkillBrandStorytellingZip}, + {FileName: marketingContentSkillName, Content: embeddata.MarketingSkillContentMarketingZip}, + {FileName: marketingImageSkillName, Content: embeddata.MarketingSkillImageGenerationZip}, + {FileName: marketingLaunchSkillName, Content: embeddata.MarketingSkillLaunchMarketingZip}, + {FileName: marketingPositioningSkillName, Content: embeddata.MarketingSkillPositioningMessagingZip}, } if err := ensureSkill(ctx, injector, skills, marketingAgent.AgentId); err != nil { return err @@ -874,12 +898,12 @@ func Run(ctx context.Context, injector *di.Injector) error { return nil } -// ensureSkill uploads `skillFileContent` as an unscoped project asset and +// ensureSkill uploads each skill archive as an unscoped project asset and // ensures a skill record bound to `agentId` exists via the skill application // service. The app layer is used (rather than the repository) because it // handles zip extraction via the core gRPC `ExtractSkill` call after the // record is created or its asset_id changes. -func ensureSkill(ctx context.Context, injector *di.Injector, skillFileContents [][]byte, agentId string) error { +func ensureSkill(ctx context.Context, injector *di.Injector, skillFiles []seedSkillFile, agentId string) error { if injector == nil || injector.SkillApp == nil { return errors.New("ensureSkill: skill service not initialized") } @@ -888,21 +912,9 @@ func ensureSkill(ctx context.Context, injector *di.Injector, skillFileContents [ } // 1) Upload the skill archive as an unscoped project asset (idempotent). - assetIds := make([]int64, 0, len(skillFileContents)) - for i, content := range skillFileContents { - assetID, _, err := ensureAsset(ctx, injector, "", - embeddedFile{bytes.NewReader(content)}, - types.FileExtraInfo{ - FileName: androidTesterSkillName, - ContentType: defaultSkillContentType, - FileExt: defaultSkillExt, - FileType: defaultSkillFileType, - }, - ) - if err != nil { - return fmt.Errorf("ensureSkill: upload asset %d: %w", i, err) - } - assetIds = append(assetIds, assetID) + assetIds, err := uploadSkillAssets(ctx, injector, skillFiles) + if err != nil { + return err } // CreateSkill / UpdateSkill expect an authenticated user in context. @@ -918,18 +930,61 @@ func ensureSkill(ctx context.Context, injector *di.Injector, skillFileContents [ return fmt.Errorf("ensureSkill: list existing skills: %w", err) } - // 3) Delete all existing skills - for _, skill := range listResp.GetData().GetSkills() { - if _, err := injector.SkillApp.DeleteSkill( - skillCtx, &skilldto.DeleteSkillRequest{Id: skill.GetId()}, - ); err != nil { - return fmt.Errorf("ensureSkill: delete existing skill %d: %w", skill.GetId(), err) + existingSkills := append([]*skilldto.Skill(nil), listResp.GetData().GetSkills()...) + sort.Slice(existingSkills, func(i, j int) bool { return existingSkills[i].GetId() < existingSkills[j].GetId() }) + + // 3) Update existing skill records in place; create rows only when adding more seed skills. + for i, assetID := range assetIds { + if err := syncSkillRecord(ctx, skillCtx, injector, existingSkills, i, assetID, agentId); err != nil { + return err } - logger.CtxInfo(ctx, "ensureSkill: deleted existing skill %d for agent %s", skill.GetId(), agentId) } - // 4) Create new skill records for each uploaded asset. - for _, assetID := range assetIds { + return nil +} + +// uploadSkillAssets uploads each skill file as an unscoped project asset and +// returns the corresponding asset IDs. +func uploadSkillAssets(ctx context.Context, injector *di.Injector, skillFiles []seedSkillFile) ([]int64, error) { + assetIds := make([]int64, 0, len(skillFiles)) + for i, skillFile := range skillFiles { + if skillFile.FileName == "" { + return nil, fmt.Errorf("ensureSkill: skill file %d missing filename", i) + } + if len(skillFile.Content) == 0 { + return nil, fmt.Errorf("ensureSkill: %s is empty", skillFile.FileName) + } + assetID, _, err := ensureAsset(ctx, injector, "", + embeddedFile{bytes.NewReader(skillFile.Content)}, + types.FileExtraInfo{ + FileName: skillFile.FileName, + ContentType: defaultSkillContentType, + FileExt: defaultSkillExt, + FileType: defaultSkillFileType, + }, + ) + if err != nil { + return nil, fmt.Errorf("ensureSkill: upload asset %d: %w", i, err) + } + assetIds = append(assetIds, assetID) + } + return assetIds, nil +} + +// syncSkillRecord creates or updates a single skill record to match the +// expected asset ID. When index exceeds the number of existing skills a new +// record is created; otherwise the existing record is updated only when the +// underlying asset has changed. +func syncSkillRecord( + ctx context.Context, + skillCtx context.Context, + injector *di.Injector, + existingSkills []*skilldto.Skill, + index int, + assetID int64, + agentId string, +) error { + if index >= len(existingSkills) { if _, err := injector.SkillApp.CreateSkill(skillCtx, &skilldto.CreateSkillRequest{ AgentId: agentId, AssetId: assetID, @@ -937,7 +992,51 @@ func ensureSkill(ctx context.Context, injector *di.Injector, skillFileContents [ return fmt.Errorf("ensureSkill: create skill: %w", err) } logger.CtxInfo(ctx, "ensureSkill: created skill for agent %s with asset %d", agentId, assetID) + return nil } + existing := existingSkills[index] + currentVersion, currentAssetID, err := currentSkillVersionForSeed(skillCtx, injector, existing.GetId()) + if err != nil { + return err + } + if currentAssetID == assetID { + logger.CtxInfo( + ctx, + "ensureSkill: skipped unchanged skill %d for agent %s asset=%d", + existing.GetId(), agentId, assetID, + ) + return nil + } + if _, err := injector.SkillApp.UpdateSkill(skillCtx, &skilldto.UpdateSkillRequest{ + Id: existing.GetId(), + AssetId: assetID, + CurrentVersion: currentVersion, + }); err != nil { + return fmt.Errorf("ensureSkill: update skill %d: %w", existing.GetId(), err) + } + logger.CtxInfo( + ctx, + "ensureSkill: updated skill %d for agent %s asset %d -> %d", + existing.GetId(), agentId, currentAssetID, assetID, + ) return nil } + +func currentSkillVersionForSeed( + ctx context.Context, + injector *di.Injector, + skillID int64, +) (string, int64, error) { + if injector.DB == nil { + return "", 0, errors.New("ensureSkill: database not initialized") + } + version, err := skillrepo.NewSkillRepo(injector.DB).GetLatestVersion(ctx, skillID) + if err != nil { + return "", 0, fmt.Errorf("ensureSkill: get latest version for skill %d: %w", skillID, err) + } + if version == nil || version.Version == "" { + return "", 0, fmt.Errorf("ensureSkill: skill %d has no current version", skillID) + } + return version.Version, version.AssetID, nil +} diff --git a/backend/configs/migrations/000002_task_runtime_tables.down.sql b/backend/configs/migrations/000002_task_runtime_tables.down.sql new file mode 100644 index 0000000..af08d73 --- /dev/null +++ b/backend/configs/migrations/000002_task_runtime_tables.down.sql @@ -0,0 +1,66 @@ +-- Copyright (c) 2026 Sico Authors +-- +-- Permission is hereby granted, free of charge, to any person obtaining a copy +-- of this software and associated documentation files (the "Software"), to deal +-- in the Software without restriction, including without limitation the rights +-- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +-- copies of the Software, and to permit persons to whom the Software is +-- furnished to do so, subject to the following conditions: +-- +-- The above copyright notice and this permission notice shall be included in +-- all copies or substantial portions of the Software. +-- +-- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +-- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +-- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +-- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +-- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +-- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +-- SOFTWARE. + +-- -------------------------------------------------------------------------- +-- Restore t_skill columns from t_skill_version, then drop t_skill_version +-- -------------------------------------------------------------------------- + +ALTER TABLE `t_skill` + ADD COLUMN `asset_id` bigint NOT NULL DEFAULT 0 COMMENT 'Associated project asset ID' AFTER `description`, + ADD COLUMN `creator_username` varchar(128) NOT NULL DEFAULT '' COMMENT 'Creator username' AFTER `asset_id`, + ADD COLUMN `status` tinyint NOT NULL DEFAULT 0 COMMENT 'Skill status: 0-UNKNOWN,1-UPLOADING,2-UPLOADED,3-FAILED' AFTER `creator_username`, + ADD COLUMN `fail_reason` varchar(1024) NOT NULL DEFAULT '' COMMENT 'Failure reason if status=FAILED' AFTER `status`; + +UPDATE `t_skill` AS s +LEFT JOIN ( + SELECT v.* + FROM `t_skill_version` AS v + INNER JOIN ( + SELECT `skill_id`, MAX(`id`) AS `id` + FROM `t_skill_version` + GROUP BY `skill_id` + ) AS latest ON latest.`id` = v.`id` +) AS latest_version ON latest_version.`skill_id` = s.`id` +SET + s.`asset_id` = COALESCE(latest_version.`asset_id`, 0), + s.`creator_username` = COALESCE(latest_version.`creator_username`, ''), + s.`fail_reason` = COALESCE(latest_version.`fail_reason`, ''), + s.`status` = COALESCE(latest_version.`status`, 0); + +ALTER TABLE `t_skill` + ADD KEY `idx_asset_id` (`asset_id`), + ADD KEY `idx_status` (`status`); + +DROP TABLE IF EXISTS `t_skill_version`; + +-- -------------------------------------------------------------------------- +-- Drop message recovery key +-- -------------------------------------------------------------------------- + +ALTER TABLE `t_message` + DROP INDEX `uniq_message_task_runtime_recovery`, + DROP COLUMN `task_runtime_recovery_key`; + +-- -------------------------------------------------------------------------- +-- Drop task runtime tables +-- -------------------------------------------------------------------------- + +DROP TABLE IF EXISTS `t_task_runtime_run`; +DROP TABLE IF EXISTS `t_task_runtime_batch`; diff --git a/backend/configs/migrations/000002_task_runtime_tables.up.sql b/backend/configs/migrations/000002_task_runtime_tables.up.sql new file mode 100644 index 0000000..dfa4330 --- /dev/null +++ b/backend/configs/migrations/000002_task_runtime_tables.up.sql @@ -0,0 +1,165 @@ +-- Copyright (c) 2026 Sico Authors +-- +-- Permission is hereby granted, free of charge, to any person obtaining a copy +-- of this software and associated documentation files (the "Software"), to deal +-- in the Software without restriction, including without limitation the rights +-- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +-- copies of the Software, and to permit persons to whom the Software is +-- furnished to do so, subject to the following conditions: +-- +-- The above copyright notice and this permission notice shall be included in +-- all copies or substantial portions of the Software. +-- +-- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +-- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +-- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +-- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +-- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +-- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +-- SOFTWARE. + +-- -------------------------------------------------------------------------- +-- Task runtime tables +-- -------------------------------------------------------------------------- + +CREATE TABLE IF NOT EXISTS `t_task_runtime_batch` ( + `id` bigint UNSIGNED NOT NULL AUTO_INCREMENT COMMENT 'Primary key ID', + `batch_id` varchar(64) NOT NULL COMMENT 'Task runtime batch ID', + `parent_conversation_id` bigint NOT NULL DEFAULT 0 COMMENT 'Parent conversation ID', + `parent_turn_id` bigint NOT NULL DEFAULT 0 COMMENT 'Parent turn ID', + `parent_tool_call_id` bigint NULL COMMENT 'Parent plan tool call ID', + `status` varchar(32) NOT NULL DEFAULT 'queued' COMMENT 'Batch status', + `reason` varchar(2000) NOT NULL DEFAULT '' COMMENT 'Delegation reason', + `join_strategy` varchar(32) NOT NULL DEFAULT 'partial_ok' COMMENT 'Batch join strategy', + `total_count` int NOT NULL DEFAULT 0 COMMENT 'Number of runs in this batch', + `counts_json` json NULL COMMENT 'Aggregated terminal counts', + `batch_json` json NOT NULL COMMENT 'Full BatchRecord JSON payload', + `created_at` bigint UNSIGNED NOT NULL DEFAULT 0 COMMENT 'Create time in milliseconds', + `updated_at` bigint UNSIGNED NOT NULL DEFAULT 0 COMMENT 'Update time in milliseconds', + `liveness_at` bigint UNSIGNED NULL COMMENT 'Owner-process batch liveness heartbeat in milliseconds', + `ended_at` bigint UNSIGNED NULL COMMENT 'End time in milliseconds', + `cancellation_reason` varchar(2000) NOT NULL DEFAULT '' COMMENT 'Cancellation reason', + PRIMARY KEY (`id`), + UNIQUE KEY `uniq_task_runtime_batch_id` (`batch_id`), + KEY `idx_task_runtime_batch_parent` (`parent_conversation_id`, `parent_turn_id`), + KEY `idx_task_runtime_batch_status` (`status`) +) ENGINE=InnoDB CHARACTER SET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='Task runtime batch metadata'; + +CREATE TABLE IF NOT EXISTS `t_task_runtime_run` ( + `id` bigint UNSIGNED NOT NULL AUTO_INCREMENT COMMENT 'Primary key ID', + `run_id` varchar(64) NOT NULL COMMENT 'Task runtime run ID', + `batch_id` varchar(64) NOT NULL COMMENT 'Task runtime batch ID', + `parent_conversation_id` bigint NOT NULL DEFAULT 0 COMMENT 'Parent conversation ID', + `parent_turn_id` bigint NOT NULL DEFAULT 0 COMMENT 'Parent turn ID', + `batch_item_index` int NOT NULL DEFAULT 0 COMMENT 'Run index inside batch', + `task_id` varchar(128) NOT NULL DEFAULT '' COMMENT 'LLM-visible task ID', + `idempotency_key` varchar(128) NOT NULL DEFAULT '' COMMENT 'Canonical idempotency key', + `status` varchar(32) NOT NULL DEFAULT 'queued' COMMENT 'Run status', + `attempt` int NOT NULL DEFAULT 1 COMMENT 'Retry attempt number', + `executor` varchar(64) NOT NULL DEFAULT '' COMMENT 'Executor backend', + `worker_id` varchar(128) NOT NULL DEFAULT '' COMMENT 'Current worker ID', + `fencing_token` varchar(128) NOT NULL DEFAULT '' COMMENT 'Current fencing token', + `queued_at` bigint UNSIGNED NOT NULL DEFAULT 0 COMMENT 'Queue time in milliseconds', + `started_at` bigint UNSIGNED NULL COMMENT 'Start time in milliseconds', + `ended_at` bigint UNSIGNED NULL COMMENT 'End time in milliseconds', + `last_error_class` varchar(64) NOT NULL DEFAULT '' COMMENT 'Last structured error class', + `last_error` mediumtext NOT NULL COMMENT 'Last error message', + `run_json` json NOT NULL COMMENT 'Full TaskRun JSON payload', + `result_json` json NULL COMMENT 'Full TaskResult JSON payload', + `latest_progress_message` varchar(1000) NOT NULL DEFAULT '' COMMENT 'Latest run progress message', + `latest_progress_at` bigint UNSIGNED NOT NULL DEFAULT 0 COMMENT 'Latest progress timestamp in milliseconds', + `created_at` bigint UNSIGNED NOT NULL DEFAULT 0 COMMENT 'Create time in milliseconds', + `updated_at` bigint UNSIGNED NOT NULL DEFAULT 0 COMMENT 'Update time in milliseconds', + PRIMARY KEY (`id`), + UNIQUE KEY `uniq_task_runtime_run_id` (`run_id`), + KEY `idx_task_runtime_run_batch` (`batch_id`, `batch_item_index`), + UNIQUE KEY `uniq_task_runtime_run_idempotency` (`idempotency_key`), + KEY `idx_task_runtime_run_status` (`status`), + KEY `idx_task_runtime_run_worker` (`worker_id`) +) ENGINE=InnoDB CHARACTER SET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='Task runtime run metadata and result'; + +-- -------------------------------------------------------------------------- +-- Message recovery key +-- -------------------------------------------------------------------------- + +ALTER TABLE `t_message` + ADD COLUMN `task_runtime_recovery_key` varchar(191) + GENERATED ALWAYS AS ( + CASE + WHEN JSON_UNQUOTE(JSON_EXTRACT(`function_context`, '$.result')) LIKE 'task_runtime_recovery_batch:%' + THEN JSON_UNQUOTE(JSON_EXTRACT(`function_context`, '$.result')) + ELSE NULL + END + ) STORED, + ADD UNIQUE KEY `uniq_message_task_runtime_recovery` ( + `conversation_id`, + `turn_id`, + `username`, + `agent_instance_id`, + `role`, + `content_type`, + `task_runtime_recovery_key` + ); + +-- -------------------------------------------------------------------------- +-- Skill versions +-- -------------------------------------------------------------------------- + +CREATE TABLE `t_skill_version` ( + `id` bigint NOT NULL AUTO_INCREMENT COMMENT 'Primary key', + `skill_id` bigint NOT NULL COMMENT 'Skill ID', + `version` varchar(32) NOT NULL DEFAULT '' COMMENT 'Timestamp version string', + `asset_id` bigint NOT NULL DEFAULT 0 COMMENT 'Associated project asset ID', + `name` varchar(256) NOT NULL DEFAULT '' COMMENT 'Skill version name', + `description` text NOT NULL COMMENT 'Skill version description', + `creator_username` varchar(128) NOT NULL DEFAULT '' COMMENT 'Version creator username', + `status` tinyint NOT NULL DEFAULT 0 COMMENT 'Skill version status: 0-UNKNOWN,1-UPLOADING,2-UPLOADED,3-FAILED', + `fail_reason` varchar(1024) NOT NULL DEFAULT '' COMMENT 'Failure reason if resolver failed', + `created_at` bigint unsigned NOT NULL DEFAULT 0 COMMENT 'Create Time in Milliseconds', + `updated_at` bigint unsigned NOT NULL DEFAULT 0 COMMENT 'Update Time in Milliseconds', + `deleted_at` datetime NULL COMMENT 'Delete Time', + PRIMARY KEY (`id`), + UNIQUE KEY `uniq_skill_version` (`skill_id`, `version`), + KEY `idx_skill_version_latest` (`skill_id`, `created_at`), + KEY `idx_skill_version_deleted` (`skill_id`, `deleted_at`), + KEY `idx_skill_version_asset_id` (`asset_id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='Skill immutable version table'; + +INSERT INTO `t_skill_version` ( + `skill_id`, + `version`, + `asset_id`, + `name`, + `description`, + `creator_username`, + `status`, + `fail_reason`, + `created_at`, + `updated_at`, + `deleted_at` +) +SELECT + `id`, + CAST(CASE + WHEN `created_at` > 0 THEN `created_at` + WHEN `updated_at` > 0 THEN `updated_at` + ELSE 1 + END AS CHAR), + `asset_id`, + `name`, + `description`, + `creator_username`, + `status`, + `fail_reason`, + `created_at`, + `updated_at`, + `deleted_at` +FROM `t_skill`; + +ALTER TABLE `t_skill` + DROP INDEX `idx_asset_id`, + DROP INDEX `idx_status`, + DROP COLUMN `asset_id`, + DROP COLUMN `creator_username`, + DROP COLUMN `status`, + DROP COLUMN `fail_reason`; diff --git a/backend/deployments/helm/values.yaml b/backend/deployments/helm/values.yaml index dee4a72..88b8a13 100644 --- a/backend/deployments/helm/values.yaml +++ b/backend/deployments/helm/values.yaml @@ -43,6 +43,7 @@ env: CORE_GRPC_ADDRESS: "sico-core:50053" REVERSE_GRPC_SERVE_ADDRESS: "0.0.0.0:50054" SEAWEEDFS_ENDPOINT: "http://sico-seaweedfs-filer:14003" + SICO_PUBLIC_ENDPOINT: "http://localhost:8080" EVENT_BUS_TYPE: kafka EVENT_BUS_TOPIC: core-backend KAFKA_BOOTSTRAP_SERVERS: "sico-kafka:9092" diff --git a/backend/go.mod b/backend/go.mod index 99aa10d..ba9251b 100644 --- a/backend/go.mod +++ b/backend/go.mod @@ -8,6 +8,7 @@ require ( github.com/cloudwego/eino v0.8.7 github.com/gin-contrib/cors v1.7.7 github.com/gin-gonic/gin v1.12.0 + github.com/go-sql-driver/mysql v1.8.1 github.com/golang-jwt/jwt/v4 v4.5.2 github.com/golang-migrate/migrate/v4 v4.19.1 github.com/google/uuid v1.6.0 @@ -27,6 +28,7 @@ require ( golang.org/x/text v0.35.0 google.golang.org/grpc v1.80.0 google.golang.org/protobuf v1.36.11 + gorm.io/datatypes v1.2.4 gorm.io/driver/mysql v1.5.7 gorm.io/gen v0.3.27 gorm.io/gorm v1.31.1 @@ -58,7 +60,6 @@ require ( github.com/go-playground/locales v0.14.1 // indirect github.com/go-playground/universal-translator v0.18.1 // indirect github.com/go-playground/validator/v10 v10.30.1 // indirect - github.com/go-sql-driver/mysql v1.8.1 // indirect github.com/goccy/go-json v0.10.5 // indirect github.com/goccy/go-yaml v1.19.2 // indirect github.com/golang/mock v1.6.0 // indirect @@ -102,6 +103,5 @@ require ( golang.org/x/tools v0.42.0 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20260406210006-6f92a3bedf2d // indirect gopkg.in/yaml.v3 v3.0.1 // indirect - gorm.io/datatypes v1.2.4 // indirect gorm.io/hints v1.1.0 // indirect ) diff --git a/backend/internal/biz/agent/impl/single_agent_core_test.go b/backend/internal/biz/agent/impl/single_agent_core_test.go index 9dfeeae..75e709e 100644 --- a/backend/internal/biz/agent/impl/single_agent_core_test.go +++ b/backend/internal/biz/agent/impl/single_agent_core_test.go @@ -164,14 +164,12 @@ func TestDeleteSingleAgentInstanceDoesNotDeleteWhenSandboxCleanupFails(t *testin pool := sandboximpl.NewPool(nil, rds) sandboxbiz.InitService(sandboximpl.NewService(pool, nil, nil)) - lease := &sandboximpl.Lease{ - SandboxID: "emulator:http://74.179.80.110:8000|3", - Type: "emulator", - ResourceID: "http://74.179.80.110:8000|3", - User: "123", - InUse: true, - } - seedAssignedSandbox(t, ctx, rds, lease) + sandboxID := "emulator:http://74.179.80.110:8000|3" + // Seed the assignment hash so HasAssignedSandboxesStrict finds a binding, + // but store corrupt JSON in the resource key so the strict lease read + // returns an unmarshal error — simulating a cleanup failure. + require.NoError(t, rds.HSet(ctx, "sandbox:assign:123", sandboxID, "emulator").Err()) + require.NoError(t, rds.Set(ctx, "sandbox:resource:"+sandboxID, "corrupt", 0).Err()) repo := &fakeSingleAgentInstanceRepo{} svc := &Service{Components: &Components{SingleAgentInstanceRepo: repo}} @@ -180,11 +178,11 @@ func TestDeleteSingleAgentInstanceDoesNotDeleteWhenSandboxCleanupFails(t *testin require.Error(t, err) require.Empty(t, repo.deleteCalls) - storedLease, getErr := rds.Get(ctx, "sandbox:resource:"+lease.SandboxID).Result() + storedLease, getErr := rds.Get(ctx, "sandbox:resource:"+sandboxID).Result() require.NoError(t, getErr) require.NotEmpty(t, storedLease) assignments, err := rds.HGetAll(ctx, "sandbox:assign:123").Result() require.NoError(t, err) - require.Contains(t, assignments, lease.SandboxID) + require.Contains(t, assignments, sandboxID) require.False(t, errors.Is(err, redis.Nil)) } diff --git a/backend/internal/biz/conversation/iconversation.go b/backend/internal/biz/conversation/iconversation.go index 1e80cb3..f9b6022 100644 --- a/backend/internal/biz/conversation/iconversation.go +++ b/backend/internal/biz/conversation/iconversation.go @@ -54,6 +54,10 @@ type Service interface { ctx context.Context, req *dto.GetUserMessageByUserAgentTurnIDRequest, ) (*dto.GetUserMessageByUserAgentTurnIDResponse, error) + ListBatchSummaries( + ctx context.Context, + req *dto.ListBatchSummariesRequest, + ) (*dto.ListBatchSummariesResponse, error) Chat(ctx context.Context, sender sse.SSESender, req *dto.ChatRequestHttp) error Reconnect(ctx context.Context, sender sse.SSESender, req *dto.ReconnectRequest) error diff --git a/backend/internal/biz/conversation/impl/batch_summaries.go b/backend/internal/biz/conversation/impl/batch_summaries.go new file mode 100644 index 0000000..9fb911d --- /dev/null +++ b/backend/internal/biz/conversation/impl/batch_summaries.go @@ -0,0 +1,88 @@ +// Copyright (c) 2026 Sico Authors +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// The above copyright notice and this permission notice shall be included in +// all copies or substantial portions of the Software. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +package impl + +import ( + "context" + "encoding/json" + + "gorm.io/gorm" +) + +// taskRuntimeBatchListRow is a read-only GORM mapping for t_task_runtime_batch +// used by the conversation batch-summary listing endpoint. +type taskRuntimeBatchListRow struct { + BatchID string `gorm:"column:batch_id"` + ParentConversationID int64 `gorm:"column:parent_conversation_id"` + ParentTurnID int64 `gorm:"column:parent_turn_id"` + Status string `gorm:"column:status"` + Reason string `gorm:"column:reason"` + TotalCount int32 `gorm:"column:total_count"` + BatchJSON string `gorm:"column:batch_json"` + CreatedAt uint64 `gorm:"column:created_at"` + UpdatedAt uint64 `gorm:"column:updated_at"` + EndedAt *uint64 `gorm:"column:ended_at"` +} + +func (taskRuntimeBatchListRow) TableName() string { return "t_task_runtime_batch" } + +// listBatchSummariesFilter narrows the t_task_runtime_batch read for one +// conversation. +type listBatchSummariesFilter struct { + ConversationID int64 + TurnID *int64 + Limit int + Offset int +} + +func listBatchSummaries( + ctx context.Context, + db *gorm.DB, + filter listBatchSummariesFilter, +) ([]taskRuntimeBatchListRow, error) { + q := db.WithContext(ctx). + Model(&taskRuntimeBatchListRow{}). + Where("parent_conversation_id = ?", filter.ConversationID) + if filter.TurnID != nil { + q = q.Where("parent_turn_id = ?", *filter.TurnID) + } + rows := make([]taskRuntimeBatchListRow, 0, filter.Limit+1) + if err := q.Order("id DESC").Limit(filter.Limit + 1).Offset(filter.Offset).Find(&rows).Error; err != nil { + return nil, err + } + return rows, nil +} + +// extractSummaryURI parses the BatchRecord JSON and returns the top-level +// ``summary_uri`` string. Missing or invalid payloads return "" — they should +// not break the HTTP listing response. +func extractSummaryURI(batchJSON string) string { + if batchJSON == "" { + return "" + } + var payload struct { + SummaryURI string `json:"summary_uri"` + } + if err := json.Unmarshal([]byte(batchJSON), &payload); err != nil { + return "" + } + return payload.SummaryURI +} diff --git a/backend/internal/biz/conversation/impl/chat.go b/backend/internal/biz/conversation/impl/chat.go index fffe97d..5fef7e2 100644 --- a/backend/internal/biz/conversation/impl/chat.go +++ b/backend/internal/biz/conversation/impl/chat.go @@ -30,6 +30,9 @@ import ( "sync" "time" + "google.golang.org/grpc" + "google.golang.org/grpc/connectivity" + "sico-backend/internal/consts" conventity "sico-backend/internal/entity/conversation/conversation" "sico-backend/internal/entity/conversation/message" @@ -49,8 +52,9 @@ import ( ) const ( - roleUser = "user" - roleAssistant = "assistant" + roleUser = "user" + roleAssistant = "assistant" + coreChatStartTimeout = 30 * time.Second ) var ( @@ -179,6 +183,7 @@ func (s *Service) sendChunkEvent( Timestamp: resp.GetTimestamp(), IsFinal: resp.GetIsFinal(), Role: roleAssistant, + ConversationID: connection.conversationId, TurnID: connection.turnId, }) if marshalErr != nil { @@ -404,9 +409,15 @@ func (s *Service) Chat(ctx context.Context, sender sse.SSESender, req *conversat } chatReq := s.buildChatRequest( - ctx, req, - username, singleAgent, agentInstance, conversation, turnID, - requestAttachments, agentAttachments, + ctx, + req, + username, + singleAgent, + agentInstance, + conversation, + turnID, + requestAttachments, + agentAttachments, ) logger.CtxInfo(ctx, "chat_stream_start conversationId=%d turnId=%d agentId=%s "+ @@ -440,7 +451,7 @@ func (s *Service) Chat(ctx context.Context, sender sse.SSESender, req *conversat safego.Go(ctx, func() { // Use WithoutCancel so the gRPC stream survives SSE disconnection, // while still propagating the OTel trace context to core. - _, streamErr := s.chatClient.StreamChat(context.WithoutCancel(ctx), chatReq) + streamErr := s.startCoreChat(ctx, chatReq) if streamErr != nil { logger.CtxError(ctx, "chat_grpc_stream_failed conversationId=%d turnId=%d agentInstanceId=%d model=%s err=%v", @@ -461,6 +472,37 @@ func (s *Service) Chat(ctx context.Context, sender sse.SSESender, req *conversat return nil } +func (s *Service) startCoreChat(ctx context.Context, chatReq *conversationdto.ChatRequest) error { + if err := s.waitCoreChatReady(ctx); err != nil { + return err + } + _, err := s.chatClient.StreamChat(context.WithoutCancel(ctx), chatReq, grpc.WaitForReady(true)) + return err +} + +func (s *Service) waitCoreChatReady(ctx context.Context) error { + if s.coreGRPC == nil { + return nil + } + waitCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), coreChatStartTimeout) + defer cancel() + + s.coreGRPC.ResetConnectBackoff() + s.coreGRPC.Connect() + for { + state := s.coreGRPC.GetState() + if state == connectivity.Ready { + return nil + } + if !s.coreGRPC.WaitForStateChange(waitCtx, state) { + if err := waitCtx.Err(); err != nil { + return err + } + return context.DeadlineExceeded + } + } +} + // Chat bridges the HTTP SSE endpoint to the gRPC ChatService StreamChat call. func (s *Service) Reconnect(ctx context.Context, sender sse.SSESender, req *conversationdto.ReconnectRequest) error { if s.chatClient == nil { @@ -684,10 +726,13 @@ func (s *Service) buildChatRequest( requestAttachments []*commondto.Attachment, agentAttachments []*commondto.Attachment, ) *conversationdto.ChatRequest { - agentInstanceName := "" - projectId := int64(0) - projectName := "" - agentRole := "" + var ( + agentInstanceName string + projectName string + agentRole string + projectId int64 + ) + if agentInstance != nil { agentInstanceName = agentInstance.GetName() projectId = agentInstance.GetProjectId() @@ -702,6 +747,7 @@ func (s *Service) buildChatRequest( } } } + return &conversationdto.ChatRequest{ Username: username, Message: buildUserChatContent(req.Message, requestAttachments), @@ -710,10 +756,10 @@ func (s *Service) buildChatRequest( AgentInstanceId: req.AgentInstanceID, AgentInstanceName: agentInstanceName, ConversationId: conversation.ID, + ProjectName: projectName, TurnId: turnID, AgentAttachments: agentAttachments, ProjectId: projectId, - ProjectName: projectName, Model: resolveAgentModel(singleAgent), AgentRole: agentRole, } diff --git a/backend/internal/biz/conversation/impl/chat_test.go b/backend/internal/biz/conversation/impl/chat_test.go index d7dd456..483db7c 100644 --- a/backend/internal/biz/conversation/impl/chat_test.go +++ b/backend/internal/biz/conversation/impl/chat_test.go @@ -45,6 +45,7 @@ const ( type mockChatClient struct { conversationrpc.ChatServiceClient mockEventBus *eventbus.MockEventBus + callOpts []grpc.CallOption } func (m *mockChatClient) StreamChat( @@ -52,6 +53,8 @@ func (m *mockChatClient) StreamChat( in *conversationdto.ChatRequest, opts ...grpc.CallOption, ) (*conversationdto.ChatDirectResponse, error) { + m.callOpts = append([]grpc.CallOption(nil), opts...) + chatResponses := []*conversationdto.ChatResponse{ { Content: &conversationdto.ChatContent{ @@ -145,6 +148,7 @@ func TestChat(t *testing.T) { } err = service.Chat(ctx, sseSender, chatRequest) require.NoError(t, err) + require.True(t, hasWaitForReadyOption(mockChatClient.callOpts)) // Verify that the SSE sender received the expected events allEvents := sseSender.Sent @@ -182,3 +186,13 @@ func TestChat(t *testing.T) { require.Equal(t, "done", sentEvents[3].Event) }) } + +func hasWaitForReadyOption(opts []grpc.CallOption) bool { + for _, opt := range opts { + option, ok := opt.(grpc.FailFastCallOption) + if ok && !option.FailFast { + return true + } + } + return false +} diff --git a/backend/internal/biz/conversation/impl/conversation.go b/backend/internal/biz/conversation/impl/conversation.go index 4c002af..991dcd2 100644 --- a/backend/internal/biz/conversation/impl/conversation.go +++ b/backend/internal/biz/conversation/impl/conversation.go @@ -27,6 +27,7 @@ import ( entity "sico-backend/internal/entity/conversation/conversation" "sico-backend/internal/shared/apperr" "sico-backend/internal/shared/errcode" + messageRepo "sico-backend/internal/store/conversation/message/repository" model "sico-backend/internal/transport/http/dto/conversation" "sico-backend/internal/transport/http/middleware" "sico-backend/pkg/logger" @@ -150,7 +151,31 @@ func (c *Service) getAgentInstanceInfo(ctx context.Context, agentInstanceID int6 } func (s *Service) GetPlan(ctx context.Context, req *model.GetPlanRequest) (*model.GetPlanResponse, error) { - response, err := s.chatClient.GetPlan(ctx, req) + conversationID, resolved, err := s.resolvePlanConversationID( + ctx, req.Username, req.AgentInstanceId, req.TurnId, req.ConversationId, + ) + if err != nil { + return nil, err + } + if !resolved { + logger.CtxWarn( + ctx, + "ambiguous plan lookup without conversation_id: username=%s agent_instance_id=%d turn_id=%d", + req.Username, + req.AgentInstanceId, + req.TurnId, + ) + return appresp.Success(&model.GetPlanResponse{ + Data: &model.GetPlanData{Status: model.PlanStatus_PLAN_STATUS_NO_PLAN}, + }), nil + } + + var planReq model.GetPlanRequest + planReq.AgentInstanceId = req.AgentInstanceId + planReq.Username = req.Username + planReq.TurnId = req.TurnId + planReq.ConversationId = conversationID + response, err := s.chatClient.GetPlan(ctx, &planReq) if err != nil { return nil, apperr.New(errcode.CommonUnavailable, "failed to query plan") } @@ -167,7 +192,29 @@ func (s *Service) GetPlan(ctx context.Context, req *model.GetPlanRequest) (*mode } func (s *Service) CancelPlan(ctx context.Context, req *model.CancelPlanRequest) (*model.CancelPlanResponse, error) { - _, err := s.chatClient.CancelPlan(ctx, req) + conversationID, resolved, err := s.resolvePlanConversationID( + ctx, req.Username, req.AgentInstanceId, req.TurnId, req.ConversationId, + ) + if err != nil { + return nil, err + } + if !resolved { + logger.CtxWarn( + ctx, + "ambiguous plan cancel without conversation_id: username=%s agent_instance_id=%d turn_id=%d", + req.Username, + req.AgentInstanceId, + req.TurnId, + ) + return appresp.Success(&model.CancelPlanResponse{}), nil + } + + var cancelReq model.CancelPlanRequest + cancelReq.AgentInstanceId = req.AgentInstanceId + cancelReq.Username = req.Username + cancelReq.TurnId = req.TurnId + cancelReq.ConversationId = conversationID + _, err = s.chatClient.CancelPlan(ctx, &cancelReq) if err != nil { return nil, apperr.New(errcode.CommonUnavailable, "failed to cancel plan") } @@ -175,6 +222,40 @@ func (s *Service) CancelPlan(ctx context.Context, req *model.CancelPlanRequest) return appresp.Success(&model.CancelPlanResponse{}), nil } +func (s *Service) resolvePlanConversationID( + ctx context.Context, + username string, + agentInstanceID int64, + turnID int64, + conversationID int64, +) (int64, bool, error) { + if conversationID != 0 { + return conversationID, true, nil + } + + role := roleUser + messages, hasMore, err := s.messageRepo.ListByFilter(ctx, &messageRepo.MessageFilter{ + Username: &username, + AgentInstanceId: &agentInstanceID, + TurnId: &turnID, + Role: &role, + IdDescending: true, + UsePagination: true, + Page: 1, + PageSize: 2, + }) + if err != nil { + return 0, false, err + } + if len(messages) > 1 || hasMore { + return 0, false, nil + } + if len(messages) > 0 { + return messages[0].ConversationId, true, nil + } + return 0, false, nil +} + func (s *Service) GenerateOnboardRecommendationTasks( ctx context.Context, req *model.GenerateOnboardRecommendationTasksRequest, ) (*model.GenerateOnboardRecommendationTasksResponse, error) { diff --git a/backend/internal/biz/conversation/impl/message.go b/backend/internal/biz/conversation/impl/message.go index e3b8ae7..07882ed 100644 --- a/backend/internal/biz/conversation/impl/message.go +++ b/backend/internal/biz/conversation/impl/message.go @@ -22,6 +22,11 @@ package impl import ( "context" + "errors" + "strings" + + "github.com/go-sql-driver/mysql" + "gorm.io/gorm" appresp "sico-backend/internal/biz/common/response" messageentity "sico-backend/internal/entity/conversation/message" @@ -31,12 +36,17 @@ import ( conversationdto "sico-backend/internal/transport/http/dto/conversation" "sico-backend/internal/transport/http/middleware" rgrpc "sico-backend/internal/transport/reverse_grpc/pb/conversation" + "sico-backend/pkg/logger" "sico-backend/pkg/ptr" ) const ( // Enable this when frontend has used the "reconnect" api. OmitMessageForOngoingTurn = false + + taskRuntimeRecoveryResultPrefix = "task_runtime_recovery_batch:" + legacyTaskRuntimeRecoveryMarkerPrefix = "") + if end < 0 { + return "", "" + } + + batchIDStart := markerStart + len(legacyTaskRuntimeRecoveryMarkerPrefix) + markerEnd := markerStart + end + len("-->") + return strings.TrimSpace(content[batchIDStart : markerStart+end]), content[markerStart:markerEnd] +} + +func stripLegacyTaskRuntimeRecoveryMessageMarker(content string) string { + _, marker := legacyTaskRuntimeRecoveryMarker(content) + if marker == "" { + return content + } + + return strings.TrimRight(strings.Replace(content, marker, "", 1), " \r\n") +} + +func isDuplicateKeyError(err error) bool { + if err == nil { + return false + } + if errors.Is(err, gorm.ErrDuplicatedKey) { + return true + } + + var mysqlErr *mysql.MySQLError + if errors.As(err, &mysqlErr) && mysqlErr.Number == mysqlDuplicateErrNum { + return true + } + + return strings.Contains(err.Error(), "Duplicate entry") +} + +func (c *Service) findExistingAssistantTextForRecoveredTaskRuntimeMessage( + ctx context.Context, + msg *messageentity.Message, + recoveryKey string, +) (*messageentity.Message, error) { + existing, _, err := c.messageRepo.ListByFilter(ctx, &messagerepo.MessageFilter{ + Username: &msg.Username, + ConversationId: &msg.ConversationId, + AgentInstanceId: &msg.AgentInstanceId, + TurnId: &msg.TurnId, + Role: ptr.Of(roleAssistant), + ContentTypeList: []conversationdto.ChatContentType{conversationdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT}, + }) + if err != nil { + return nil, err + } + + var matchingRecovery *messageentity.Message + var artifactText *messageentity.Message + for _, item := range existing { + if isTaskRuntimeArtifactAssistantTextMessage(item) && artifactText == nil { + artifactText = item + } + if taskRuntimeRecoveryMessageKey(item) == recoveryKey { + matchingRecovery = item + } + } + if artifactText != nil { + return artifactText, nil + } + + return matchingRecovery, nil +} + +func (c *Service) RpcCreateMessage(ctx context.Context, req *rgrpc.CreateMessageRequest) (*rgrpc.CreateMessageResponse, error) { message := req.Message // if message type is Plan and there is already // a plan in with same (agent_instance_id, username, turn_id), // we should omit this insertion if message.ContentType == conversationdto.ChatContentType_CHAT_CONTENT_TYPE_PLAN { - existing, _, err := c.messageRepo.ListByFilter(ctx, &messagerepo.MessageFilter{ - Username: &message.Username, - AgentInstanceId: &message.AgentInstanceId, - TurnId: &message.TurnId, - ContentTypeList: []conversationdto.ChatContentType{ - conversationdto.ChatContentType_CHAT_CONTENT_TYPE_PLAN, - }, - }) + resp, err := c.deduplicatePlanMessage(ctx, message) if err != nil { return nil, err } - if len(existing) > 0 { - return appresp.Success(&rgrpc.CreateMessageResponse{ - Data: &rgrpc.CreateMessageData{Id: existing[0].Id}, - }), nil + if resp != nil { + return resp, nil + } + } + + // Task-runtime recovery deduplication: if this is a recovered assistant text message, + // check if a final artifact-containing message already exists for this turn. + recoveryKey := c.getRecoveryKey(ctx, message) + if recoveryKey != "" { + resp, err := c.deduplicateRecoveryMessage(ctx, message, recoveryKey) + if err != nil { + return nil, err + } + if resp != nil { + return resp, nil } } created, err := c.messageRepo.Create(ctx, message) if err != nil { - return nil, err + return c.handleCreateMessageError(ctx, message, recoveryKey, err) } return &rgrpc.CreateMessageResponse{ Data: &rgrpc.CreateMessageData{Id: created.Id}, }, nil } +func (c *Service) deduplicatePlanMessage( + ctx context.Context, message *messageentity.Message, +) (*rgrpc.CreateMessageResponse, error) { + existing, _, err := c.messageRepo.ListByFilter(ctx, &messagerepo.MessageFilter{ + Username: &message.Username, + AgentInstanceId: &message.AgentInstanceId, + TurnId: &message.TurnId, + ContentTypeList: []conversationdto.ChatContentType{ + conversationdto.ChatContentType_CHAT_CONTENT_TYPE_PLAN, + }, + }) + if err != nil { + logger.CtxError(ctx, + "chat_plan_dedupe_lookup_failed turnId=%d agentInstanceId=%d contentType=%s err=%v", + message.TurnId, message.AgentInstanceId, message.ContentType.String(), err) + return nil, err + } + if len(existing) > 0 { + return appresp.Success(&rgrpc.CreateMessageResponse{ + Data: &rgrpc.CreateMessageData{Id: existing[0].Id}, + }), nil + } + return nil, nil +} + +func (c *Service) getRecoveryKey(_ context.Context, message *messageentity.Message) string { + if message.ContentType == conversationdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT && message.Role == roleAssistant { + return taskRuntimeRecoveryMessageKey(message) + } + return "" +} + +func (c *Service) deduplicateRecoveryMessage( + ctx context.Context, message *messageentity.Message, recoveryKey string, +) (*rgrpc.CreateMessageResponse, error) { + existing, err := c.findExistingAssistantTextForRecoveredTaskRuntimeMessage(ctx, message, recoveryKey) + if err != nil { + return nil, err + } + if existing != nil { + return appresp.Success(&rgrpc.CreateMessageResponse{ + Data: &rgrpc.CreateMessageData{Id: existing.Id}, + }), nil + } + return nil, nil +} + +func (c *Service) handleCreateMessageError( + ctx context.Context, message *messageentity.Message, recoveryKey string, err error, +) (*rgrpc.CreateMessageResponse, error) { + if recoveryKey != "" && isDuplicateKeyError(err) { + existing, findErr := c.findExistingAssistantTextForRecoveredTaskRuntimeMessage( + ctx, message, recoveryKey, + ) + if findErr != nil { + return nil, findErr + } + if existing != nil { + return appresp.Success(&rgrpc.CreateMessageResponse{ + Data: &rgrpc.CreateMessageData{Id: existing.Id}, + }), nil + } + } + if isDuplicateKeyError(err) { + logger.CtxWarn(ctx, + "chat_message_create_duplicate conversationId=%d turnId=%d agentInstanceId=%d contentType=%s", + message.ConversationId, message.TurnId, message.AgentInstanceId, message.ContentType.String()) + } else { + logger.CtxError(ctx, + "chat_message_create_failed conversationId=%d turnId=%d agentInstanceId=%d "+ + "contentType=%s err=%v", + message.ConversationId, message.TurnId, + message.AgentInstanceId, message.ContentType.String(), err) + } + return nil, err +} + func (c *Service) RpcListUserMessageByUserAgentTurnID( ctx context.Context, req *rgrpc.ListUserMessageByUserAgentTurnIDRequest, @@ -328,3 +609,76 @@ func (c *Service) RpcListUserMessageByUserAgentTurnID( Data: msgs, }), nil } + +// ListBatchSummaries returns delegated-task batches for a single conversation. +// The caller MUST own the conversation (creator_username == JWT subject), so a +// missing/foreign conversation is reported as a generic NotFound to avoid +// leaking existence. +func (c *Service) ListBatchSummaries( + ctx context.Context, + req *conversationdto.ListBatchSummariesRequest, +) (*conversationdto.ListBatchSummariesResponse, error) { + if c.db == nil { + return appresp.Success(&conversationdto.ListBatchSummariesResponse{ + Data: &conversationdto.ListBatchSummariesData{Items: []*conversationdto.BatchSummaryItem{}}, + }), nil + } + username := middleware.MustGetUsernameFromCtx(ctx) + conv, err := c.conversationRepo.GetByID(ctx, req.GetConversationId()) + if err != nil { + return nil, err + } + if conv == nil || conv.CreatorUsername != username { + return nil, apperr.New(errcode.CommonNotFound, "conversation not found") + } + pageSize := int(req.GetPageSize()) + if pageSize <= 0 { + pageSize = 20 + } + page := int(req.GetPage()) + if page <= 0 { + page = 1 + } + filter := listBatchSummariesFilter{ + ConversationID: req.GetConversationId(), + Limit: pageSize, + Offset: (page - 1) * pageSize, + } + if req.TurnId != nil { + v := req.GetTurnId() + filter.TurnID = &v + } + rows, err := listBatchSummaries(ctx, c.db, filter) + if err != nil { + return nil, err + } + hasMore := len(rows) > pageSize + if hasMore { + rows = rows[:pageSize] + } + items := make([]*conversationdto.BatchSummaryItem, 0, len(rows)) + for i := range rows { + r := rows[i] + item := &conversationdto.BatchSummaryItem{ + BatchId: r.BatchID, + ParentConversationId: r.ParentConversationID, + ParentTurnId: r.ParentTurnID, + Status: r.Status, + Reason: r.Reason, + TotalCount: r.TotalCount, + SummaryUri: extractSummaryURI(r.BatchJSON), + CreatedAt: int64(r.CreatedAt), + UpdatedAt: int64(r.UpdatedAt), + } + if r.EndedAt != nil { + item.EndedAt = int64(*r.EndedAt) + } + items = append(items, item) + } + return appresp.Success(&conversationdto.ListBatchSummariesResponse{ + Data: &conversationdto.ListBatchSummariesData{ + Items: items, + HasMore: hasMore, + }, + }), nil +} diff --git a/backend/internal/biz/conversation/impl/service.go b/backend/internal/biz/conversation/impl/service.go index cd1ae09..fadaaaf 100644 --- a/backend/internal/biz/conversation/impl/service.go +++ b/backend/internal/biz/conversation/impl/service.go @@ -26,6 +26,7 @@ import ( "time" "github.com/redis/go-redis/v9" + "gorm.io/gorm" "sico-backend/internal/biz/agent" "sico-backend/internal/biz/project" @@ -52,6 +53,7 @@ type Components struct { Storage storage.Storage CoreGRPC coregrpc.Connection Cache *redis.Client + DB *gorm.DB } type ChatConnection struct { @@ -83,10 +85,12 @@ type Service struct { projectSvc project.Service idGen idgen.IDGenerator storage storage.Storage + coreGRPC coregrpc.Connection chatClient conversationrpc.ChatServiceClient chatConnections map[ChatConnectionIdentifier][]*ChatConnection eventBusSubscription eventbus.EventBusSubscription cache *redis.Client + db *gorm.DB } // NewService wires dependencies into a conversation service implementation. @@ -103,9 +107,11 @@ func NewService(c *Components) *Service { projectSvc: c.ProjectService, idGen: c.IDGenerator, storage: c.Storage, + coreGRPC: c.CoreGRPC, chatClient: chatClient, chatConnections: make(map[ChatConnectionIdentifier][]*ChatConnection), cache: c.Cache, + db: c.DB, } _ = svc.SubscribeTopic() diff --git a/backend/internal/biz/conversation/impl/service_test.go b/backend/internal/biz/conversation/impl/service_test.go index 5e56393..e250d76 100644 --- a/backend/internal/biz/conversation/impl/service_test.go +++ b/backend/internal/biz/conversation/impl/service_test.go @@ -25,11 +25,15 @@ import ( "testing" "github.com/stretchr/testify/require" + "google.golang.org/grpc" + "gorm.io/gorm" convEntity "sico-backend/internal/entity/conversation/conversation" msgEntity "sico-backend/internal/entity/conversation/message" convMock "sico-backend/internal/store/conversation/conversation/repository/mock" + messagerepo "sico-backend/internal/store/conversation/message/repository" msgMock "sico-backend/internal/store/conversation/message/repository/mock" + conversationrpc "sico-backend/internal/transport/grpc/pb/conversation" convdto "sico-backend/internal/transport/http/dto/conversation" "sico-backend/internal/transport/http/middleware" rgrpc "sico-backend/internal/transport/reverse_grpc/pb/conversation" @@ -49,6 +53,68 @@ func newTestConversationService() *Service { } } +type recordingPlanChatClient struct { + conversationrpc.ChatServiceClient + getPlanRequests []*convdto.GetPlanRequest + cancelPlanRequests []*convdto.CancelPlanRequest +} + +type racingDuplicateMessageRepo struct { + messagerepo.MessageRepo + stored *msgEntity.Message + createCalls int + listCalls int +} + +func (r *racingDuplicateMessageRepo) Create(_ context.Context, msg *msgEntity.Message) (*msgEntity.Message, error) { + r.createCalls++ + var stored msgEntity.Message + stored.TurnId = msg.TurnId + stored.ConversationId = msg.ConversationId + stored.Username = msg.Username + stored.AgentInstanceId = msg.AgentInstanceId + stored.Role = msg.Role + stored.ContentType = msg.ContentType + stored.Content = msg.Content + stored.FunctionContext = msg.FunctionContext + stored.Ext = msg.Ext + stored.Attachments = msg.Attachments + stored.CreatedAt = msg.CreatedAt + stored.UpdatedAt = msg.UpdatedAt + stored.Id = 99 + r.stored = &stored + return nil, gorm.ErrDuplicatedKey +} + +func (r *racingDuplicateMessageRepo) ListByFilter( + _ context.Context, + _ *messagerepo.MessageFilter, +) ([]*msgEntity.Message, bool, error) { + r.listCalls++ + if r.listCalls == 1 || r.stored == nil { + return []*msgEntity.Message{}, false, nil + } + return []*msgEntity.Message{r.stored}, false, nil +} + +func (c *recordingPlanChatClient) GetPlan( + _ context.Context, + req *convdto.GetPlanRequest, + _ ...grpc.CallOption, +) (*convdto.GetPlanResponse, error) { + c.getPlanRequests = append(c.getPlanRequests, req) + return &convdto.GetPlanResponse{Data: &convdto.GetPlanData{Status: convdto.PlanStatus_PLAN_STATUS_RUNNING}}, nil +} + +func (c *recordingPlanChatClient) CancelPlan( + _ context.Context, + req *convdto.CancelPlanRequest, + _ ...grpc.CallOption, +) (*convdto.CancelPlanResponse, error) { + c.cancelPlanRequests = append(c.cancelPlanRequests, req) + return &convdto.CancelPlanResponse{}, nil +} + // region Conversation CRUD func TestCreateConversation(t *testing.T) { @@ -116,6 +182,137 @@ func TestListConversationsStatusCount(t *testing.T) { require.NoError(t, err) } +func TestGetPlanWithoutConversationIDResolvesUniqueTurn(t *testing.T) { + service := newTestConversationService() + chatClient := &recordingPlanChatClient{} + service.chatClient = chatClient + ctx := ctxWithUser("alice") + conv, err := service.conversationRepo.Create(ctx, &convEntity.Conversation{ + CreatorUsername: "alice", + AgentInstanceID: 42, + }) + require.NoError(t, err) + _, err = service.messageRepo.Create(ctx, &msgEntity.Message{ + ConversationId: conv.ID, + TurnId: 7, + Username: "alice", + AgentInstanceId: 42, + Role: roleUser, + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "hello", + }) + require.NoError(t, err) + + resp, err := service.GetPlan(ctx, &convdto.GetPlanRequest{ + AgentInstanceId: 42, + Username: "alice", + TurnId: 7, + }) + + require.NoError(t, err) + require.NotNil(t, resp.Data) + require.Equal(t, convdto.PlanStatus_PLAN_STATUS_RUNNING, resp.Data.Status) + require.Len(t, chatClient.getPlanRequests, 1) + require.Equal(t, conv.ID, chatClient.getPlanRequests[0].ConversationId) +} + +func TestGetPlanWithoutConversationIDDoesNotGuessWhenTurnMessageMissing(t *testing.T) { + service := newTestConversationService() + chatClient := &recordingPlanChatClient{} + service.chatClient = chatClient + ctx := ctxWithUser("alice") + _, err := service.conversationRepo.Create(ctx, &convEntity.Conversation{ + CreatorUsername: "alice", + AgentInstanceID: 42, + }) + require.NoError(t, err) + + resp, err := service.GetPlan(ctx, &convdto.GetPlanRequest{ + AgentInstanceId: 42, + Username: "alice", + TurnId: 7, + }) + + require.NoError(t, err) + require.NotNil(t, resp.Data) + require.Equal(t, convdto.PlanStatus_PLAN_STATUS_NO_PLAN, resp.Data.Status) + require.Empty(t, chatClient.getPlanRequests) +} + +func TestGetPlanWithoutConversationIDDoesNotGuessAmbiguousTurn(t *testing.T) { + service := newTestConversationService() + chatClient := &recordingPlanChatClient{} + service.chatClient = chatClient + ctx := ctxWithUser("alice") + firstConv, err := service.conversationRepo.Create(ctx, &convEntity.Conversation{ + CreatorUsername: "alice", + AgentInstanceID: 42, + }) + require.NoError(t, err) + secondConv, err := service.conversationRepo.Create(ctx, &convEntity.Conversation{ + CreatorUsername: "alice", + AgentInstanceID: 42, + }) + require.NoError(t, err) + for _, conversationID := range []int64{firstConv.ID, secondConv.ID} { + _, err = service.messageRepo.Create(ctx, &msgEntity.Message{ + ConversationId: conversationID, + TurnId: 7, + Username: "alice", + AgentInstanceId: 42, + Role: roleUser, + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "hello", + }) + require.NoError(t, err) + } + + resp, err := service.GetPlan(ctx, &convdto.GetPlanRequest{ + AgentInstanceId: 42, + Username: "alice", + TurnId: 7, + }) + + require.NoError(t, err) + require.NotNil(t, resp.Data) + require.Nil(t, resp.Data.Plan) + require.Equal(t, convdto.PlanStatus_PLAN_STATUS_NO_PLAN, resp.Data.Status) + require.Empty(t, chatClient.getPlanRequests) +} + +func TestCancelPlanWithoutConversationIDDoesNotGuessAmbiguousTurn(t *testing.T) { + service := newTestConversationService() + chatClient := &recordingPlanChatClient{} + service.chatClient = chatClient + ctx := ctxWithUser("alice") + for range 2 { + conv, err := service.conversationRepo.Create(ctx, &convEntity.Conversation{ + CreatorUsername: "alice", + AgentInstanceID: 42, + }) + require.NoError(t, err) + _, err = service.messageRepo.Create(ctx, &msgEntity.Message{ + ConversationId: conv.ID, + TurnId: 7, + Username: "alice", + AgentInstanceId: 42, + Role: roleUser, + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "hello", + }) + require.NoError(t, err) + } + + _, err := service.CancelPlan(ctx, &convdto.CancelPlanRequest{ + AgentInstanceId: 42, + Username: "alice", + TurnId: 7, + }) + + require.NoError(t, err) + require.Empty(t, chatClient.cancelPlanRequests) +} + // region RPC Message Operations func TestRpcCreateMessage(t *testing.T) { @@ -171,6 +368,254 @@ func TestRpcCreateMessage(t *testing.T) { require.NoError(t, err) require.Equal(t, firstId, resp2.Data.Id) }) + + t.Run("duplicate recovered task runtime message skips creation", func(t *testing.T) { + ctx := context.Background() + recoveryKey := taskRuntimeRecoveryResultPrefix + "batch-1" + + resp1, err := service.RpcCreateMessage(ctx, &rgrpc.CreateMessageRequest{ + Message: &msgEntity.Message{ + ConversationId: 3, + TurnId: 30, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "Task execution finished after recovery.", + FunctionContext: &convdto.FunctionContext{Result: recoveryKey}, + }, + }) + require.NoError(t, err) + firstId := resp1.Data.Id + + resp2, err := service.RpcCreateMessage(ctx, &rgrpc.CreateMessageRequest{ + Message: &msgEntity.Message{ + ConversationId: 3, + TurnId: 30, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "Updated recovered content", + FunctionContext: &convdto.FunctionContext{Result: recoveryKey}, + }, + }) + require.NoError(t, err) + require.Equal(t, firstId, resp2.Data.Id) + }) + + t.Run("duplicate recovered task runtime insert race returns existing", func(t *testing.T) { + ctx := context.Background() + repo := &racingDuplicateMessageRepo{MessageRepo: msgMock.NewMockMessageRepo()} + service := newTestConversationService() + service.messageRepo = repo + + resp, err := service.RpcCreateMessage(ctx, &rgrpc.CreateMessageRequest{ + Message: &msgEntity.Message{ + ConversationId: 4, + TurnId: 40, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "Task execution finished after recovery.", + FunctionContext: &convdto.FunctionContext{ + Result: taskRuntimeRecoveryResultPrefix + "batch-race", + }, + }, + }) + + require.NoError(t, err) + require.Equal(t, int64(99), resp.Data.Id) + require.Equal(t, 1, repo.createCalls) + require.Equal(t, 2, repo.listCalls) + }) + + t.Run("recovered task runtime message skips when assistant final already has artifact", func(t *testing.T) { + ctx := context.Background() + normal, err := service.messageRepo.Create(ctx, &msgEntity.Message{ + ConversationId: 5, + TurnId: 50, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "The task did not run.\n\nExecution summary: "+ + "http://localhost:8080/storage/task-runtime/batch/summary.html", + }) + require.NoError(t, err) + + resp, err := service.RpcCreateMessage(ctx, &rgrpc.CreateMessageRequest{ + Message: &msgEntity.Message{ + ConversationId: 5, + TurnId: 50, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "Task execution finished after recovery.", + FunctionContext: &convdto.FunctionContext{ + Result: taskRuntimeRecoveryResultPrefix + "batch-existing-final", + }, + }, + }) + + require.NoError(t, err) + require.Equal(t, normal.Id, resp.Data.Id) + messages, _, err := service.messageRepo.ListByFilter(ctx, &messagerepo.MessageFilter{ + ConversationId: ptr.Of(int64(5)), + TurnId: ptr.Of(int64(50)), + AgentInstanceId: ptr.Of(int64(42)), + Username: ptr.Of("alice"), + Role: ptr.Of("assistant"), + }) + require.NoError(t, err) + require.Len(t, messages, 1) + }) + + t.Run("recovered task runtime message is kept when assistant text has no artifact", func(t *testing.T) { + ctx := context.Background() + normal, err := service.messageRepo.Create(ctx, &msgEntity.Message{ + ConversationId: 6, + TurnId: 60, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "The task finished, but I could not render the artifact link.", + }) + require.NoError(t, err) + + resp, err := service.RpcCreateMessage(ctx, &rgrpc.CreateMessageRequest{ + Message: &msgEntity.Message{ + ConversationId: 6, + TurnId: 60, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "Task execution finished after recovery.", + FunctionContext: &convdto.FunctionContext{ + Result: taskRuntimeRecoveryResultPrefix + "batch-missing-artifact", + }, + }, + }) + + require.NoError(t, err) + require.NotEqual(t, normal.Id, resp.Data.Id) + messages, _, err := service.messageRepo.ListByFilter(ctx, &messagerepo.MessageFilter{ + ConversationId: ptr.Of(int64(6)), + TurnId: ptr.Of(int64(60)), + AgentInstanceId: ptr.Of(int64(42)), + Username: ptr.Of("alice"), + Role: ptr.Of("assistant"), + }) + require.NoError(t, err) + require.Len(t, messages, 2) + }) + + t.Run("normalized recovered text strips legacy internal marker", func(t *testing.T) { + marker := legacyTaskRuntimeRecoveryMarkerPrefix + "batch-2 -->" + contentType, content, include := NormalizeMessageContent(&msgEntity.Message{ + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Role: "assistant", + Content: "Task execution finished after recovery.\n\n" + marker, + }) + + require.True(t, include) + require.Equal(t, convdto.MessageContentType_MESSAGE_CONTENT_TYPE_MARKDOWN, contentType) + require.Equal(t, "Task execution finished after recovery.", content) + }) + + t.Run("recovered task runtime function context is hidden from listing", func(t *testing.T) { + item := buildMessageItem(&msgEntity.Message{ + ConversationId: 3, + TurnId: 30, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "Task execution finished after recovery.", + FunctionContext: &convdto.FunctionContext{Result: taskRuntimeRecoveryResultPrefix + "batch-3"}, + }) + + require.NotNil(t, item) + require.Equal(t, "Task execution finished after recovery.", item.Content) + require.Empty(t, item.GetFunctionContext().GetResult()) + }) +} + +func TestBuildMessageResponseSuppressesRedundantRecoveryMessage(t *testing.T) { + recovery := &msgEntity.Message{ + ConversationId: 3, + TurnId: 30, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "Task execution finished after recovery.", + FunctionContext: &convdto.FunctionContext{Result: taskRuntimeRecoveryResultPrefix + "batch-3"}, + } + normal := &msgEntity.Message{ + ConversationId: 3, + TurnId: 30, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "The task did not run.\n\nExecution summary: " + + "http://localhost:8080/storage/task-runtime/batch-3/summary.html", + } + + resp := buildMessageResponse([]*msgEntity.Message{recovery, normal}, false) + require.Len(t, resp.Messages, 1) + require.Equal(t, normal.Content, resp.Messages[0].Content) + + plainFinal := &msgEntity.Message{ + ConversationId: 3, + TurnId: 30, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "The task finished, but no artifact URL was available in this message.", + } + plainResp := buildMessageResponse([]*msgEntity.Message{recovery, plainFinal}, false) + require.Len(t, plainResp.Messages, 2) + require.Equal(t, "Task execution finished after recovery.", plainResp.Messages[0].Content) + require.Equal(t, plainFinal.Content, plainResp.Messages[1].Content) + + noLinkFieldText := &msgEntity.Message{ + ConversationId: 3, + TurnId: 30, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "The report_url field was not available in the final digest.", + } + noLinkFieldResp := buildMessageResponse([]*msgEntity.Message{recovery, noLinkFieldText}, false) + require.Len(t, noLinkFieldResp.Messages, 2) + require.Equal(t, "Task execution finished after recovery.", noLinkFieldResp.Messages[0].Content) + require.Equal(t, noLinkFieldText.Content, noLinkFieldResp.Messages[1].Content) + + legacyRecovery := &msgEntity.Message{ + ConversationId: 3, + TurnId: 30, + Username: "alice", + AgentInstanceId: 42, + Role: "assistant", + ContentType: convdto.ChatContentType_CHAT_CONTENT_TYPE_TEXT, + Content: "Task execution finished after recovery.\n\n" + + legacyTaskRuntimeRecoveryMarkerPrefix + "batch-legacy -->", + } + legacyResp := buildMessageResponse([]*msgEntity.Message{legacyRecovery, normal}, false) + require.Len(t, legacyResp.Messages, 1) + require.Equal(t, normal.Content, legacyResp.Messages[0].Content) + + recoveryOnlyResp := buildMessageResponse([]*msgEntity.Message{recovery}, false) + require.Len(t, recoveryOnlyResp.Messages, 1) + require.Equal(t, "Task execution finished after recovery.", recoveryOnlyResp.Messages[0].Content) } func TestRpcListUserMessageByUserAgentTurnID(t *testing.T) { @@ -268,6 +713,9 @@ func TestListMessagesByUserAndAgent(t *testing.T) { require.NoError(t, err) require.NotNil(t, resp.Data) require.Len(t, resp.Data.Messages, 2) + for _, item := range resp.Data.Messages { + require.Equal(t, conv.ID, item.ConversationId) + } }) t.Run("list by turn id", func(t *testing.T) { diff --git a/backend/internal/biz/sandbox/impl/pool.go b/backend/internal/biz/sandbox/impl/pool.go index fd61c26..4386267 100644 --- a/backend/internal/biz/sandbox/impl/pool.go +++ b/backend/internal/biz/sandbox/impl/pool.go @@ -1750,36 +1750,27 @@ redis.call('SET', KEYS[1], updated) return updated `) -// AcquireAvailableLease finds the first sandbox of the given type assigned to -// the instance that is NOT currently in use, atomically marks it as InUse=true, -// and returns it. Returns nil if all assigned sandboxes are in use or none exist. -func (p *Pool) AcquireAvailableLease(ctx context.Context, instanceID, sandboxType string) (*Lease, error) { - return p.acquireAvailableLease(ctx, instanceID, sandboxType, nil) -} - -func (p *Pool) AcquireAvailableLeaseFromResources( - ctx context.Context, instanceID, sandboxType string, availableResources map[string]*Resource, +// AcquireAssignedLease tries, in the given priority order, to acquire one of the +// candidate sandboxes that is assigned to the instance. +// +// candidateSandboxIDs are pre-filtered by the caller to available resources of +// the requested OS (see Service.appliableResourcesForOS) and ordered by +// scheduling priority. This routine intersects them with the instance's +// assignment set and acquires the first that can be leased; the assignment hash +// value (concrete type) is irrelevant here — selection is purely OS-driven +// upstream and identity-driven (assigned to this instance) here. +func (p *Pool) AcquireAssignedLease( + ctx context.Context, instanceID string, candidateSandboxIDs []string, ) (*Lease, error) { - allowedResourceIDs := make(map[string]struct{}, len(availableResources)) - for resourceID, resource := range availableResources { - if resource == nil || resource.Status != ResourceStatusAvailable || resourceID == "" { - continue - } - allowedResourceIDs[resourceID] = struct{}{} + if instanceID == "" || len(candidateSandboxIDs) == 0 { + return nil, nil } - return p.acquireAvailableLease(ctx, instanceID, sandboxType, allowedResourceIDs) -} - -func (p *Pool) acquireAvailableLease( - ctx context.Context, - instanceID, sandboxType string, - allowedResourceIDs map[string]struct{}, -) (*Lease, error) { - if instanceID == "" || sandboxType == "" { - return nil, nil + assignments, err := p.rds.HGetAll(ctx, assignKey(instanceID)).Result() + if err != nil { + return nil, err } - if allowedResourceIDs != nil && len(allowedResourceIDs) == 0 { + if len(assignments) == 0 { return nil, nil } @@ -1789,17 +1780,8 @@ func (p *Pool) acquireAvailableLease( } expectedOwnerPattern := `"User":` + string(expectedOwnerJSON) - aKey := assignKey(instanceID) - assignments, err := p.rds.HGetAll(ctx, aKey).Result() - if err != nil { - return nil, err - } - - for sandboxID, sType := range assignments { - if sType != sandboxType { - continue - } - if !assignmentMatchesAllowedResources(sandboxID, allowedResourceIDs) { + for _, sandboxID := range candidateSandboxIDs { + if _, assigned := assignments[sandboxID]; !assigned { continue } @@ -1815,20 +1797,6 @@ func (p *Pool) acquireAvailableLease( return nil, nil } -func assignmentMatchesAllowedResources(sandboxID string, allowedResourceIDs map[string]struct{}) bool { - if allowedResourceIDs == nil { - return true - } - - parts := strings.SplitN(sandboxID, ":", 2) - if len(parts) != 2 { - return false - } - - _, ok := allowedResourceIDs[parts[1]] - return ok -} - func (p *Pool) tryAcquireAssignedLease(ctx context.Context, sandboxID, expectedOwnerPattern string) (*Lease, error) { resKey := resourceKeyPrefix + sandboxID cdKey := cooldownKeyPrefix + sandboxID @@ -1848,7 +1816,7 @@ func (p *Pool) tryAcquireAssignedLease(ctx context.Context, sandboxID, expectedO var lease Lease if jsonErr := json.Unmarshal([]byte(updatedJSON), &lease); jsonErr != nil { - logger.CtxError(ctx, "AcquireAvailableLease: failed to parse lease JSON: %v", jsonErr) + logger.CtxError(ctx, "AcquireAssignedLease: failed to parse lease JSON: %v", jsonErr) return nil, nil } @@ -1904,7 +1872,7 @@ const resourceSnapshotKeyPrefix = "sandbox:snapshot:resource:" const resourcePendingShrinkKeyPrefix = "sandbox:snapshot:resource:pending-shrink:" const resourceSnapshotLeaderKey = "sandbox:snapshot:resource:leader" const cooldownKeyPrefix = "sandbox:cooldown:" -const releaseCooldown = 5 * time.Second +const releaseCooldown = 3 * time.Second func resourceKey(t, id string) string { return resourceKeyPrefix + t + ":" + id } diff --git a/backend/internal/biz/sandbox/impl/reverse_rpc.go b/backend/internal/biz/sandbox/impl/reverse_rpc.go index 581c2d6..c2e5afc 100644 --- a/backend/internal/biz/sandbox/impl/reverse_rpc.go +++ b/backend/internal/biz/sandbox/impl/reverse_rpc.go @@ -40,15 +40,17 @@ func (s *Service) RpcApplySandbox( return &sandboxRgrpc.ApplySandboxResponse{Code: 1, Msg: "instanceId is required"}, nil } - sandboxType := strings.TrimSpace(req.GetType()) - if sandboxType == "" { + sandboxOS := strings.TrimSpace(req.GetType()) + if sandboxOS == "" { return &sandboxRgrpc.ApplySandboxResponse{Code: 1, Msg: "type is required"}, nil } - if !enum.IsValidSandboxType(sandboxType) { - return &sandboxRgrpc.ApplySandboxResponse{Code: 1, Msg: "invalid sandbox type: " + sandboxType}, nil + // Scheduling is OS-only: the type field carries an OS selector (e.g. + // "windows") and ApplySandbox resolves it to a concrete pool. + if !enum.IsOSSelector(sandboxOS) { + return &sandboxRgrpc.ApplySandboxResponse{Code: 1, Msg: "invalid sandbox os: " + sandboxOS}, nil } - appliedSandbox, err := s.ApplySandbox(ctx, instanceID, sandboxType) + appliedSandbox, err := s.ApplySandbox(ctx, instanceID, sandboxOS) if err != nil { return &sandboxRgrpc.ApplySandboxResponse{Code: 1, Msg: err.Error()}, nil } @@ -56,7 +58,7 @@ func (s *Service) RpcApplySandbox( applied, appliedSandboxID := getApplyOutcome(appliedSandbox) msg := "success" if !applied { - msg = "no available sandbox for requested type" + msg = "no available sandbox for requested os" } providerBaseURL, deviceID := getApplyMetadata(appliedSandbox) diff --git a/backend/internal/biz/sandbox/impl/reverse_rpc_test.go b/backend/internal/biz/sandbox/impl/reverse_rpc_test.go index c505e25..71d37c8 100644 --- a/backend/internal/biz/sandbox/impl/reverse_rpc_test.go +++ b/backend/internal/biz/sandbox/impl/reverse_rpc_test.go @@ -54,7 +54,7 @@ func TestRpcApplySandboxReturnsVNCURL(t *testing.T) { resp, err := svc.RpcApplySandbox(ctx, &sandboxRgrpc.ApplySandboxRequest{ InstanceId: lease.User, - Type: lease.Type, + Type: osForLease(lease), }) require.NoError(t, err) require.Equal(t, int32(0), resp.GetCode()) diff --git a/backend/internal/biz/sandbox/impl/service.go b/backend/internal/biz/sandbox/impl/service.go index 8d593fd..e0afff0 100644 --- a/backend/internal/biz/sandbox/impl/service.go +++ b/backend/internal/biz/sandbox/impl/service.go @@ -66,38 +66,42 @@ func NewService(pool *Pool, instanceRepo agentrepo.SingleAgentInstanceRepository // ==================== New Simplified APIs ==================== -// ApplySandbox picks an available (InUse=false) sandbox of the requested type -// from the pool pre-assigned to this instance, marks it InUse=true, and returns it. +// ApplySandbox picks an available (InUse=false) sandbox supplying the requested +// OS from the pool pre-assigned to this instance, marks it InUse=true, and +// returns it. // If all assigned sandboxes are in use or none are assigned, returns an informational response. -func (s *Service) ApplySandbox(ctx context.Context, instanceID, sandboxType string) (map[string]interface{}, error) { - if instanceID == "" || sandboxType == "" { - return nil, apperr.New(errcode.CommonInvalidParam, "instanceID and sandboxType are required") +func (s *Service) ApplySandbox(ctx context.Context, instanceID, sandboxOS string) (map[string]interface{}, error) { + if instanceID == "" || sandboxOS == "" { + return nil, apperr.New(errcode.CommonInvalidParam, "instanceID and sandbox os are required") } - appliableResourcesByID, snapshotAge, err := s.listAppliableResources(ctx, sandboxType) + os, err := resolveSandboxOS(sandboxOS) if err != nil { return nil, err } - logger.CtxInfo( - ctx, - "ApplySandbox: using shared sandbox snapshot for type=%s age=%s", - sandboxType, snapshotAge.Round(time.Millisecond), - ) - lease, err := s.Pool.AcquireAvailableLeaseFromResources( - ctx, instanceID, sandboxType, appliableResourcesByID, - ) + // Selection is OS-only: gather every available resource that can supply the + // OS, across all enabled providers, in scheduling-priority order (managed + // pools before a person's physical machine). + candidateIDs, resourcesByID, snapshotAge, err := s.appliableResourcesForOS(ctx, os) if err != nil { return nil, err } + if len(candidateIDs) == 0 { + logger.CtxInfo(ctx, "No available sandbox for os %s and instance %s (all in use or none assigned)", + os, instanceID) + return nil, nil + } + logger.CtxInfo(ctx, "ApplySandbox: os=%s candidates=%d age=%s", + os, len(candidateIDs), snapshotAge.Round(time.Millisecond)) + lease, err := s.Pool.AcquireAssignedLease(ctx, instanceID, candidateIDs) + if err != nil { + return nil, err + } if lease == nil { - // No available sandbox found - return empty result (not an error) - logger.CtxInfo( - ctx, - "No available sandbox of type %s for instance %s (all in use or none assigned)", - sandboxType, instanceID, - ) + logger.CtxInfo(ctx, "No available sandbox for os %s and instance %s (all in use or none assigned)", + os, instanceID) return nil, nil } if strings.TrimSpace(lease.User) != instanceID { @@ -109,7 +113,7 @@ func (s *Service) ApplySandbox(ctx context.Context, instanceID, sandboxType stri return nil, apperr.New(errcode.CommonConflict, "sandbox assignment owner changed, please retry") } - if resource := appliableResourcesByID[lease.ResourceID]; resource != nil { + if resource := resourcesByID[lease.SandboxID]; resource != nil { s.mergeLeaseMetadata(ctx, lease, resource.Metadata) } @@ -131,14 +135,14 @@ func (s *Service) ApplySandbox(ctx context.Context, instanceID, sandboxType stri } logger.CtxInfo(ctx, "Sandbox applied for instance %s: sandbox_id=%s, type=%s, endpoint=%s", - instanceID, lease.SandboxID, sandboxType, endpoint) + instanceID, lease.SandboxID, lease.Type, endpoint) return result, nil } -// ReleaseSandbox resets the sandbox first while the lease is still in-use, -// then marks it as no longer in use with a cooldown period. -// If reset fails, the lease stays in-use so the dirty sandbox cannot be reused. +// ReleaseSandbox marks a sandbox as no longer in use with a cooldown period. +// The task runtime resets sandboxes on acquire, so release intentionally avoids +// provider reset work to prevent back-to-back release/acquire reset throttling. func (s *Service) ReleaseSandbox(ctx context.Context, instanceID, sandboxID string) error { if instanceID == "" || sandboxID == "" { return apperr.New(errcode.CommonInvalidParam, "instanceID and sandboxID are required") @@ -163,18 +167,6 @@ func (s *Service) ReleaseSandbox(ctx context.Context, instanceID, sandboxID stri return nil } - prov, ok := s.Pool.GetProvider(lease.Type) - if !ok || prov == nil { - return apperr.New( - errcode.SandboxProviderUnavailable, - fmt.Sprintf("sandbox provider unavailable for type %s", lease.Type), - ) - } - if resetErr := prov.ResetResource(ctx, lease.ResourceID); resetErr != nil { - logger.CtxWarn(ctx, "Sandbox reset failed during release: sandbox_id=%s, err=%v", sandboxID, resetErr) - return apperr.New(errcode.SandboxResetFailed, fmt.Sprintf("failed to reset sandbox %s: %v", sandboxID, resetErr)) - } - lease, err = s.Pool.ReleaseLease(ctx, instanceID, sandboxID) if err != nil { return err @@ -241,13 +233,6 @@ func resourcesByIDWithStatus(resources []*Resource, status ResourceStatus) map[s return filtered } -// appliableResources returns resources that an existing owner may use. -// During the missing-resource grace period we still allow apply against a -// resource that remains logically assigned to the same instance. -func appliableResources(resources []*Resource) map[string]*Resource { - return resourcesByIDWithStatus(resources, ResourceStatusAvailable) -} - // allocatableResources returns resources eligible for new assignment. // Resources in the grace period (MissingSinceAt != nil) are excluded even // when their snapshot status is still available — they are kept visible in @@ -294,15 +279,6 @@ func (s *Service) listSnapshotResources( return resources, age, nil } -func (s *Service) listAppliableResources(ctx context.Context, sandboxType string) (map[string]*Resource, time.Duration, error) { - resources, age, err := s.listSnapshotResources(ctx, sandboxType) - if err != nil { - return nil, age, err - } - - return appliableResources(resources), age, nil -} - func (s *Service) listAllocatableResources(ctx context.Context, sandboxType string) (map[string]*Resource, time.Duration, error) { resources, age, err := s.listSnapshotResources(ctx, sandboxType) if err != nil { @@ -1307,18 +1283,26 @@ func (s *Service) unassignRetryAfterRaceRelease( } // GetInstanceSandboxesWithStatus returns all sandboxes for an instance with type, status, and endpoints. -// If typeFilter is non-empty, only sandboxes of that type are returned. +// osFilter, when non-empty, is an OS selector (e.g. "windows"): only leases whose +// resolved OS matches are returned. Selection is OS-only — concrete sandbox types +// are an internal detail and never a filter here. func (s *Service) GetInstanceSandboxesWithStatus( - ctx context.Context, instanceID, typeFilter string, + ctx context.Context, instanceID, osFilter string, ) ([]map[string]interface{}, error) { instanceID = strings.TrimSpace(instanceID) if instanceID == "" { return nil, apperr.New(errcode.CommonInvalidParam, "instanceID is required") } - typeFilter = strings.TrimSpace(typeFilter) - if typeFilter != "" && !enum.IsValidSandboxType(typeFilter) { - return nil, apperr.New(errcode.CommonInvalidParam, "invalid sandbox type: "+typeFilter) + osFilter = strings.TrimSpace(osFilter) + var os enum.SandboxOS + hasFilter := false + if osFilter != "" { + parsed, err := resolveSandboxOS(osFilter) + if err != nil { + return nil, err + } + os, hasFilter = parsed, true } allLeases, err := s.loadAssignedLeasesStrict(ctx, instanceID) @@ -1326,9 +1310,17 @@ func (s *Service) GetInstanceSandboxesWithStatus( return nil, err } + // An OS filter matches physical leases on their metadata["os"], so refresh + // from the live snapshot before filtering — otherwise a lease whose stored + // metadata is stale could be wrongly excluded, diverging from what + // ApplySandbox (which always reads the fresh snapshot) would select. + if hasFilter { + s.refreshLeaseMetadata(ctx, allLeases...) + } + filteredLeases := make([]*Lease, 0, len(allLeases)) for _, lease := range allLeases { - if typeFilter != "" && lease.Type != typeFilter { + if hasFilter && !leaseMatchesOS(lease, os) { continue } filteredLeases = append(filteredLeases, lease) @@ -1337,7 +1329,11 @@ func (s *Service) GetInstanceSandboxesWithStatus( return []map[string]interface{}{}, nil } - s.refreshLeaseMetadata(ctx, filteredLeases...) + // With a filter we already refreshed every lease above; otherwise refresh the + // surviving subset here. + if !hasFilter { + s.refreshLeaseMetadata(ctx, filteredLeases...) + } resourceStatusByID, displayNames, err := s.loadResourceStatusAndNames(ctx, instanceID) if err != nil { diff --git a/backend/internal/biz/sandbox/impl/service_list_resources_test.go b/backend/internal/biz/sandbox/impl/service_list_resources_test.go index ec3ca4f..ce0a274 100644 --- a/backend/internal/biz/sandbox/impl/service_list_resources_test.go +++ b/backend/internal/biz/sandbox/impl/service_list_resources_test.go @@ -1,4 +1,4 @@ -// Copyright (c) 2026 Sico Authors +// Copyright (c) 2026 Sico Authors // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal @@ -34,9 +34,7 @@ import ( "github.com/redis/go-redis/v9" "github.com/stretchr/testify/require" - "sico-backend/internal/shared/apperr" "sico-backend/internal/shared/enum" - "sico-backend/internal/shared/errcode" ) type fakeProvider struct { @@ -132,12 +130,12 @@ func newTestPoolWithProviders(rds *redis.Client, missingHideAfter time.Duration, } return &Pool{ - providers: providerMap, - rds: rds, - refreshInterval: 15 * time.Second, - providerFailureCount: make(map[string]int), - providerLastSuccessAt: make(map[string]time.Time), - missingLeaseMarkAfter: 30 * time.Second, + providers: providerMap, + rds: rds, + refreshInterval: 15 * time.Second, + providerFailureCount: make(map[string]int), + providerLastSuccessAt: make(map[string]time.Time), + missingLeaseMarkAfter: 30 * time.Second, missingLeaseHideAfter: missingHideAfter, missingLeaseDeleteAfter: 24 * time.Hour, instanceID: newPoolInstanceID(), @@ -224,6 +222,13 @@ func testEmulatorResource(status ResourceStatus) *Resource { } } +// osForLease returns the OS a lease supplies, for use as an OS scheduling +// selector. Apply/list speak OS only, so tests pass this rather than lease.Type. +func osForLease(lease *Lease) string { + os, _ := enum.ResolveResourceOS(lease.Type, lease.Metadata) + return os.String() +} + func TestListAllResourcesDoesNotIncludeLeaseWithoutSnapshot(t *testing.T) { t.Parallel() @@ -370,7 +375,7 @@ func TestTransientProviderEmptyResponsePreservesAvailableDuringGrace(t *testing. Metadata: map[string]string{"adbAddress": "74.179.80.110:16704"}, } - // First refresh: provider returns both resources → snapshot contains 2 available + // First refresh: provider returns both resources → snapshot contains 2 available provider := &fakeProvider{ providerType: enum.SandboxTypeEmulator.String(), responses: [][]*Resource{ @@ -392,7 +397,7 @@ func TestTransientProviderEmptyResponsePreservesAvailableDuringGrace(t *testing. require.Equal(t, true, emulatorResources[0]["allocatable"]) require.Equal(t, true, emulatorResources[1]["allocatable"]) - // Second refresh: provider returns empty once → keep last accepted snapshot untouched. + // Second refresh: provider returns empty once → keep last accepted snapshot untouched. require.NoError(t, pool.refreshResources(ctx)) result, err = svc.ListAllResources(ctx) @@ -406,7 +411,7 @@ func TestTransientProviderEmptyResponsePreservesAvailableDuringGrace(t *testing. "resource %s should not be allocatable for new assignment while shrink is pending", r["resource_id"]) } - // Third refresh: provider returns both → resources still available, missingSinceAt cleared + // Third refresh: provider returns both → resources still available, missingSinceAt cleared require.NoError(t, pool.refreshResources(ctx)) result, err = svc.ListAllResources(ctx) @@ -534,7 +539,7 @@ func TestAssignSandboxRejectsGracePeriodResource(t *testing.T) { // Refresh 1: resource is available. // Refresh 2: provider returns empty (resource enters grace period). - // AssignSandbox should fail — grace-period resources are visible but not allocatable. + // AssignSandbox should fail — grace-period resources are visible but not allocatable. provider := &fakeProvider{ providerType: enum.SandboxTypeEmulator.String(), responses: [][]*Resource{ @@ -591,7 +596,7 @@ func TestApplySandboxAllowsGracePeriodAssignedResource(t *testing.T) { require.NoError(t, pool.refreshResources(ctx)) svc := &Service{Pool: pool} - result, err := svc.ApplySandbox(ctx, lease.User, lease.Type) + result, err := svc.ApplySandbox(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.NotNil(t, result) require.Equal(t, lease.SandboxID, result["sandbox_id"]) @@ -690,7 +695,7 @@ func TestRefreshResourcesDoesNotConfirmShrinkAcrossProviderError(t *testing.T) { require.ErrorIs(t, getErr, redis.Nil) // Provider error clears pending shrink but snapshot still has MissingSinceAt - // from the pending delay — resource remains non-allocatable (safe behavior). + // from the pending delay — resource remains non-allocatable (safe behavior). result, err = svc.ListAllResources(ctx) require.NoError(t, err) emulatorResources = result[enum.SandboxTypeEmulator.String()].([]map[string]interface{}) @@ -750,8 +755,8 @@ func TestRefreshResourcesRequiresSameShrinkCandidateToConfirm(t *testing.T) { emulatorResources := result[enum.SandboxTypeEmulator.String()].([]map[string]interface{}) require.Len(t, emulatorResources, 2) - // After refresh 3: candidate changed (B→A missing), pending shrink resets. - // A is missing → allocatable=false; B recovered → allocatable=true. + // After refresh 3: candidate changed (B→A missing), pending shrink resets. + // A is missing → allocatable=false; B recovered → allocatable=true. allocatableByID := map[string]bool{} for _, resource := range emulatorResources { allocatableByID[resource["resource_id"].(string)] = resource["allocatable"].(bool) @@ -837,7 +842,7 @@ func TestRefreshResourcesReleasesLeadershipWhenAllProvidersFail(t *testing.T) { leaderPool.instanceID = "leader-pod" followerPool.instanceID = "follower-pod" - // Leader has no previous snapshots and all providers fail → error + release. + // Leader has no previous snapshots and all providers fail → error + release. err := leaderPool.refreshResources(ctx) require.Error(t, err) require.Equal(t, 1, leaderEmulator.calls) @@ -1002,7 +1007,7 @@ func TestListAllResourcesHidesUnavailableResourceAfterGracePeriod(t *testing.T) require.NoError(t, err) require.Len(t, listResult[enum.SandboxTypeEmulator.String()].([]map[string]interface{}), 0) - instanceResult, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, lease.Type) + instanceResult, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.Len(t, instanceResult, 1) require.Equal(t, string(ResourceStatusUnavailable), instanceResult[0]["status"]) @@ -1049,7 +1054,7 @@ func TestRefreshResourcesHidesExplicitlyUnhealthyResourceAfterGracePeriod(t *tes require.NoError(t, err) require.Len(t, listResult[enum.SandboxTypeEmulator.String()].([]map[string]interface{}), 0) - instanceResult, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, lease.Type) + instanceResult, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.Len(t, instanceResult, 1) require.Equal(t, string(ResourceStatusUnavailable), instanceResult[0]["status"]) @@ -1075,7 +1080,7 @@ func TestApplySandboxSkipsUnavailableSnapshotResource(t *testing.T) { } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - result, err := svc.ApplySandbox(ctx, lease.User, lease.Type) + result, err := svc.ApplySandbox(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.Nil(t, result) require.Equal(t, 0, provider.calls) @@ -1105,7 +1110,7 @@ func TestApplySandboxUsesFreshSnapshotWithoutProviderCall(t *testing.T) { } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - result, err := svc.ApplySandbox(ctx, lease.User, lease.Type) + result, err := svc.ApplySandbox(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.NotNil(t, result) require.Equal(t, lease.SandboxID, result["sandbox_id"]) @@ -1139,7 +1144,7 @@ func TestApplySandboxUsesStaleSnapshotWithoutProviderCall(t *testing.T) { } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - result, err := svc.ApplySandbox(ctx, lease.User, lease.Type) + result, err := svc.ApplySandbox(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.NotNil(t, result) require.Equal(t, 0, provider.calls) @@ -1168,7 +1173,7 @@ func TestApplySandboxFailsWhenSnapshotUnavailable(t *testing.T) { } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - result, err := svc.ApplySandbox(ctx, lease.User, lease.Type) + result, err := svc.ApplySandbox(ctx, lease.User, osForLease(lease)) require.Nil(t, result) require.Error(t, err) require.ErrorContains(t, err, "snapshot unavailable") @@ -1311,7 +1316,7 @@ func TestUnassignSandboxReleasesInUseSandboxBeforeDeleting(t *testing.T) { err := svc.UnassignSandbox(ctx, lease.User, lease.SandboxID) require.NoError(t, err) - require.Equal(t, 1, provider.resetCalls) + require.Equal(t, 0, provider.resetCalls) _, getErr := rds.Get(ctx, resourceKeyPrefix+lease.SandboxID).Result() require.ErrorIs(t, getErr, redis.Nil) @@ -1347,7 +1352,7 @@ func TestApplySandboxUsesStaleSnapshotWhenProviderWouldFail(t *testing.T) { } svc := &Service{Pool: newTestPoolWithProviders(rds, time.Minute, emulatorProvider)} - result, err := svc.ApplySandbox(ctx, lease.User, lease.Type) + result, err := svc.ApplySandbox(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.NotNil(t, result) require.Equal(t, 0, emulatorProvider.calls) @@ -1422,7 +1427,7 @@ func TestGetInstanceSandboxesWithStatusKeepsProviderMissingResourceAssignedDurin require.NoError(t, pool.refreshResources(ctx)) svc := &Service{Pool: pool} - result, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, lease.Type) + result, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.Len(t, result, 1) require.Equal(t, string(ResourceStatusAssigned), result[0]["status"]) @@ -1447,7 +1452,7 @@ func TestGetInstanceSandboxesWithStatusReturnsErrorWhenSnapshotUnavailable(t *te } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - result, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, lease.Type) + result, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, osForLease(lease)) require.Nil(t, result) require.Error(t, err) require.ErrorContains(t, err, "failed to load sandbox status") @@ -1475,7 +1480,7 @@ func TestGetInstanceSandboxesWithStatusReturnsErrorWhenAssignedLeaseIsUnreadable } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - result, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, lease.Type) + result, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, osForLease(lease)) require.Nil(t, result) require.Error(t, err) require.ErrorContains(t, err, "invalid character") @@ -1498,7 +1503,7 @@ func TestGetInstanceSandboxesWithStatusReturnsEmptyWithoutAssignments(t *testing } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - result, err := svc.GetInstanceSandboxesWithStatus(ctx, "123", enum.SandboxTypeEmulator.String()) + result, err := svc.GetInstanceSandboxesWithStatus(ctx, "123", enum.SandboxOSAndroid.String()) require.NoError(t, err) require.Empty(t, result) } @@ -1567,7 +1572,7 @@ func TestMissingAssignmentIsHiddenFromDashboardButStillAssignedBeforeDeleteWindo require.NoError(t, err) require.Len(t, listResult[enum.SandboxTypeEmulator.String()].([]map[string]interface{}), 0) - instanceResult, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, lease.Type) + instanceResult, err := svc.GetInstanceSandboxesWithStatus(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.Len(t, instanceResult, 1) require.Equal(t, string(ResourceStatusUnavailable), instanceResult[0]["status"]) @@ -1636,7 +1641,7 @@ func TestResolveResourceByHashFallsBackToLeaseMetadataWithoutSnapshot(t *testing require.Equal(t, 0, provider.calls) } -func TestReleaseSandboxKeepsLeaseInUseUntilResetCompletes(t *testing.T) { +func TestReleaseSandboxDoesNotResetBeforeReleasingLease(t *testing.T) { t.Parallel() ctx := context.Background() @@ -1654,49 +1659,26 @@ func TestReleaseSandboxKeepsLeaseInUseUntilResetCompletes(t *testing.T) { time.Now(), testEmulatorResource(ResourceStatusAvailable), ) - resetStarted := make(chan struct{}) - allowReset := make(chan struct{}) provider := &fakeProvider{ providerType: enum.SandboxTypeEmulator.String(), resetFn: func(context.Context, string) error { - select { - case <-resetStarted: - default: - close(resetStarted) - } - <-allowReset - return nil + return errors.New("reset should not be called") }, } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - errCh := make(chan error, 1) - go func() { - errCh <- svc.ReleaseSandbox(ctx, lease.User, lease.SandboxID) - }() - - <-resetStarted - - applied, err := svc.ApplySandbox(ctx, lease.User, lease.Type) - require.NoError(t, err) - require.Nil(t, applied) + require.NoError(t, svc.ReleaseSandbox(ctx, lease.User, lease.SandboxID)) storedLease := loadLease(t, ctx, rds, lease.SandboxID) - require.True(t, storedLease.InUse) - - close(allowReset) - require.NoError(t, <-errCh) - - storedLease = loadLease(t, ctx, rds, lease.SandboxID) require.False(t, storedLease.InUse) cooldownExists, err := rds.Exists(ctx, cooldownKeyPrefix+lease.SandboxID).Result() require.NoError(t, err) require.Equal(t, int64(1), cooldownExists) - require.Equal(t, 1, provider.resetCalls) + require.Equal(t, 0, provider.resetCalls) } -func TestReleaseSandboxReturnsResetFailedAndKeepsLeaseInUse(t *testing.T) { +func TestReleaseSandboxDoesNotRequireConfiguredProvider(t *testing.T) { t.Parallel() ctx := context.Background() @@ -1710,27 +1692,16 @@ func TestReleaseSandboxReturnsResetFailedAndKeepsLeaseInUse(t *testing.T) { lease.InUse = true seedLease(t, ctx, rds, lease) - provider := &fakeProvider{ - providerType: enum.SandboxTypeEmulator.String(), - resetFn: func(context.Context, string) error { - return errors.New("reset boom") - }, - } - svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} + svc := &Service{Pool: newTestPool(rds, nil, time.Minute)} - err := svc.ReleaseSandbox(ctx, lease.User, lease.SandboxID) - require.Error(t, err) - ae, ok := apperr.As(err) - require.True(t, ok) - require.Equal(t, errcode.SandboxResetFailed, ae.Code()) + require.NoError(t, svc.ReleaseSandbox(ctx, lease.User, lease.SandboxID)) storedLease := loadLease(t, ctx, rds, lease.SandboxID) - require.True(t, storedLease.InUse) + require.False(t, storedLease.InUse) cooldownExists, cdErr := rds.Exists(ctx, cooldownKeyPrefix+lease.SandboxID).Result() require.NoError(t, cdErr) - require.Equal(t, int64(0), cooldownExists) - require.Equal(t, 1, provider.resetCalls) + require.Equal(t, int64(1), cooldownExists) } func TestRefreshResourcesClearsMissingStateWhenProviderRecovers(t *testing.T) { @@ -1843,7 +1814,7 @@ func TestApplySandboxDoesNotAcquireLeaseOwnedByAnotherInstance(t *testing.T) { } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - result, err := svc.ApplySandbox(ctx, "123", lease.Type) + result, err := svc.ApplySandbox(ctx, "123", osForLease(lease)) require.NoError(t, err) require.Nil(t, result) @@ -1996,7 +1967,7 @@ func TestApplySandboxReturnsNilWhenSnapshotIsStaleAndDegraded(t *testing.T) { } svc := &Service{Pool: newTestPool(rds, provider, time.Minute)} - result, err := svc.ApplySandbox(ctx, lease.User, lease.Type) + result, err := svc.ApplySandbox(ctx, lease.User, osForLease(lease)) require.NoError(t, err) require.Nil(t, result, "no sandbox should be available when snapshot is stale and degraded") require.Equal(t, 0, provider.calls) @@ -2066,7 +2037,7 @@ func TestStaleSnapshotDegradationIsPerType(t *testing.T) { require.NoError(t, rds.Close()) }) - // emulator snapshot is stale (>60s) → should degrade to unhealthy + // emulator snapshot is stale (>60s) → should degrade to unhealthy seedSnapshot( t, ctx, rds, enum.SandboxTypeEmulator.String(), time.Now().Add(-65*time.Second), testEmulatorResource(ResourceStatusAvailable), @@ -2123,7 +2094,7 @@ func TestPendingShrinkMergesCurrentResourceState(t *testing.T) { // First refresh: both resources available. require.NoError(t, pool.refreshResources(ctx)) - // Second refresh: A becomes unhealthy, B disappears → pending shrink. + // Second refresh: A becomes unhealthy, B disappears → pending shrink. require.NoError(t, pool.refreshResources(ctx)) result, err := svc.ListAllResources(ctx) @@ -2145,7 +2116,7 @@ func TestPendingShrinkMergesCurrentResourceState(t *testing.T) { require.Equal(t, false, allocatableByID[resourceA.ResourceID], "unhealthy resource should not be allocatable") - // B: missing, should have MissingSinceAt → not allocatable. + // B: missing, should have MissingSinceAt → not allocatable. require.Equal(t, string(ResourceStatusAvailable), statusByID[resourceB.ResourceID], "missing resource keeps previous status during pending shrink") require.Equal(t, false, allocatableByID[resourceB.ResourceID], diff --git a/backend/internal/biz/sandbox/impl/service_os_routing.go b/backend/internal/biz/sandbox/impl/service_os_routing.go new file mode 100644 index 0000000..a6e476e --- /dev/null +++ b/backend/internal/biz/sandbox/impl/service_os_routing.go @@ -0,0 +1,107 @@ +// Copyright (c) 2026 Sico Authors +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// The above copyright notice and this permission notice shall be included in +// all copies or substantial portions of the Software. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +package impl + +import ( + "context" + "fmt" + "time" + + "sico-backend/internal/shared/apperr" + "sico-backend/internal/shared/enum" + "sico-backend/internal/shared/errcode" +) + +// resolveSandboxOS parses a scheduling selector into the OS capability it names. +// +// Scheduling (apply / acquire / instance listing) speaks OS only: a task asks +// for an OS (e.g. "android") and the scheduler matches whatever concrete sandbox +// can supply it. Concrete sandbox types stay an internal detail of providers, +// snapshots, leases and resource-proxy paths — never a scheduling input. An +// unrecognized selector is an invalid-param error so a typo fails fast. +func resolveSandboxOS(selector string) (enum.SandboxOS, error) { + if os, ok := enum.ParseSandboxOS(selector); ok { + return os, nil + } + + return "", apperr.New(errcode.CommonInvalidParam, "invalid sandbox os: "+selector) +} + +// leaseMatchesOS reports whether a lease supplies the given OS. A lease's OS is +// resolved from its concrete type (fixed-OS types) or, for physical devices, +// from its metadata["os"] — see enum.ResolveResourceOS. +func leaseMatchesOS(lease *Lease, os enum.SandboxOS) bool { + if lease == nil { + return false + } + + resolved, ok := enum.ResolveResourceOS(lease.Type, lease.Metadata) + return ok && resolved == os +} + +// appliableResourcesForOS collects the currently-available resources that can +// supply os, across every enabled provider, and returns them as: +// - ordered: candidate sandboxIDs ("{type}:{resourceID}") in scheduling +// priority order (fixed-OS types first, then physical) so apply prefers a +// disposable managed pool over a person's real machine; +// - byID: the same resources keyed by sandboxID, for metadata merging. +// +// The OS filter is the single source of selection: a fixed-OS type matches by +// its type, a physical device by its metadata["os"]. Providers disabled in this +// deployment have no snapshot and contribute nothing — no special casing needed. +func (s *Service) appliableResourcesForOS( + ctx context.Context, os enum.SandboxOS, +) ([]string, map[string]*Resource, time.Duration, error) { + resources, age, ok, err := s.Pool.loadSnapshotResources(ctx, "") + if err != nil { + return nil, nil, age, apperr.New(errcode.SandboxProviderUnavailable, + fmt.Sprintf("failed to load sandbox resources: %v", err)) + } + if !ok { + return nil, nil, age, apperr.New(errcode.SandboxProviderUnavailable, + "sandbox resource snapshot unavailable") + } + + byID := make(map[string]*Resource, len(resources)) + byType := make(map[string][]string) + for _, resource := range resources { + if resource == nil || resource.ResourceID == "" { + continue + } + if resource.Status != ResourceStatusAvailable { + continue + } + resolved, matched := enum.ResolveResourceOS(resource.Type, resource.Metadata) + if !matched || resolved != os { + continue + } + sandboxID := resource.Type + ":" + resource.ResourceID + byID[sandboxID] = resource + byType[resource.Type] = append(byType[resource.Type], sandboxID) + } + + var ordered []string + for _, t := range enum.EligibleTypesForOS(os) { + ordered = append(ordered, byType[t]...) + } + + return ordered, byID, age, nil +} diff --git a/backend/internal/biz/sandbox/impl/service_os_routing_test.go b/backend/internal/biz/sandbox/impl/service_os_routing_test.go new file mode 100644 index 0000000..2a2e38a --- /dev/null +++ b/backend/internal/biz/sandbox/impl/service_os_routing_test.go @@ -0,0 +1,124 @@ +// Copyright (c) 2026 Sico Authors +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// The above copyright notice and this permission notice shall be included in +// all copies or substantial portions of the Software. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +package impl + +import ( + "context" + "testing" + "time" + + "github.com/alicebob/miniredis/v2" + "github.com/redis/go-redis/v9" + "github.com/stretchr/testify/require" + + "sico-backend/internal/shared/enum" +) + +func TestResolveSandboxOS(t *testing.T) { + // An OS selector parses to its capability. + os, err := resolveSandboxOS("android") + require.NoError(t, err) + require.Equal(t, enum.SandboxOSAndroid, os) + + // A concrete sandbox type is no longer a valid scheduling selector. + _, err = resolveSandboxOS(enum.SandboxTypeEmulator.String()) + require.Error(t, err) + + // A typo fails fast. + _, err = resolveSandboxOS("bogus") + require.Error(t, err) +} + +func TestLeaseMatchesOS(t *testing.T) { + // A fixed-OS type resolves by its type, regardless of metadata. + require.True(t, leaseMatchesOS(&Lease{Type: enum.SandboxTypeEmulator.String()}, enum.SandboxOSAndroid)) + + // An unknown type never matches. + require.False(t, leaseMatchesOS(&Lease{Type: "bogus"}, enum.SandboxOSAndroid)) + + // Nil lease never matches. + require.False(t, leaseMatchesOS(nil, enum.SandboxOSAndroid)) +} + +func TestAppliableResourcesForOSOrdersManagedBeforePhysical(t *testing.T) { + t.Parallel() + + ctx := context.Background() + mr := miniredis.RunT(t) + rds := redis.NewClient(&redis.Options{Addr: mr.Addr()}) + t.Cleanup(func() { + require.NoError(t, rds.Close()) + }) + + seedSnapshot(t, ctx, rds, enum.SandboxTypeEmulator.String(), time.Now(), testEmulatorResource(ResourceStatusAvailable)) + + svc := &Service{Pool: newTestPool(rds, &fakeProvider{ + providerType: enum.SandboxTypeEmulator.String(), + }, time.Minute)} + + ordered, byID, _, err := svc.appliableResourcesForOS(ctx, enum.SandboxOSAndroid) + require.NoError(t, err) + + require.Equal(t, []string{ + enum.SandboxTypeEmulator.String() + ":" + testEmulatorResource(ResourceStatusAvailable).ResourceID, + }, ordered) + require.Contains(t, byID, enum.SandboxTypeEmulator.String()+":"+testEmulatorResource(ResourceStatusAvailable).ResourceID) +} + +func TestApplySandboxReturnsNilWhenNoResourceSuppliesOS(t *testing.T) { + t.Parallel() + + ctx := context.Background() + mr := miniredis.RunT(t) + rds := redis.NewClient(&redis.Options{Addr: mr.Addr()}) + t.Cleanup(func() { + require.NoError(t, rds.Close()) + }) + + // The emulator pool is enabled but its snapshot is empty. + seedSnapshot(t, ctx, rds, enum.SandboxTypeEmulator.String(), time.Now()) + svc := &Service{Pool: newTestPool(rds, &fakeProvider{ + providerType: enum.SandboxTypeEmulator.String(), + }, time.Minute)} + + result, err := svc.ApplySandbox(ctx, "instance-1", enum.SandboxOSAndroid.String()) + require.NoError(t, err) + require.Nil(t, result) +} + +func TestApplySandboxRejectsNonOSSelector(t *testing.T) { + t.Parallel() + + ctx := context.Background() + mr := miniredis.RunT(t) + rds := redis.NewClient(&redis.Options{Addr: mr.Addr()}) + t.Cleanup(func() { + require.NoError(t, rds.Close()) + }) + + svc := &Service{Pool: newTestPool(rds, &fakeProvider{ + providerType: enum.SandboxTypeEmulator.String(), + }, time.Minute)} + + // A concrete sandbox type is no longer accepted by apply. + _, err := svc.ApplySandbox(ctx, "instance-1", enum.SandboxTypeEmulator.String()) + require.Error(t, err) +} diff --git a/backend/internal/biz/sandbox/isandbox.go b/backend/internal/biz/sandbox/isandbox.go index f4423d0..4d159c2 100644 --- a/backend/internal/biz/sandbox/isandbox.go +++ b/backend/internal/biz/sandbox/isandbox.go @@ -31,9 +31,10 @@ import ( type Service interface { // ==================== Client APIs (require X-Sico-* auth) ==================== - // ApplySandbox returns an available pre-assigned sandbox for the instance by type. - // Marks the sandbox as in-use. Returns empty result if none available. - ApplySandbox(ctx context.Context, instanceID, sandboxType string) (map[string]interface{}, error) + // ApplySandbox returns an available pre-assigned sandbox supplying the + // requested OS for the instance. Marks the sandbox as in-use. Returns empty + // result if none available. + ApplySandbox(ctx context.Context, instanceID, sandboxOS string) (map[string]interface{}, error) // ReleaseSandbox marks a sandbox as no longer in use so it can be re-acquired. ReleaseSandbox(ctx context.Context, instanceID, sandboxID string) error @@ -66,8 +67,8 @@ type Service interface { UnassignSandbox(ctx context.Context, instanceID string, sandboxID string) error // GetInstanceSandboxesWithStatus returns sandboxes for an instance including type, status, and endpoints. - // If typeFilter is empty, returns all types. - GetInstanceSandboxesWithStatus(ctx context.Context, instanceID, typeFilter string) ([]map[string]interface{}, error) + // osFilter, when non-empty, is an OS selector (e.g. "windows"); empty returns all. + GetInstanceSandboxesWithStatus(ctx context.Context, instanceID, osFilter string) ([]map[string]interface{}, error) // CleanupInstanceSandboxes removes sandbox bindings for an instance. // In-use sandboxes are released first, then all matching leases are unassigned. diff --git a/backend/internal/biz/skill/impl/service.go b/backend/internal/biz/skill/impl/service.go index f467aa2..c80e2a1 100644 --- a/backend/internal/biz/skill/impl/service.go +++ b/backend/internal/biz/skill/impl/service.go @@ -24,6 +24,9 @@ import ( "context" "errors" "fmt" + "net/url" + "os" + "strings" "time" "gorm.io/gorm" @@ -41,7 +44,12 @@ import ( "sico-backend/pkg/logger" ) -const extractSkillTimeout = 30 * time.Second +// 180s average for one try; 3 retries with backoff should be sufficient for most cases. +const extractSkillTimeout = 180 * time.Second + +const getSkillVersionLimit = 5 + +type skillDownloadURLBuilder func(ctx context.Context, assetID int64) (string, error) type Components struct { SkillRepo repository.SkillRepository @@ -51,11 +59,13 @@ type Components struct { type Service struct { *Components - grpcClient skillgrpc.SkillServiceClient + grpcClient skillgrpc.SkillServiceClient + buildDownloadURLFunc skillDownloadURLBuilder } func NewService(c *Components) *Service { svc := &Service{Components: c} + svc.buildDownloadURLFunc = svc.buildDownloadURL if c != nil && c.CoreGRPC != nil { svc.grpcClient = skillgrpc.NewSkillServiceClient(c.CoreGRPC) } @@ -85,11 +95,8 @@ func (s *Service) CreateSkill(ctx context.Context, req *skill.CreateSkillRequest creator := middleware.MustGetUsernameFromCtx(ctx) rec := &repository.SkillModel{ - ProjectID: req.ProjectId, - AgentID: req.AgentId, - AssetID: req.AssetId, - Status: statusToDB(skill.SkillStatus_SKILL_STATUS_UPLOADING), - CreatorUsername: creator, + ProjectID: req.ProjectId, + AgentID: req.AgentId, } id, err := s.SkillRepo.Create(ctx, rec) @@ -98,9 +105,14 @@ func (s *Service) CreateSkill(ctx context.Context, req *skill.CreateSkillRequest } rec.ID = id - s.extractAndUpdateSkill(ctx, rec) + version, err := s.extractAndCreateVersion(ctx, rec, req.AssetId, creator) + if err != nil { + return nil, err + } - // Re-read to get updated status/name/description + if err := s.updateSkillDisplay(ctx, rec.ID, version.Name, version.Description); err != nil { + return nil, err + } rec, _ = s.SkillRepo.GetByID(ctx, id) return appresp.Success(&skill.CreateSkillResponse{ @@ -119,7 +131,8 @@ func (s *Service) GetSkill(ctx context.Context, req *skill.GetSkillRequest) (*sk return appresp.Success(&skill.GetSkillResponse{ Data: &skill.GetSkillData{ - Skill: skillModelToDTO(rec), + Skill: skillModelToDTO(rec), + Versions: s.skillVersionDTOs(ctx, rec), }, }), nil } @@ -133,22 +146,49 @@ func (s *Service) UpdateSkill(ctx context.Context, req *skill.UpdateSkillRequest return nil, err } - rec.AssetID = req.AssetId - rec.Status = statusToDB(skill.SkillStatus_SKILL_STATUS_UPLOADING) - rec.FailReason = "" - - if err := s.SkillRepo.Update(ctx, rec); err != nil { + creator := middleware.MustGetUsernameFromCtx(ctx) + sourceVersion, err := s.validateCurrentVersion(ctx, rec.ID, req.GetCurrentVersion()) + if err != nil { return nil, err } - s.extractAndUpdateSkill(ctx, rec) - rec, _ = s.SkillRepo.GetByID(ctx, req.Id) + version, err := s.writeManualVersion(ctx, rec, sourceVersion, req.GetAssetId(), req.GetFiles(), req.GetActions(), creator) + if err != nil { + return nil, err + } + if err := s.updateSkillDisplay(ctx, rec.ID, version.Name, version.Description); err != nil { + return nil, err + } return appresp.Success(&skill.UpdateSkillResponse{ - Data: &skill.UpdateSkillData{Skill: skillModelToDTO(rec)}, + Data: &skill.UpdateSkillData{ + SkillId: rec.ID, + Version: version.Version, + Name: version.Name, + Description: version.Description, + AssetId: version.AssetID, + }, }), nil } +func (s *Service) validateCurrentVersion( + ctx context.Context, skillID int64, currentVersion string, +) (*repository.SkillVersionModel, error) { + currentVersion = strings.TrimSpace(currentVersion) + if currentVersion == "" { + return nil, apperr.New(errcode.CommonInvalidParam, "currentVersion is required") + } + + version, err := s.SkillRepo.GetVersion(ctx, skillID, currentVersion) + if err != nil { + if errors.Is(err, gorm.ErrRecordNotFound) { + return nil, apperr.New(errcode.CommonConflict, "currentVersion does not exist") + } + return nil, err + } + return version, nil +} + func (s *Service) DeleteSkill(ctx context.Context, req *skill.DeleteSkillRequest) (*skill.DeleteSkillResponse, error) { rec, err := s.SkillRepo.GetByID(ctx, req.Id) if err != nil { @@ -161,6 +201,9 @@ func (s *Service) DeleteSkill(ctx context.Context, req *skill.DeleteSkillRequest if err := s.SkillRepo.Delete(ctx, req.Id); err != nil { return nil, err } + if err := s.SkillRepo.DeleteVersions(ctx, req.Id); err != nil { + return nil, err + } // Delete skill files from FS via core gRPC; ignore errors if skill doesn't exist in FS. if s.grpcClient != nil { @@ -196,6 +239,7 @@ func (s *Service) ListSkills(ctx context.Context, req *skill.ListSkillRequest) ( for _, rec := range records { result = append(result, skillModelToDTO(rec)) } + s.setLatestVersions(ctx, result) return appresp.Success(&skill.ListSkillResponse{ Data: &skill.ListSkillData{ @@ -206,133 +250,229 @@ func (s *Service) ListSkills(ctx context.Context, req *skill.ListSkillRequest) ( }), nil } -func (s *Service) GetSkillDetails( - ctx context.Context, req *skill.GetSkillDetailsRequest, -) (*skill.GetSkillDetailsResponse, error) { - rec, err := s.SkillRepo.GetByID(ctx, req.Id) - if err != nil { - if errors.Is(err, gorm.ErrRecordNotFound) { - return nil, apperr.New(errcode.CommonNotFound, "skill not found") - } - return nil, err - } +// ---------- Extraction ---------- +func (s *Service) extractAndCreateVersion( + ctx context.Context, + rec *repository.SkillModel, + assetID int64, + creator string, +) (*repository.SkillVersionModel, error) { + if rec == nil || rec.ID == 0 { + return nil, apperr.New(errcode.CommonInvalidParam, "skill is required") + } if s.grpcClient == nil { return nil, apperr.New(errcode.CommonUnavailable, "core gRPC client not initialized") } - if rec.Status != statusToDB(skill.SkillStatus_SKILL_STATUS_UPLOADED) { - return nil, apperr.New(errcode.CommonInvalidParam, "skill not yet uploaded") - } - - grpcReq := &skillgrpc.GetSkillDetailsGrpcRequest{ - SkillId: rec.ID, - ProjectId: rec.ProjectID, - AgentId: rec.AgentID, - } - - resp, err := s.grpcClient.GetSkillDetails(ctx, grpcReq) - if err != nil { - return nil, err - } - if resp == nil { - return nil, apperr.New(errcode.CommonInternalError, "empty response from core skill service") - } - if resp.Code != 0 { - msg := resp.Msg - if msg == "" { - msg = "failed to fetch skill details" - } - return nil, apperr.New(errcode.CommonInternalError, msg) - } - - files := make([]*skill.SkillFile, 0, len(resp.Files)) - for _, f := range resp.Files { - files = append(files, &skill.SkillFile{ - Path: f.Path, - Content: f.Content, - }) - } - - return appresp.Success(&skill.GetSkillDetailsResponse{ - Data: &skill.GetSkillDetailsData{Files: files}, - }), nil -} - -// ---------- Extraction ---------- - -func (s *Service) extractAndUpdateSkill(ctx context.Context, rec *repository.SkillModel) { - if rec == nil || rec.ID == 0 || s.grpcClient == nil { - return - } - - downloadURL, err := s.buildDownloadURL(ctx, rec) + downloadURL, err := s.skillDownloadURL(ctx, assetID) if err != nil || downloadURL == "" { logger.CtxWarn(ctx, "skill: failed to build download URL: id=%d err=%v", rec.ID, err) - s.updateSkillRecord(ctx, rec.ID, skill.SkillStatus_SKILL_STATUS_FAILED, "failed to build download URL", "", "") - return + return nil, apperr.New(errcode.CommonInvalidParam, "failed to build download URL") } reqCtx, cancel := context.WithTimeout(ctx, extractSkillTimeout) defer cancel() - + versionString := newSkillVersionString() req := &skillgrpc.ExtractSkillRequest{ SkillId: rec.ID, ProjectId: rec.ProjectID, AgentId: rec.AgentID, DownloadUrl: downloadURL, + Version: versionString, } resp, err := s.grpcClient.ExtractSkill(reqCtx, req) if err != nil { - failReason := err.Error() logger.CtxWarn(ctx, "skill: gRPC extract skill failed: id=%d err=%v", rec.ID, err) - s.updateSkillRecord(ctx, rec.ID, skill.SkillStatus_SKILL_STATUS_FAILED, failReason, "", "") - return + return nil, err } - if resp == nil || resp.Code != 0 { failReason := "skill extraction failed" if resp != nil && resp.Message != "" { failReason = resp.Message } - s.updateSkillRecord(ctx, rec.ID, skill.SkillStatus_SKILL_STATUS_FAILED, failReason, "", "") - return + return nil, apperr.New(errcode.CommonInternalError, failReason) } - s.updateSkillRecord(ctx, rec.ID, skill.SkillStatus_SKILL_STATUS_UPLOADED, "", resp.Name, resp.Description) + version := &repository.SkillVersionModel{ + SkillID: rec.ID, + Version: versionString, + AssetID: assetID, + Name: resp.Name, + Description: resp.Description, + CreatorUsername: creator, + Status: statusToDB(skill.SkillStatus_SKILL_STATUS_UPLOADED), + } + versionID, err := s.SkillRepo.CreateVersion(ctx, version) + if err != nil { + return nil, err + } + version.ID = versionID + return version, nil } -func (s *Service) updateSkillRecord( +func (s *Service) writeManualVersion( ctx context.Context, - skillID int64, status skill.SkillStatus, - failReason string, name string, description string, -) { - if s.SkillRepo == nil { - return + rec *repository.SkillModel, + sourceVersion *repository.SkillVersionModel, + assetID int64, + files []*skill.SkillFile, + actions []*skill.SkillAction, + creator string, +) (*repository.SkillVersionModel, error) { + if err := s.validateManualVersionRequest(sourceVersion, assetID, files, actions); err != nil { + return nil, err } - rec, err := s.SkillRepo.GetByID(ctx, skillID) + name, description, err := skillVersionMetadata(sourceVersion, files) if err != nil { - return + return nil, err + } + + if assetID == 0 && len(files) == 0 { + assetID = sourceVersion.AssetID + } + downloadURL, err := s.manualVersionDownloadURL(ctx, rec, sourceVersion, assetID, files, actions) + if err != nil { + return nil, err + } + + versionString := newSkillVersionString() + reqCtx, cancel := context.WithTimeout(ctx, extractSkillTimeout) + defer cancel() + resp, err := s.grpcClient.WriteSkillVersion(reqCtx, &skillgrpc.WriteSkillVersionRequest{ + SkillId: rec.ID, + ProjectId: rec.ProjectID, + AgentId: rec.AgentID, + Version: versionString, + Files: files, + Actions: actions, + DownloadUrl: downloadURL, + SourceVersion: sourceVersion.Version, + AssetId: assetID, + }) + if err != nil { + return nil, err } - rec.Status = statusToDB(status) - rec.FailReason = failReason - if name != "" { - rec.Name = name + if resp == nil || resp.Code != 0 { + msg := "failed to write skill version" + if resp != nil && resp.Msg != "" { + msg = resp.Msg + } + return nil, apperr.New(errcode.CommonInternalError, msg) } - if description != "" { - rec.Description = description + if resp.GetName() != "" { + name = resp.GetName() } - if err := s.SkillRepo.Update(ctx, rec); err != nil { - logger.CtxWarn(ctx, "skill: failed to update status: id=%d err=%v", skillID, err) + if resp.GetDescription() != "" { + description = resp.GetDescription() + } + if resp.GetAssetId() != 0 { + assetID = resp.GetAssetId() + } + version := &repository.SkillVersionModel{ + SkillID: rec.ID, + Version: versionString, + AssetID: assetID, + Name: name, + Description: description, + CreatorUsername: creator, + Status: statusToDB(skill.SkillStatus_SKILL_STATUS_UPLOADED), } + versionID, err := s.SkillRepo.CreateVersion(ctx, version) + if err != nil { + return nil, err + } + version.ID = versionID + return version, nil } -func (s *Service) buildDownloadURL(ctx context.Context, rec *repository.SkillModel) (string, error) { - if rec.AssetID == 0 || s.ProjectRepo == nil { +func (s *Service) validateManualVersionRequest( + sourceVersion *repository.SkillVersionModel, + assetID int64, + files []*skill.SkillFile, + actions []*skill.SkillAction, +) error { + if s.grpcClient == nil { + return apperr.New(errcode.CommonUnavailable, "core gRPC client not initialized") + } + if sourceVersion == nil { + return apperr.New(errcode.CommonInvalidParam, "sourceVersion is required") + } + if assetID == 0 && len(files) == 0 && len(actions) == 0 { + return apperr.New(errcode.CommonInvalidParam, "assetId, files, or actions are required") + } + return nil +} + +func skillVersionMetadata( + sourceVersion *repository.SkillVersionModel, + files []*skill.SkillFile, +) (string, string, error) { + if len(files) == 0 { + return sourceVersion.Name, sourceVersion.Description, nil + } + + name, description, found, err := skillMetadataFromFiles(files) + if err != nil { + return "", "", err + } + if !found { + return sourceVersion.Name, sourceVersion.Description, nil + } + return name, description, nil +} + +func (s *Service) manualVersionDownloadURL( + ctx context.Context, + rec *repository.SkillModel, + sourceVersion *repository.SkillVersionModel, + assetID int64, + files []*skill.SkillFile, + actions []*skill.SkillAction, +) (string, error) { + if assetID == sourceVersion.AssetID && len(files) == 0 && len(actions) == 0 { + logger.CtxInfo( + ctx, + "skill: update requested with unchanged asset id: id=%d version=%s asset=%d", + rec.ID, sourceVersion.Version, assetID, + ) + } + if assetID == 0 || len(files) > 0 || len(actions) > 0 { + return "", nil + } + + downloadURL, err := s.skillDownloadURL(ctx, assetID) + if err != nil || downloadURL == "" { + logger.CtxWarn(ctx, "skill: failed to build download URL: id=%d err=%v", rec.ID, err) + return "", apperr.New(errcode.CommonInvalidParam, "failed to build download URL") + } + return downloadURL, nil +} + +func (s *Service) updateSkillDisplay(ctx context.Context, skillID int64, name, description string) error { + rec, err := s.SkillRepo.GetByID(ctx, skillID) + if err != nil { + return err + } + rec.Name = name + rec.Description = description + return s.SkillRepo.Update(ctx, rec) +} + +func (s *Service) skillDownloadURL(ctx context.Context, assetID int64) (string, error) { + if s.buildDownloadURLFunc != nil { + return s.buildDownloadURLFunc(ctx, assetID) + } + + return s.buildDownloadURL(ctx, assetID) +} + +func (s *Service) buildDownloadURL(ctx context.Context, assetID int64) (string, error) { + if assetID == 0 || s.ProjectRepo == nil { return "", nil } - asset, err := s.ProjectRepo.GetProjectAsset(ctx, rec.AssetID) + asset, err := s.ProjectRepo.GetProjectAsset(ctx, assetID) if err != nil { return "", err } @@ -349,13 +489,129 @@ func skillModelToDTO(rec *repository.SkillModel) *skill.Skill { if rec == nil { return nil } + return &skill.Skill{ + Id: rec.ID, + ProjectId: rec.ProjectID, + AgentId: rec.AgentID, + Name: rec.Name, + Description: rec.Description, + CreatedAt: rec.CreatedAt, + UpdatedAt: rec.UpdatedAt, + } +} + +func (s *Service) setLatestVersions(ctx context.Context, skills []*skill.Skill) { + skillIDs := make([]int64, 0, len(skills)) + for _, item := range skills { + if item != nil && item.Id != 0 { + skillIDs = append(skillIDs, item.Id) + } + } + latestVersions, err := s.SkillRepo.ListLatestVersionsBySkillIDs(ctx, skillIDs) + if err != nil { + logger.CtxWarn(ctx, "skill: failed to load latest versions for list: err=%v", err) + return + } + for _, item := range skills { + if item == nil { + continue + } + if latest := latestVersions[item.Id]; latest != nil { + item.Version = latest.Version + } + } +} + +func (s *Service) skillVersionDetails( + ctx context.Context, rec *repository.SkillModel, requestedVersions []string, +) map[string]*skill.SkillVersion { + if s.grpcClient == nil || len(requestedVersions) == 0 { + return nil + } + resp, err := s.grpcClient.GetSkillDetails(ctx, &skillgrpc.GetSkillDetailsGrpcRequest{ + SkillId: rec.ID, + ProjectId: rec.ProjectID, + AgentId: rec.AgentID, + Versions: requestedVersions, + }) + if err != nil || resp == nil || resp.Code != 0 { + if err != nil { + logger.CtxWarn(ctx, "skill: failed to load skill details: id=%d err=%v", rec.ID, err) + } + return nil + } + details := make(map[string]*skill.SkillVersion, len(resp.Versions)+1) + for _, version := range resp.Versions { + if version != nil && version.Version != "" { + details[version.Version] = version + } + } + if resp.Version != nil && resp.Version.Version != "" { + details[resp.Version.Version] = resp.Version + } + return details +} + +func (s *Service) skillVersionDTOs(ctx context.Context, rec *repository.SkillModel) []*skill.SkillVersion { + versions, err := s.SkillRepo.ListLatestVersions(ctx, rec.ID, getSkillVersionLimit) + if err != nil { + logger.CtxWarn(ctx, "skill: failed to list versions: id=%d err=%v", rec.ID, err) + return nil + } + items := make([]*skill.SkillVersion, 0, len(versions)) + for _, version := range versions { + items = append(items, s.skillVersionModelToDTO(ctx, version)) + } + + details := s.skillVersionDetails(ctx, rec, uploadedVersionStrings(versions)) + if len(details) == 0 { + return items + } + for _, item := range items { + if detail := details[item.Version]; detail != nil { + mergeSkillVersionDTO(item, detail) + } + } + return items +} + +func uploadedVersionStrings(versions []*repository.SkillVersionModel) []string { + values := make([]string, 0, len(versions)) + for _, version := range versions { + if version == nil || version.Status != statusToDB(skill.SkillStatus_SKILL_STATUS_UPLOADED) { + continue + } + if strings.TrimSpace(version.Version) == "" { + continue + } + values = append(values, version.Version) + } + return values +} + +func (s *Service) skillVersionModelToDTO(ctx context.Context, rec *repository.SkillVersionModel) *skill.SkillVersion { + if rec == nil { + return nil + } + url := "" + if rec.AssetID != 0 { + var err error + url, err = s.skillDownloadURL(ctx, rec.AssetID) + if err != nil { + logger.CtxWarn(ctx, + "skill: failed to build version download URL: id=%d version=%s asset=%d err=%v", + rec.SkillID, rec.Version, rec.AssetID, err) + } + url = publicSkillDownloadURL(url) + } + return &skill.SkillVersion{ Id: rec.ID, - ProjectId: rec.ProjectID, - AgentId: rec.AgentID, + SkillId: rec.SkillID, + Version: rec.Version, + Url: url, Name: rec.Name, Description: rec.Description, - AssetId: rec.AssetID, CreatorUsername: rec.CreatorUsername, Status: statusFromDB(rec.Status), FailReason: rec.FailReason, @@ -363,3 +619,84 @@ func skillModelToDTO(rec *repository.SkillModel) *skill.Skill { UpdatedAt: rec.UpdatedAt, } } + +func publicSkillDownloadURL(rawURL string) string { + rawURL = strings.TrimSpace(rawURL) + if rawURL == "" { + return rawURL + } + publicEndpoint := strings.TrimRight(os.Getenv("SICO_PUBLIC_ENDPOINT"), "/") + if publicEndpoint == "" { + return rawURL + } + if parsed, err := url.Parse(rawURL); err == nil && parsed.Scheme != "" && parsed.Host != "" { + path := "/" + strings.TrimLeft(parsed.Path, "/") + if strings.HasPrefix(path, "/storage/") { + return publicEndpoint + path + } + return publicEndpoint + "/storage" + path + } + if strings.HasPrefix(rawURL, "/") { + return publicEndpoint + rawURL + } + return publicEndpoint + "/" + strings.TrimLeft(rawURL, "/") +} + +func mergeSkillVersionDTO(base, detail *skill.SkillVersion) *skill.SkillVersion { + if detail == nil { + return base + } + if base == nil { + return detail + } + base.Actions = detail.Actions + return base +} + +func newSkillVersionString() string { + return fmt.Sprintf("%d", time.Now().UnixMilli()) +} + +func skillMetadataFromFiles(files []*skill.SkillFile) (string, string, bool, error) { + for _, file := range files { + if file == nil || normalizedSkillFilePath(file.GetPath()) != "SKILL.md" { + continue + } + metadata := parseFrontmatter(file.GetContent()) + name := strings.TrimSpace(metadata["name"]) + description := strings.TrimSpace(metadata["description"]) + if name == "" || description == "" { + return "", "", true, apperr.New( + errcode.CommonInvalidParam, "SKILL.md must contain non-empty name and description", + ) + } + return name, description, true, nil + } + return "", "", false, nil +} + +func normalizedSkillFilePath(path string) string { + path = strings.TrimSpace(strings.ReplaceAll(path, "\\", "/")) + path = strings.TrimPrefix(path, "original/") + return path +} + +func parseFrontmatter(content string) map[string]string { + lines := strings.Split(content, "\n") + result := map[string]string{} + if len(lines) == 0 || strings.TrimSpace(lines[0]) != "---" { + return result + } + for _, line := range lines[1:] { + line = strings.TrimSpace(line) + if line == "---" { + break + } + key, value, ok := strings.Cut(line, ":") + if !ok { + continue + } + result[strings.TrimSpace(key)] = strings.Trim(strings.TrimSpace(value), "\"'") + } + return result +} diff --git a/backend/internal/biz/skill/impl/service_test.go b/backend/internal/biz/skill/impl/service_test.go new file mode 100644 index 0000000..22b3839 --- /dev/null +++ b/backend/internal/biz/skill/impl/service_test.go @@ -0,0 +1,204 @@ +// Copyright (c) 2026 Sico Authors +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// The above copyright notice and this permission notice shall be included in +// all copies or substantial portions of the Software. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +package impl + +import ( + "context" + "errors" + "testing" + + "google.golang.org/grpc" + + repository "sico-backend/internal/store/skill/repository" + skillgrpc "sico-backend/internal/transport/grpc/pb/skill" + "sico-backend/internal/transport/http/dto/skill" +) + +func TestSkillVersionModelToDTOConvertsAssetIDToURL(t *testing.T) { + svc := &Service{ + buildDownloadURLFunc: func(_ context.Context, assetID int64) (string, error) { + if assetID != 123 { + t.Fatalf("unexpected asset id: %d", assetID) + } + return "https://assets.example/123.zip", nil + }, + } + + got := svc.skillVersionModelToDTO(context.Background(), &repository.SkillVersionModel{ + ID: 7, + SkillID: 9, + Version: "v1", + AssetID: 123, + }) + + if got.GetUrl() != "https://assets.example/123.zip" { + t.Fatalf("expected url to be populated, got %q", got.GetUrl()) + } +} + +func TestSkillVersionModelToDTOExpandsPartialStorageURL(t *testing.T) { + t.Setenv("SICO_PUBLIC_ENDPOINT", "http://localhost:8080") + svc := &Service{ + buildDownloadURLFunc: func(_ context.Context, assetID int64) (string, error) { + if assetID != 123 { + t.Fatalf("unexpected asset id: %d", assetID) + } + return "/storage/default_space/asset.zip", nil + }, + } + + got := svc.skillVersionModelToDTO(context.Background(), &repository.SkillVersionModel{ + ID: 7, + SkillID: 9, + Version: "v1", + AssetID: 123, + }) + + if got.GetUrl() != "http://localhost:8080/storage/default_space/asset.zip" { + t.Fatalf("expected full public url, got %q", got.GetUrl()) + } +} + +func TestSkillVersionModelToDTOExpandsInternalStorageURL(t *testing.T) { + t.Setenv("SICO_PUBLIC_ENDPOINT", "http://localhost:8080") + svc := &Service{ + buildDownloadURLFunc: func(_ context.Context, assetID int64) (string, error) { + if assetID != 123 { + t.Fatalf("unexpected asset id: %d", assetID) + } + return "http://sico-seaweedfs-filer:14003/default_space/asset.zip", nil + }, + } + + got := svc.skillVersionModelToDTO(context.Background(), &repository.SkillVersionModel{ + ID: 7, + SkillID: 9, + Version: "v1", + AssetID: 123, + }) + + if got.GetUrl() != "http://localhost:8080/storage/default_space/asset.zip" { + t.Fatalf("expected full public url, got %q", got.GetUrl()) + } +} + +type fakeSkillRepo struct { + createVersionCalls int + createdVersions []*repository.SkillVersionModel +} + +func (f *fakeSkillRepo) Create(context.Context, *repository.SkillModel) (int64, error) { return 0, nil } +func (f *fakeSkillRepo) Update(context.Context, *repository.SkillModel) error { return nil } +func (f *fakeSkillRepo) GetByID(context.Context, int64) (*repository.SkillModel, error) { + return nil, errors.New("not implemented") +} +func (f *fakeSkillRepo) List(context.Context, *repository.SkillFilter) ([]*repository.SkillModel, int64, error) { + return nil, 0, nil +} +func (f *fakeSkillRepo) Delete(context.Context, int64) error { return nil } +func (f *fakeSkillRepo) CreateVersion(_ context.Context, version *repository.SkillVersionModel) (int64, error) { + f.createVersionCalls++ + f.createdVersions = append(f.createdVersions, version) + return int64(f.createVersionCalls), nil +} +func (f *fakeSkillRepo) GetLatestVersion(context.Context, int64) (*repository.SkillVersionModel, error) { + return nil, errors.New("not implemented") +} +func (f *fakeSkillRepo) GetVersion(context.Context, int64, string) (*repository.SkillVersionModel, error) { + return nil, errors.New("not implemented") +} +func (f *fakeSkillRepo) ListLatestVersionsBySkillIDs(context.Context, []int64) (map[int64]*repository.SkillVersionModel, error) { + return nil, nil +} +func (f *fakeSkillRepo) ListLatestVersions(context.Context, int64, int) ([]*repository.SkillVersionModel, error) { + return nil, nil +} +func (f *fakeSkillRepo) DeleteVersions(context.Context, int64) error { return nil } + +type fakeSkillGrpcClient struct { + writeResp *skillgrpc.WriteSkillVersionResponse + writeErr error +} + +func (f *fakeSkillGrpcClient) ExtractSkill( + context.Context, *skillgrpc.ExtractSkillRequest, ...grpc.CallOption, +) (*skillgrpc.ExtractSkillResponse, error) { + return nil, errors.New("not implemented") +} + +func (f *fakeSkillGrpcClient) GetSkillDetails( + context.Context, *skillgrpc.GetSkillDetailsGrpcRequest, ...grpc.CallOption, +) (*skillgrpc.GetSkillDetailsGrpcResponse, error) { + return nil, errors.New("not implemented") +} + +func (f *fakeSkillGrpcClient) DeleteSkillFromFS( + context.Context, *skillgrpc.DeleteSkillFromFSRequest, ...grpc.CallOption, +) (*skillgrpc.DeleteSkillFromFSResponse, error) { + return nil, errors.New("not implemented") +} + +func (f *fakeSkillGrpcClient) WriteSkillVersion( + context.Context, *skillgrpc.WriteSkillVersionRequest, ...grpc.CallOption, +) (*skillgrpc.WriteSkillVersionResponse, error) { + return f.writeResp, f.writeErr +} + +func TestWriteManualVersionRejectsInvalidSkillMarkdownWithoutCreatingVersion(t *testing.T) { + repo := &fakeSkillRepo{} + svc := &Service{Components: &Components{SkillRepo: repo}, grpcClient: &fakeSkillGrpcClient{}} + rec := &repository.SkillModel{ID: 283, Name: "ai-3d-model", Description: "existing"} + source := &repository.SkillVersionModel{Version: "v1", AssetID: 10, Name: "ai-3d-model", Description: "existing"} + files := []*skill.SkillFile{{ + Path: "SKILL.md", + Content: "---\nname: ai-3d-model\ndescription: >\nnot indented\n---\n# Skill\n", + }} + + _, err := svc.writeManualVersion(context.Background(), rec, source, 0, files, nil, "tester@example.com") + if err == nil { + t.Fatal("expected invalid SKILL.md to fail") + } + if repo.createVersionCalls != 0 { + t.Fatalf("CreateVersion calls = %d, want 0", repo.createVersionCalls) + } +} + +func TestWriteManualVersionRejectsCoreActionFailureWithoutCreatingVersion(t *testing.T) { + repo := &fakeSkillRepo{} + svc := &Service{ + Components: &Components{SkillRepo: repo}, + grpcClient: &fakeSkillGrpcClient{writeResp: &skillgrpc.WriteSkillVersionResponse{ + Code: 1, + Msg: "invalid actions manifest", + }}, + } + rec := &repository.SkillModel{ID: 283, Name: "ai-3d-model", Description: "existing"} + source := &repository.SkillVersionModel{Version: "v1", AssetID: 10, Name: "ai-3d-model", Description: "existing"} + actions := []*skill.SkillAction{{Name: "run", AdvancedSettings: "{bad json"}} + + _, err := svc.writeManualVersion(context.Background(), rec, source, 0, nil, actions, "tester@example.com") + if err == nil { + t.Fatal("expected core action validation failure to fail") + } + if repo.createVersionCalls != 0 { + t.Fatalf("CreateVersion calls = %d, want 0", repo.createVersionCalls) + } +} diff --git a/backend/internal/biz/skill/iskill.go b/backend/internal/biz/skill/iskill.go index dc766ed..4a78432 100644 --- a/backend/internal/biz/skill/iskill.go +++ b/backend/internal/biz/skill/iskill.go @@ -33,7 +33,6 @@ type Service interface { UpdateSkill(ctx context.Context, req *skill.UpdateSkillRequest) (*skill.UpdateSkillResponse, error) DeleteSkill(ctx context.Context, req *skill.DeleteSkillRequest) (*skill.DeleteSkillResponse, error) ListSkills(ctx context.Context, req *skill.ListSkillRequest) (*skill.ListSkillResponse, error) - GetSkillDetails(ctx context.Context, req *skill.GetSkillDetailsRequest) (*skill.GetSkillDetailsResponse, error) } var defaultSvc Service diff --git a/backend/internal/biz/taskruntime/impl/errors.go b/backend/internal/biz/taskruntime/impl/errors.go new file mode 100644 index 0000000..f3ffeca --- /dev/null +++ b/backend/internal/biz/taskruntime/impl/errors.go @@ -0,0 +1,64 @@ +// Copyright (c) 2026 Sico Authors +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +package impl + +import ( + "errors" + "fmt" + + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" + "gorm.io/gorm" + + taskruntimerepo "sico-backend/internal/store/taskruntime/repository" +) + +// duplicateKeyToken is the stable substring legacy Python clients fall back to +// when classifying AlreadyExists responses; it must stay in the error message. +const duplicateKeyToken = "duplicate key" + +// responseSuccess is the Msg field every successful reverse-RPC response carries. +const responseSuccess = "success" + +// internalError wraps a transport/DB-level failure as a gRPC Internal error. +// It is intentionally opaque about the underlying error structure: leaking +// gorm error types across the wire would be a coupling smell. +func internalError(op string, err error) error { + return status.Errorf(codes.Internal, "%s: %s", op, err.Error()) +} + +// translateError maps repository errors to gRPC status errors with appropriate +// codes. This is the single chokepoint between the persistence layer and the +// wire: record-not-found becomes NotFound, a stale fencing token becomes +// FailedPrecondition, and a unique-constraint collision becomes AlreadyExists +// (which Python clients translate into a re-read via lookup_idempotent). +func translateError(op string, err error) error { + if err == nil { + return nil + } + if errors.Is(err, gorm.ErrRecordNotFound) { + return status.Error(codes.NotFound, fmt.Sprintf("%s: %s", op, err.Error())) + } + if errors.Is(err, taskruntimerepo.ErrStaleToken) { + return status.Error(codes.FailedPrecondition, fmt.Sprintf("%s: %s", op, err.Error())) + } + if errors.Is(err, taskruntimerepo.ErrDuplicate) || taskruntimerepo.IsDuplicateKey(err) { + return status.Error(codes.AlreadyExists, fmt.Sprintf("%s: %s: %s", op, duplicateKeyToken, err.Error())) + } + return internalError(op, err) +} diff --git a/backend/internal/biz/taskruntime/impl/errors_test.go b/backend/internal/biz/taskruntime/impl/errors_test.go new file mode 100644 index 0000000..69d0528 --- /dev/null +++ b/backend/internal/biz/taskruntime/impl/errors_test.go @@ -0,0 +1,93 @@ +// Copyright (c) 2026 Sico Authors +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +package impl + +import ( + "errors" + "fmt" + "testing" + + "github.com/go-sql-driver/mysql" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" + "gorm.io/gorm" + + taskruntimerepo "sico-backend/internal/store/taskruntime/repository" +) + +func TestTranslateErrorPassesThroughNil(t *testing.T) { + if got := translateError("op", nil); got != nil { + t.Fatalf("expected nil for nil input, got %v", got) + } +} + +func TestTranslateErrorMapsRecordNotFoundToNotFound(t *testing.T) { + got := translateError("RpcGetRun", gorm.ErrRecordNotFound) + st, ok := status.FromError(got) + if !ok { + t.Fatalf("translateError did not return a status error: %v", got) + } + if st.Code() != codes.NotFound { + t.Fatalf("expected NotFound, got %s", st.Code()) + } +} + +func TestTranslateErrorMapsStaleTokenToFailedPrecondition(t *testing.T) { + // Errors that wrap the stale-token sentinel via fmt.Errorf("%w: ...") must + // still be detected after crossing the repository boundary. + wrapped := fmt.Errorf("%w: run abc", taskruntimerepo.ErrStaleToken) + got := translateError("RpcHeartbeatBatch", wrapped) + st, ok := status.FromError(got) + if !ok { + t.Fatalf("translateError did not return a status error: %v", got) + } + if st.Code() != codes.FailedPrecondition { + t.Fatalf("expected FailedPrecondition, got %s", st.Code()) + } +} + +func TestTranslateErrorMapsDuplicateSentinelToAlreadyExists(t *testing.T) { + got := translateError("RpcCreateRun", taskruntimerepo.ErrDuplicate) + st, ok := status.FromError(got) + if !ok { + t.Fatalf("translateError did not return a status error: %v", got) + } + if st.Code() != codes.AlreadyExists { + t.Fatalf("expected AlreadyExists, got %s", st.Code()) + } +} + +func TestTranslateErrorMapsRawMySQLDuplicateToAlreadyExists(t *testing.T) { + // A raw driver duplicate (e.g. from a unique-constraint update that never + // went through the create path) must still map to AlreadyExists. + inner := &mysql.MySQLError{Number: 1062, Message: "Duplicate entry"} + wrapped := fmt.Errorf("create: %w", inner) + got := translateError("RpcCreateRun", wrapped) + st, _ := status.FromError(got) + if st.Code() != codes.AlreadyExists { + t.Fatalf("expected AlreadyExists for wrapped duplicate, got %s", st.Code()) + } +} + +func TestTranslateErrorDefaultIsInternal(t *testing.T) { + got := translateError("RpcCreateRun", errors.New("kaboom")) + st, _ := status.FromError(got) + if st.Code() != codes.Internal { + t.Fatalf("expected Internal, got %s", st.Code()) + } +} diff --git a/backend/internal/biz/taskruntime/impl/service.go b/backend/internal/biz/taskruntime/impl/service.go new file mode 100644 index 0000000..f20ffdb --- /dev/null +++ b/backend/internal/biz/taskruntime/impl/service.go @@ -0,0 +1,226 @@ +// Copyright (c) 2026 Sico Authors +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +// Package impl is the reverse-gRPC transport adapter for the task runtime. It +// decodes the JSON document payloads Core sends, delegates persistence to the +// store-layer repository, and translates repository errors into gRPC status +// codes. All database access lives in internal/store/taskruntime. +package impl + +import ( + "context" + + taskruntimerepo "sico-backend/internal/store/taskruntime/repository" + rgrpc "sico-backend/internal/transport/reverse_grpc/pb/taskruntime" +) + +// Service implements the reverse task-runtime gRPC server on top of the +// task-runtime persistence repository. +type Service struct { + rgrpc.UnimplementedReverseTaskRuntimeRPCServer + repo taskruntimerepo.TaskRuntimeRepository +} + +// NewService builds the reverse task-runtime service over the given repository. +func NewService(repo taskruntimerepo.TaskRuntimeRepository) *Service { + return &Service{repo: repo} +} + +func (s *Service) RpcCreateBatch(ctx context.Context, req *rgrpc.CreateBatchRequest) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.CreateBatch(ctx, req.GetBatchJson()); err != nil { + return nil, translateError("RpcCreateBatch", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcUpdateBatch(ctx context.Context, req *rgrpc.UpdateBatchRequest) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.UpdateBatch(ctx, req.GetBatchJson()); err != nil { + return nil, translateError("RpcUpdateBatch", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcGetBatch(ctx context.Context, req *rgrpc.GetBatchRequest) (*rgrpc.GetBatchResponse, error) { + batchJSON, found, err := s.repo.GetBatch(ctx, req.GetBatchId()) + if err != nil { + return nil, internalError("RpcGetBatch", err) + } + + return &rgrpc.GetBatchResponse{BatchJson: batchJSON, Found: found, Code: 0, Msg: responseSuccess}, nil +} + +func (s *Service) RpcCreateRun(ctx context.Context, req *rgrpc.CreateRunRequest) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.CreateRun(ctx, req.GetRunJson()); err != nil { + return nil, translateError("RpcCreateRun", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcUpdateRun(ctx context.Context, req *rgrpc.UpdateRunRequest) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.UpdateRun(ctx, req.GetRunJson()); err != nil { + return nil, translateError("RpcUpdateRun", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcReopenRunForRetry( + ctx context.Context, + req *rgrpc.ReopenRunForRetryRequest, +) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.ReopenRunForRetry(ctx, req.GetRunJson(), req.GetExpectedAttempt()); err != nil { + return nil, translateError("RpcReopenRunForRetry", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcLookupIdempotent(ctx context.Context, req *rgrpc.LookupIdempotentRequest) (*rgrpc.GetRunResponse, error) { + runJSON, found, err := s.repo.LookupIdempotent(ctx, req.GetIdempotencyKey()) + if err != nil { + return nil, internalError("RpcLookupIdempotent", err) + } + + return &rgrpc.GetRunResponse{RunJson: runJSON, Found: found, Code: 0, Msg: responseSuccess}, nil +} + +func (s *Service) RpcClaimRun(ctx context.Context, req *rgrpc.ClaimRunRequest) (*rgrpc.ClaimRunResponse, error) { + tokenJSON, err := s.repo.ClaimRun(ctx, req.GetRunId(), req.GetWorkerId()) + if err != nil { + return nil, translateError("RpcClaimRun", err) + } + + return &rgrpc.ClaimRunResponse{TokenJson: tokenJSON, Code: 0, Msg: responseSuccess}, nil +} + +func (s *Service) RpcHeartbeatBatch( + ctx context.Context, + req *rgrpc.HeartbeatBatchRequest, +) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.HeartbeatBatch(ctx, req.GetBatchId()); err != nil { + return nil, translateError("RpcHeartbeatBatch", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcSetRunProgress( + ctx context.Context, + req *rgrpc.SetRunProgressRequest, +) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.SetRunProgress(ctx, req.GetRunId(), req.GetMessage(), req.GetTs()); err != nil { + return nil, translateError("RpcSetRunProgress", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcWriteResult(ctx context.Context, req *rgrpc.WriteResultRequest) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.WriteResult(ctx, req.GetRunId(), req.GetTokenJson(), req.GetResultJson()); err != nil { + return nil, translateError("RpcWriteResult", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcCancelBatch(ctx context.Context, req *rgrpc.CancelBatchRequest) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.CancelBatch(ctx, req.GetBatchId(), req.GetReason()); err != nil { + return nil, translateError("RpcCancelBatch", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcCancelRun(ctx context.Context, req *rgrpc.CancelRunRequest) (*rgrpc.EmptyTaskRuntimeResponse, error) { + if err := s.repo.CancelRun(ctx, req.GetRunId(), req.GetReason()); err != nil { + return nil, translateError("RpcCancelRun", err) + } + + return emptyOK(), nil +} + +func (s *Service) RpcGetRun(ctx context.Context, req *rgrpc.GetRunRequest) (*rgrpc.GetRunResponse, error) { + runJSON, found, err := s.repo.GetRun(ctx, req.GetRunId()) + if err != nil { + return nil, internalError("RpcGetRun", err) + } + + return &rgrpc.GetRunResponse{RunJson: runJSON, Found: found, Code: 0, Msg: responseSuccess}, nil +} + +func (s *Service) RpcGetTaskDetail(ctx context.Context, req *rgrpc.GetTaskDetailRequest) (*rgrpc.GetTaskDetailResponse, error) { + detail, found, err := s.repo.GetTaskDetail(ctx, req.GetRunId(), req.GetView()) + if err != nil { + return nil, internalError("RpcGetTaskDetail", err) + } + + return &rgrpc.GetTaskDetailResponse{ + RunJson: detail.RunJSON, + ResultJson: detail.ResultJSON, + View: req.GetView(), + Content: detail.Content, + ArtifactsJson: detail.ArtifactsJSON, + Found: found, + Code: 0, + Msg: responseSuccess, + }, nil +} + +func (s *Service) RpcListBatchRuns(ctx context.Context, req *rgrpc.ListBatchRunsRequest) (*rgrpc.ListBatchRunsResponse, error) { + runsJSON, err := s.repo.ListBatchRuns(ctx, req.GetBatchId()) + if err != nil { + return nil, internalError("RpcListBatchRuns", err) + } + + return &rgrpc.ListBatchRunsResponse{RunsJson: runsJSON, Code: 0, Msg: responseSuccess}, nil +} + +func (s *Service) RpcListBatchesByTurn( + ctx context.Context, + req *rgrpc.ListBatchesByTurnRequest, +) (*rgrpc.ListBatchesByTurnResponse, error) { + batchesJSON, err := s.repo.ListBatchesByTurn( + ctx, + req.GetParentConversationId(), + req.GetParentTurnId(), + req.GetActiveOnly(), + ) + if err != nil { + return nil, internalError("RpcListBatchesByTurn", err) + } + + return &rgrpc.ListBatchesByTurnResponse{BatchesJson: batchesJSON, Code: 0, Msg: responseSuccess}, nil +} + +func (s *Service) RpcSweepStaleRuns( + ctx context.Context, + req *rgrpc.SweepStaleRunsRequest, +) (*rgrpc.SweepStaleRunsResponse, error) { + staleRunsJSON, err := s.repo.SweepStaleRuns(ctx, req.GetBeforeTs()) + if err != nil { + return nil, internalError("RpcSweepStaleRuns", err) + } + + return &rgrpc.SweepStaleRunsResponse{StaleRunsJson: staleRunsJSON, Code: 0, Msg: responseSuccess}, nil +} + +func emptyOK() *rgrpc.EmptyTaskRuntimeResponse { + return &rgrpc.EmptyTaskRuntimeResponse{Code: 0, Msg: responseSuccess} +} diff --git a/backend/internal/biz/taskruntime/init.go b/backend/internal/biz/taskruntime/init.go new file mode 100644 index 0000000..b2b327a --- /dev/null +++ b/backend/internal/biz/taskruntime/init.go @@ -0,0 +1,45 @@ +// Copyright (c) 2026 Sico Authors +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +package taskruntime + +import ( + "github.com/google/wire" + + "sico-backend/internal/biz/taskruntime/impl" + taskruntimerepo "sico-backend/internal/store/taskruntime/repository" + "sico-backend/pkg/logger" +) + +var defaultSvc Service + +// Default returns the singleton task runtime service. +func Default() Service { return defaultSvc } + +func InitService(svc *impl.Service) Service { + defaultSvc = svc + logger.Info("Task runtime service initialized") + return defaultSvc +} + +// ProviderSet wires the task runtime persistence service: the store-layer +// repository, the reverse-gRPC adapter over it, and the singleton installer. +var ProviderSet = wire.NewSet( + taskruntimerepo.NewTaskRuntimeRepo, + impl.NewService, + InitService, +) diff --git a/backend/internal/biz/taskruntime/itaskruntime.go b/backend/internal/biz/taskruntime/itaskruntime.go new file mode 100644 index 0000000..119319a --- /dev/null +++ b/backend/internal/biz/taskruntime/itaskruntime.go @@ -0,0 +1,25 @@ +// Copyright (c) 2026 Sico Authors +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. + +package taskruntime + +import taskruntimeRgrpc "sico-backend/internal/transport/reverse_grpc/pb/taskruntime" + +// Service is the task runtime persistence contract exposed to Core over reverse gRPC. +type Service interface { + taskruntimeRgrpc.ReverseTaskRuntimeRPCServer +} diff --git a/backend/internal/consts/consts.go b/backend/internal/consts/consts.go index 81d894d..04f3dcb 100644 --- a/backend/internal/consts/consts.go +++ b/backend/internal/consts/consts.go @@ -48,7 +48,6 @@ const ( // Sandbox-related environment variables const ( - SandboxResetOnRelease = "SANDBOX_RESET_ON_RELEASE" SandboxResetCooldownSeconds = "SANDBOX_RESET_COOLDOWN_SECONDS" SandboxEmulatorBaseURL = "SANDBOX_EMULATOR_BASE_URL" diff --git a/backend/internal/di/app/providers.go b/backend/internal/di/app/providers.go index b7e2fec..92544e1 100644 --- a/backend/internal/di/app/providers.go +++ b/backend/internal/di/app/providers.go @@ -31,6 +31,7 @@ import ( "sico-backend/internal/biz/rbac" "sico-backend/internal/biz/sandbox" "sico-backend/internal/biz/skill" + "sico-backend/internal/biz/taskruntime" ) var ProviderSet = wire.NewSet( @@ -42,4 +43,5 @@ var ProviderSet = wire.NewSet( sandbox.ProviderSet, llmhubs.ProviderSet, skill.ProviderSet, + taskruntime.ProviderSet, ) diff --git a/backend/internal/di/injector.go b/backend/internal/di/injector.go index 6f2ce97..0aea24f 100644 --- a/backend/internal/di/injector.go +++ b/backend/internal/di/injector.go @@ -32,6 +32,7 @@ import ( "sico-backend/internal/biz/rbac" "sico-backend/internal/biz/sandbox" "sico-backend/internal/biz/skill" + "sico-backend/internal/biz/taskruntime" "sico-backend/internal/infra/coregrpc" "sico-backend/internal/infra/idgen" "sico-backend/internal/infra/storage" @@ -52,4 +53,5 @@ type Injector struct { SandboxApp sandbox.Service SkillApp skill.Service LLMHubApp llmhubs.Service + TaskRuntimeApp taskruntime.Service } diff --git a/backend/internal/di/wire_gen.go b/backend/internal/di/wire_gen.go index f346757..617e773 100644 --- a/backend/internal/di/wire_gen.go +++ b/backend/internal/di/wire_gen.go @@ -23,6 +23,8 @@ import ( impl6 "sico-backend/internal/biz/sandbox/impl" "sico-backend/internal/biz/skill" impl7 "sico-backend/internal/biz/skill/impl" + "sico-backend/internal/biz/taskruntime" + impl8 "sico-backend/internal/biz/taskruntime/impl" "sico-backend/internal/di/infra" repository4 "sico-backend/internal/store/agent/singleagent/repository" repository5 "sico-backend/internal/store/conversation/conversation/repository" @@ -33,6 +35,7 @@ import ( "sico-backend/internal/store/rbac/enforcer" repository2 "sico-backend/internal/store/rbac/repository" repository7 "sico-backend/internal/store/skill/repository" + repository9 "sico-backend/internal/store/taskruntime/repository" ) // Injectors from wire.go: @@ -130,6 +133,7 @@ func BuildInjector(ctx context.Context) (*Injector, func(), error) { Storage: storage, CoreGRPC: clientConn, Cache: client, + DB: db, } conversationService := conversation.InitService(components4) emulatorProvider := impl6.NewEmulatorProvider() @@ -152,6 +156,9 @@ func BuildInjector(ctx context.Context) (*Injector, func(), error) { modelRegistryRepository := repository8.NewModelRegistryRepo(db) modelRegistrySecretRepository := repository8.NewModelRegistrySecretRepo(db) llmhubsService := llmhubs.InitService(db, clientConn, modelRegistryRepository, modelRegistrySecretRepository, singleAgentLLMHubConfigRepository) + taskRuntimeRepository := repository9.NewTaskRuntimeRepo(db) + service3 := impl8.NewService(taskRuntimeRepository) + taskruntimeService := taskruntime.InitService(service3) injector := &Injector{ DB: db, Cache: client, @@ -166,6 +173,7 @@ func BuildInjector(ctx context.Context) (*Injector, func(), error) { SandboxApp: sandboxService, SkillApp: skillService, LLMHubApp: llmhubsService, + TaskRuntimeApp: taskruntimeService, } return injector, func() { cleanup3() diff --git a/backend/internal/embeddata/embed.go b/backend/internal/embeddata/embed.go index d31d035..8455861 100644 --- a/backend/internal/embeddata/embed.go +++ b/backend/internal/embeddata/embed.go @@ -69,6 +69,20 @@ var AndroidTesterSkillZip = sync.OnceValues(func() ([]byte, error) { return buildZipFromFS(AndroidTesterSkillFS, AndroidTesterSkillRoot) }) +// TestCasesRewriteSkillFS embeds the test-cases-rewrite orchestrator skill tree. +// +//go:embed all:skills/test-cases-rewrite +var TestCasesRewriteSkillFS embed.FS + +// TestCasesRewriteSkillRoot is the path prefix used inside TestCasesRewriteSkillFS. +const TestCasesRewriteSkillRoot = "skills/test-cases-rewrite" + +// TestCasesRewriteSkillZip lazily builds a deterministic zip archive from the +// embedded test-cases-rewrite directory. +var TestCasesRewriteSkillZip = sync.OnceValues(func() ([]byte, error) { + return buildZipFromFS(TestCasesRewriteSkillFS, TestCasesRewriteSkillRoot) +}) + // -------------- 3D Artist -------------- //go:embed icons/avatar-3d-artist.svg diff --git a/backend/internal/embeddata/skills/android-tester/Makefile b/backend/internal/embeddata/skills/android-tester/Makefile index e53ace0..c8da284 100644 --- a/backend/internal/embeddata/skills/android-tester/Makefile +++ b/backend/internal/embeddata/skills/android-tester/Makefile @@ -1,4 +1,4 @@ -.PHONY: install install-project install-adb package +.PHONY: install install-project install-adb test package install: install-project install-adb @@ -12,6 +12,9 @@ else sh scripts/install-adb.sh endif +test: + uv run --with pytest pytest tests + package: ifeq ($(OS),Windows_NT) powershell -ExecutionPolicy Bypass -File scripts/package-skill.ps1 diff --git a/backend/internal/embeddata/skills/android-tester/README.md b/backend/internal/embeddata/skills/android-tester/README.md index 1aef3bf..a90a0d5 100644 --- a/backend/internal/embeddata/skills/android-tester/README.md +++ b/backend/internal/embeddata/skills/android-tester/README.md @@ -38,134 +38,137 @@ Priority (highest wins): CLI arg > env var > `config.env` > built-in default. ## Usage -The CLI has two subcommands: - -- `android-tester run` — run a single test instruction on one device. -- `android-tester batch` — run a set of test cases from a JSON document, sharded across multiple devices (one async worker per device). - ```sh -android-tester run --device-id --instructions "" - -android-tester batch --file --devices [ ...] +android-tester --device-id --instructions "" ``` -### Common arguments (both subcommands) +### Arguments -| Argument | Required | Default | Description | -|---|---|---|---| -| `-o`, `--output-dir` | no | `./output/` (run) / `./output` (batch) | Output directory. In `batch`, each task gets a `` subdirectory under this root. | -| `--sico-endpoint` | no | `SICO_ENDPOINT` env var | Sico platform base URL | -| `--sico-app-name` | no | `sico` | Sico application name used to construct API paths | -| `--sico-agent-instance-id` | no | `SICO_AGENT_INSTANCE_ID` env var | Agent instance ID for X-Sico-Context header | -| `--llmhub-model` | no | `gpt5.4` | LLM model identifier | -| `--llmhub-model-image-size` | no | — | LLM perceived image size as `WIDTHxHEIGHT` (e.g. `1024x768`). When set, action coordinates are rescaled to match the actual screenshot size | -| `--model-auto-resize-width` | no | `768` | Target width (in pixels) for downscaling screenshots before sending them to the LLM. Set to `0` to disable. Only takes effect when `--llmhub-model-image-size` is unset. | -| `--log-level` | no | `WARNING` | Logging level (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`) | -| `--telemetry` / `--no-telemetry` | no | enabled | Enable or disable telemetry collection | -| `--reflector` / `--no-reflector` | no | disabled | Enable or disable the reflector step after each action | -| `--max-no-progress-steps` | no | `6` | Stop after this many steps without progress | -| `--max-repetitive-actions` | no | `5` | Stop after this many identical consecutive actions | -| `--n-retries-if-failed` | no | `0` | Re-run the whole pipeline up to this many additional times on failure | - -### `run` arguments +Argument values such as `--instructions`, `--task-name`, `--precondition`, and `--keep-app-state` are literal CLI strings, not paths to workspace files. | Argument | Required | Default | Description | |---|---|---|---| -| `--device-id` | yes | — | ADB device serial or `host:port` (e.g. `10.0.0.5:5555`) | -| `--instructions` | yes | — | Natural-language test instruction to execute | -| `--task-id` | no | auto-generated UUID | Unique task identifier | -| `--task-name` | no | — | Human-readable label for the test run | -| `--device-name` | no | same as `--device-id` | Friendly device name used in logs | - -### `batch` arguments - -| Argument | Required | Default | Description | -|---|---|---|---| -| `--file` | one of `--file`/`--test-cases` | — | Path to a JSON file with test cases. Use `-` to read JSON from stdin. | -| `--test-cases` | one of `--file`/`--test-cases` | — | Inline JSON document with the same shape as `--file`. | -| `--devices` | yes | — | One or more ADB device serials or `host:port` entries. One async worker is spawned per device; cases are pulled from a shared queue. | - -#### Test-cases JSON format +| `-o`, `--output-dir` | no | `./output/` | Directory for output files (screenshots, logs, report). | +| `--device-id` | yes | — | ADB device serial or `host:port` (e.g. `10.0.0.5:5555`). | +| `--instructions` | yes | — | Natural-language test instruction to execute. | +| `--task-id` | no | auto-generated UUID | Unique task identifier. | +| `--task-name` | no | — | Human-readable label for the test run. | +| `--device-name` | no | same as `--device-id` | Friendly device name used in logs. | +| `--precondition` | no | — | One atomic precondition in `label: description` form; the `label` is short, lowercase, hyphenated. **Repeatable** — supply once per precondition. On first use of a label, the precondition is established from its description and a reusable script is recorded under `/preconditions/