From e52a8ea16bf155192fb8d67763dd1c7e12106bda Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=94=98=E9=9C=96?= Date: Thu, 15 Oct 2026 11:20:17 +0800 Subject: [PATCH 1/5] docs(rfc): propose source plugin framework --- docs/en/rfcs/0000-source-plugin-system.md | 1120 +++++++++++++++++++++ docs/zh/rfcs/0000-source-plugin-system.md | 1120 +++++++++++++++++++++ 2 files changed, 2240 insertions(+) create mode 100644 docs/en/rfcs/0000-source-plugin-system.md create mode 100644 docs/zh/rfcs/0000-source-plugin-system.md diff --git a/docs/en/rfcs/0000-source-plugin-system.md b/docs/en/rfcs/0000-source-plugin-system.md new file mode 100644 index 000000000..5e94fb50b --- /dev/null +++ b/docs/en/rfcs/0000-source-plugin-system.md @@ -0,0 +1,1120 @@ +--- +title: Source plugin framework +description: Develop and use custom Source plugins to extend data types, Projections, and collection logic with a common PowerContext integration. +--- + +- Proposal Name: `source_plugin_framework` +- Start Date: 2026-10-14 +- RFC PR: Not assigned + +# Summary + +This proposal introduces a Source plugin framework for PowerContext so users can develop or use plugins to define custom Source types, data models, and Projections, as well as collection logic for different data origins, without modifying PowerContext core. It supports two data-ingestion paths: `write` runs plugin-defined validation and transformation inside Server and synchronously saves complete content supplied by the caller; `collect` uses independently running plugins to continuously read external resources, with collection targets added and managed at runtime. The framework handles plugin loading, lifecycle management, and the submission pipeline, reusing existing Source persistence rules and authorization; plugins decide what to collect, when to run, and how to retry. Applications and other hosts use the public MCP to discover and use built-in and custom Source capabilities through a common interface. + +This RFC specifies proposed behavior. Examples of interfaces and deployment commands do not imply availability in a released version. + +**Example conventions:** Commands, configuration, and code illustrate intended usage and interface relationships rather than final released syntax. Proposed interface names, module paths, and configuration layouts may be adjusted during implementation. Changes to public interfaces or usage steps must be explained in the implementation PR and reflected in this RFC; runnable usage is documented in the user guide and examples shipped with the implementation. Responsibility boundaries, permissions, persistence, synchronous returns, and repeat-call semantics are design constraints; changing them requires a rationale and design review in the PR. Existing public interfaces remain subject to their established contracts. + +# Motivation + +Data brought into PowerContext has different business meanings. Data from different business domains varies in structure, Source identity rules, and Projection requirements; built-in Source types cannot cover every requirement. If every customization requires a core change, business extensions depend on PowerContext's own release and maintenance schedule. Users need to develop Source plugins independently or configure and use plugins supplied by others. + +Defining a data type alone does not complete data integration. The same business data may come from systems with different authentication, APIs, and update patterns. Plugin authors need to implement the reading, filtering, or incremental collection their data origin requires and choose execution timing according to external access limits and business requirements. They should also be able to reuse PowerContext's submission and progress handling instead of reimplementing that common logic in every plugin. + +Users also obtain data in different ways. An application may already hold complete content that only needs saving, or resources may remain in an external system whose changes need continuous collection. Both needs should share Source data definitions and Projections. Saving existing content should complete directly; ongoing collection needs to run independently and continue after the application or host that initiated it exits. + +Collection scope changes during use. For example, a user may add a newly created document to a running collection instance or have the same instance serve another authorized Scope. Users should be able to add targets, inspect existing submissions, and stop or resume collection while reusing the instance and its progress, without deploying a separate plugin instance for every resource or Scope. + +These capabilities need to serve business applications, automation workflows, coding agents, and other hosts. Custom Sources should have discovery and invocation consistent with built-in capabilities, and adding a plugin should not require a separate integration implementation for every host. A common entry point and shared collection instances let users reuse existing plugins and continue using the same data integration capabilities across applications and workflows. + +# 1. Overview + +Hosts use built-in and custom Source capabilities through **the same public MCP on PC Server**, with different execution paths for the two operations: + +- **write**: the host already has complete content. PC Server uses loaded SourceDefinition and Projection implementations to save it synchronously and return the result; no plugin instance or management service is started. +- **collect**: plugins continuously read external resources. A shared background management service manages targets, and each plugin instance executes in its own background Runtime process. By default, Server startup discovers instances in a configured directory and automatically starts the local independent management service and Runtimes. Multiple hosts share instances. Remote Runtimes can connect to the same management service, which can also run on another machine. + +Both paths reuse SourceDefinition and Projection contracts. The same Source type can be produced by host writes and plugin collection. + +**The framework provides SourceDefinition (output type, identity, and Projection contracts) and the Source saving pipeline; collect plugins decide what to collect, when to execute, when to submit, and when to exit.** collect reuses the existing Worker and checkpoint pipeline, with batch execution invoked by the plugin. write validation and transformation execute within the current Server call. + +One management service serves one Server. Multiple instances, Scopes, and execution machines are supported; management clusters, automatic migration, general-purpose task scheduling, and in-process Python code replacement are out of scope. This document specifies the target design. Appendix A describes directory discovery and automatic startup. + +# 2. Architecture, connections, and storage + +## Component organization + +```mermaid +flowchart TB + H[Host and optional host integration] + subgraph PC[PC Server process] + M[Public Source MCP] + W[Loaded write Definition and Projection] + S[Source and Projection storage / checkpoint] + M -->|write| W + W --> S + end + G[Independent management service] + subgraph RT[Runtime process per collect instance] + R[Target synchronization and lifecycle] + A[Plugin apply and Connector] + C[PluginContext.run_batch] + K[Existing RemoteConnectorWorker] + R -->|targets and control state| A + A -->|select a batch| C + C --> K + K -->|Connector.run| A + end + H --> M + M -->|collect management| G + G -->|synchronize| R + K -->|Source submissions and checkpoint| S +``` + +Arrows represent business calls or data flow. write implementations run inside the Server process; collect Context, Worker, and user Connector execute inside the corresponding Runtime process. The management service delivers targets and control state without relaying Source data or scheduling batches. Management messages are delivered in responses to polls initiated by the receiver; network connection directions are described below. + +| Component | Responsibility and location | +| --- | --- | +| Public Source MCP | PC Server's `/mcp`; authenticates hosts, checks Scope permissions, and unifies discovery, submission, queries, and enable/disable operations | +| PC Server data pipeline | Loads custom write Definition and Projection code and registers Definitions; saves Source / Projection and checkpoint data and provides reads. Built-in and custom write use the Server saving pipeline directly | +| Shared management service | Independent background process; stores instances, collection targets, and results; executes Server-authorized management requests; checks Scope ownership and execution conditions; synchronizes collect capabilities and instance state to Server | +| Runtime | Outer framework in each collect instance process. A stable instance_id identifies an account, configuration, and execution environment that can serve multiple authorized Scopes. Loads the plugin, maintains management connectivity, synchronizes targets and control state, and creates and ends business runs | +| PluginContext (Context below) | Context for one business run, passed as ctx to apply; owns the Connector and business resources and provides the SDK (interfaces plugins call), including run_batch, submission, and cleanup | +| RemoteConnectorWorker | Existing execution pipeline; called by `run_batch()` to create a session, execute the Connector, and handle submission receipts and checkpoint CAS | +| apply / Connector | Plugin business code; apply decides timing, while Connector.run(session) reads or transforms data and submits it | +| Host-specific plugin | Optional adapter for Hooks, Scope defaults, and context injection; does not host Source instances or duplicate public MCP management logic | + +Only Server is needed for built-in capabilities or custom write. N collect instances add one management service and N instance processes; Server startup prepares these independent services. System services host background processes. Installation/startup commands return when finished, and closing Codex does not stop collection. + +## Startup loading and adding instances at runtime + +**write: load implementations local to Server.** The deployer prepares trusted plugin code and dependencies on the Server machine and configures an entry. Server's generic loader imports it, reads `definitions` and write capability declarations, validates input, Definition, Projection, and name/version conflicts, and exposes capabilities only after SourceDefinition registration succeeds. Deployment requires `server.admin`; the CLI triggers loading, but the Server process imports the code. Registering SourceDefinition structures and contracts through an API does not transfer Python functions and cannot establish that write is executable. + +Server saves successfully loaded entries and required configuration references, then imports and validates them again on restart. Explicit additions and repeated loading of identical configuration are supported; differing configuration conflicts. Existing implementations are replaced through the restart/update procedure in section 5, without in-process Python replacement. Interfaces and examples appear in section 4. This path creates no collect instance, Runtime, or target. + +**collect: load an instance and start an independent Runtime.** The following configuration and deployment flow apply to collect. + +The deployer specifies a trusted directory through Server's `source_plugins_dir`. Each `powercontext.yml` declares an instance: instance ID, code entry, plugin business configuration, and external credential references. Its `id` becomes the runtime `instance_id`, for example `yuque-personal`. It is unique within a PC deployment and stable across restarts, and is not a code name, Scope ID, or Source ID. Bundled code does not execute merely because it is installed; putting instance configuration in this directory authorizes loading. Different instance IDs can reuse the same code without copying it. + +`powercontext.yml` may provide initial `targets`, each containing `scope_id`, a capability, and business input. On first startup, if targets have not been initialized for this instance, deployment adds them to the management service through normal add rules and marks the instance initialized. Subsequently, management targets and control state are authoritative. File targets do not overwrite or supplement them or participate in instance configuration conflict detection. An empty target set does not trigger reimport, nor does management connection failure imply an uninitialized instance. + +Server's deployment flow scans the directory once at startup and supports explicit instance loading at runtime. It prepares independent management and Runtime services through the local service manager and generates and saves their connection configuration. Behavior is summarized below; Appendix A gives usage steps. + +| Situation | Behavior | +| --- | --- | +| No local instances and management service not enabled | Run only Server; explicitly loading the first local instance later starts management on demand | +| First valid instance configuration | Start one management service as needed, validate code contracts and configuration, register Definitions and the instance, then start business execution. Preparation failures show their reason without blocking built-in functionality | +| New instance while Server is running | Prepare code, dependencies, and complete `powercontext.yml`, then explicitly load it. Placing files alone does not start it; existing instances are not restarted | +| Server restart or repeated load of the same plugin instance ID | Reuse registration and process by configured id. Identical configuration preserves control intent and does not call apply again in a surviving Runtime. Required process restarts follow section 5; differing configuration conflicts and requires its update procedure | +| Directory removed or all instances stopped | File deletion is not unload and does not delete targets, history, or the management database. An already started management service keeps waiting; instance disablement is explicit | + +Stopping Server does not implicitly uninstall independent services or delete their state. Submissions fail or await recovery while Server is unavailable, subject to section 5. Remote code and Runtime deployment belong to the execution machine; local loading does not perform remote shell execution or code distribution. + +## Component communication and connection directions + +| Initiator | Receiver | Content | +| --- | --- | --- | +| Host | PC Server | `/mcp` and existing data-read interfaces | +| Management service | PC Server | Establish the management channel, register capabilities, receive management requests, and return results | +| Runtime | Management service | Instance authentication, resource identity resolution, target synchronization, control state and collection result reports | +| Runtime | PC Server | Worker Source submissions and checkpoint reads/updates | +| Plugin business code | External system | Read resources and check external business permissions | + +The management channel uses **HTTP polling and reporting initiated by the management service**: it receives pending messages from Server and posts results back. Appendix C lists paths. Thus Server hands requests to management at the business level, while management always initiates network requests to Server. Server need not connect to management's listening address; Runtimes must still be able to reach it. + +After connecting, management sends Server its complete collect capability list: names, versions, input requirements, associated instances, instance states, and public scope configuration needed for discovery. This is the capability catalog; management synchronizes changes proactively. Server merges it with built-in and locally loaded write capabilities and answers capability and business-instance discovery directly under the caller's PC permissions. It does not ask management on every discovery. collect add, target queries, control views, and enable/disable operations still use the management channel. The catalog does not replace operation-time authorization and execution checks. write executes locally on Server. + +## Persistent data and storage ownership + +| Owner | Storage and contents | +| --- | --- | +| Management service | Maintains `/management.sqlite3` without a separate database service. Stores instances, capabilities, collect targets, control state, and collection results; user authorization remains in PC Server | +| Runtime machine | Keeps only startup files: plugin code, startup configuration, and credential references. Target lists, current batches, and receipts are in process memory | +| PC Server | Existing data storage holds final Source, Projection, and checkpoints; also saves successful write code entries and required configuration references for restart. Code and dependencies remain on Server's filesystem; imported objects are in memory. Repeated write uses existing Source storage rules | + +Management SQLite uses two tables: + +| Table | Contents | +| --- | --- | +| `records` | Deployment bindings, instances, capability declarations, collect targets, and received submission records (Source references and timestamps). Scope ownership, creator records, and control state belong to their corresponding records | +| `follow_identity` | Maps stable collect resource identities to target_id to reuse targets on repeated add | + +write inputs and responses are used only within the current synchronous call. Server persists the resulting Source and Projection. + +**Management stores what to collect; PC Server stores what has been collected; Runtime performs current execution.** Management persists running collect targets, synchronized by Runtime on startup and reconnection. Initial file targets are not a replica of running state. Management owns desired instance/target control state; PC Server owns Source and checkpoint data. Management stores the latest received execution state and collection results for host queries. + +Runtime process exit loses in-memory work. Restart reruns apply if enabled, using Server checkpoint data to determine subsequent collection positions. Plugins may persist data for their own business needs. + +Source submission and result reporting to management are separate actions. A lost report does not remove a saved Source, but management may lack its submission record; the framework does not guarantee repair after restart. Management and Server databases need durable storage; Runtime needs only its startup files. System service managers start and stop processes without storing collection business state. + +# 3. Public MCP: unified host usage + +Hosts connect only to PC Server's `/mcp`. Public MCP provides capability discovery, Source writes, target management, and instance controls. Host-specific plugins provide connection, identity, and Scope context without connecting directly to management or Runtime. + +This section describes the public MCP's **proposed interfaces**, explaining each method's required inputs, complete call/response flow, and result and error meanings. JSON illustrates proposed request and response formats; requests use the MCP `tools/call` parameter structure. Replace Scope, instance, and target IDs with real values. Method and field names may change during implementation review with corresponding RFC updates; authorization checks and behavior remain governed by the design below. + +## Objects to understand before calling + +| Object | Meaning | +| --- | --- | +| capability | Callable business operation with name, version, description, input_schema (business parameter structure), and kind. It is not a Source type; collect may produce several Source types, while write associates its input with a Definition | +| kind | Operation type declared by a capability: write synchronously saves content the host already has; collect accepts an external location or selection criteria and establishes ongoing collection. The capability declaration determines the operation type | +| instance_id | ID of the collect plugin instance, such as yuque-personal; represents an account, configuration, and execution environment and may serve multiple authorized Scopes. write does not use it | +| target_id | Persistent collect relationship ID for queries, stopping, and resuming; not a Source ID | +| SourceRef | Reference to a saved Source, with source_type and source_id. Use it with this call's scope_id to read through existing Server data interfaces | +| enabled | Desired target/instance control state; not confirmation that the execution side completed the transition | +| capability_version | Version of the capability's input and behavior contract, selected from discovery; separate from the output SourceDefinition version | + +Invalid parameters, insufficient permissions, and processing failures return errors through the original MCP call. Steps that have not passed their checks are not executed. + +Server authenticates the host on every call and applies operation-specific PC permissions; scope_id specifies data ownership. For collect, management checks target Scope ownership, instance service range, and execution conditions. Plugins handle external accounts, credentials, and business permissions during actual access. + +Server answers capability and business-instance discovery directly. Calls requiring management use this handoff: + +**Management requests share one handoff mechanism:** Server delivers a message in a response to management's `channel/poll`; management processes it and posts to Server's `channel/reply`, after which Server completes the corresponding MCP call. The framework matches the exchange with a temporary message_id, which hosts neither supply nor retain. “Server → management” below describes message flow, not a reverse network connection. Runtime also initiates its management connections. Appendix C explains communication details. + +| Method | When hosts use it | +| --- | --- | +| `pc_source_capabilities` | Discover available operations, inputs, and executable instances | +| `pc_source_add` | Write existing content or add an ongoing collection target | +| `pc_source_status` | Read collect target results or control state | +| `pc_source_target_set_enabled` | Stop or resume one collect target | +| `pc_source_instances` | Discover instances or read instance control state | +| `pc_source_instance_set_enabled` | Enable or disable an instance's entire business run | + +## pc_source_capabilities: discover capabilities and input requirements + +Call when the host needs to choose a capability, understand input, or confirm available instances. Discovery only reads management information; it does not load code, start apply, or access external resources. + +**Parameters:** scope_id and operation; business discovery here uses `operation="add"`. capability and instance_id can narrow results. + +```json +{ + "name": "pc_source_capabilities", + "arguments": {"scope_id": "scp_demo", "operation": "add"} +} +``` + +**Call-to-response flow:** + +1. The host calls Server. Server checks Scope read permission and determines whether the caller can contribute. Discovery must not claim writable access when it is absent. +2. Server reads built-in capabilities, loaded custom write capabilities, and the collect catalog synchronized by management. It filters using public visibility, instance Scope ranges, and current PC permissions. This discovery sends no management query. +3. write availability depends on the local implementation being loaded and ready; collect uses catalog instance state and management connection availability. Visible offline or disabled instances may still appear, marked unavailable. +4. Server returns directly through the original MCP call, without channel/reply. Results contain only public information, not instance Config or external credentials. + +**Host result:** capability name, version, kind, description, input_schema, PC permissions, and availability; collect additionally includes visible instances, a suggested default, and running state. Permissions and running state are separate: write permission does not imply Runtime connectivity; connectivity does not imply apply is processing work or external resources are accessible. + +The host selects a capability and calls `pc_source_add`. collect also requires explicit instance_id. A default is only a discovery suggestion; retries must not select another account. write selects no instance. The chosen capability version is fixed as well. capability_version prevents upgrades from interpreting original calls and persistent collect targets under incompatible input rules. The same version must preserve inputs, operation meaning, and resource identity rules. + +Without management configured, built-in and loaded custom write capabilities remain discoverable. Management disconnection marks collect unavailable for new management requests rather than failing all discovery or reporting stale cached states as executable. Unavailability does not imply Runtime stopped existing collection. Discovery reflects call-time state; add checks again. + +## pc_source_add: write content or add a collection target + +**Parameters:** scope_id, capability, capability_version, and input; collect also requires instance_id, while write does not. Input follows the discovered input_schema; capability kind determines the operation. + +Server checks current `scope.contribute` and `scope.read`. write requires the capability version and associated Definition/Projection implementations to be loaded and validated on Server. New collect work also requires registered capability/version, completed SourceDefinition deployment validation, a valid Runtime management connection, enabled=true, and recently confirmed actual=running. A connected outer Runtime whose apply has ended cannot accept new collection work. + +### write: the original call waits for Source persistence + +For example, a host with existing text saves it through built-in `content.inline / 2`: + +```json +{ + "name": "pc_source_add", + "arguments": { + "scope_id": "scp_demo", + "capability": "content.inline", + "capability_version": "2", + "input": {"source_id": "meeting-note-001", "text": "Text to save"} + } +} +``` + +source_id is the stable identity of this data and remains unchanged on repeated submission. Custom write uses the same tool with its public capability and business input, without instance_id. + +```mermaid +sequenceDiagram + participant H as Host + participant S as Server / Source MCP + participant D as Loaded implementation + participant DB as Source storage + H->>S: pc_source_add(scope, capability, version, input) + S->>S: Authenticate / authorize / validate capability + S->>D: Validate input / resolve Source / project + D-->>S: Source and Projection values + S->>DB: Validate output and save using existing identity rules + DB-->>S: Actual save result + S->>S: Recheck read permission + S-->>H: SourceRef and save result, or error +``` + +**Call-to-response flow:** + +1. The host submits complete content. Server authenticates, checks Scope permissions, capability version, and input requirements, then locates the associated loaded Definition. +2. Server validates input through the Definition input model, calls `resolve()` to form a Source with stable identity, then calls Projection `project()` implementations. Custom functions come from the loaded plugin module; adding a type needs no type-specific import in Server code. +3. Server validates output contracts and saves Source and Projection through existing write rules. Identical identity and data reuse storage; identical identity with different data conflicts. +4. Server obtains the actual persistence result, checks current read permission, and returns SourceRef through the original MCP call. Input, transformation, or storage failures return errors. The call waits for the actual result rather than returning acceptance first; no host query is required. + +Both built-in and custom write execute this flow inside Server; custom implementations must be loaded first. write accepts complete content and does not continuously fetch remote resources. + +**Successful response example:** + +```json +{ + "kind": "write", + "result": { + "outcome": "saved", + "source_refs": [{"source_type": "content", "source_id": "meeting-note-001"}] + } +} +``` + +SourceRef comes from actual persistence receipts. saved means required writes completed; unchanged means storage confirmed an identical Source. Hosts read content with SourceRef through existing Server data interfaces. + +**Repeated calls and failures:** write reuses `(scope_id, source_type, source_id)` rules: identical identity and data reuse; identical identity with different data conflicts; different IDs create distinct Sources. The underlying submitter owns stable identity: built-in adapters provide it, and custom plugins derive it through their Definition. If a host must supply a resource ID, input_schema says so. Repeated calls may rerun validation or transformation; there is no exactly-once guarantee for external side effects. + +Timeout, disconnection, or Server exit can leave the result unconfirmed and do not prove nothing was saved. Server cleans up current processing when it detects an invalidated call; an already started save may succeed, and reconnection does not automatically replay it. Retries retain the same identity and actual submitted data, without new random IDs or changing metadata. Lost responses do not roll back saved data. + +### collect: return the collection relationship, then continue processing + +For example, add newly created Yuque document B to the running yuque-personal instance: + +```json +{ + "name": "pc_source_add", + "arguments": { + "scope_id": "scp_demo", + "capability": "yuque.document", + "capability_version": "1", + "instance_id": "yuque-personal", + "input": {"url": "https://www.yuque.com/team/book/doc-b"} + } +} +``` + +**Call-to-response flow:** + +1. Server checks host identity, Scope permissions, and input, then hands the call and trusted caller identity to management through the channel. +2. Management invokes plugin `identify_target(request, config)` through Runtime to obtain target_key (stable resource identity) and normalized_input (full normalized input including collection options). This entry is independent of apply and resolves identity from input and instance configuration. +3. Management looks up “Scope + instance + capability and version + target_key.” Matching identity/configuration reuses a target; differing configuration returns target-config-conflict. New targets pass instance Scope-range and readiness checks before target and identity index are persisted together. Uniqueness constraints handle concurrent duplicates. Creator identity is recorded but does not determine target identity or management authorization. +4. Management returns the target through reply. Subject to current read permission, Server completes the original MCP call with target_id, tracking state, and existing collection results. This confirms the relationship, not that external content has been saved. + +**First-creation response example:** + +```json +{ + "kind": "collect", + "target_id": "tgt_b", + "enabled": true, + "tracking": "enabled", + "source_refs": [], + "last_submitted_at": null +} +``` + +**Collection after the response:** Runtime synchronizes targets from management. Existing apply obtains B through a later `ctx.targets.list()` and chooses its own batches and timing. Worker executes Connector; plugin credentials read Yuque, and data is submitted directly to Server. SDK associates targets and Sources using actual successful receipts. Runtime reports Source references and timestamps to management for persistence. Reports describe received submissions. Worker handles batches and checkpoints under existing rules. Hosts may call `pc_source_status` for progress; querying neither triggers nor sustains collection. + +Adding B does not restart the instance or recreate A's business client. Repeated add of an existing target does not reset progress, trigger extra collection, or resume stopping/stopped targets, and always requires current PC permissions. Identity confirmation requires a reachable Runtime, but reusing an existing target does not require apply to be running. If identity cannot be confirmed, return an error rather than guessing a new target. Management can still query or stop an existing target_id. + +If the acceptance response is lost, repeating the original Scope, instance, capability version, and business input returns the original target. Within one Scope, different users reuse the same instance/capability-version/resource identity with identical configuration. Different Scopes, instances, or capability versions do not merge. Stopped targets retain their identities. This version provides neither hard target deletion nor automatic merging of historical duplicates. + +## pc_source_status: query collection results or control state + +This method queries collect targets and submission records saved by management. + +**Parameters:** scope_id and target_id. The default view reads collection state; `view="control"` reads only control fields for subsequent control operations. + +```json +{ + "name": "pc_source_status", + "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b"} +} +``` + +**Call-to-response flow:** + +1. Server authenticates the host. The default view requires current scope.read; control requires scope.contribute. +2. After checking Scope permissions, Server forwards the query through the management channel. Management verifies actual target Scope matches the requested Scope; mismatches are rejected without revealing another Scope's target. +3. Management reads its target and received collection records without identify_target, a reachable Runtime, or waiting for another Worker batch. +4. Management replies to Server, which completes the original MCP call with permitted results. Collection data still requires current read permission; control responses include only authorized control fields. + +**Host result:** + +| View | Returned content and meaning | +| --- | --- | +| Default | target_id, enabled, tracking, source_refs, last_submitted_at, latest confirmation time, and instance availability. source_refs contains received actual submission associations; last_submitted_at is their latest submission time. Without records, these are an empty list and null | +| control | target_id, enabled, tracking; no input, SourceRef, or business error body. scope.contribute permits this view without additionally requiring scope.read | + +tracking represents enabled / stopping / stopped for the collection relationship. source_refs and last_submitted_at reflect confirmed, reported submissions, not complete collection. Without new submissions, they do not distinguish unprocessed, absent, or unchanged content. Submission reports can be lost, so this view is not a complete Server data inventory. An offline instance retains past records but shows offline / unknown availability; past submissions do not establish current collection. + +The call is repeatable without business side effects. Unreachable management returns unavailable, not a fabricated latest state from Server cache. Authorized hosts can still read Sources already saved on Server. + +## pc_source_target_set_enabled: stop or resume a target + +**Parameters:** scope_id, target_id, and enabled. Hosts express stop/resume intent directly. + +```json +{ + "name": "pc_source_target_set_enabled", + "arguments": { + "scope_id": "scp_demo", + "target_id": "tgt_b", + "enabled": false + } +} +``` + +**Call-to-response flow:** + +1. Server checks scope.contribute for the target Scope and forwards the call through the management channel. +2. Management verifies target Scope ownership and, when resuming, the instance service range, then saves enabled while preserving target identity, existing Sources, and checkpoint. Setting the current state again creates no action. +3. Management returns control state through Server to the original MCP call. Runtime synchronizes target control state and applies it to collection. + +The response confirms persisted intent, not completed execution. Resuming a target does not enable a disabled instance or rerun an ended apply. + +## pc_source_instances: discover instances or read instance control state + +This method manages collect instances. Server-local write capabilities have no associated instance. + +capabilities starts from “what can be done”; instances starts from “which instances are available and how is this instance running.” It loads no code and starts no instance. + +**Two parameter combinations:** + +| View | Parameters | Purpose | +| --- | --- | --- | +| Business | scope_id, operation="add" | Find visible instances serving this Scope, PC permissions, and running state | +| Control | instance_id, view="control" | Read authorized control state for an instance without specifying Scope | + +```json +{ + "name": "pc_source_instances", + "arguments": {"scope_id": "scp_demo", "operation": "add"} +} +``` + +```json +{ + "name": "pc_source_instances", + "arguments": {"instance_id": "yuque-personal", "view": "control"} +} +``` + +**Call-to-response flow:** + +1. Server authenticates. Business view checks scope.read and determines current write permission; control view checks server.admin. +2. Server directly answers business view from synchronized public visibility, Scope ranges, and instance state without asking management. Catalog and connection handling match capabilities. +3. After server.admin authorization, Server forwards control queries to management without additional Scope permission requirements. Management reads persistent instance control state and latest execution reports, replies to Server, and Server returns to the host. Neither view reruns apply or accesses external accounts. + +**Host result:** business view returns instance IDs, capabilities, PC permissions, and running state. Control view returns instance_id, enabled, actual, availability, and latest update time, without business Config, target inputs, or Source results. + +Desired enabled=true does not guarantee actual=running. Normal apply completion can be actual=complete; unhandled errors can be failed, while outer Runtime remains online. Control view distinguishes desired and actual state; enabled alone must not be described as collecting. + +Calls are read-only and repeatable. Business view uses Server's synchronized public information; invalid management connectivity marks the custom collect portion unavailable. Old catalogs do not guarantee current permission or executability, and records are withheld when visibility cannot be confirmed. Control view requires reachable management. If only Runtime is offline, persistent control state remains readable but running availability is unknown/offline. Built-in and custom write need no plugin instances. + +## pc_source_instance_set_enabled: enable or disable instance business execution + +This method controls apply / Context inside Runtime, affecting all Scopes and targets served by the instance. It is not a command to install, uninstall, or start a remote system process. + +**Parameters:** instance_id and enabled. Hosts express intent directly. + +```json +{ + "name": "pc_source_instance_set_enabled", + "arguments": { + "instance_id": "yuque-personal", + "enabled": false + } +} +``` + +**Call-to-response flow:** + +1. Server authenticates and checks server.admin before forwarding instance control. An instance may serve multiple Scopes; one Scope's permission cannot control the entire instance. +2. Management checks instance existence and transition conditions, then transactionally sets desired state in processing order. Same-state requests return current results. Disable saves enabled=false and, if execution confirmation is outstanding, stopping. Enable requires the old run confirmed stopped, then saves enabled=true, creates a new run_id (business-run identity), and waits for Runtime execution. +3. Management replies with persistent control state; Server returns it to the host. Fields such as instance_id, enabled, and actual do not mean the system process was installed or business execution started successfully. +4. Runtime synchronizes and applies desired state. Disable blocks new batches, cancels and awaits apply and owned subtasks, and closes Context/business resources. It reports stopped only after completion; outer Runtime stays online. Enable creates a new Context, calls apply, and reports actual state. Hosts may use instances control view for confirmation; querying does not drive transitions. + +An offline Runtime permits saving disable intent but not reporting completed stopping. Cleanup failure or surviving old tasks prevents creating a second Context. Management disconnection prevents confirmation of control changes. Re-enabling processes only still-enabled targets; it does not resume individually stopped targets or reset their checkpoints. + +Within one Runtime process, repeated enable while already enabled does not rerun apply, even after complete or failed. Rerun requires explicit disable, confirmed stopped, then explicit enable. After Runtime is shut down or uninstalled, management cannot remotely start its system process. See sections 2 and 5 for deployment. + +## Operation retries + +Repeated collect add reuses targets; repeated write reuses underlying Source rules; controls directly set desired state. Setting the same state creates no action, and different intents take effect in processing order. Communication timeout does not roll back saved data or control changes. Hosts may retry or read state according to each method's rules. + +# 4. Plugin interfaces and data pipeline + +This section is for plugin authors. write provides input models, Definition, and Projection implementations that Server can load. collect additionally provides apply and Connector, controls its own timing, and submits through Worker. Section 3 describes host calls; section 5 describes loading, process controls, and restart behavior. + +## Contracts provided by plugins + +Plugin code can load from a local directory without publishing a wheel. write code and dependencies reside on the Server machine; collect code and dependencies reside on the Runtime machine. Both may reuse Definition and Projection implementations from one package without splitting packages. + +**Shared data contracts.** SourceDefinition declares its input model, Source model, name, and version. `resolve(value)` converts input to a Source with stable identity; `project(source)` on each implementation in `projections` produces its Projection. `definitions` exports these objects. Capability declarations contain name, version, description, kind, and input_schema and describe callable operations; Definitions describe how results are saved. One collect capability can submit several Source types, and several capabilities can reuse one Source type. + +**write loading contract.** Each write capability explicitly associates a Definition name and version, with input_schema matching its input model. Server can validate input and call the associated Definition directly. Plugins export `definitions` and `capabilities` for Server to load. Capability names cannot override built-ins; identical name/version combinations must preserve input and operation semantics. + +The following **interface sketch** illustrates how Server obtains the Projection function by relating plugin exports to Server loading. Concrete class names, module paths, and supporting code may change during implementation; write association fields and deployment entry support remain to be implemented. Suppose a deployer prepares this importable local package: + +```text +/srv/powercontext/custom_sources/tickets/ + __init__.py + ticket_definition.py +``` + +`ticket_definition.py` implements TicketCapture, TicketDefinition, and TicketProjection. The key relationship is shown below, omitting the existing Source model and resolve implementation: + +```python +class TicketProjection: + # Also declare Projection name, version, input Source class, and output model. + def project(self, source): + return TextEvidence( + source_type="example-ticket", + source_id=source.name, + content=f"{source.capture.title}\n\n{source.capture.body}", + ) + + +class TicketDefinition: + name = "example-ticket" + version = "1" + input_class = TicketCapture + projections = (TicketProjection(),) + # resolve() produces TicketSource with stable identity; other members follow SourceDefinition. +``` + +The plugin's own `__init__.py` exports objects. Type-specific imports belong in the plugin: + +```python +from .ticket_definition import TicketCapture, TicketDefinition + +definitions = (TicketDefinition(),) +capabilities = ({ + "name": "ticket.write", + "version": "1", + "kind": "write", + "description": "Save ticket content supplied by the host", + "input_schema": TicketCapture.model_json_schema(), + "definition_name": "example-ticket", + "definition_version": "1", +},) +``` + +Server's configured entry points to this package, imported by a generic loader. This shows only the imported object relationship, not a complete loader: + +```python +module = importlib.import_module(entry) # For example tickets, from trusted deployment configuration. +registry = SourceDefinitionRegistry(module.definitions) +# registry.resolve / registry.project call these actual Python objects at runtime. +``` + +Server can execute TicketProjection.project because **it imported the local module containing that object**. Adding a type adds a module and entry configuration, not a hardcoded type-specific Server import. The deployer makes code available on Server's module search path and installs dependencies in that environment. The generic loader also resolves file-path entries. Implementations validate and transform current input without starting collection tasks during import. + +**Registering SourceDefinition and loading code are separate actions.** The existing registration API saves type structure, identity, and Projection contracts, not Python implementations. Server's write loader validates code, contracts, and conflicts and publishes capabilities only after successful registration. Registration alone cannot execute write. collect still registers Definitions with `server.admin` during deployment or upgrade. Daily Worker execution uses only Scope execution permissions, without registering each batch or temporarily acquiring admin credentials. Missing or mismatched Definitions produce explicit errors. + +**Additional collect runtime contracts:** + +| Export or object | Plugin definition | +| --- | --- | +| `plugin_api_version` | Interface version used with Runtime, currently the string `"2"` | +| `Config` | Pydantic model defining and validating configuration needed by plugin logic, such as external credentials or credential references | +| `capabilities / definitions` | Nonempty collect capability declarations and SourceDefinition collection; capability input describes external locations or selection criteria | +| `connector_name / connector_version` | Stable Connector name and compatibility version for checkpoint ownership; cannot change randomly on restart | +| `identify_target(request, config)` | Derives stable resource identity and normalized collection parameters from input | +| `apply(ctx, config)` | Async business entry; creates clients, connects a Connector, and decides batches, waits, retries, and exit | +| `Connector.run(session)` | Processes and submits the batch, returning ConnectorRunCompletion (batch completion and next checkpoint) | + +collect Config remains with the instance. Plugins handle external credentials and permissions. Capability descriptions and input_schema expose only what hosts need to call them; catalogs, results, and ordinary logs do not expose secrets. Same-version collect capabilities must also preserve resource identity rules. + +**Identifying the same resource for collect:** Runtime invokes identify_target during target acceptance, independently of apply's collection loop. The request contains capability, capability_version, and input. The plugin returns: + +| Result | Meaning | +| --- | --- | +| target_key | Stable resource identity used by management to reuse a target | +| normalized_input | Complete normalized collection parameters, preserving all options that affect collection, for detecting configuration conflicts on the same target | + +For example, a plugin may remove URL fragments used only for page navigation and identify two addresses as the same target. identify_target resolves resource identity from input and instance configuration; content and readability are handled during actual collection. input_schema must require sufficient identifiers, and equivalence must not be promised when it cannot be established. + +## collect: from apply to Source persistence + +`ctx` is the PluginContext passed by Runtime to apply. It holds the Connector and business resources and exposes the SDK below. One Context connects exactly one Connector. Worker creates `session` per batch, providing its Scope, current checkpoint, and submission capability. + +One apply can execute multiple batches. **One `run_batch()` is one batch, fixed to one Scope and a selected set of content.** A collect target may continue producing Sources over many batches. SDK associates targets with actual submissions; hosts do not generate batch IDs. + +The diagram shows one batch's call order. apply and Connector on the left are plugin code; SDK and existing RemoteConnectorWorker are in the middle, and PC Server is on the right. Arrows indicate calls/returns, not process counts. It shows successful collect submission, and submit can be called repeatedly. binding identifies this instance's execution within a Scope; isolation is explained at the end of this section. + +```mermaid +sequenceDiagram + participant A as Plugin apply + participant C as Plugin Connector + participant SDK as PluginContext / SDK + participant W as RemoteConnectorWorker + participant S as PC Server + A->>SDK: targets.list() + SDK-->>A: Current target snapshot + A->>SDK: run_batch(scope_id, targets) + SDK->>W: Run batch Connector and binding + W->>S: Read checkpoint + S-->>W: Current checkpoint + W->>C: run(session) + C->>C: Read external resources + C->>SDK: targets.submit(target, session, value) + SDK->>S: session.submit / Source and Projection + S-->>SDK: Actual submission receipt + SDK-->>C: ConnectorItemOutcome + Note over SDK: Associate target and SourceRef for Runtime reporting + C-->>W: ConnectorRunCompletion + W->>S: CAS checkpoint under completion and receipt rules + W-->>SDK: Batch result + SDK-->>A: Batch result + Note over A: Plugin decides wait, next batch, or exit +``` + +apply obtains work, selects entries for one Scope, and calls run_batch. Worker reads checkpoint and invokes Connector.run; Connector reads the external system and transforms data as needed, then submits through SDK. The plugin decides waiting and subsequent batches. + +| Step | collect interface | +| --- | --- | +| Obtain currently eligible work | `ctx.targets.list()` | +| Execute the selected batch | `ctx.run_batch(scope_id=..., targets=selected)` | +| Submit one item | `ctx.targets.submit(...)` | + +list returns a target snapshot for current processing: + +| Field | Meaning | +| --- | --- | +| target_id | Corresponding target persisted by management | +| scope_id, instance_id, capability, capability_version | Ownership and capability contract | +| input | Normalized collection parameters confirmed by identify_target | +| source_refs | Actual submission associations received by management, available for plugin reference; not proof of complete collection | + +After initial synchronization, ctx.targets.list / run_batch may use the last in-memory target/control snapshot while management is disconnected. Without an initial snapshot, they report unavailable rather than pretending there is no work. New targets appear after synchronization. + +Batches in one Context execute sequentially without overlap. Each selects targets from one Scope without concurrently modifying its snapshot. Managed mode explicitly passes the selected set; autonomous collection uses an explicit empty set, and omission does not mean all targets. Direct execution retains `run_batch(scope_id=...)`. + +submit accepts the current entry, session, batch-unique item_id, Definition name, and business value. SDK checks instance, Scope, and batch ownership, wraps `session.submit(...)`, and associates the actual receipt with that entry. Returned ConnectorItemOutcome describes the submission result; plugins must check acceptance. Processed data goes directly to PC Server rather than through management. + +Actual collect submission associations are reported to management for persistence. Appendix B shows full apply and Connector code. + +## collect receipts and batch completion + +Plugins maintain loops, pagination, and retries and submit through `ctx.targets.submit(...)`. Submission receipts describe persistence results; ConnectorRunCompletion describes batch completion. An ongoing collection target may produce Sources across multiple batches. + +| Action | Meaning | +| --- | --- | +| submit receives a successful receipt | Server accepted one item; SDK associates its actual SourceRef with the collect target | +| Connector returns ConnectorRunCompletion | Declares batch completion and next checkpoint; Worker validates and processes progress under existing rules | +| collect submission report reaches management | Saves confirmed Source references and timestamps for optional host queries, without waiting for the whole target or other targets in that batch to finish | + +Without new submissions, the framework does not infer whether a target is unprocessed, unchanged, or empty. Plugins decide retries and continuation; unhandled exceptions end apply as described in section 5. Source saving, checkpoint updates, and reporting are separate: incomplete batches or lost reports do not undo saved Sources, and reports are not guaranteed to be restored after restart. + +## Checkpoint isolation, advancement, and new targets + +ConnectorBinding describes which instance's Connector executes in which Scope. This design uses instance_id as binding_id. PC Server stores checkpoints by `(scope_id, binding_id)` and verifies connector_name / connector_version. ConnectorBinding differs from the permission system's role-granting Scope Binding. + +```mermaid +flowchart TB + subgraph PC[PC Server persists checkpoints] + A["Scope 1 + instance A: checkpoint"] + B["Scope 2 + instance A: checkpoint"] + C["Scope 1 + instance B: checkpoint"] + end + T1[Instance A / Scope 1 / target 1] --> A + T2[Instance A / Scope 1 / target 2] --> A + T3[Instance A / Scope 2 / target 3] --> B + T4[Instance B / Scope 1 / target 4] --> C + A --> P[Plugin-defined format may include per-target progress] +``` + +Server stores every checkpoint in the diagram; plugins define the content format. Multiple targets or capabilities in the same instance and Scope share one checkpoint. The framework does not automatically create a cursor per target; plugins may record individual target progress inside it. Different instances or Scopes do not share checkpoints. + +Worker reads checkpoint at batch start, and Connector returns the next value based on business progress. Updates use CAS: save the new value only if Server still has the previously read value, preventing overwrites of concurrent progress. Plugins must not advance past unprocessed data. + +For example, A has been collected through time T when document B, older than T, is added. The plugin must not skip B using A's cursor. It recognizes the new target, initializes B's progress, and backfills it using a plugin-defined representation. Stopping/resuming targets does not automatically clear checkpoints; changing Connector versions cannot bypass compatibility checks on existing progress. + +After Runtime process restart, targets are resynchronized and Worker reads Server's current checkpoint. Plugins decide whether to continue or recollect; the framework does not restore individual steps from the previous batch. + +# 5. Lifecycle, restarts, and permissions + +Runtime, instance, and target lifecycle in this section applies to collect; apply still controls collection timing. write implementations load and release with Server without separate instance controls. New implementations can be loaded explicitly, and identical repeat loads reuse registration. Existing code replacement takes effect through controlled Server restart. Saved entries are revalidated on startup; failed capabilities are not marked executable, while other valid capabilities remain usable. + +## Runtime connectivity and apply execution + +In managed background mode, each instance has one Runtime process. Runtime maintains management connectivity and synchronizes targets/control state. **Context is the Connector, clients, and business tasks owned by one apply run.** Business waits or sleep do not block Runtime from receiving management operations. + +Instances distinguish three things: + +| Information | Meaning | +| --- | --- | +| enabled | Desired control state stored by management; true does not mean apply is currently running | +| actual | Latest reported business state, such as running, normal completion (complete), failure (failed), stopping, or stopped | +| Connection availability | Whether Runtime can communicate; disconnected means offline / unknown, not that an old actual state proves ongoing execution | + +**Background Runtime remains online after apply ends.** It can receive controls but does not rerun apply because a target is added, connectivity recovers, or enable is repeated. Plugins that continuously handle new targets keep their own loop inside apply. + +```mermaid +flowchart TD + S[Runtime starts and syncs instance state and targets] --> E{enabled?} + E -->|No| I[Remain online waiting] + E -->|Yes, old process exited| R[Create Context and run apply] + R -->|Normal return| C[Clean up Context, report complete] + R -->|Unhandled error| F[Clean up Context, report failed] + C --> I + F --> I + R -->|Disable| D[Cancel and await tasks, close resources] + I -->|Disable| D + I -->|Enable from disabled, old run confirmed stopped| R + D -->|Cleanup complete| T[Confirm stopped, Runtime stays online] + T -->|Explicit enable| R + I -->|New target, reconnect, or enable while already enabled| I + D -->|Cleanup failed| U[Report error, no second Context] +``` + +Uninstalling the system service exits Runtime; starting it again requires a deployment operation on the execution machine. + +To rerun an ended apply within the same process, explicitly disable, confirm stopped, then enable. Direct `pc-source run` exits when apply ends; direct and background execution cannot occupy the same instance identity simultaneously. + +Management saves desired state, latest actual state, and update time. Internal run_id distinguishes apply reports; only the current connection/run's messages are accepted so late reports cannot overwrite newer state. Hosts do not supply run_id, and Runtime does not use it for cross-process business recovery. + +## Instance disablement and resource cleanup + +To disable an instance, management saves enabled=false. Runtime blocks new batches, cancels and awaits apply and its subtasks, and closes Context. After cleanup, it reports stopped; outer Runtime remains online awaiting enable. + +Business subtasks must belong to apply, for example through TaskGroup. Resources registered with `ctx.enter_async_context` / `ctx.on_dispose` close with Context. Plugins cannot leave detached background tasks or swallow cancellation. If tasks remain or cleanup fails, do not report stopped or create a second Context. + +Instance disablement does not affect loaded write capabilities or undo saved Sources. An offline Runtime cannot be claimed stopped; unreachable management cannot confirm persistence of new control intent. + +## Disconnections, restarts, and data recovery + +**Reconnection preserves process memory; process restart creates a new business run.** Server or management restart is not Runtime restart. The system service manager maintains processes, not Worker invocations. + +| Situation | Subsequent behavior | +| --- | --- | +| Runtime lacks an initial target snapshot | collect list / run_batch reports unavailable; plugin waits or exits. Process restarts also need targets first | +| Synchronized Runtime loses management | collect may start new batches using the last in-memory target/control state on plugin timing. Server still checks submission permissions; reports may be lost | +| Same Runtime process reconnects | Synchronize latest instance/target control before further processing. Apply synchronized control state; do not rerun active or ended apply | +| Runtime process restarts | In-memory batches and undelivered reports are lost. After synchronization, if enabled, create Context and execute apply even after previous complete / failed. Worker reads Server checkpoint | +| Only management-to-Server channel fails, while Runtime still connects to both | Host collect management is unavailable. Existing collection may continue under synchronized state. Built-ins and loaded custom write remain independently usable | + +Do not start another Runtime when the old process's exit cannot be confirmed. Heartbeat loss is not grounds for cross-machine takeover. apply and external side effects are not guaranteed exactly once; repeat submissions use underlying Source identity rules. + +Runtime retains startup files only, with execution information in memory. Section 2 defines storage ownership. Recovery uses management targets/control state and Server Sources/checkpoints. + +collect **Source saving, checkpoint update, and submission reporting** are three separate actions; section 4 describes normal processing. Failures are handled separately: + +| Failure point | Recovery and effect | +| --- | --- | +| Source saved before checkpoint update | Later collection may repeat; identical identity/data reuse storage. Plugins maintain stable Source identity and handle changed data according to their Definition | +| Checkpoint update response lost | Read actual Server progress next time. Do not claim this update confirmed, but actual successful submission receipts may still be reported | +| Source and checkpoint saved, report lost | Data and progress remain valid; management may lack submission records. Do not roll back checkpoint, and do not guarantee report repair after restart | + +Missing management records do not prove Server lacks Sources; authorized hosts can still read through Server data interfaces. collect continues on plugin timing without restoring previous batch steps. Server restart interrupts in-flight write. After restart implementations reload, but old calls are neither resumed nor replayed. Saved Sources remain; retries retain the same identity and data. + +## Adding instances, configuration changes, and upgrades + +| Change | Handling | +| --- | --- | +| New instance | Prepare code/dependencies; deployment registers Definitions and instance and starts Runtime. Publish new capabilities without restarting existing instances | +| Add a Scope to a running instance | Grant its execution identity the appropriate Scope Binding and update service range. After Runtime sync, hosts can add targets in that Scope while preserving Context | +| Change business configuration or code | Disable and confirm stopped, stop old Runtime, then validate and update code, Config, and registration. Start the new Runtime and explicitly enable. External credential changes use the same configuration-update flow; plugins interpret their business meaning | + +Scope Binding is Server's relation granting an identity a role in a Scope, not section 4's checkpoint ConnectorBinding. Passing a new scope_id grants no permissions and does not require a separate plugin instance per Scope. + +Confirm old Runtime exit before updating collect code/configuration; retain Sources, targets, and checkpoints. For write, first handle in-flight Server calls, then load new code through controlled restart, preserving capability/Definition compatibility and Source identity without migrating old calls. collect upgrades check three compatibility dimensions: + +| Retained data or contract | Requirement | +| --- | --- | +| Target capability/version, input, and identity rules | New code can still handle registered targets, including stopped ones; a v2-only entry cannot take over v1 targets directly | +| SourceDefinition, ConnectorBinding, and checkpoint format | Remain compatible with existing Sources and progress; do not bypass checks by changing versions or resetting cursors | +| Targets and received submission records | Preserve records and Scope ownership under current PC permissions; do not require business reruns to recreate history | + +If compatibility cannot be established, remain disabled without deleting records or resetting progress. Deployers can supply compatible code or explicitly address incompatible targets and progress; the first version provides no automatic migration, and a new instance does not automatically inherit old targets. + +After validation, the deployer updates registration through configure while keeping it disabled, then explicitly enables the new Runtime after successful validation. Failure leaves it non-executable without creating a second Context. Ordinary repeated startup/loading only reuses identical configuration; conflicting configuration must fail explicitly rather than substitute for the update procedure. Appendix C lists management interfaces. + +## Permission responsibilities and operation conditions + +**Reuse PC Server's existing Access authorization.** Principal is the user or service identity authenticated by Server. Hosts cannot claim another identity in input. + +| Component | Responsibility | +| --- | --- | +| PC Server | Authenticate hosts and check current Scope/Server permissions before forwarding management requests. Check write persistence under the host identity, every collect submission under its execution identity, and current read permission when returning data | +| Management service | Accept only Server management requests over authenticated channels; verify actual target Scope, instance service range, and execution conditions | +| Plugin | Interpret private configuration/external credentials and handle business permissions when accessing external systems; identify_target resolves identity only and does not authorize | + +| Operation | PC Server permission | +| --- | --- | +| Capability and business-instance discovery | scope.read; indicate whether scope.contribute is currently present | +| write and new/repeated collect | scope.contribute plus scope.read because responses contain data or existing Source references | +| Query collect targets/submissions or read Sources | scope.read | +| collect control view, target stop/resume | scope.contribute; only control fields returned, without additional scope.read | +| Load Server-local write; instance control view, loading, configuration, unload, or whole-instance enable/disable | server.admin; actual process operations still belong to deployment tools on the execution machine | +| Grant users/execution identities Scope permissions | Existing Access authorization rules | + +All scope.contribute holders can control that Scope's targets, not only creators; creator identity is informational. An instance may serve multiple Scopes, so whole-instance control requires server.admin, not one Scope's permission. Instance control responses contain no business Config, target inputs, or Source data. + +Server forwards the authorized operation together with Scope, target_id, or instance_id. Management verifies actual ownership to prevent using an authorized Scope to address another Scope's target; this adds no new authorization model. Management channels and instance connections still authenticate. Runtime can synchronize only its own targets and report its own state, not impersonate Server management commands. + +Permission changes/revocation occur only in PC Server Access. Current read permissions remain checked while write waits and before returning. Revoked read permission prevents returning results but does not imply an accepted save did not occur. Continuous collect uses its own PC execution identity; creator permission changes do not automatically disable existing targets. Normal controls stop targets, and Server checks execution permission on every submission. + +On external access refusal, the collect plugin decides to wait, retry, or end apply. Server returns write validation, resolve, project, or storage errors through the original call. + +## Credentials and initial authorization + +AuthenticationProvider verifies Server credentials and maps them to Principal. Local deployment may create or reuse execution identities through a deployment-controlled Provider. Externally managed Servers keep their own Provider; administrators pre-provision identities where needed. Deployment does not overwrite authentication configuration or promise automatic credential issuance. + +| Credential | Purpose | Location | +| --- | --- | --- | +| Deployment | Load write, register Definitions, create Scopes, grant roles, and configure instances | Deployment secret storage, not distributed to Runtime | +| Host | Call Server as the host user/service identity | Host secure configuration | +| Management channel | Authenticate management to Server on its dedicated channel | Management service | +| PC execution | Per-instance Principal submits to authorized Scopes and handles checkpoints | Execution machine, without server.admin | +| Instance connection | Runtime proves instance identity to management for its own work/reports | Execution machine | +| External system | Plugin accesses Yuque, ticket systems, etc. | Plugin-managed storage/interpretation; Config may reference files, environment, or secret services | + +write loading requires admin permission, but daily writes use the current host identity without creating another execution identity or credential. collect deployment uses separate Bearer credentials mapped by Provider to explicit identities. Before collect business execution, deployment: + +1. Validates Server, deployment identity, permissions, and target Scope. Confirms actual host/execution Principals rather than treating credential-file existence as validation. +2. Registers SourceDefinition with deployment credentials and creates/reuses execution Scope Bindings through existing Access interfaces. Daily execution needs neither Definition registration nor server.admin. +3. Configures management channel identity, registers execution location and served Scopes, reuses Server Access for management authorization, and supplies Runtime only required credentials/references. +4. Checks actual execution identity and Scope permissions before allowing work in that Scope. An unauthorized instance may be registered but cannot accept that Scope's collection. Host MCP connectivity/authorization follows host integration procedures. + +These steps use existing `POST /v1/source-definitions/register`, `POST /v1/access/bindings/create`, `GET /v1/access/me`, and `POST /v1/access/check`, without another registration or OAuth system. + +Automatic local deployment generates internal configuration without bypassing authorization. Deployment credentials do not reach collect Runtime. Only administrators load trusted write code, while daily writes do not borrow deployment permissions. Credential values are excluded from command arguments, ordinary logs, and capability catalogs. Rotation uses Provider and secret references; invalid identities cannot remain in use. Cross-machine communication uses TLS or equivalent authenticated secure entry points. + +# Appendix A: usage scenarios—startup and adding resources dynamically + +These scenarios first show collect user operations and component interactions by deployment location, then write loading/calls. For local collect, Server startup scanning and `source load` prepare shared management and Runtime without separately running both `install` commands. For cross-machine deployment, deployers prepare full configuration, start management, complete registration/authorization, then start remote Runtime. Service installation commands here use Linux `systemd --user`; Server must already have authentication configured and PC Access enabled. + +Commands and configuration in this appendix are **usage illustrations** showing deployment locations, operation order, and component interactions. Final commands and arguments are documented in the user guide shipped with the implementation. Run shell commands on the indicated machine. JSON uses MCP `tools/call` parameters for section 3's **proposed interfaces**, sent by the host to Server. Replace example Scopes, paths, and document URLs with real values. + +## Scenario 1: start Codex, Server, and plugins on one machine + +**1. Prepare the instance directory on the Server machine.** Assume bundled code, directory `/srv/powercontext/source-plugins/yuque-personal`, and external credentials kept separately in a protected file. `powercontext.yml`: + +```yaml +id: yuque-personal +plugin: yuque +config: + token_file: /srv/powercontext/secrets/yuque.token + base_url: https://www.yuque.com + poll_interval: 60 +# Optional targets for first initialization. +targets: + - scope_id: scp_demo + capability: yuque.document + capability_version: "1" + input: + url: https://www.yuque.com/team/book/doc-a +``` + +The plugin defines `config`, including business parameters such as external credential references. Target Scopes must exist, and both initialization caller and execution identity need appropriate permissions. Initial targets use normal add rules; management becomes authoritative after initialization. Later file changes do not overwrite or supplement targets, and neither empty lists nor connection failures trigger reimport. Omit targets if Scopes are not ready; complete Scope authorization/service-range expansion below, then add through public MCP. + +**2. Configure the plugin directory and start Server in the background.** Keep existing database, authentication, and deployment settings in `pc.env`, adding: + +```bash +POWERCONTEXT_SERVER_SOURCE_PLUGINS_DIR=/srv/powercontext/source-plugins +``` + +From the PowerContext repository root: + +```bash +uv run powercontext service install --env-file ./pc.env +``` + +`service install` installs and starts Server's background service, then returns. Startup performs the following local deployment without separate `manager install` or `runtime install`: + +| Order | Components and interactions | +| --- | --- | +| 1 | Server deployment scans trusted directories, reads instance configuration, and prepares local service configuration and internal identity references | +| 2 | Start/reuse independent management; management initiates its authenticated Server channel | +| 3 | Validate plugin declarations, register Definition/instance with deployment permission, and confirm execution Scope permissions and served range | +| 4 | Start Runtime, connecting to management and Server; Runtime creates Context and executes apply, with capabilities and instance state synchronized to Server | +| 5 | If targets are not initialized, import file targets through normal add after instance readiness. Plugin obtains them through ctx.targets.list() and decides collection timing | + +Deployment permissions come from Server deployment configuration, not daily Runtime. Existing instance restarts do not reimport targets; management retains runtime additions and stop/resume state. Built-in and custom write need no management service when no collect instances exist. + +**3. Install host integration and connect to the same Server.** If Codex is installed but PowerContext integration is not, run from repository root: + +```bash +uv run powercontext setup codex --source . --server-url http://127.0.0.1:8000 +POWERCONTEXT_CODEX_SCOPE_ID='scp_demo' codex +``` + +Prepare host credentials through existing host authentication. The second command starts a new Codex session from this shell. Desktop hosts use their own integration configuration and establish a new session; they do not inherit another terminal's environment. Verify the session Scope through `resolve_scope_binding` and explicitly pass the same Scope to Source MCP calls below. + +Hosts access only Server `/mcp`, not management or Runtime directly. Closing Codex does not stop background plugins; adding a Source plugin does not require reinstalling host integration. + +## Scenario 2: collect a new document through a running instance + +Assume `yuque-personal` already collects A and needs document B added. + +**1. Codex creates B through an authorized Yuque tool and obtains its actual URL.** For example, with Yuque CLI: + +```bash +yuque create doc --namespace team/book --title 'Document B' --body-file /tmp/doc-b.md --yes --json +``` + +Public Source MCP neither creates Yuque documents nor monitors all external tool operations. + +**2. Codex discovers capabilities and instances.** + +```json +{ + "name": "pc_source_capabilities", + "arguments": {"scope_id": "scp_demo", "operation": "add"} +} +``` + +Server returns capabilities, versions, schemas, and instance state based on current PC permissions and synchronized catalog. Choose the returned `yuque.document` version and `yuque-personal` instance. + +**3. Codex adds B.** + +```json +{ + "name": "pc_source_add", + "arguments": { + "scope_id": "scp_demo", + "capability": "yuque.document", + "capability_version": "1", + "instance_id": "yuque-personal", + "input": {"url": "https://www.yuque.com/team/book/doc-b"} + } +} +``` + +Server checks current Scope permissions and forwards to management. Runtime invokes identify_target to confirm resource identity; management creates/reuses the target and returns target_id. The response establishes ongoing collection, not that B is saved. Repeating the same Scope, instance, capability version, and resource identity reuses the target without resuming stopped targets. + +**4. The running plugin obtains and processes B.** Runtime synchronizes targets; apply reads B through `ctx.targets.list()` and calls `run_batch()` on its own schedule. Worker executes Connector, which reads Yuque and submits directly to Server. SDK automatically associates actual receipts, and Runtime reports Source references/timestamps to management. + +**5. Call query/control tools only when the user needs results or wants to stop tracking.** Queries are not needed to sustain collection: + +```json +{ + "name": "pc_source_status", + "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b"} +} +``` + +source_refs and last_submitted_at describe confirmed and reported submissions, not complete collection of B. Stop B using the actual returned target_id: + +```json +{ + "name": "pc_source_target_set_enabled", + "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b", "enabled": false} +} +``` + +Any scope.contribute holder can stop/resume Scope targets, not only their creator; reading submissions requires scope.read. The response confirms saved intent; a disconnected Runtime may still collect from its last synchronized state. Resume uses the same target_id and enabled=true. Repeated add is not resume. + +## Scenario 3: add plugins, Scopes, and remote execution + +**Add a local custom plugin instance.** Prepare code, dependencies, and an independent instance configuration: + +```text +/srv/powercontext/source-plugins/tickets-team/ + powercontext.yml + plugin.py +``` + +```yaml +id: tickets-team +plugin: ./plugin.py +scope_ids: [scp_demo] +config: + api_base: https://tickets.example.org + token_file: /srv/powercontext/secrets/tickets.token +``` + +Business parameters must be declared by the plugin's Config. The instance ID is deployment-unique, and a local entry needs no published wheel. Once Server is running with the plugin directory configured, execute on the same machine using its `pc.env`: + +```bash +uv run powercontext source load --config /srv/powercontext/source-plugins/tickets-team/powercontext.yml --env-file ./pc.env +``` + +`load` uses protected local deployment credentials, prepares/reuses shared management, registers Definitions and the instance, configures execution Scope permissions, generates RuntimeConfig, and installs/starts Runtime. First loading also imports optional initial targets. It calls the service-installation logic used by the two `install` commands, so this scenario needs only `load`. Background execution continues after the command returns, without restarting Server or existing instances. Prepare plugin dependencies beforehand. + +Adding files alone while running does not start an instance. Repeat loading neither overwrites initialized targets nor restarts an instance with identical configuration. Hosts then call `pc_source_capabilities` again and use the new input_schema with `pc_source_add`, without modifying the MCP tool list. + +The three plugin configuration files have different purposes: + +- `powercontext.yml`: user-authored instance configuration with ID, entry, business config, served Scopes, and optional initial targets; read by `source load` or startup scanning. +- `source-manager.yml`: management deployment configuration (ManagerConfig), including Server address/credential file, management ID, listening address, and durable state directory; read by `manager install`. +- `plugin-runtime.yml`: full deployment configuration for one plugin process (RuntimeConfig), including plugin directory, business config, Server/management addresses, both connection credential files, and configuration digest; read by `runtime install`. An instance powercontext.yml cannot substitute for it. + +`pc.env` is Server environment configuration used by load for connection addresses and local deployment directories. + +**Add a Scope to the same instance.** An administrator grants the execution identity permissions through existing Access and expands the instance service range through `POST /v1/source-management/configure`. Hosts also need corresponding PC permissions. Add targets with the new scope_id without recreating Context or editing initialized file targets. + +**Manual cross-machine deployment.** For example, machine A runs Server and management, while B can reach the ticket system and runs `tickets-team`. Codex connects only to A's public MCP. A's `source load` does not install code or start processes on B. + +Prepare reachable addresses and connection credentials. With deployment permissions, register Definitions on Server, register instance ID, execution location, configuration digest, and served Scopes with management, and grant execution Scope permissions through PC Access. B's code, dependencies, and RuntimeConfig must match registration. If management has not started, install it first as below, register the instance, then install Runtime. + +```bash +# Machine A: manually deploy shared management; skip if it already exists. +uv run powercontext source manager install --config ./source-manager.yml + +# Machine B: after registration and authorization, start the prepared Runtime. +uv run powercontext source runtime install --config ./plugin-runtime.yml +``` + +These commands only install/start their background services; they do not install dependencies or automatically register Definitions/instances or authorize identities. Deployment generates or prepares RuntimeConfig from that information. Current source load orchestrates local deployment without remote configuration distribution or SSH execution. + +Management initiates requests to Server; Runtime initiates requests to management, Server, and external sources. Cross-machine connections need reachable secure endpoints, not loopback-only defaults. Management may run on a third machine with the same responsibilities; each Server has one shared management service. + +**Custom write: load implementations, then save complete content directly.** For example, a host already has complete ticket content to save as an `example-ticket` Source: + +1. Prepare section 4's local `tickets` package and dependencies on Server's machine, importable from Server's Python environment. +2. Supply entry configuration and trigger Server loading with administrator permission. The configuration sketch is `entry: tickets`; loading validates/registers Definitions, saves entry references, and exposes `ticket.write / 1`. +3. The host connects to the same `/mcp`, discovers and submits below without an instance, target, or manager/runtime configuration: + +```json +{ + "name": "pc_source_capabilities", + "arguments": {"scope_id": "scp_demo", "operation": "add"} +} +``` + +After discovery confirms `ticket.write / 1` and its input model: + +```json +{ + "name": "pc_source_add", + "arguments": { + "scope_id": "scp_demo", + "capability": "ticket.write", + "capability_version": "1", + "input": { + "namespace": "team", + "project": "support", + "ticket_id": "T-100", + "title": "Login failure", + "body": "A user reports being unable to log in." + } + } +} +``` + +Server validates TicketCapture, calls TicketDefinition.resolve and TicketProjection.project, saves, and returns actual SourceRef through this call. Hosts read Source/Projection through the reference without status queries. This example's Definition derives stable identity from the complete ticket snapshot, so identical-content retries reuse Sources. Continuous ticket-change collection uses collect above; both paths may reuse the same Definition. + +# Appendix B: plugin collection pipeline reference + +This collect **interface sketch** relates identity confirmation, plugin loops, submissions, and checkpoints; concrete class names, module paths, and helper functions may change during implementation. Plugins implement `BusinessClient`, `DocumentDefinition`, `TemporaryReadError`, and Config external-credential fields. It is not a complete runnable client and does not replace SDK changes still to be implemented. + +Plugins declare `plugin_api_version = "2"`, Config, capabilities, definitions, and stable Connector name/version. Capabilities describe collect input/business contracts; identify_target confirms resource identity. + +For collect input `{"document_id":"A"}`, an identity entry can return: + +```python +async def identify_target(request, config): + # document_id is a nonempty business ID required by input_schema. + document_id = request.input["document_id"].strip() + if not document_id: + raise ValueError("document_id must not be blank") + return { + "target_key": document_id, + "normalized_input": {"document_id": document_id}, + } +``` + +This entry resolves resource identity. If there are additional collection options, normalized_input must preserve all parameters affecting collection. Omitting options must not equate different requirements. + +```python +import asyncio + +from powercontext.source_management import ManagementError +from powercontext.sources import ( + ConnectorRunCompletion, + ConnectorRunStatus, + ConnectorSubmissionStatus, +) + +plugin_api_version = "2" +connector_name = "document-reader" +connector_version = "1" +document_definition = DocumentDefinition() +definitions = (document_definition,) + + +class BusinessConnector: + name = connector_name + version = connector_version + source_definitions = frozenset({document_definition.name}) + + def __init__(self, ctx, client): + self.ctx = ctx + self.client = client + self.targets = () + + async def run(self, session): + batch_complete = True + for target in self.targets: + try: + document = await self.client.read(target.input) + if document is None: + # Confirmed absent content; network or permission errors must not become None. + continue + receipt = await self.ctx.targets.submit( + target, + session=session, + item_id=target.target_id, # One item per target in this example, unique within the batch. + definition_name=document_definition.name, + value=document, + ) + if receipt.status != ConnectorSubmissionStatus.ACCEPTED: + batch_complete = False + except TemporaryReadError: + # Retry in a later batch; do not swallow cancellation or other unhandled errors. + batch_complete = False + return ConnectorRunCompletion( + status=ConnectorRunStatus.COMPLETE if batch_complete else ConnectorRunStatus.INCOMPLETE, + checkpoint=session.checkpoint, # Full reads in this example, without advancing an incremental cursor. + ) + + +async def apply(ctx, config): + client = await ctx.enter_async_context(BusinessClient(config)) + connector = BusinessConnector(ctx, client) + ctx.connect(connector) + while True: + try: + targets = await ctx.targets.list() + for scope_id in sorted({target.scope_id for target in targets}): + selected = tuple(target for target in targets if target.scope_id == scope_id) + connector.targets = selected + await ctx.run_batch(scope_id=scope_id, targets=selected) + except ManagementError as error: + # Wait for an initial target snapshot; propagate other configuration/permission errors. + if error.code not in {"targets-unavailable", "management-unavailable"}: + raise + await asyncio.sleep(config.poll_interval) +``` + +SDK records actual submission associations automatically. Connector batch results serve the Worker/checkpoint pipeline; section 5 defines disconnection and restart behavior. + +Incremental/paginated plugins calculate checkpoints themselves and may maintain target-specific progress inside the instance/Scope checkpoint. New targets need backfill rather than being skipped because another target's cursor is newer. Repeated data must produce stable Source identity, with failure retries following existing saving rules. + +Section 5 defines full cleanup and stop behavior. write loading/calls are in section 4 and Appendix A. + +# Appendix C: management communication and instance configuration reference + +This appendix serves deployers and framework implementers. Hosts use public MCP; plugins use SDK to obtain targets and submit data. Paths and configuration fields below are **interface sketches**; connection directions, permissions, and state semantics remain governed by the main design. + +## Channels and identity + +Sections 2 and 3 define connection directions and request handoff; section 5 defines authentication and permission responsibilities. Management and Runtime each use their own credentials to connect upstream. + +| Interface | Purpose | +| --- | --- | +| Server `/v1/source-management/channel/open`, `catalog`, `poll`, `reply`, `close` | Authenticate management, synchronize catalog, receive/respond to messages. Once catalog is ready, Server answers discovery directly | +| Server `POST /v1/source-management/configure` | After server.admin checks, register or explicitly update instances through the management channel; does not upload arbitrary Python code | +| Management `POST /v1/source-management/instances/{instance_id}/sync` | Runtime authenticates with instance credentials and synchronizes its collect targets/control state; SDK handles pagination | +| Management `POST /v1/source-management/instances/{instance_id}/results` | Runtime reports actual Source references/timestamps for its instance. Duplicate reports reuse associations | + +On reconnection, management authenticates again and synchronizes its complete catalog. Server checks capability names, versions, kind, and input contracts for consistency with existing capabilities. Management message size, in-flight count, and waiting time are bounded. + +## Instance configuration and updates + +`registration` in `POST /v1/source-management/configure` is complete instance registration; `update=true` requests an explicit update. Section 2 defines initial target import; section 5 defines configuration updates and upgrades. + +Expanding an already authorized Scope range can preserve Context; shrinking the range or changing execution configuration follows the disable/update procedure. diff --git a/docs/zh/rfcs/0000-source-plugin-system.md b/docs/zh/rfcs/0000-source-plugin-system.md new file mode 100644 index 000000000..737b0365b --- /dev/null +++ b/docs/zh/rfcs/0000-source-plugin-system.md @@ -0,0 +1,1120 @@ +--- +title: Source 插件框架 +description: 支持用户开发和使用自定义 Source 插件，扩展数据类型、Projection 与采集逻辑，并统一接入 PowerContext。 +--- + +- Proposal Name: `source_plugin_framework` +- Start Date: 2026-10-14 +- RFC PR: 尚未分配 + +# 概要 + +本提案为 PowerContext 引入 Source 插件框架，支持用户开发或直接使用插件，自定义 Source type、数据模型与 Projection，以及面向不同数据来源的采集逻辑，无需修改 PowerContext 核心代码。框架支持两种数据接入方式：`write` 由 Server 执行插件定义的校验与转换，同步保存调用方已有的完整内容；`collect` 由独立运行的插件持续读取外部资料，并支持在运行中新增和管理采集目标。框架负责插件加载、生命周期管理和数据提交管道，复用已有的 Source 保存规则与权限体系；插件决定采集内容、执行节奏和重试策略。应用和各类宿主通过公共 MCP 统一发现和使用内置与自定义 Source 能力。 + +本文规定提议的设计行为。接口和部署命令示例不表示对应功能已经在发布版本中可用。 + +**示例约定：** 本文中的命令、配置和代码用于说明预期使用方式与接口关系，不代表最终发布的语法。实现过程中可以调整拟议接口的命名、模块路径和配置布局；涉及公共接口或使用步骤的调整，应在实现 PR 中说明并同步本文，可执行用法以随实现发布的用户指南和示例为准。职责边界、权限、持久化、同步返回和重复调用语义等属于设计约束；如需改变，应在 PR 中说明理由并接受设计评审。已有公开接口仍遵循其既有契约。 + +# 动机 + +用户接入 PowerContext 的数据具有不同的业务含义。不同业务的数据在结构、Source 身份规则和 Projection 需求上存在差异，内置 Source type 难以覆盖所有需求。如果这些定制都依赖修改核心代码，业务扩展就会受 PowerContext 自身的发布和维护节奏限制。用户需要能够独立开发 Source 插件，也能够直接配置和使用他人提供的插件。 + +定义数据类型还不足以完成数据接入。同一种业务数据可能来自不同系统，各自有不同的认证方式、接口和更新规律。插件作者需要根据来源实现所需的读取、筛选或增量采集逻辑，并根据外部系统的访问限制和业务要求决定执行节奏；同时，应能复用 PowerContext 的提交和进度处理能力，避免每个插件重复实现这些通用逻辑。 + +用户获取数据的方式也不同：有时应用已经持有完整内容，只需将其保存；有时资料仍在外部系统，需要持续获取后续变化。这两类需求应能共用相同的 Source 数据定义与 Projection。保存已有内容应当可以直接完成，持续采集则需要独立运行，不随发起操作的应用或宿主退出而中断。 + +采集范围会在使用过程中变化。例如，用户需要把新建文档加入正在运行的采集实例，或让同一实例服务另一个获授权的 Scope。用户应能新增目标、查看已有提交并启停采集，复用已有实例与采集进度，无需为每份资料或每个 Scope 单独部署一套插件。 + +这些能力需要同时服务业务应用、自动化工作流、coding agent 等不同宿主。用户使用自定义 Source 时，应有与内置能力一致的发现和调用方式；新增插件也不应要求每种宿主分别实现对接逻辑。通过统一的使用入口和共享的采集实例，用户能够复用已有插件，并在不同应用和工作流中持续使用同一套数据接入能力。 + +# 1. 方案概览 + +宿主通过 **PC Server 的同一个公共 MCP** 使用内置和自定义 Source 能力，两类操作采用不同执行方式： + +- **write**：宿主已有完整内容，PC Server 使用已加载的 SourceDefinition 和 Projection 实现，同步保存并返回结果；不启动插件实例或管理服务。 +- **collect**：插件从外部系统持续读取资料，由一个共享的后台管理服务管理目标，每个插件实例在自己的 Runtime 后台进程中执行。默认在 Server 启动时发现指定目录中的实例，自动启动本机独立管理服务和 Runtime；多个宿主共享实例。远端 Runtime 也可接入同一个管理服务，管理服务亦可异机部署。 + +两条路径复用 SourceDefinition 与 Projection 契约，同一种 Source type 可以同时由宿主写入和插件采集产生。 + +**框架提供 SourceDefinition（输出类型、身份与 Projection 契约）和 Source 保存管道；collect 插件决定采集什么、何时执行、何时提交以及何时退出。** collect 复用已有 Worker 与 checkpoint 管道，由插件调用批次执行。write 的校验和转换则在当前 Server 调用内执行。 + +一个管理服务只服务一个 Server。支持多实例、多 Scope、多执行机器；不做管理集群、自动迁移、通用任务调度或进程内 Python 代码热替换。本文规定目标设计；目录发现与自动启动的目标使用步骤见附录 A。 + +# 2. 架构、连接与存储 + +## 组件如何组织 + +```mermaid +flowchart TB + H[宿主和可选宿主集成] + subgraph PC[PC Server 进程] + M[公共 Source MCP] + W[已加载的 write Definition 和 Projection] + S[Source / Projection 存储与 checkpoint] + M -->|write| W + W --> S + end + G[独立管理服务] + subgraph RT[每个 collect 实例的 Runtime 进程] + R[目标同步与生命周期] + A[用户 apply 和 Connector] + C[PluginContext.run_batch] + K[已有 RemoteConnectorWorker] + R -->|目标与控制状态| A + A -->|选择批次| C + C --> K + K -->|Connector.run| A + end + H --> M + M -->|collect 管理| G + G -->|同步| R + K -->|Source 提交和 checkpoint| S +``` + +图中箭头表示业务调用或数据流向。write 的实现运行在 Server 进程内；collect 的 Context、Worker 和用户 Connector 均在对应 Runtime 进程内。管理服务交付目标与控制状态，不转存 Source 数据，也不调度采集批次。管理消息通过接收方主动发起的轮询响应交付，网络连接方向见下文。 + +| 组件 | 职责与运行位置 | +| --- | --- | +| 公共 Source MCP | 位于 PC Server 的 `/mcp`；认证宿主、检查 Scope 权限，统一发现、提交、查询和启停接口 | +| PC Server 数据管道 | 加载自定义 write 的 Definition 与 Projection 代码，注册 Definition；保存 Source / Projection 和 checkpoint，提供数据读取。内置和自定义 write 直接使用 Server 内的保存管道 | +| 共享管理服务 | 独立后台进程；保存实例、采集目标和结果，执行 Server 已授权的管理请求，检查 Scope 归属与运行条件，向 Server 同步 collect 能力和实例状态 | +| Runtime | 每个 collect 实例进程的外层框架；实例由稳定的 instance_id 标识，代表一份账号、配置和执行环境，可服务多个获授权的 Scope。Runtime 加载插件、维护管理连接、同步目标与启停状态，管理业务运行的创建和结束 | +| PluginContext（下文简称 Context） | 一次业务运行的上下文对象，作为 ctx 传给 apply；持有 Connector 和业务资源，提供 SDK（插件调用框架的一组接口），包括 run_batch、提交和清理 | +| RemoteConnectorWorker | 已有执行管道；由 `run_batch()` 调用，建立 session、执行 Connector、处理提交回执及 checkpoint CAS | +| apply / Connector | 插件业务代码；apply 决定节奏，Connector.run(session) 读取或转换数据并提交 | +| 宿主专属插件 | 可选适配层，仅负责 Hook、Scope 默认值和上下文注入；不托管 Source 实例、不复制公共 MCP 的管理实现 | + +只使用内置能力或自定义 write 时只需 Server。使用 N 个 collect 实例时增加一个管理服务和 N 个实例进程；Server 的启动流程负责准备这些独立服务。系统服务托管后台进程，安装或启动命令完成后退出，关闭 Codex 不会停止已启动的采集。 + +## 启动加载与运行时新增实例 + +**write：加载 Server 本地实现。** 部署者先在 Server 机器准备可信插件代码与依赖，并配置代码入口。Server 的通用加载器导入入口，读取其 `definitions` 和 write 能力声明，校验输入、Definition、Projection 与名称版本冲突，完成 SourceDefinition 注册后才公开能力。部署操作要求 `server.admin`；命令行只触发加载，真正导入代码的是 Server 进程。仅通过 API 注册 SourceDefinition 的结构与契约，并不会把 Python 函数传给 Server，不能据此宣称 write 已可执行。 + +成功加载的入口与必要配置引用由 Server 保存，重启时重新导入和校验。支持显式新增和一致配置的重复加载；配置不一致时报冲突，修改已有实现按第 5 节重启更新，不做进程内 Python 代码替换。接口和示例见第 4 节；此路径不创建 collect 实例、Runtime 或目标。 + +**collect：加载实例并启动独立 Runtime。** 以下配置与部署流程用于 collect。 + +部署者通过 Server 的 `source_plugins_dir` 指定可信插件目录。目录中的每个 `powercontext.yml` 声明一个实例：插件实例 ID、代码入口、插件业务配置和外部凭据引用。配置中的 `id` 对应运行时的 `instance_id`，例如 `yuque-personal`；它在同一 PC 部署内唯一，重启后保持不变，不是插件代码名称、Scope ID 或 Source ID。随包代码不会仅因“已安装”而执行；将实例配置放入该目录表示部署者允许加载它。同一份代码可由不同 ID 的配置复用，无需复制代码。 + +`powercontext.yml` 可选配置初始 `targets`（每项包含 `scope_id`、能力及业务输入）；首次启动时，若该实例尚未初始化目标，部署流程按正常添加规则写入管理服务，并在实例记录中标记已初始化。此后以管理服务中的目标和启停状态为准，文件中的 `targets` 不再用于覆盖、补充目标或判定实例配置冲突；目标为空不重新导入，管理服务连接失败也不视为未初始化。 + +Server 的部署流程在启动时扫描一次指定目录，也支持运行中显式加载实例；通过本机服务管理器准备独立管理服务和 Runtime，生成并保存所需连接配置。行为如下，具体操作见附录 A。 + +| 情况 | 行为 | +| --- | --- | +| 没有本地实例，也未启用管理服务 | 只运行 Server；之后显式加载首个本地实例时，按需启动管理服务 | +| 首次加载有效实例配置 | 按需启动一个管理服务，校验代码契约与配置、注册 Definition、登记实例，再启动业务；准备失败的实例显示原因，不阻塞内置功能 | +| Server 运行中新增实例 | 部署者准备好代码、依赖和完整的 `powercontext.yml` 后显式触发加载；只放入目录不会自动启动，已有实例不重启 | +| Server 重启或再次加载同一插件实例 ID | 按配置中的 id 复用已有登记和进程；配置一致时不重置启停意图；已有 Runtime 存活时不重复调用 apply，进程需要重启时按第 5 节启动规则处理；配置不同则报冲突，更新须走第 5 节流程 | +| 删除目录或所有实例停止 | 不把文件删除当作卸载，不删除目标、历史或管理数据库；已经启动的管理服务继续等待，实例停用按显式管理操作执行 | + +停止 Server 不隐式卸载独立服务或删除状态；数据提交在 Server 不可用时失败或等待恢复，仍遵守第 5 节的断线与重启规则。远端代码和 Runtime 由执行机器部署，本机加载不承担远程 shell 或代码分发。 + +## 组件通信与连接方向 + +| 发起方 | 接收方 | 通信内容 | +| --- | --- | --- | +| 宿主 | PC Server | `/mcp`、现有数据读取等接口 | +| 管理服务 | PC Server | 建立管理通道、注册能力、获取管理请求并回传处理结果 | +| Runtime | 管理服务 | 实例认证、资料身份确认、采集目标同步、启停状态和采集结果报告 | +| Runtime | PC Server | Worker 提交 Source、读取和更新 checkpoint | +| 插件业务 | 外部系统 | 获取资料、验证外部业务权限 | + +当前管理通道使用**管理服务主动发起的 HTTP 轮询和回报**：管理服务从 Server 领取待处理消息，再向 Server 回传结果；具体路径见附录 C。因此，业务上是 Server 把请求交给管理服务，网络上却始终是管理服务访问 Server。Server 无需连接管理服务的监听地址；管理服务仍须能被 Runtime 访问。 + +连接建立后，管理服务把自己管理的完整 collect 能力清单发送给 Server，包括能力名称、版本、输入要求、对应实例、实例状态与发现所需的公开范围配置。这份清单称为能力目录；发生变化时管理服务主动同步。Server 将该目录与本地内置、已加载的 write 能力合并，结合当前调用者的 PC 权限直接回答能力和业务实例发现，不为每次发现再询问管理服务。collect 的添加、目标查询、控制视图和启停仍经管理通道处理；目录不代替实际操作的权限与执行条件检查。write 由 Server 本地处理。 + +## 持久化数据与存储归属 + +| 保存方 | 存储与内容 | +| --- | --- | +| 管理服务 | 自己维护 `/management.sqlite3`，无需独立数据库服务；保存实例、能力、collect 目标、启停状态与采集结果；用户授权统一保存在 PC Server | +| Runtime 所在机器 | 仅保留插件代码、启动配置、凭据引用等启动所需文件；目标列表、当前批次和提交回执保存在进程内存中 | +| PC Server | 已有数据存储保存最终 Source、Projection 与 checkpoint；另外保存成功加载的 write 代码入口和必要配置引用，供重启恢复。代码与依赖留在 Server 文件系统，导入后的实现对象留在内存；write 的重复提交沿用已有 Source 存储规则 | + +管理服务的 SQLite 按以下两张表组织： + +| 表 | 保存内容 | +| --- | --- | +| `records` | 部署绑定、实例、能力声明、collect（采集）目标及已收到的提交记录（Source 引用、提交时间）；Scope 归属、创建者记录与控制状态保存在对应记录中 | +| `follow_identity` | collect 的稳定资料身份与 target_id 的对应关系，用于重复添加时复用同一目标 | + +write 的输入和应答仅在当前同步调用中使用，最终 Source 与 Projection 由 Server 持久保存。 + +**管理服务保存“需要采什么”，PC Server 保存“已经采到什么”，Runtime 负责当前执行。** collect 的运行目标由管理服务持久保存，Runtime 启动和重连时同步；文件中的初始 targets 不作为运行状态副本。实例和目标的期望启停状态以管理服务为准，Source 与 checkpoint 以 PC Server 为准；最近收到的执行状态和采集结果由管理服务保存，供宿主查询。 + +Runtime 进程退出时内存工作丢失；重启后若实例仍启用，就重新执行 apply，通过 Server 的 checkpoint 决定后续采集位置。插件可按业务需要自行持久保存数据。 + +Source 提交和向管理服务报告结果是两个独立动作。Source 已保存但结果报告丢失时，数据仍在 Server，管理服务可能缺少对应的提交记录；框架不保证重启后补齐。管理服务和 Server 的数据库须持久保存，Runtime 只需保留启动文件。系统服务管理器负责进程启停，不保存采集业务状态。 + +# 3. 公共 MCP：宿主如何统一使用 + +宿主只连接 PC Server 的 `/mcp`。公共 MCP 负责能力发现、Source 写入、采集目标管理和实例启停；宿主专属插件只提供连接、身份与 Scope 上下文，不直接连接管理服务或 Runtime。 + +本节描述公共 MCP 的**拟议接口**，按每个方法说明必要输入、从调用到返回的流程，以及结果和错误的含义。以下 JSON 是拟议请求与返回格式的示例，其中请求使用 MCP `tools/call` 参数结构；Scope、实例和目标 ID 均须替换为实际值。方法名和字段名可在实现评审中调整并同步本文，权限检查和行为语义仍按正文规定。 + +## 调用前需要理解的对象 + +| 对象 | 含义 | +| --- | --- | +| capability | 可调用的业务能力，包含名称、版本、说明、input_schema（业务参数结构）和 kind。它不是 Source type；collect 能力可以生成多个 Source type，write 能力关联其输入对应的 Definition | +| kind | 能力声明的操作类型：write 接收宿主已有内容，同步保存并返回；collect 接收外部资料位置或选择条件，建立持续采集关系。操作类型由能力声明确定 | +| instance_id | 执行 collect 能力的插件实例 ID，例如 yuque-personal；代表账号、配置和执行环境，可服务多个获授权 Scope。write 不使用 instance_id | +| target_id | collect 目标的持久 ID，标识持续采集关系，用于查询、停止和恢复；不是 Source ID | +| SourceRef | 已保存 Source 的引用，包含 source_type 和 source_id；结合本次 scope_id，通过 Server 已有数据接口读取内容 | +| enabled | 目标或实例的期望启停状态；不等于执行端已完成启停 | +| capability_version | 能力输入与行为契约的版本，宿主使用发现返回的版本；与输出 SourceDefinition 的版本分开 | + +参数错误、权限不足或处理失败均通过原 MCP 调用返回错误；不执行尚未通过检查的后续步骤。 + +所有调用由 Server 认证宿主身份，并按操作检查 PC 权限；scope\_id 指定数据归属。collect 由管理服务核对目标所属 Scope、实例可服务范围与执行条件。外部系统的账号、凭据及业务权限由插件在实际访问时处理。 + +能力和业务实例发现直接由 Server 回答；需要管理服务处理的调用使用以下交接方式。 + +**管理请求的交接方式统一：** Server 将消息交给管理服务主动发起的 `channel/poll` 响应；管理服务处理后调用 Server 的 `channel/reply` 回传，Server 再结束对应的 MCP 调用。框架用临时 message\_id 匹配问答，宿主无需填写或保存。下文“Server → 管理服务”表示业务消息流向，不表示 Server 反向建立网络连接。Runtime 同样主动连接管理服务；通信细节见附录 C。 + +| 方法 | 宿主使用时机 | +| --- | --- | +| `pc_source_capabilities` | 了解能做什么、参数如何填写、哪些实例可以执行 | +| `pc_source_add` | 写入已有内容，或添加持续采集目标 | +| `pc_source_status` | 查询 collect 目标的采集结果或控制状态 | +| `pc_source_target_set_enabled` | 停止或恢复一个 collect 目标 | +| `pc_source_instances` | 查看可用实例，或读取实例控制状态 | +| `pc_source_instance_set_enabled` | 启用或停用整个实例的业务运行 | + +## pc_source_capabilities：发现能力与输入要求 + +宿主不知道应选哪个能力、如何填写 input，或想确认可用实例时调用。发现只读管理信息，不加载新代码、不启动 apply，也不访问外部资料。 + +**调用参数：** scope\_id、operation；本节的业务发现使用 `operation="add"`。可用 capability、instance\_id 缩小查询范围。 + +```json +{ + "name": "pc_source_capabilities", + "arguments": {"scope_id": "scp_demo", "operation": "add"} +} +``` + +**从调用到返回：** + +1. 宿主向 Server 发出调用。Server 检查当前 Scope 的读取权限，并取得调用者是否具有写入权限的判断；没有写入权限时，发现结果不能宣称可写。 +2. Server 读取内置能力、已加载的自定义 write 能力，以及管理服务主动同步的 collect 目录；根据公开范围、实例可服务 Scope 和当前调用者 PC 权限筛选结果。本次发现不再向管理服务发送查询。 +3. write 根据 Server 本地实现是否加载就绪标注能否执行；collect 根据目录中的实例状态及管理连接可用性标注。可见但离线或停用的实例仍可显示，并明确不可用。 +4. Server 将上述结果直接经原 MCP 调用返回宿主，不经过 channel/reply。返回只包含允许公开的信息，不暴露实例 Config 或外部凭据。 + +**宿主获得什么：** 每个能力的名称、版本、kind、说明和 input\_schema，以及 PC 权限和可用性；collect 另包含可见实例、默认实例建议与运行状态。权限和运行状态分开表达：有写入权限不等于 Runtime 在线，在线也不等于 apply 正在处理工作，更不保证外部资料可访问。 + +宿主据此选择能力，再调用 `pc_source_add`。collect 还须明确传 instance\_id；默认实例仅为发现建议，不在重试时重新选择账号。write 不选择实例。能力版本也固定为本次选定版本。capability\_version 用于防止插件升级后以不兼容的输入规则解释原调用和持久 collect 目标；同一版本须保持输入、操作含义和资料身份规则兼容。 + +未配置管理服务时仍返回内置和已加载的自定义 write 能力。管理服务断线时明确说明 collect 部分暂不能接收新管理请求，不能将整次发现变成失败，也不能把缓存中的旧状态报为可执行。这里的不可用不表示 Runtime 已停止已有采集；发现结果只是调用时的状态，add 仍会重新检查。 + +## pc_source_add：写入内容或添加采集目标 + +**调用参数：** scope\_id、capability、capability\_version、input；collect 还须 instance\_id，write 不需要。input 按发现所得 input\_schema 填写，操作类型由 capability 的 kind 决定。 + +Server 对 add 检查当前 `scope.contribute` 和 `scope.read`。write 要求能力版本、对应 Definition 与 Projection 实现已在 Server 加载并通过校验。新建 collect 工作还要求能力与版本已登记、SourceDefinition 部署校验完成、Runtime 管理连接有效、实例 enabled=true 且最近确认 actual=running。只有外层 Runtime 在线、apply 已结束的实例不能接下新采集工作。 + +### write：原调用等待 Source 保存结果 + +例如宿主已有一段正文，使用内置 `content.inline / 2` 保存： + +```json +{ + "name": "pc_source_add", + "arguments": { + "scope_id": "scp_demo", + "capability": "content.inline", + "capability_version": "2", + "input": {"source_id": "meeting-note-001", "text": "需要保存的正文"} + } +} +``` + +source\_id 是这份数据的稳定身份，重复提交时保持不变。自定义 write 使用同一工具，选择其公开能力和业务输入，不传 instance\_id。 + +```mermaid +sequenceDiagram + participant H as 宿主 + participant S as Server / Source MCP + participant D as 已加载实现 + participant DB as Source 存储 + H->>S: pc_source_add(scope, capability, version, input) + S->>S: 认证 / 授权 / 校验能力 + S->>D: 校验输入 / resolve Source / project + D-->>S: Source 和 Projection 值 + S->>DB: 校验输出并按已有身份规则保存 + DB-->>S: 真实保存结果 + S->>S: 再检查读取权限 + S-->>H: SourceRef 与保存结果，或错误 +``` + +**从调用到返回：** + +1. 宿主提交已有完整内容。Server 认证并检查 Scope 权限、能力版本和输入要求，找到该能力关联的、已加载的 Definition 实现。 +2. Server 使用 Definition 的输入模型校验 input，调用 `resolve()` 形成带稳定身份的 Source，再调用对应 Projection 的 `project()` 形成投影数据。自定义函数来自已加载的插件模块，无需修改 Server 代码添加某个类型的 import。 +3. Server 校验输出契约，沿已有 Source 写入规则保存 Source 和 Projection；同身份同数据复用，同身份不同数据报冲突。 +4. Server 取得真实保存结果，检查调用者当前读取权限，经原 MCP 调用返回 SourceRef；输入、转换或保存失败则返回错误。原调用等待实际结果，不先返回“已受理”，宿主无需主动查询。 + +内置与自定义 write 都在 Server 内完成上述流程；自定义类型需要预先加载实现。write 接收完整内容，不承担远程资料的持续获取。 + +**成功返回示例：** + +```json +{ + "kind": "write", + "result": { + "outcome": "saved", + "source_refs": [{"source_type": "content", "source_id": "meeting-note-001"}] + } +} +``` + +SourceRef 以实际保存回执为准。saved 表示所需写入已完成；unchanged 表示底层确认已有相同 Source。宿主使用 SourceRef 和现有 Server 数据接口读取正文。 + +**重复调用与异常：** write 复用底层 `(scope_id, source_type, source_id)` 规则：同身份、同数据复用；同身份、不同数据报冲突；不同 ID 保存为不同 Source。稳定身份由底层提交方负责，内置适配层提供，自定义插件按 Definition 生成；需要宿主提供资料 ID 时在 input\_schema 中说明。重复调用可以再次执行校验或转换，不承诺外部副作用只发生一次。 + +调用超时、连接中断或 Server 退出时，结果可能未确认，不能据此判断 Source 没有保存。Server 检测到调用失效后清理当前处理；已经开始的保存可能成功，重连不自动重放。宿主需要重试时保持相同身份和实际提交数据，不能重新生成随机 ID 或变化的元数据。已保存的数据不因应答丢失而回滚。 + +### collect：先返回采集关系，插件随后持续处理 + +例如把新建的语雀文档 B 加入已经运行的 yuque-personal 实例： + +```json +{ + "name": "pc_source_add", + "arguments": { + "scope_id": "scp_demo", + "capability": "yuque.document", + "capability_version": "1", + "instance_id": "yuque-personal", + "input": {"url": "https://www.yuque.com/team/book/doc-b"} + } +} +``` + +**从调用到返回：** + +1. Server 检查宿主身份、Scope 权限和输入，将调用及可信调用者身份经管理通道交给管理服务。 +2. 管理服务通过 Runtime 调用插件的 `identify_target(request, config)`，取得 target\_key（稳定资料标识）和 normalized\_input（包含采集选项的规范化输入）。该入口独立于 apply，根据输入与实例配置确认资料身份。 +3. 管理服务按“Scope + 实例 + 能力及版本 + target\_key”查找目标。同身份同配置复用，同身份不同配置返回 target-config-conflict。新目标须通过实例可服务 Scope 与就绪检查，再将目标和身份索引一并持久保存；唯一约束处理并发重复。创建者仅作记录，不参与目标身份或管理授权判断。 +4. 管理服务通过 reply 返回目标。Server 在当前读取权限允许时，经原 MCP 调用把 target\_id、跟踪状态和已有采集结果返回宿主。到这里确认的是采集关系已建立，不是外部资料已保存。 + +**首次创建的返回示例：** + +```json +{ + "kind": "collect", + "target_id": "tgt_b", + "enabled": true, + "tracking": "enabled", + "source_refs": [], + "last_submitted_at": null +} +``` + +**返回之后如何采集：** Runtime 从管理服务同步目标，现有 apply 在后续 `ctx.targets.list()` 中取得 B，自行决定分批和节奏。Worker 执行 Connector，插件使用自己的凭据读取语雀并提交；数据直接存到 Server。SDK 根据实际成功提交回执关联目标与 Source，由 Runtime 向管理服务报告 Source 引用和提交时间，管理服务收到后持久保存。这些记录表示已收到的提交。Worker 仍按已有规则处理批次与 checkpoint。宿主需要了解进展时调用 `pc_source_status`；查询既不触发采集，也不是维持跟踪的必要动作。 + +新增 B 不重启实例，也不重建文档 A 的业务客户端。对已有目标的重复 add 不重置进度、不额外采集、不恢复 stopping/stopped 目标；调用始终需要当前 PC 权限。重复身份确认需要 Runtime 可达，但复用已有目标不要求 apply 正在运行；无法确认身份时明确报错，不猜测新建。已有 target\_id 的查询或停止仍可由管理服务处理。 + +受理响应丢失时，宿主可用原 Scope、实例、能力版本和业务 input 再次调用，取得原目标。同一 Scope 内，不同用户添加相同实例、能力版本及资料身份且配置一致时复用同一目标；不同 Scope、实例或能力版本不合并。停止目标仍占原身份，本版不提供目标硬删除或自动合并历史重复记录。 + +## pc_source_status：查询采集结果或控制状态 + +此方法查询管理服务已保存的 collect 目标和提交记录。 + +**调用参数：** scope\_id、target\_id；默认读取采集状态，`view="control"` 只读取启停字段，供后续控制操作使用。 + +```json +{ + "name": "pc_source_status", + "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b"} +} +``` + +**从调用到返回：** + +1. Server 认证宿主。默认视图检查当前 scope.read；control 视图检查 scope.contribute。 +2. Server 完成相应 Scope 权限检查后，经管理通道转交目标查询。管理服务核对目标实际所属 Scope 与请求 Scope 一致；不一致时拒绝，不返回其他 Scope 的目标信息。 +3. 管理服务读取自己的目标记录和已经收到的采集结果，不调用 identify\_target、不要求 Runtime 在线、不等待新的 Worker 批次。 +4. 管理服务经 reply 返回，Server 将允许读取的结果交给原 MCP 调用。采集结果返回仍须当前读取权限；control 响应只包含有权查看的控制字段。 + +**宿主获得什么：** + +| 视图 | 返回内容与含义 | +| --- | --- | +| 默认 | target_id、enabled、tracking、source_refs、last_submitted_at，以及最近确认时间和实例可用性。source_refs 是管理服务已收到的实际提交关联，last_submitted_at 是这些记录中的最近提交时间；没有记录时分别为空列表和 null | +| control | target_id、enabled、tracking；不包含 input、SourceRef 或业务错误正文。具有 scope.contribute 的调用者可读取控制状态，不额外要求 scope.read | + +tracking 表示采集关系的 enabled / stopping / stopped。source\_refs 和 last\_submitted\_at 仅反映已经确认并报告的提交事实，不表示目标已全部采完；没有新提交时，不区分尚未处理、暂无资料或内容未变化。提交报告可能丢失，因此此视图不是 Server 全部数据的清单。实例失联时保留已有提交记录，但可用性显示 offline / unknown，不能用历史提交表示此刻仍在采集。 + +该调用可以重复执行且无业务副作用。管理服务不可达时返回不可用，不由 Server 缓存伪造最新状态；这不影响有权限的宿主直接读取 Server 已保存的 Source。 + +## pc_source_target_set_enabled：停止或恢复一个目标 + +**调用参数：** scope\_id、target\_id、enabled。宿主直接表达停止或恢复意图。 + +```json +{ + "name": "pc_source_target_set_enabled", + "arguments": { + "scope_id": "scp_demo", + "target_id": "tgt_b", + "enabled": false + } +} +``` + +**从调用到返回：** + +1. Server 检查目标 Scope 的 scope.contribute，经管理通道转交调用。 +2. 管理服务核对目标所属 Scope，恢复时检查实例可服务范围，再保存 enabled，保留目标身份、已有 Source 与 checkpoint；重复设置当前状态不产生新动作。 +3. 管理服务返回控制状态，Server 经原 MCP 调用返回宿主。Runtime 同步后按目标启停状态处理采集。 + +返回表示启停意图已保存，不代表执行端已完成操作。恢复目标不会自动启用已停用的实例，也不会重新调用已经结束的 apply。 + +## pc_source_instances：发现实例或读取实例控制状态 + +本方法管理 collect 实例；Server 内的 write 能力没有对应实例。 + +capabilities 以“能做什么”为入口；instances 以“哪些实例可用、某实例当前怎样运行”为入口。此方法不加载代码或启动实例。 + +**调用参数有两种组合：** + +| 视图 | 参数 | 用途 | +| --- | --- | --- | +| 业务视图 | scope_id、operation="add" | 查找能服务当前 Scope 的可见实例、PC 权限与运行状态 | +| 控制视图 | instance_id、view="control" | 读取有权管理的某个实例的控制状态；不要求指定 Scope | + +```json +{ + "name": "pc_source_instances", + "arguments": {"scope_id": "scp_demo", "operation": "add"} +} +``` + +```json +{ + "name": "pc_source_instances", + "arguments": {"instance_id": "yuque-personal", "view": "control"} +} +``` + +**从调用到返回：** + +1. Server 认证宿主。业务视图检查 scope.read，并取得当前 Scope 写权限判断；控制视图检查 server.admin。 +2. 业务视图由 Server 根据已同步目录中的公开范围、可服务 Scope 和实例状态直接返回，不再询问管理服务；目录与连接可用性的处理同 capabilities。 +3. Server 通过 server.admin 检查后，将控制视图查询经管理通道交给管理服务，不额外要求 Scope 权限。管理服务读取指定实例的持久控制状态与最近执行报告，经 reply 返回 Server，再交给宿主。两种视图都不重新调用 apply 或访问外部账号。 + +**宿主获得什么：** 业务视图返回实例 ID、可用能力、PC 权限及运行状态；控制视图返回 instance\_id、enabled、actual、可用性和最近更新时间，不包含业务 Config、目标输入或 Source 结果。 + +期望 enabled=true 不保证 actual=running；apply 正常结束可以是 actual=complete，未处理异常可以是 failed。Runtime 外层仍可能在线。控制视图用于了解期望状态与实际运行状态的区别，不能因“已启用”就宣称正在采集。 + +调用只读，可重复执行。业务视图按 Server 已同步的公开信息回答，管理连接失效时明确标注自定义部分不可用，不用旧目录保证当前权限或可执行性；无法确认可见范围时不暴露记录。控制视图依赖管理服务可达；仅 Runtime 失联时仍可读取持久控制状态，但运行可用性须标明未知或离线。内置与自定义 write 能力都不需要插件实例。 + +## pc_source_instance_set_enabled：启停整个实例的业务运行 + +此方法控制 Runtime 内的 apply / Context，影响该实例服务的所有 Scope 和目标；它不是安装、卸载或启动远端系统进程的命令。 + +**调用参数：** instance\_id、enabled。宿主直接表达启用或停用意图。 + +```json +{ + "name": "pc_source_instance_set_enabled", + "arguments": { + "instance_id": "yuque-personal", + "enabled": false + } +} +``` + +**从调用到返回：** + +1. Server 认证宿主并检查 server.admin，再将实例控制调用交给管理服务；实例可能服务多个 Scope，单个 Scope 的权限不能用于启停整个实例。 +2. 管理服务检查实例存在及切换条件，在事务内按实际处理顺序设置期望状态。同状态返回当前结果；停用保存 enabled=false，并在仍需执行端确认时标为 stopping；启用要求旧业务运行已经确认 stopped，再保存 enabled=true、生成新的 run\_id（一次业务运行的身份）并等待 Runtime 执行。 +3. 管理服务通过 reply 将持久控制状态交回 Server，再返回宿主。响应包含 instance\_id、enabled、actual 等控制字段，不表示系统进程已经安装或业务已经启动成功。 +4. Runtime 同步后执行期望状态。停用阻止新批次，取消并等待 apply 及所属子任务，关闭 Context 和业务资源；全部完成后报告 stopped，外层 Runtime 继续在线。启用创建新的 Context 并调用 apply，报告实际运行状态。宿主需要确认时调用 instances 的 control 视图；查询不驱动启停。 + +仅 Runtime 失联时可以保存停用意图，但不能报告已停止；清理失败或旧任务未退出时不启动第二份 Context。管理服务失联时不能确认控制变更。重新启用后只处理仍 enabled 的目标，不恢复已经单独停止的目标，不重置目标 checkpoint。 + +同一 Runtime 进程内，已 enabled 时重复启用不会再调用 apply，即使本次 apply 已 complete 或 failed。需要重新运行时明确停用，确认 stopped，再显式启用。关闭或卸载整个 Runtime 后，管理服务不能远程替它启动系统进程；部署操作见第 2、5 节。 + +## 操作重试 + +collect 重复 add 复用目标，write 重复提交复用底层 Source 规则，启停直接设置期望状态；相同状态不产生新动作，不同意图按实际处理顺序生效。通信超时不表示已保存的数据或控制变更被撤销，宿主可按各方法的规则重试或读取当前状态。 + +# 4. 插件接口与数据管道 + +本节面向插件作者。write 提供 Server 可以加载的输入模型、Definition 与 Projection 实现；collect 在这些数据契约之外提供 apply 和 Connector，自行决定采集节奏，通过 Worker 提交。宿主调用方式见第 3 节，加载、进程启停和重启行为见第 5 节。 + +## 插件需要提供的契约 + +插件代码可以从本地目录加载，无需发布 wheel。write 的代码与依赖部署在 Server 机器；collect 的代码与依赖部署在 Runtime 机器。两者可以复用同一个代码包中的 Definition 和 Projection，无需强制拆成两个包。 + +**共同的数据契约。** SourceDefinition 声明输入模型、Source 模型、名称和版本；`resolve(value)` 将输入转换成具有稳定身份的 Source，`projections` 中各实现的 `project(source)` 生成相应 Projection。`definitions` 导出这些实现对象。能力声明包含 name、version、description、kind 和 input\_schema，描述宿主能调用什么；Definition 描述结果如何保存。collect 的一个能力可以提交多种 Source，多个能力也可以复用同一 Source type。 + +**write 的加载契约。** 每项 write 能力明确关联一个 Definition 的名称与版本，input\_schema 对应它的输入模型；Server 可以据此直接校验 input 并调用该 Definition。插件导出 `definitions` 和 `capabilities` 供 Server 加载。能力名称不得覆盖内置能力，同名同版本须保持输入和操作含义兼容。 + +下面通过**接口示意**展示“Server 如何得到 Projection 函数”。示例说明插件导出对象与 Server 加载的关系，具体类名、模块路径和辅助代码可随实现调整；write 能力关联字段和部署入口尚待实现。假设部署者准备一个可导入的本地包： + +```text +/srv/powercontext/custom_sources/tickets/ + __init__.py + ticket_definition.py +``` + +`ticket_definition.py` 实现 TicketCapture、TicketDefinition 和 TicketProjection。其中关键关系如下（省略已有 Source 模型和 resolve 实现）： + +```python +class TicketProjection: + # 同时声明 Projection 名称、版本、输入 Source 类与输出模型。 + def project(self, source): + return TextEvidence( + source_type="example-ticket", + source_id=source.name, + content=f"{source.capture.title}\n\n{source.capture.body}", + ) + + +class TicketDefinition: + name = "example-ticket" + version = "1" + input_class = TicketCapture + projections = (TicketProjection(),) + # resolve() 生成稳定身份的 TicketSource；其余成员遵循 SourceDefinition。 +``` + +插件自己的 `__init__.py` 导出对象，类型专属 import 写在插件中： + +```python +from .ticket_definition import TicketCapture, TicketDefinition + +definitions = (TicketDefinition(),) +capabilities = ({ + "name": "ticket.write", + "version": "1", + "kind": "write", + "description": "保存宿主提供的工单内容", + "input_schema": TicketCapture.model_json_schema(), + "definition_name": "example-ticket", + "definition_version": "1", +},) +``` + +Server 配置入口指向这个包，由通用加载器导入。以下只展示导入对象关系，不是完整加载器实现： + +```python +module = importlib.import_module(entry) # 例如 tickets；来自受信任部署配置。 +registry = SourceDefinitionRegistry(module.definitions) +# 运行时 registry.resolve / registry.project 调用这些实际 Python 对象。 +``` + +因此，Server 能执行 TicketProjection.project，是因为**它已导入包含该对象的本地模块**。新增类型只增加模块与入口配置，不在 Server 中硬编码类型专属 import。部署者须将代码加入 Server 的模块搜索路径，并准备相同环境中的依赖；文件路径入口也由通用加载器解析。实现只校验和转换当前输入，不在导入时启动采集任务。 + +**注册 SourceDefinition 与加载代码是两个动作。** 现有注册 API 保存类型结构、身份和 Projection 契约，不传输 Python 实现。Server 的 write 加载流程校验代码、契约与冲突，并在注册成功后才发布能力；仅有注册记录不能执行 write。collect 仍在部署或升级阶段使用 `server.admin` 注册 Definition，日常 Worker 只使用 Scope 执行权限，不每批注册或临时取得管理员凭据；Definition 缺失或不匹配时明确报错。 + +**collect 额外提供的运行契约：** + +| 导出项或对象 | 插件需要定义的内容 | +| --- | --- | +| `plugin_api_version` | 与 Runtime 对接的插件接口版本，当前为字符串 `"2"` | +| `Config` | Pydantic 配置模型，用于定义和校验插件自身逻辑所需配置，例如访问外部系统的凭据或凭据引用 | +| `capabilities / definitions` | 非空的 collect 能力声明与 SourceDefinition 集合；能力输入描述外部资料位置或选择条件 | +| `connector_name / connector_version` | Connector 的稳定名称与兼容版本，用于核对 checkpoint 归属，不能随重启随机变化 | +| `identify_target(request, config)` | 用输入确定稳定资料身份和规范化采集参数 | +| `apply(ctx, config)` | 异步业务入口；建立客户端、connect Connector，自行决定分批、等待、重试和退出 | +| `Connector.run(session)` | 处理并提交本批数据，返回 ConnectorRunCompletion（本批是否完成及下一 checkpoint） | + +collect 的 Config 留在实例侧，外部凭据与业务权限由插件自行处理。能力描述和 input\_schema 只公开宿主调用所需信息；目录、结果和普通日志不暴露秘密。同版本 collect 能力还须保持资料身份规则兼容。 + +**collect 如何识别同一份资料：** Runtime 在受理目标时调用 identify\_target，独立于 apply 的采集循环。request 包含 capability、capability\_version 和 input；插件返回： + +| 返回项 | 含义 | +| --- | --- | +| target_key | 稳定资料标识，管理服务据此复用同一目标 | +| normalized_input | 规范化后的完整采集参数，保留全部影响采集的选项，用于检查同一目标是否发生配置冲突 | + +例如，插件可以去掉文档 URL 中仅用于页面定位的锚点，将两个地址识别为同一目标。identify\_target 根据输入和实例配置确认资料身份，文档内容与可读性在实际采集时处理。input\_schema 应要求足够的资料标识，无法确认相同时，不提前承诺合并。 + +## collect：从 apply 到 Source 保存 + +`ctx` 是 Runtime 传给 apply 的 PluginContext，持有 Connector 和业务资源，并提供下表中的 SDK 接口。一个 Context 只 connect 一个 Connector。`session` 由 Worker 为每批创建，提供本批 Scope、当前 checkpoint 和提交能力。 + +一次 apply 可以执行多个批次；**一次 `run_batch()` 就是一批，固定一个 Scope 和一组待处理内容。** 同一 collect 目标可以在多个批次中持续产生 Source，SDK 关联目标与实际提交，宿主不生成批次 ID。 + +下图展示一次批次调用的顺序。左侧 apply 和 Connector 都是插件代码，中间是 SDK 与已有 RemoteConnectorWorker，右侧是 PC Server；箭头表示调用或返回，不表示独立进程数量。图中展示 collect 的成功提交路径，submit 可以重复调用。binding 表示本实例在某个 Scope 的执行归属，隔离规则见本节末尾。 + +```mermaid +sequenceDiagram + participant A as 插件 apply + participant C as 插件 Connector + participant SDK as PluginContext / SDK + participant W as RemoteConnectorWorker + participant S as PC Server + A->>SDK: targets.list() + SDK-->>A: 当前目标快照 + A->>SDK: run_batch(scope_id, targets) + SDK->>W: 执行本批 Connector 与 binding + W->>S: 读取 checkpoint + S-->>W: 当前 checkpoint + W->>C: run(session) + C->>C: 读取外部资料 + C->>SDK: targets.submit(target, session, value) + SDK->>S: session.submit / Source 与 Projection + S-->>SDK: 真实提交回执 + SDK-->>C: ConnectorItemOutcome + Note over SDK: 关联 target 与 SourceRef，供 Runtime 报告 + C-->>W: ConnectorRunCompletion + W->>S: 按完成与回执规则 CAS 更新 checkpoint + W-->>SDK: 批次结果 + SDK-->>A: 批次结果 + Note over A: 插件决定等待、下一批或退出 +``` + +apply 先取得待处理内容，选择同一 Scope 的条目，再调用 run\_batch。Worker 读取 checkpoint，调用 Connector.run；Connector 按业务需要读取外部系统并转换数据，再通过 SDK 提交。等待和下一批由插件决定。 + +| 步骤 | collect 接口 | +| --- | --- | +| 获取当前可处理内容 | `ctx.targets.list()` | +| 执行所选批次 | `ctx.run_batch(scope_id=..., targets=selected)` | +| 提交一项数据 | `ctx.targets.submit(...)` | + +list 返回的是本次处理使用的目标快照，字段含义如下： + +| 字段 | 含义 | +| --- | --- | +| target_id | 对应管理服务持久保存的目标 | +| scope_id、instance_id、capability、capability_version | 归属与能力契约 | +| input | identify_target 确认后的规范化采集参数 | +| source_refs | 管理服务已收到的真实提交关联，供插件按需参考；不代表目标已采完 | + +collect 首次同步成功后，ctx.targets.list / run\_batch 可在管理服务断线期间继续使用内存中最后同步的目标与启停状态；没有初始快照时明确报不可用，不用空列表冒充没有工作。新增目标在同步后的快照中出现。 + +同一 Context 的批次顺序执行，不重叠；一批选择同一 Scope 的 targets，不并发改写本批快照。管理模式必须显式传入所选集合；自主采集使用显式空集合，遗漏参数不表示执行所有目标。独立运行入口保留 `run_batch(scope_id=...)`。 + +submit 接收当前条目、session、本批唯一的 item\_id、Definition 名称和业务 value。SDK 校验实例、Scope 与批次归属，包装 `session.submit(...)`，并把实际提交回执关联到该条目。返回的 ConnectorItemOutcome 表示这一项提交的处理结果，插件必须检查是否被接受。处理后的数据直接进入 PC Server，不经管理服务转存。 + +collect 的实际提交关联报告给管理服务保存。完整 apply 与 Connector 代码见附录 B。 + +## collect 的提交回执与批次完成 + +插件自行维护循环、分页和重试，通过 `ctx.targets.submit(...)` 提交数据。提交回执表示数据保存结果，ConnectorRunCompletion 表示本批完成情况；持续采集目标可以跨多个批次产生 Source。 + +| 动作 | 含义 | +| --- | --- | +| submit 收到成功回执 | 一项数据已由 Server 接受；SDK 将真实 SourceRef 关联到对应 collect 目标 | +| Connector 返回 ConnectorRunCompletion | 声明本批是否完成及下一 checkpoint；Worker 按已有规则核对并处理进度 | +| collect 提交记录报告管理服务 | 保存已确认的 Source 引用与提交时间，供宿主按需查看；不等待该目标“全部采完”，也不要求同批其他目标完成 | + +没有新提交时，框架不推断目标是尚未处理、没有变化还是暂无资料。失败后是否重试、何时继续仍由插件决定；未处理异常按第 5 节结束当前 apply。Source 保存、checkpoint 更新和向管理服务报告提交记录是独立动作：批次未完成或报告丢失不撤销已保存 Source，报告也不保证重启后补齐。 + +## checkpoint 的隔离、推进与新增目标 + +ConnectorBinding 表示“哪个实例的 Connector 在哪个 Scope 执行”。本方案用 instance\_id 作为 binding\_id，PC Server 按 `(scope_id, binding_id)` 保存 checkpoint，并核对 connector\_name / connector\_version。这里的 ConnectorBinding 不同于权限系统中授予角色的 Scope Binding。 + +```mermaid +flowchart TB + subgraph PC[PC Server 保存 checkpoint] + A["Scope 1 + 实例 A：checkpoint"] + B["Scope 2 + 实例 A：checkpoint"] + C["Scope 1 + 实例 B：checkpoint"] + end + T1[实例 A / Scope 1 / 目标 1] --> A + T2[实例 A / Scope 1 / 目标 2] --> A + T3[实例 A / Scope 2 / 目标 3] --> B + T4[实例 B / Scope 1 / 目标 4] --> C + A --> P[内容格式由插件定义，可包含逐目标进度] +``` + +图中的每个 checkpoint 由 Server 保存，内容格式由插件定义。同实例、同 Scope 的多个目标或能力共用一份 checkpoint，框架不会为每个 target 自动创建独立游标；插件可以在其中记录各目标的进度。不同实例或不同 Scope 的 checkpoint 互不共享。 + +Worker 在批次开始时读取 checkpoint，Connector 根据实际业务进度返回下一值；需要更新时，Worker 使用 CAS（仅当 Server 中仍是本次读取的旧值才写入新值），避免覆盖并发进度。插件不能在尚有未处理资料时推进游标而跳过它们。 + +例如文档 A 已采集至时间 T，后来加入一篇早于 T 的文档 B。插件不能直接用 A 的时间游标跳过 B，应识别新增目标，为 B 初始化进度并补采；具体记录方式由插件定义。停止或恢复目标不自动清空 checkpoint，Connector 升级也不能靠换版本号绕过已有进度的兼容性检查。 + +Runtime 进程重启后重新同步 targets，由 Worker 读取 Server 当前 checkpoint，插件决定继续或重新采集；框架不恢复上次批次的具体执行步骤。 + +# 5. 生命周期、重启与权限 + +本节的 Runtime、实例与目标生命周期用于 collect，采集节奏仍由 apply 决定。write 实现随 Server 加载和释放，没有独立实例启停：新增实现可显式加载，一致配置重复加载复用已有登记；已有代码替换通过受控重启 Server 生效。启动时重新校验已保存入口，加载失败的能力不标为可执行，其他有效能力照常使用。 + +## Runtime 在线与 apply 运行 + +后台管理模式下，每个实例有一个 Runtime 进程。Runtime 维护管理连接、同步目标和启停状态；其中的 **Context 是一次 apply 运行所持有的 Connector、客户端和业务任务的集合**。业务等待或 sleep 不阻塞 Runtime 接收管理操作。 + +实例需要区分三件事： + +| 信息 | 含义 | +| --- | --- | +| enabled | 管理服务保存的期望启停状态；为 true 不代表 apply 此刻一定在运行 | +| actual | Runtime 最近报告的业务状态，例如正在运行、正常结束（complete）、异常结束（failed）、正在停止（stopping）或已停止（stopped） | +| 连接可用性 | Runtime 是否仍可通信；失联显示 offline / unknown，不能用旧的 actual 宣称业务仍在执行 | + +**apply 结束后，后台 Runtime 仍保持在线。** 它可以接收控制操作，但不会因新增目标、通信重连或重复启用而自动再次调用 apply。持续处理新增目标的插件，需要在 apply 内保持自己的采集循环。 + +```mermaid +flowchart TD + S[Runtime 启动并同步实例状态与目标] --> E{enabled?} + E -->|否| I[保持在线等待] + E -->|是，旧进程已退出| R[创建 Context 并执行 apply] + R -->|正常返回| C[清理 Context，报告 complete] + R -->|未处理异常| F[清理 Context，报告 failed] + C --> I + F --> I + R -->|停用| D[取消并等待任务，关闭资源] + I -->|停用| D + I -->|从 disabled 启用，旧业务已确认停止| R + D -->|清理完成| T[确认 stopped，Runtime 仍在线] + T -->|显式启用| R + I -->|新增目标、重连或已 enabled 时重复启用| I + D -->|清理失败| U[报告错误，不创建第二份 Context] +``` + +卸载系统服务会使 Runtime 进程退出；再次启动需要在执行机器执行部署操作。 + +若要在同一进程中重新执行已经结束的 apply，显式停用、确认 stopped，再启用。直接运行的 `pc-source run` 则在 apply 结束后退出；它与后台模式不能同时占用同一实例身份。 + +管理服务保存期望状态、最近实际状态及更新时间。框架用内部运行标识（run\_id）区分每次 apply 的报告，只接收当前连接和当前运行的消息，避免迟到报告覆盖新状态。宿主无需提供该标识，Runtime 也不用它跨进程恢复业务。 + +## 实例停用与资源清理 + +停用实例时，管理服务保存 enabled=false；Runtime 阻止新批次，取消并等待 apply 及其子任务，关闭 Context。清理完成后报告 stopped，外层 Runtime 保持在线，等待再次启用。 + +业务子任务必须归属于 apply，例如使用 TaskGroup；资源通过 `ctx.enter_async_context` / `ctx.on_dispose` 登记，随 Context 关闭。插件不能留下脱离生命周期的后台任务，也不能吞掉取消异常。任务未结束或资源清理失败时，不报告 stopped，不创建第二份 Context。 + +实例停用不影响 Server 已加载的 write 能力，也不撤销已经保存的 Source。Runtime 失联时不能宣称业务已经停止；管理服务不可达时不能确认新的启停意图已保存。 + +## 断线、重启与数据恢复 + +**连接重连保留当前进程内存，进程重启则创建新的业务运行。** Server 或管理服务重启不等于 Runtime 重启。系统服务管理器负责进程存活，不负责调用 Worker。 + +| 情况 | 后续行为 | +| --- | --- | +| Runtime 尚未取得初始目标快照 | collect 的 list / run_batch 报不可用，插件决定等待或退出；进程重启后也须先取得目标 | +| 已同步的 Runtime 与管理服务断线 | collect 可使用内存中最后同步的目标和启停状态开启新批次，节奏仍由插件决定；提交继续受 Server 权限检查，提交报告可能丢失 | +| 同一 Runtime 进程重连 | 先同步最新实例和目标启停状态，再允许后续处理；按同步后的启停状态处理，不重跑仍在运行或已经结束的 apply | +| Runtime 进程重启 | 内存中的批次和未送达报告丢失；重新同步后，实例仍启用就创建 Context、执行 apply，即使上次状态为 complete / failed；Worker 读取 Server checkpoint | +| 仅管理服务与 Server 的通道中断，Runtime 仍连接两者 | 宿主的 collect 管理操作不可用；已有 collect 可在当前同步状态允许下继续。内置功能与已加载的自定义 write 独立可用 | + +同身份旧进程是否退出无法确认时，不启动另一份 Runtime；心跳丢失不构成跨机器接管的依据。框架不保证 apply 或外部副作用只执行一次，重复提交沿用底层 Source 身份规则。 + +Runtime 只保留启动所需文件，当前执行信息在内存中；持久存储归属见第 2 节。恢复依赖管理服务中的目标与启停状态，以及 Server 中的 Source 和 checkpoint。 + +collect 的 **Source 保存、checkpoint 更新、提交记录报告**是三个独立动作，正常处理关系见第 4 节。故障时分别处理： + +| 故障位置 | 恢复依据与影响 | +| --- | --- | +| Source 已保存，checkpoint 尚未更新 | 下次可能重新采集；同身份同数据由底层复用。插件须保持 Source 身份稳定，数据变化按 Definition 的规则处理 | +| checkpoint 更新响应丢失 | 下次读取 Server 中实际保存的进度；不能声称本次进度已确认，但已有成功提交回执仍可报告 | +| Source 和 checkpoint 已保存，提交报告丢失 | 数据和进度有效；管理服务可能缺少对应提交记录，不回退 checkpoint，也不保证重启后补齐报告 | + +因此，管理服务没有提交记录，不代表 Server 没有 Source；有权限的宿主仍可通过 Server 数据接口读取。collect 按插件节奏继续，不恢复上次批次的具体执行步骤。Server 重启会中断进行中的 write；重启后重新加载实现，不恢复或重放旧调用。已保存 Source 不受影响，重试仍沿用相同 Source 身份与数据。 + +## 新增实例、修改配置与升级 + +| 变更 | 处理方式 | +| --- | --- | +| 新增实例 | 准备代码和依赖，由部署流程注册 Definition、登记实例并启动 Runtime；发布新能力，不重启已有实例 | +| 给运行实例增加 Scope | 部署者授予执行身份相应 Scope Binding，并更新实例可服务范围；Runtime 同步后，宿主可添加该 Scope 的目标，原 Context 保留 | +| 修改业务配置或升级代码 | 停用并确认 stopped，停止旧 Runtime，再校验和更新代码、Config 与登记；启动新 Runtime 后显式启用。外部凭据变更按同一配置更新流程处理，其业务含义由插件解释 | + +Scope Binding 是 Server 权限系统中授予身份某个 Scope 角色的关系，不是第 4 节用于 checkpoint 的 ConnectorBinding。仅提交新的 scope\_id 不会授予权限，也不要求按 Scope 新建插件实例。 + +collect 代码或配置更新前须确认旧 Runtime 已退出；已有 Source、目标和 checkpoint 保留。write 实现更新则先处置 Server 在途调用，再通过受控重启加载新代码；须保持能力与 Definition 契约兼容，不迁移旧调用或改变已有 Source 身份。collect 升级需核对三类兼容性： + +| 保留的数据或契约 | 要求 | +| --- | --- | +| collect 目标的 capability/version、输入与身份规则 | 新代码仍能处理已登记的目标，包括已停止目标；只支持 v2 的入口不能直接接管 v1 目标 | +| SourceDefinition、ConnectorBinding 与 checkpoint 格式 | 兼容已有 Source 和进度，不通过更换版本或重置游标绕过检查 | +| 目标与已收到的提交记录 | 保留原记录和 Scope 归属，访问统一检查当前 PC 权限，不要求重新执行业务来重建历史 | + +无法确认兼容时，保持停用，不删除记录或重置进度。部署者可准备兼容代码，或明确处理不兼容的目标与进度；第一版不提供自动迁移，新实例也不会自动继承旧目标。 + +通过检查后，部署者用 configure 接口更新登记，仍保持 disabled；新 Runtime 校验成功后再显式启用。失败保持不可执行，不并行启动第二份 Context。普通重复启动或加载只复用一致配置，配置冲突须明确报错，不能代替更新流程；管理接口见附录 C。 + +## 权限职责与操作条件 + +**用户授权统一复用 PC Server 的 Access 体系。** Principal 是 Server 认证得到的用户或服务身份，宿主不能在业务 input 中自行声明它。 + +| 负责方 | 职责 | +| --- | --- | +| PC Server | 认证宿主并检查当前 Scope 或 Server 权限，授权通过后才转交管理请求；write 按宿主身份检查保存权限，collect 每次提交检查实例执行身份；返回数据时检查当前读取权限 | +| 管理服务 | 只接受认证通道中的 Server 管理请求，核对目标实际所属 Scope、实例可服务范围和运行条件，再执行操作 | +| 插件 | 解释自身配置与外部凭据，在访问外部系统时处理业务权限；identify_target 只确认资料身份，不参与授权 | + +| 操作 | PC Server 检查的权限 | +| --- | --- | +| 能力与业务实例发现 | scope.read，并在结果中说明当前是否具备 scope.contribute | +| write、新建或重复 collect | scope.contribute；本接口应答包含数据或已有 Source 引用，因此同时检查 scope.read | +| 查询 collect 目标及提交结果、读取 Source | scope.read | +| collect control 视图、停止或恢复目标 | scope.contribute；仅返回启停字段，不额外要求 scope.read | +| 加载 Server 本地 write 实现；实例 control 视图、加载、配置、卸载或启停整个实例 | server.admin；实际进程操作仍由执行机器的部署工具完成 | +| 为用户或执行身份授予 Scope 权限 | 沿用现有 Access 接口的授权规则 | + +同一 Scope 内，具有 scope.contribute 的用户都可以控制该 Scope 的采集目标，不限于创建者；创建者只作记录。实例可能服务多个 Scope，因此控制整个实例需要 server.admin，不能以某个 Scope 的权限代替。实例控制响应不含业务 Config、目标输入或 Source 数据。 + +Server 将已授权操作与请求中的 Scope、target\_id 或 instance\_id 一并传给管理服务。管理服务必须核对实际资源归属，防止用有权限的 Scope 指向另一 Scope 的目标；这个检查不增加新的授权模型。实例接入与管理通道仍需认证，Runtime 只能同步自身目标和报告自身状态，不能冒充 Server 下发管理操作。 + +修改或撤销用户权限只操作 PC Server 的 Access 体系。write 等待期间和最终返回前仍检查当前读取权限；权限已撤销就拒绝返回结果，但不声称已接受的写入未发生。collect 的持续执行使用实例自身的 PC 执行身份，创建者权限变化不自动停用既有目标；停止目标使用正常控制操作，实例提交权限则由 Server 每次检查。 + +collect 遇到外部系统拒绝访问时，由插件决定等待、重试或结束 apply。write 的输入校验、resolve、project 或保存失败由 Server 经原调用返回错误。 + +## 凭据与初始化授权 + +AuthenticationProvider 是 Server 验证凭据并映射 Principal 的组件。本机部署流程可用部署控制的 Provider 创建或复用执行身份；连接外部管理的 Server 时，沿用它的 Provider，必要时由管理员预置身份，不覆盖认证配置或承诺自动签发凭据。 + +| 凭据 | 用途 | 保存位置 | +| --- | --- | --- | +| 部署凭据 | 加载 write 实现、注册 Definition、创建 Scope、授予角色与配置实例 | 部署工具的秘密存储，不下发 Runtime | +| 宿主凭据 | 以宿主用户或服务身份调用 Server | 宿主安全配置 | +| 管理通道凭据 | 管理服务向 Server 认证并建立专用通道 | 管理服务 | +| PC 执行凭据 | 每实例独立 Principal，向获授权的 Scope 提交 Source、处理 checkpoint | 执行机器，不保留 server.admin | +| 实例连接凭据 | Runtime 向管理服务证明实例身份，同步本实例工作与报告 | 执行机器 | +| 外部系统凭据 | 插件访问语雀、工单等业务系统 | 插件自行保存和解释，可在 Config 中引用文件、环境或秘密服务 | + +write 的部署加载需要管理员权限，日常保存使用当前宿主身份，不另外创建 PC 执行身份或发放执行凭据。collect 部署使用独立 Bearer 凭据，经 Provider 映射到明确身份。启动 collect 业务前，部署流程完成： + +1. 验证 Server、部署标识、部署权限和目标 Scope；实际确认宿主及执行凭据对应的 Principal，不以文件存在代替验证。 +2. 使用部署凭据注册 SourceDefinition，并通过现有 Access 接口创建或复用执行身份的 Scope Binding；日常执行无需注册 Definition 或取得 server.admin。 +3. 配置管理通道身份，登记实例执行位置与可服务 Scope；管理操作授权复用 Server Access，Runtime 只取得自身所需凭据或引用。 +4. 用执行凭据检查实际身份和 Scope 权限，通过后才允许处理该 Scope。没有 Scope 授权的实例可以登记，但不能受理该 Scope 的采集。宿主 MCP 的连接与授权沿用宿主集成流程。 + +上述步骤使用现有的 `POST /v1/source-definitions/register`、`POST /v1/access/bindings/create`、`GET /v1/access/me` 和 `POST /v1/access/check`，不另建用户注册或 OAuth 系统。 + +本机自动部署生成内部连接配置，不跳过授权。部署凭据不下发 collect Runtime；Server 只允许管理员加载受信任的 write 代码，日常 write 不借用部署权限保存数据；凭据值不写入命令参数、普通日志或能力目录。轮换沿用 Provider 与秘密引用，失效身份不能继续使用；跨机通信使用 TLS 或等效的已认证安全入口。 + +# 附录 A：使用场景——启动与动态加入资料 + +以下先按部署位置展示 collect 的用户操作与组件交互，随后补充 write 的加载与调用。本机 collect 部署时，Server 启动扫描和 `source load` 会准备共享管理服务及 Runtime，无须再手动执行两个 `install`；跨机器部署时，部署者准备完整部署配置，启动共享管理服务，完成登记和授权后再启动远端 Runtime。本文中的服务安装命令使用 Linux `systemd --user`；PC Server 须已配置认证并启用 PC Access。 + +本附录的命令和配置为**使用示意**，说明部署位置、操作顺序和组件交互；最终命令与参数以随实现发布的用户指南为准。Shell 命令在标明的机器执行；JSON 为第 3 节**拟议接口**的 MCP `tools/call` 参数，由宿主发送到 Server。示例 Scope、路径和文档 URL 必须替换为实际值。 + +## 场景一：在同一台机器启动 Codex、Server 和插件 + +**1\. 在 Server 所在机器准备实例目录。** 假设代码随包提供，目录为 `/srv/powercontext/source-plugins/yuque-personal`，外部凭据单独保存在受保护文件中。`powercontext.yml` 如下： + +```yaml +id: yuque-personal +plugin: yuque +config: + token_file: /srv/powercontext/secrets/yuque.token + base_url: https://www.yuque.com + poll_interval: 60 +# 可选：首次初始化的采集目标。 +targets: + - scope_id: scp_demo + capability: yuque.document + capability_version: "1" + input: + url: https://www.yuque.com/team/book/doc-a +``` + +`config` 由插件定义，包含外部凭据引用等业务参数；targets 中的 Scope 必须存在，且初始化调用方及实例执行身份具备相应权限。初始目标按正常添加规则写入管理服务，初始化后以管理服务为准；后续改文件不会覆盖或补充目标，空目标列表或连接失败也不会触发重新导入。尚未准备好 Scope 时可省略 targets；之后先完成下文的 Scope 授权与实例范围扩展，再通过公共 MCP 添加。 + +**2\. 配置插件目录并启动 Server 后台服务。** 复用已有 `pc.env` 中的数据库、认证与部署配置，增加目录配置： + +```bash +POWERCONTEXT_SERVER_SOURCE_PLUGINS_DIR=/srv/powercontext/source-plugins +``` + +在 PowerContext 仓库根目录执行： + +```bash +uv run powercontext service install --env-file ./pc.env +``` + +`service install` 安装并启动 Server 后台服务后返回终端；Server 启动时执行以下本机部署流程，无须另行执行 `manager install` 或 `runtime install`： + +| 顺序 | 组件与交互 | +| --- | --- | +| 1 | Server 部署流程扫描可信目录，读取实例配置，准备本机服务配置与内部身份引用 | +| 2 | 启动或复用独立管理服务；管理服务主动连接 Server，建立认证后的管理通道 | +| 3 | 部署流程校验插件声明，使用部署权限注册 Definition、登记实例，并确认执行身份的 Scope 权限与实例可服务范围 | +| 4 | 启动 Runtime，连接管理服务和 Server；Runtime 创建 Context、执行 apply，能力与实例状态同步到 Server | +| 5 | 若实例尚未初始化目标，部署流程在实例就绪后按正常添加规则导入文件中的 targets；插件从 ctx.targets.list() 取得目标，自行决定何时采集 | + +部署权限通过 Server 部署配置准备，不交给日常 Runtime。已有实例重启不会重新导入 targets；运行中添加、停止或恢复的目标状态由管理服务保留。没有 collect 实例时，内置与自定义 write 能力无需启动管理服务。 + +**3\. 安装宿主集成并连接同一个 Server。** Codex 已安装，但 PowerContext 集成尚未安装时，在仓库根目录执行： + +```bash +uv run powercontext setup codex --source . --server-url http://127.0.0.1:8000 +POWERCONTEXT_CODEX_SCOPE_ID='scp_demo' codex +``` + +按现有宿主认证流程准备宿主凭据；第二条命令适用于从当前 Shell 启动的新 Codex 会话。桌面宿主应使用自己的集成配置并重新建立会话，不会继承另一终端的环境变量。通过 `resolve_scope_binding` 核对会话 Scope，并为下面的 Source MCP 调用显式传入同一 Scope。 + +宿主只访问 Server `/mcp`，不直接连接管理服务或 Runtime。插件后台进程不会随 Codex 关闭而停止；新增 Source 插件不要求重新安装宿主集成。 + +## 场景二：新建文档，交给已运行实例采集 + +假设 `yuque-personal` 已经采集文档 A，需要加入文档 B。 + +**1\. Codex 使用已授权的语雀工具创建 B，取得真实 URL。** 例如语雀 CLI： + +```bash +yuque create doc --namespace team/book --title '文档 B' --body-file /tmp/doc-b.md --yes --json +``` + +公共 Source MCP 不创建语雀文档，也不监听所有外部工具操作。 + +**2\. Codex 发现能力和可用实例。** + +```json +{ + "name": "pc_source_capabilities", + "arguments": {"scope_id": "scp_demo", "operation": "add"} +} +``` + +Server 根据当前 PC 权限和已同步目录返回能力、版本、输入 schema 与实例状态。选择发现返回的 `yuque.document` 版本和 `yuque-personal` 实例。 + +**3\. Codex 添加 B。** + +```json +{ + "name": "pc_source_add", + "arguments": { + "scope_id": "scp_demo", + "capability": "yuque.document", + "capability_version": "1", + "instance_id": "yuque-personal", + "input": {"url": "https://www.yuque.com/team/book/doc-b"} + } +} +``` + +Server 检查当前 Scope 权限，再经管理通道转交管理服务；Runtime 调用插件的 identify\_target 确认资料身份，管理服务创建或复用目标并返回 target\_id。响应表示已建立持续采集关系，不表示 B 已保存；相同 Scope、实例、能力版本及资料身份的重复添加复用目标，不恢复已停止目标。 + +**4\. 已运行的插件取得并处理 B。** Runtime 同步 targets，apply 从 `ctx.targets.list()` 取得 B，按自身节奏调用 `run_batch()`；Worker 执行 Connector，插件读取语雀并提交，Source 直接保存到 Server。SDK 自动关联真实提交回执，由 Runtime 向管理服务报告 Source 引用和提交时间。 + +**5\. 用户需要查看结果或停止跟踪时，再调用相应工具。** 查询不是维持采集的必要动作： + +```json +{ + "name": "pc_source_status", + "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b"} +} +``` + +返回的 source\_refs 与 last\_submitted\_at 只表示已确认并报告的提交，不表示 B 已全部采完。使用实际返回的 target\_id 停止 B： + +```json +{ + "name": "pc_source_target_set_enabled", + "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b", "enabled": false} +} +``` + +有 scope.contribute 的用户可以停止或恢复该 Scope 的目标，不限创建者；读取提交结果需要 scope.read。返回表示启停意图已保存；断线中的 Runtime 可能仍按最后同步状态采集。恢复使用相同 target\_id 和 enabled=true，重复 add 不代替恢复。 + +## 场景三：新增插件、Scope 与异机运行 + +**新增本地自定义插件实例。** 部署者准备代码、依赖和一份独立的实例配置，示例目录如下： + +```text +/srv/powercontext/source-plugins/tickets-team/ + powercontext.yml + plugin.py +``` + +```yaml +id: tickets-team +plugin: ./plugin.py +scope_ids: [scp_demo] +config: + api_base: https://tickets.example.org + token_file: /srv/powercontext/secrets/tickets.token +``` + +其中业务参数须由这个插件的 Config 声明。实例 ID 在部署内唯一，入口使用本地路径，无需发布 wheel。Server 已配置上述插件目录并启动后，在同一台机器使用其 `pc.env` 执行： + +```bash +uv run powercontext source load --config /srv/powercontext/source-plugins/tickets-team/powercontext.yml --env-file ./pc.env +``` + +`load` 使用本机受保护的部署凭据，准备或复用共享管理服务，注册 Definition、登记实例、配置执行身份的 Scope 权限，生成 RuntimeConfig 并安装启动 Runtime；首次加载时还会导入可选的初始 targets。它内部调用两个 `install` 所用的服务安装逻辑，因此本场景只需执行 `load`。命令返回后后台进程继续运行，不重启 Server 或已有实例；插件依赖须事先准备。 + +运行中只放入目录不会自动启动，重复加载不覆盖已初始化 targets，也不会重启配置相同的实例。加载后宿主重新调用 `pc_source_capabilities`，按新能力的 `input_schema` 调用 `pc_source_add`，无需修改 MCP 工具清单。 + +三个插件配置文件的用途不同： + +- `powercontext.yml`：用户编写的实例配置，声明实例 ID、插件入口、业务配置、可服务 Scope 和可选的初始 targets，供 `source load` 或 Server 启动扫描读取。 +- `source-manager.yml`：管理服务的部署配置（ManagerConfig），包含 Server 地址及凭据文件、管理服务 ID、监听地址和持久数据目录，供 `manager install` 读取。 +- `plugin-runtime.yml`：单个插件进程的完整部署配置（RuntimeConfig），包含插件目录、业务配置、Server 与管理服务地址、两套连接凭据文件及配置摘要，供 `runtime install` 读取，不能用实例 `powercontext.yml` 替代。 + +`pc.env` 是 Server 的环境配置，供 `load` 确定连接地址与本地部署目录等信息。 + +**给同一实例增加 Scope。** 管理员通过已有 Access 接口授予执行身份所需权限，并经 `POST /v1/source-management/configure` 扩展实例可服务范围；宿主也需要相应 PC 权限。随后添加携带新 scope\_id 的目标即可，不重建 Context，不修改已初始化文件中的 targets。 + +**跨机器手动部署。** 例如机器 A 运行 Server 和共享管理服务，机器 B 能访问工单系统并运行 `tickets-team`；Codex 只连接机器 A 的公共 MCP。机器 A 上的 `source load` 不会到机器 B 安装代码或启动进程。 + +部署者先准备机器间的连接凭据和可访问地址；使用部署权限向 Server 注册 Definition，向管理服务登记实例的 ID、执行位置、配置摘要及可服务 Scope，并通过 PC Access 授予执行身份相应 Scope 权限。机器 B 的插件代码、依赖和 RuntimeConfig 必须与登记信息一致。管理服务尚未启动时，先完成下面的管理服务安装，再登记实例，最后安装 Runtime。 + +```bash +# 机器 A：手动部署共享管理服务；已有管理服务时跳过。 +uv run powercontext source manager install --config ./source-manager.yml + +# 机器 B：登记和授权完成后，启动已准备好的插件 Runtime。 +uv run powercontext source runtime install --config ./plugin-runtime.yml +``` + +这两条命令只安装并启动对应的后台服务，不安装插件依赖，也不自动完成 Definition 注册、实例登记或授权。RuntimeConfig 由部署阶段根据上述信息生成或准备；当前 `source load` 只编排本机部署，不提供远端配置分发或 SSH 执行。 + +管理服务主动访问 Server；Runtime 主动访问管理服务、Server 和外部数据源。跨机器连接需要可访问的安全入口，不能直接使用只监听本机的默认地址。管理服务也可部署在第三台机器，职责不变；每个 Server 只对接一个共享管理服务。 + +**自定义 write：加载实现后直接保存内容。** 例如宿主已经获得工单完整内容，只需保存为 `example-ticket` Source： + +1. 部署者在 Server 机器准备第 4 节的 `tickets` 本地包及依赖，确保 Server 的 Python 环境能够导入它。 +2. 部署者提供入口配置并以管理员权限触发 Server 加载。配置契约示意为 `entry: tickets`；加载流程校验并注册 Definition，保存入口引用，公开 `ticket.write / 1`。 +3. 宿主连接同一个 `/mcp`，执行下面的发现和提交，不创建实例或 target，不准备 manager / runtime 配置： + +```json +{ + "name": "pc_source_capabilities", + "arguments": {"scope_id": "scp_demo", "operation": "add"} +} +``` + +确认发现返回 `ticket.write / 1` 及对应输入模型后： + +```json +{ + "name": "pc_source_add", + "arguments": { + "scope_id": "scp_demo", + "capability": "ticket.write", + "capability_version": "1", + "input": { + "namespace": "team", + "project": "support", + "ticket_id": "T-100", + "title": "登录失败", + "body": "用户反馈无法登录。" + } + } +} +``` + +Server 校验 TicketCapture，调用 TicketDefinition.resolve 和 TicketProjection.project，保存后沿本次调用返回真实 SourceRef。宿主使用该引用读取 Source 或 Projection；无须调用状态查询。此示例的 Definition 根据完整工单快照生成稳定身份，相同内容重试复用已有 Source。若需要持续获取工单变化，则使用前述 collect 场景；两者可以复用同一个 Definition。 + +# 附录 B：插件采集链路参考 + +下面是 collect 的**接口示意**，展示资料身份确认、插件自循环、提交与 checkpoint 的关系；具体类名、模块路径和辅助函数可随实现调整。`BusinessClient`、`DocumentDefinition`、`TemporaryReadError` 和 Config 的外部凭据字段由插件实现；它不是可直接运行的完整客户端，也不能替代当前代码尚未完成的 SDK 调整。 + +插件声明 `plugin_api_version = "2"`、Config、capabilities、definitions 和稳定 Connector 名称与版本。capabilities 描述 collect 的输入与业务契约，并提供 identify\_target 确认资料身份。 + +例如一种 collect 输入为 `{"document_id":"A"}`，其身份入口可写为以下返回值示意： + +```python +async def identify_target(request, config): + # document_id 为 input_schema 要求的非空业务 ID。 + document_id = request.input["document_id"].strip() + if not document_id: + raise ValueError("document_id must not be blank") + return { + "target_key": document_id, + "normalized_input": {"document_id": document_id}, + } +``` + +该入口用于确认资料身份。实际能力有其他采集选项时，normalized\_input 须保留全部影响采集的参数；不能通过省略参数把不同要求当成同一配置。 + +```python +import asyncio + +from powercontext.source_management import ManagementError +from powercontext.sources import ( + ConnectorRunCompletion, + ConnectorRunStatus, + ConnectorSubmissionStatus, +) + +plugin_api_version = "2" +connector_name = "document-reader" +connector_version = "1" +document_definition = DocumentDefinition() +definitions = (document_definition,) + + +class BusinessConnector: + name = connector_name + version = connector_version + source_definitions = frozenset({document_definition.name}) + + def __init__(self, ctx, client): + self.ctx = ctx + self.client = client + self.targets = () + + async def run(self, session): + batch_complete = True + for target in self.targets: + try: + document = await self.client.read(target.input) + if document is None: + # 已确认没有资料；网络或权限错误不能伪装成 None。 + continue + receipt = await self.ctx.targets.submit( + target, + session=session, + item_id=target.target_id, # 本例每个目标提交一项，批内唯一。 + definition_name=document_definition.name, + value=document, + ) + if receipt.status != ConnectorSubmissionStatus.ACCEPTED: + batch_complete = False + except TemporaryReadError: + # 此插件选择后续批次重试，不吞掉取消或其他未处理异常。 + batch_complete = False + return ConnectorRunCompletion( + status=ConnectorRunStatus.COMPLETE if batch_complete else ConnectorRunStatus.INCOMPLETE, + checkpoint=session.checkpoint, # 本例全量读取，不推进增量游标。 + ) + + +async def apply(ctx, config): + client = await ctx.enter_async_context(BusinessClient(config)) + connector = BusinessConnector(ctx, client) + ctx.connect(connector) + while True: + try: + targets = await ctx.targets.list() + for scope_id in sorted({target.scope_id for target in targets}): + selected = tuple(target for target in targets if target.scope_id == scope_id) + connector.targets = selected + await ctx.run_batch(scope_id=scope_id, targets=selected) + except ManagementError as error: + # 尚无初始目标快照时等待；其他配置、权限等错误继续抛出。 + if error.code not in {"targets-unavailable", "management-unavailable"}: + raise + await asyncio.sleep(config.poll_interval) +``` + +SDK 自动记录真实提交关联，Connector 返回的批次结果用于 Worker 与 checkpoint 管线；断线和重启行为见第 5 节。 + +增量或分页插件自行计算 checkpoint，可在同一实例、同一 Scope 的 checkpoint 中维护各目标进度；新增目标必须补采，不能因旧目标的游标较新而跳过。插件提交同一数据须生成稳定的 Source 身份，异常重试沿用已有 Source 保存规则。 + +完整 collect 资源清理及停止规则见第 5 节。write 的实现加载与调用见第 4 节和附录 A。 + +# 附录 C：管理通信与实例配置参考 + +本附录供部署者与框架实现者查阅；宿主使用公共 MCP，插件通过 SDK 获取目标和提交。下列路径与配置字段属于**接口示意**，调用方向、权限和状态语义仍遵循正文约定。 + +## 通道与身份 + +连接方向与请求交接见第 2、3 节，身份认证和权限职责见第 5 节。管理服务和 Runtime 分别使用自己的凭据连接上游。 + +| 接口 | 用途 | +| --- | --- | +| Server `/v1/source-management/channel/open`、`catalog`、`poll`、`reply`、`close` | 管理服务认证建连、同步能力目录、领取消息并应答；目录就绪后，发现由 Server 直接回答 | +| Server `POST /v1/source-management/configure` | Server 检查 server.admin 后登记或明确更新实例；经管理通道执行，不上传任意 Python 代码 | +| 管理服务 `POST /v1/source-management/instances/{instance_id}/sync` | Runtime 使用实例凭据同步自身 collect 目标和控制状态；分页由 SDK 处理 | +| 管理服务 `POST /v1/source-management/instances/{instance_id}/results` | Runtime 报告本实例实际提交的 Source 引用与时间，重复报告复用相同关联 | + +重连后，管理服务重新认证并同步完整能力目录；Server 校验能力名称、版本、kind 和输入契约与已有能力的一致性。管理通道的消息大小、在途数量与等待时间有界。 + +## 实例配置与更新 + +`POST /v1/source-management/configure` 的 registration 表示完整实例登记，`update=true` 表示显式更新。初始 targets 的导入规则见第 2 节；配置更新和升级流程见第 5 节。 + +仅扩展已获授权的 Scope 范围可以保留原 Context；缩减范围或改变执行配置按停用更新流程处理。 From a144d8df2d4e7c3700898ef7f68ba16e017370ec Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=94=98=E9=9C=96?= Date: Thu, 15 Oct 2026 12:44:25 +0800 Subject: [PATCH 2/5] docs(rfc): streamline source plugin proposal --- docs/en/rfcs/0000-source-plugin-system.md | 997 ++++------------------ docs/zh/rfcs/0000-source-plugin-system.md | 997 ++++------------------ 2 files changed, 296 insertions(+), 1698 deletions(-) diff --git a/docs/en/rfcs/0000-source-plugin-system.md b/docs/en/rfcs/0000-source-plugin-system.md index 5e94fb50b..58ee9b7c2 100644 --- a/docs/en/rfcs/0000-source-plugin-system.md +++ b/docs/en/rfcs/0000-source-plugin-system.md @@ -29,16 +29,14 @@ These capabilities need to serve business applications, automation workflows, co # 1. Overview -Hosts use built-in and custom Source capabilities through **the same public MCP on PC Server**, with different execution paths for the two operations: +Hosts use built-in and custom Source capabilities through **the same public MCP on PC Server**: -- **write**: the host already has complete content. PC Server uses loaded SourceDefinition and Projection implementations to save it synchronously and return the result; no plugin instance or management service is started. -- **collect**: plugins continuously read external resources. A shared background management service manages targets, and each plugin instance executes in its own background Runtime process. By default, Server startup discovers instances in a configured directory and automatically starts the local independent management service and Runtimes. Multiple hosts share instances. Remote Runtimes can connect to the same management service, which can also run on another machine. +- **write**: the host supplies complete content. Server executes loaded SourceDefinition and Projection implementations, saves synchronously, and returns. +- **collect**: plugins continuously read external resources. One independent background plugin management service stores collection targets; each instance executes in its own plugin instance Runtime process and can serve multiple authorized Scopes. Management and plugin instance Runtimes can run on different machines, with instances shared by hosts. -Both paths reuse SourceDefinition and Projection contracts. The same Source type can be produced by host writes and plugin collection. +Both paths reuse SourceDefinition, Projection, and existing Source persistence rules. The same Source type can originate from write or collect. The framework provides loading, lifecycle management, and the data pipeline; plugins decide collection content, timing, batching, retries, and exit. -**The framework provides SourceDefinition (output type, identity, and Projection contracts) and the Source saving pipeline; collect plugins decide what to collect, when to execute, when to submit, and when to exit.** collect reuses the existing Worker and checkpoint pipeline, with batch execution invoked by the plugin. write validation and transformation execute within the current Server call. - -One management service serves one Server. Multiple instances, Scopes, and execution machines are supported; management clusters, automatic migration, general-purpose task scheduling, and in-process Python code replacement are out of scope. This document specifies the target design. Appendix A describes directory discovery and automatic startup. +One plugin management service serves one Server. This proposal excludes management clustering, automatic migration, general task scheduling, and in-process Python code replacement. Default local deployment prepares independent background services during Server startup. Installation or startup commands return when done; host exit does not stop collection. # 2. Architecture, connections, and storage @@ -54,8 +52,8 @@ flowchart TB M -->|write| W W --> S end - G[Independent management service] - subgraph RT[Runtime process per collect instance] + G[Independent plugin management service] + subgraph RT[Plugin instance Runtime process] R[Target synchronization and lifecycle] A[Plugin apply and Connector] C[PluginContext.run_batch] @@ -71,167 +69,92 @@ flowchart TB K -->|Source submissions and checkpoint| S ``` -Arrows represent business calls or data flow. write implementations run inside the Server process; collect Context, Worker, and user Connector execute inside the corresponding Runtime process. The management service delivers targets and control state without relaying Source data or scheduling batches. Management messages are delivered in responses to polls initiated by the receiver; network connection directions are described below. +Arrows show business calls and data flow. write implementations run in Server; collect Context, Worker, and Connector run in the plugin instance Runtime process. -| Component | Responsibility and location | +| Component | Responsibility | | --- | --- | -| Public Source MCP | PC Server's `/mcp`; authenticates hosts, checks Scope permissions, and unifies discovery, submission, queries, and enable/disable operations | -| PC Server data pipeline | Loads custom write Definition and Projection code and registers Definitions; saves Source / Projection and checkpoint data and provides reads. Built-in and custom write use the Server saving pipeline directly | -| Shared management service | Independent background process; stores instances, collection targets, and results; executes Server-authorized management requests; checks Scope ownership and execution conditions; synchronizes collect capabilities and instance state to Server | -| Runtime | Outer framework in each collect instance process. A stable instance_id identifies an account, configuration, and execution environment that can serve multiple authorized Scopes. Loads the plugin, maintains management connectivity, synchronizes targets and control state, and creates and ends business runs | -| PluginContext (Context below) | Context for one business run, passed as ctx to apply; owns the Connector and business resources and provides the SDK (interfaces plugins call), including run_batch, submission, and cleanup | -| RemoteConnectorWorker | Existing execution pipeline; called by `run_batch()` to create a session, execute the Connector, and handle submission receipts and checkpoint CAS | -| apply / Connector | Plugin business code; apply decides timing, while Connector.run(session) reads or transforms data and submits it | -| Host-specific plugin | Optional adapter for Hooks, Scope defaults, and context injection; does not host Source instances or duplicate public MCP management logic | +| Public Source MCP | Server `/mcp`; authenticates hosts, checks PC permissions, and provides discovery, submission, query, and controls | +| PC Server data pipeline | Loads write implementations, registers Definitions, and saves and reads Sources, Projections, and checkpoints | +| Plugin management service | Stores plugin instances, capabilities, targets, desired control state, and received submission records; delivers management requests without scheduling collection or relaying Source data | +| Plugin instance Runtime | Outer framework of each plugin instance; loads plugins, maintains management connectivity, synchronizes targets and controls, and creates and ends business runs | +| PluginContext (Context) | Context for one business run, passed as ctx to apply; owns Connector, clients, and resources, and provides plugin interfaces such as run_batch | +| RemoteConnectorWorker | Existing batch pipeline; creates sessions, calls Connector.run, and processes receipts and checkpoint CAS | +| apply / Connector | Plugin business code; apply decides timing, while Connector reads, transforms, and submits batch data | -Only Server is needed for built-in capabilities or custom write. N collect instances add one management service and N instance processes; Server startup prepares these independent services. System services host background processes. Installation/startup commands return when finished, and closing Codex does not stop collection. +write alone requires only Server. N plugin instances add one plugin management service and N plugin instance Runtime processes, hosted by the system service manager. ## Startup loading and adding instances at runtime -**write: load implementations local to Server.** The deployer prepares trusted plugin code and dependencies on the Server machine and configures an entry. Server's generic loader imports it, reads `definitions` and write capability declarations, validates input, Definition, Projection, and name/version conflicts, and exposes capabilities only after SourceDefinition registration succeeds. Deployment requires `server.admin`; the CLI triggers loading, but the Server process imports the code. Registering SourceDefinition structures and contracts through an API does not transfer Python functions and cannot establish that write is executable. - -Server saves successfully loaded entries and required configuration references, then imports and validates them again on restart. Explicit additions and repeated loading of identical configuration are supported; differing configuration conflicts. Existing implementations are replaced through the restart/update procedure in section 5, without in-process Python replacement. Interfaces and examples appear in section 4. This path creates no collect instance, Runtime, or target. - -**collect: load an instance and start an independent Runtime.** The following configuration and deployment flow apply to collect. +**write:** The deployer prepares trusted code and dependencies on the Server machine and configures an entry with `server.admin`. Server's generic loader imports the module, validates Definitions, Projections, capabilities, and name/version conflicts, and publishes capabilities after registration succeeds. The registration API stores definition contracts rather than Python implementations and cannot replace code loading. Entry and required configuration references are persisted for reimport on Server restart. -The deployer specifies a trusted directory through Server's `source_plugins_dir`. Each `powercontext.yml` declares an instance: instance ID, code entry, plugin business configuration, and external credential references. Its `id` becomes the runtime `instance_id`, for example `yuque-personal`. It is unique within a PC deployment and stable across restarts, and is not a code name, Scope ID, or Source ID. Bundled code does not execute merely because it is installed; putting instance configuration in this directory authorizes loading. Different instance IDs can reuse the same code without copying it. +**collect:** Server scans the plugin directory specified through an environment variable once at startup. Each `powercontext.yml` declares an instance `id`, code entry, business configuration, credential references, and serviceable Scopes. id is instance_id, unique within a PC deployment and stable across restarts. Several instances can use the same local code without publishing a wheel; placing configuration in the trusted directory authorizes loading. -`powercontext.yml` may provide initial `targets`, each containing `scope_id`, a capability, and business input. On first startup, if targets have not been initialized for this instance, deployment adds them to the management service through normal add rules and marks the instance initialized. Subsequently, management targets and control state are authoritative. File targets do not overwrite or supplement them or participate in instance configuration conflict detection. An empty target set does not trigger reimport, nor does management connection failure imply an uninitialized instance. +Deployment prepares or reuses the plugin management service, validates the plugin, registers Definitions and the instance, grants permissions, and starts an independent plugin instance Runtime. Adding an instance while running requires explicit loading, without continuous directory scanning or restarting other instances. Repeated loading of the same ID and matching configuration reuses registration and processes without resetting desired state. Conflicting configuration is rejected; updates follow section 5. -Server's deployment flow scans the directory once at startup and supports explicit instance loading at runtime. It prepares independent management and Runtime services through the local service manager and generates and saves their connection configuration. Behavior is summarized below; Appendix A gives usage steps. - -| Situation | Behavior | -| --- | --- | -| No local instances and management service not enabled | Run only Server; explicitly loading the first local instance later starts management on demand | -| First valid instance configuration | Start one management service as needed, validate code contracts and configuration, register Definitions and the instance, then start business execution. Preparation failures show their reason without blocking built-in functionality | -| New instance while Server is running | Prepare code, dependencies, and complete `powercontext.yml`, then explicitly load it. Placing files alone does not start it; existing instances are not restarted | -| Server restart or repeated load of the same plugin instance ID | Reuse registration and process by configured id. Identical configuration preserves control intent and does not call apply again in a surviving Runtime. Required process restarts follow section 5; differing configuration conflicts and requires its update procedure | -| Directory removed or all instances stopped | File deletion is not unload and does not delete targets, history, or the management database. An already started management service keeps waiting; instance disablement is explicit | +Configuration may contain initial `targets`, each specifying Scope, capability, and business input. First target initialization imports them using normal add rules and records initialization. Management is authoritative thereafter; the file no longer replaces or adds targets. Empty targets or connection failure do not trigger reimport, and changes to initial targets do not constitute instance configuration conflicts. -Stopping Server does not implicitly uninstall independent services or delete their state. Submissions fail or await recovery while Server is unavailable, subject to section 5. Remote code and Runtime deployment belong to the execution machine; local loading does not perform remote shell execution or code distribution. +No plugin instances means management need not start. Failure to prepare one instance does not block built-ins. Removing a directory, stopping Server, or disabling all instances does not implicitly uninstall independent services or delete data. Remote machines deploy their own code and plugin instance Runtimes; local loading does not distribute code or invoke remote shells. ## Component communication and connection directions -| Initiator | Receiver | Content | +| Connection initiator | Receiver | Content | | --- | --- | --- | -| Host | PC Server | `/mcp` and existing data-read interfaces | -| Management service | PC Server | Establish the management channel, register capabilities, receive management requests, and return results | -| Runtime | Management service | Instance authentication, resource identity resolution, target synchronization, control state and collection result reports | -| Runtime | PC Server | Worker Source submissions and checkpoint reads/updates | -| Plugin business code | External system | Read resources and check external business permissions | +| Host | Server | Public MCP and data reads | +| Plugin management service | Server | Authentication, capability catalog synchronization, request polling, and replies | +| Plugin instance Runtime | Plugin management service | Instance authentication, resource identification, target/control synchronization, and submission reports | +| Plugin instance Runtime | Server | Worker Source submission and checkpoint reads/updates | +| Plugin business code | External system | Resource access and external business authorization | -The management channel uses **HTTP polling and reporting initiated by the management service**: it receives pending messages from Server and posts results back. Appendix C lists paths. Thus Server hands requests to management at the business level, while management always initiates network requests to Server. Server need not connect to management's listening address; Runtimes must still be able to reach it. +The management channel uses active HTTP polling: management calls Server's `channel/poll` to receive requests and `channel/reply` to return results. Server matches replies to the original MCP call using temporary message IDs. Business requests flow from Server to management, but management initiates all network connections. Plugin instance Runtime likewise initiates connections to management, which must be reachable by plugin instance Runtime. -After connecting, management sends Server its complete collect capability list: names, versions, input requirements, associated instances, instance states, and public scope configuration needed for discovery. This is the capability catalog; management synchronizes changes proactively. Server merges it with built-in and locally loaded write capabilities and answers capability and business-instance discovery directly under the caller's PC permissions. It does not ask management on every discovery. collect add, target queries, control views, and enable/disable operations still use the management channel. The catalog does not replace operation-time authorization and execution checks. write executes locally on Server. +On connection or reconnection, management proactively synchronizes the complete **capability catalog**: capability names, versions, input requirements, instances, state, and public visibility. Changes trigger updates. Server combines this catalog with built-ins and loaded write capabilities and checks current PC permissions to answer discovery directly. Actual collect management operations still use the channel; cached discovery does not replace operation-time authorization and execution checks. ## Persistent data and storage ownership -| Owner | Storage and contents | +| Owner | Authoritative data and recovery basis | | --- | --- | -| Management service | Maintains `/management.sqlite3` without a separate database service. Stores instances, capabilities, collect targets, control state, and collection results; user authorization remains in PC Server | -| Runtime machine | Keeps only startup files: plugin code, startup configuration, and credential references. Target lists, current batches, and receipts are in process memory | -| PC Server | Existing data storage holds final Source, Projection, and checkpoints; also saves successful write code entries and required configuration references for restart. Code and dependencies remain on Server's filesystem; imported objects are in memory. Repeated write uses existing Source storage rules | +| Plugin management service | Its own SQLite stores instances, capabilities, collect targets, desired state, and received submission records, plus last reported execution state and confirmation time | +| PC Server | Existing storage holds Sources, Projections, checkpoints, and user authorization; additionally stores write entries and configuration references, while code and dependencies remain on its filesystem | +| Plugin instance Runtime | Persists only code, startup configuration, and credential references; target snapshots, batches, and undelivered reports remain in memory and are resynchronized after restart | -Management SQLite uses two tables: +Management's `records` stores deployment bindings, instances, capabilities, targets, and submission records (Source references and times). `follow_identity` maps stable collect resource identities to target_id for repeat-add reuse. write uses existing Source storage rules without a separate acceptance ledger. -| Table | Contents | -| --- | --- | -| `records` | Deployment bindings, instances, capability declarations, collect targets, and received submission records (Source references and timestamps). Scope ownership, creator records, and control state belong to their corresponding records | -| `follow_identity` | Maps stable collect resource identities to target_id to reuse targets on repeated add | - -write inputs and responses are used only within the current synchronous call. Server persists the resulting Source and Projection. - -**Management stores what to collect; PC Server stores what has been collected; Runtime performs current execution.** Management persists running collect targets, synchronized by Runtime on startup and reconnection. Initial file targets are not a replica of running state. Management owns desired instance/target control state; PC Server owns Source and checkpoint data. Management stores the latest received execution state and collection results for host queries. - -Runtime process exit loses in-memory work. Restart reruns apply if enabled, using Server checkpoint data to determine subsequent collection positions. Plugins may persist data for their own business needs. - -Source submission and result reporting to management are separate actions. A lost report does not remove a saved Source, but management may lack its submission record; the framework does not guarantee repair after restart. Management and Server databases need durable storage; Runtime needs only its startup files. System service managers start and stop processes without storing collection business state. +**Management stores what to collect; Server stores what has been collected; plugin instance Runtime executes.** Source persistence and submission reporting are separate actions: a lost report does not undo data, and management does not guarantee report backfill after restart. Its result view is not a complete inventory of Server data; both databases require persistent storage. Plugins may persist additional business data as needed. # 3. Public MCP: unified host usage -Hosts connect only to PC Server's `/mcp`. Public MCP provides capability discovery, Source writes, target management, and instance controls. Host-specific plugins provide connection, identity, and Scope context without connecting directly to management or Runtime. - -This section describes the public MCP's **proposed interfaces**, explaining each method's required inputs, complete call/response flow, and result and error meanings. JSON illustrates proposed request and response formats; requests use the MCP `tools/call` parameter structure. Replace Scope, instance, and target IDs with real values. Method and field names may change during implementation review with corresponding RFC updates; authorization checks and behavior remain governed by the design below. - ## Objects to understand before calling | Object | Meaning | | --- | --- | -| capability | Callable business operation with name, version, description, input_schema (business parameter structure), and kind. It is not a Source type; collect may produce several Source types, while write associates its input with a Definition | -| kind | Operation type declared by a capability: write synchronously saves content the host already has; collect accepts an external location or selection criteria and establishes ongoing collection. The capability declaration determines the operation type | -| instance_id | ID of the collect plugin instance, such as yuque-personal; represents an account, configuration, and execution environment and may serve multiple authorized Scopes. write does not use it | -| target_id | Persistent collect relationship ID for queries, stopping, and resuming; not a Source ID | -| SourceRef | Reference to a saved Source, with source_type and source_id. Use it with this call's scope_id to read through existing Server data interfaces | -| enabled | Desired target/instance control state; not confirmation that the execution side completed the transition | -| capability_version | Version of the capability's input and behavior contract, selected from discovery; separate from the output SourceDefinition version | +| capability | Callable behavior with name, version, description, input_schema (business input structure), and kind; not a Source type, and one collect capability may produce several types | +| kind | write receives complete content and saves synchronously; collect establishes continuous collection; declared by the capability | +| capability_version | Version of input and operation contracts, including resource identity rules for collect; separate from the output SourceDefinition version | +| instance_id | collect executor with its configuration, external credentials, and environment; may serve multiple Scopes; write has no instance | +| target_id | Durable collection relationship ID for querying and controls, not a Source ID | +| SourceRef | source_type and source_id of a saved Source, read with its Scope through existing data interfaces | +| enabled / actual | Desired control state / last confirmed business state; neither replaces connection availability | -Invalid parameters, insufficient permissions, and processing failures return errors through the original MCP call. Steps that have not passed their checks are not executed. +Hosts call only Server. After authentication and PC authorization, Server handles write and catalog discovery directly or delivers collect requests through the section 2 channel. Management checks resource Scope and execution conditions and replies; Server then completes the original MCP call. Returning data requires current read permission. Business input cannot declare caller identity; plugins handle external permissions during actual access. Section 5 centralizes operation permissions. -Server authenticates the host on every call and applies operation-specific PC permissions; scope_id specifies data ownership. For collect, management checks target Scope ownership, instance service range, and execution conditions. Plugins handle external accounts, credentials, and business permissions during actual access. - -Server answers capability and business-instance discovery directly. Calls requiring management use this handoff: - -**Management requests share one handoff mechanism:** Server delivers a message in a response to management's `channel/poll`; management processes it and posts to Server's `channel/reply`, after which Server completes the corresponding MCP call. The framework matches the exchange with a temporary message_id, which hosts neither supply nor retain. “Server → management” below describes message flow, not a reverse network connection. Runtime also initiates its management connections. Appendix C explains communication details. - -| Method | When hosts use it | -| --- | --- | -| `pc_source_capabilities` | Discover available operations, inputs, and executable instances | -| `pc_source_add` | Write existing content or add an ongoing collection target | -| `pc_source_status` | Read collect target results or control state | -| `pc_source_target_set_enabled` | Stop or resume one collect target | -| `pc_source_instances` | Discover instances or read instance control state | -| `pc_source_instance_set_enabled` | Enable or disable an instance's entire business run | +The following describes proposed methods, essential parameters, call flows, and return semantics. ## pc_source_capabilities: discover capabilities and input requirements -Call when the host needs to choose a capability, understand input, or confirm available instances. Discovery only reads management information; it does not load code, start apply, or access external resources. - -**Parameters:** scope_id and operation; business discovery here uses `operation="add"`. capability and instance_id can narrow results. - -```json -{ - "name": "pc_source_capabilities", - "arguments": {"scope_id": "scp_demo", "operation": "add"} -} -``` +**Input:** scope_id and operation (`add` for business discovery), optionally filtered by capability or instance_id. -**Call-to-response flow:** +**Flow:** Host calls → Server checks Scope read permission and whether writing is permitted → filters local capabilities and the synchronized collect catalog by visibility and serviceable Scope → directly returns names, versions, kind, input_schema, permissions, instances, and availability. -1. The host calls Server. Server checks Scope read permission and determines whether the caller can contribute. Discovery must not claim writable access when it is absent. -2. Server reads built-in capabilities, loaded custom write capabilities, and the collect catalog synchronized by management. It filters using public visibility, instance Scope ranges, and current PC permissions. This discovery sends no management query. -3. write availability depends on the local implementation being loaded and ready; collect uses catalog instance state and management connection availability. Visible offline or disabled instances may still appear, marked unavailable. -4. Server returns directly through the original MCP call, without channel/reply. Results contain only public information, not instance Config or external credentials. +Discovery does not access external resources or invoke plugin authorization prechecks. Visible offline or disabled instances may appear as unavailable. A disconnected management channel prevents cached information from promising collect execution; records are withheld when visibility cannot be established. Missing management does not affect built-ins or loaded write capabilities. -**Host result:** capability name, version, kind, description, input_schema, PC permissions, and availability; collect additionally includes visible instances, a suggested default, and running state. Permissions and running state are separate: write permission does not imply Runtime connectivity; connectivity does not imply apply is processing work or external resources are accessible. - -The host selects a capability and calls `pc_source_add`. collect also requires explicit instance_id. A default is only a discovery suggestion; retries must not select another account. write selects no instance. The chosen capability version is fixed as well. capability_version prevents upgrades from interpreting original calls and persistent collect targets under incompatible input rules. The same version must preserve inputs, operation meaning, and resource identity rules. - -Without management configured, built-in and loaded custom write capabilities remain discoverable. Management disconnection marks collect unavailable for new management requests rather than failing all discovery or reporting stale cached states as executable. Unavailability does not imply Runtime stopped existing collection. Discovery reflects call-time state; add checks again. +Hosts use the discovered contract to fill input. collect requires explicit instance_id; a suggested default is only a recommendation, and retries do not reselect instance or version. Discovery is a snapshot; add still checks conditions. ## pc_source_add: write content or add a collection target -**Parameters:** scope_id, capability, capability_version, and input; collect also requires instance_id, while write does not. Input follows the discovered input_schema; capability kind determines the operation. - -Server checks current `scope.contribute` and `scope.read`. write requires the capability version and associated Definition/Projection implementations to be loaded and validated on Server. New collect work also requires registered capability/version, completed SourceDefinition deployment validation, a valid Runtime management connection, enabled=true, and recently confirmed actual=running. A connected outer Runtime whose apply has ended cannot accept new collection work. +**Input:** scope_id, capability, capability_version, and input; collect additionally requires instance_id. Server checks `scope.contribute` and `scope.read`, then dispatches by capability kind. ### write: the original call waits for Source persistence -For example, a host with existing text saves it through built-in `content.inline / 2`: - -```json -{ - "name": "pc_source_add", - "arguments": { - "scope_id": "scp_demo", - "capability": "content.inline", - "capability_version": "2", - "input": {"source_id": "meeting-note-001", "text": "Text to save"} - } -} -``` - -source_id is the stable identity of this data and remains unchanged on repeated submission. Custom write uses the same tool with its public capability and business input, without instance_id. - ```mermaid sequenceDiagram participant H as Host @@ -248,300 +171,96 @@ sequenceDiagram S-->>H: SourceRef and save result, or error ``` -**Call-to-response flow:** +1. Server checks capability version and loading readiness, then validates complete input with the associated Definition's input model. +2. It calls `resolve()` to produce a stable Source identity and `project()` for Projections, validates output, and saves through the existing pipeline. +3. After receiving actual persistence receipts, it returns SourceRef and saved / unchanged through the original MCP call. Validation, transformation, or persistence failures return errors. The host need not query processing status separately. -1. The host submits complete content. Server authenticates, checks Scope permissions, capability version, and input requirements, then locates the associated loaded Definition. -2. Server validates input through the Definition input model, calls `resolve()` to form a Source with stable identity, then calls Projection `project()` implementations. Custom functions come from the loaded plugin module; adding a type needs no type-specific import in Server code. -3. Server validates output contracts and saves Source and Projection through existing write rules. Identical identity and data reuse storage; identical identity with different data conflicts. -4. Server obtains the actual persistence result, checks current read permission, and returns SourceRef through the original MCP call. Input, transformation, or storage failures return errors. The call waits for the actual result rather than returning acceptance first; no host query is required. +Built-in and custom write share this path. The same `(scope_id, source_type, source_id)` and data reuse the Source; different data conflict, while different IDs represent different Sources. The underlying submitter owns stable identity: built-in adapters provide it, and custom plugins derive it through Definitions. If the host must provide a resource ID, input_schema says so. -Both built-in and custom write execute this flow inside Server; custom implementations must be loaded first. write accepts complete content and does not continuously fetch remote resources. - -**Successful response example:** - -```json -{ - "kind": "write", - "result": { - "outcome": "saved", - "source_refs": [{"source_type": "content", "source_id": "meeting-note-001"}] - } -} -``` - -SourceRef comes from actual persistence receipts. saved means required writes completed; unchanged means storage confirmed an identical Source. Hosts read content with SourceRef through existing Server data interfaces. - -**Repeated calls and failures:** write reuses `(scope_id, source_type, source_id)` rules: identical identity and data reuse; identical identity with different data conflicts; different IDs create distinct Sources. The underlying submitter owns stable identity: built-in adapters provide it, and custom plugins derive it through their Definition. If a host must supply a resource ID, input_schema says so. Repeated calls may rerun validation or transformation; there is no exactly-once guarantee for external side effects. - -Timeout, disconnection, or Server exit can leave the result unconfirmed and do not prove nothing was saved. Server cleans up current processing when it detects an invalidated call; an already started save may succeed, and reconnection does not automatically replay it. Retries retain the same identity and actual submitted data, without new random IDs or changing metadata. Lost responses do not roll back saved data. +A timeout or lost response does not imply nothing was saved; retries preserve identity and actual data. Server does not recover or replay interrupted calls. Repeated calls may repeat validation and transformation and do not guarantee exactly-once external effects. ### collect: return the collection relationship, then continue processing -For example, add newly created Yuque document B to the running yuque-personal instance: - -```json -{ - "name": "pc_source_add", - "arguments": { - "scope_id": "scp_demo", - "capability": "yuque.document", - "capability_version": "1", - "instance_id": "yuque-personal", - "input": {"url": "https://www.yuque.com/team/book/doc-b"} - } -} -``` - -**Call-to-response flow:** - -1. Server checks host identity, Scope permissions, and input, then hands the call and trusted caller identity to management through the channel. -2. Management invokes plugin `identify_target(request, config)` through Runtime to obtain target_key (stable resource identity) and normalized_input (full normalized input including collection options). This entry is independent of apply and resolves identity from input and instance configuration. -3. Management looks up “Scope + instance + capability and version + target_key.” Matching identity/configuration reuses a target; differing configuration returns target-config-conflict. New targets pass instance Scope-range and readiness checks before target and identity index are persisted together. Uniqueness constraints handle concurrent duplicates. Creator identity is recorded but does not determine target identity or management authorization. -4. Management returns the target through reply. Subject to current read permission, Server completes the original MCP call with target_id, tracking state, and existing collection results. This confirms the relationship, not that external content has been saved. +1. Server checks permissions and input and passes the request with trusted caller identity to management through the channel. +2. Management asks plugin instance Runtime to invoke plugin `identify_target`, obtaining stable `target_key` and complete `normalized_input`, defined in section 4. +3. It looks up **Scope + instance + capability and version + target_key**. Matching identity and configuration reuse a target; conflicting configuration is rejected. New targets additionally require validated Definition deployment, a serviceable Scope, active management connectivity, enabled=true, and running instance business execution before target and identity association are persisted. +4. Management replies; Server returns target_id, enabled, tracking, existing Source references, and latest submission time. A new target has no references or submission time. The response confirms a collection relationship, not saved external content. -**First-creation response example:** - -```json -{ - "kind": "collect", - "target_id": "tgt_b", - "enabled": true, - "tracking": "enabled", - "source_refs": [], - "last_submitted_at": null -} -``` +After the response, plugin instance Runtime synchronizes targets. Existing apply discovers additions through `ctx.targets.list()` and calls run_batch at plugin-selected times. Worker executes Connector and saves Sources directly to Server. SDK associates actual receipts, and plugin instance Runtime reports references and times to management. Hosts choose whether to query progress; queries do not drive collection. -**Collection after the response:** Runtime synchronizes targets from management. Existing apply obtains B through a later `ctx.targets.list()` and chooses its own batches and timing. Worker executes Connector; plugin credentials read Yuque, and data is submitted directly to Server. SDK associates targets and Sources using actual successful receipts. Runtime reports Source references and timestamps to management for persistence. Reports describe received submissions. Worker handles batches and checkpoints under existing rules. Hosts may call `pc_source_status` for progress; querying neither triggers nor sustains collection. - -Adding B does not restart the instance or recreate A's business client. Repeated add of an existing target does not reset progress, trigger extra collection, or resume stopping/stopped targets, and always requires current PC permissions. Identity confirmation requires a reachable Runtime, but reusing an existing target does not require apply to be running. If identity cannot be confirmed, return an error rather than guessing a new target. Management can still query or stop an existing target_id. - -If the acceptance response is lost, repeating the original Scope, instance, capability version, and business input returns the original target. Within one Scope, different users reuse the same instance/capability-version/resource identity with identical configuration. Different Scopes, instances, or capability versions do not merge. Stopped targets retain their identities. This version provides neither hard target deletion nor automatic merging of historical duplicates. +Repeated add does not reset progress, trigger extra collection, or resume a stopped target. Different users in the same Scope reuse matching identity and configuration; different Scopes, instances, or versions do not merge. Identification requires reachable plugin instance Runtime; reusing an existing target does not require running apply. Failure to establish identity returns an error rather than guessing a new target. Stopped targets retain identity; target hard deletion is outside this proposal. ## pc_source_status: query collection results or control state -This method queries collect targets and submission records saved by management. +**Input:** scope_id and target_id; the default view reads collection status, while `view="control"` reads control fields. -**Parameters:** scope_id and target_id. The default view reads collection state; `view="control"` reads only control fields for subsequent control operations. +**Flow:** Server checks the view's permissions → management verifies the target's actual Scope → reads persistent target and received submission records → reply → Server returns to the host. Queries do not call identify_target, wait for a new batch, or require online plugin instance Runtime. -```json -{ - "name": "pc_source_status", - "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b"} -} -``` - -**Call-to-response flow:** - -1. Server authenticates the host. The default view requires current scope.read; control requires scope.contribute. -2. After checking Scope permissions, Server forwards the query through the management channel. Management verifies actual target Scope matches the requested Scope; mismatches are rejected without revealing another Scope's target. -3. Management reads its target and received collection records without identify_target, a reachable Runtime, or waiting for another Worker batch. -4. Management replies to Server, which completes the original MCP call with permitted results. Collection data still requires current read permission; control responses include only authorized control fields. - -**Host result:** - -| View | Returned content and meaning | +| View | Return | | --- | --- | -| Default | target_id, enabled, tracking, source_refs, last_submitted_at, latest confirmation time, and instance availability. source_refs contains received actual submission associations; last_submitted_at is their latest submission time. Without records, these are an empty list and null | -| control | target_id, enabled, tracking; no input, SourceRef, or business error body. scope.contribute permits this view without additionally requiring scope.read | +| Default | target_id, enabled, tracking, source_refs, last_submitted_at, last confirmation time, and instance availability | +| control | target_id, enabled, tracking; no input, Source references, or business error bodies | -tracking represents enabled / stopping / stopped for the collection relationship. source_refs and last_submitted_at reflect confirmed, reported submissions, not complete collection. Without new submissions, they do not distinguish unprocessed, absent, or unchanged content. Submission reports can be lost, so this view is not a complete Server data inventory. An offline instance retains past records but shows offline / unknown availability; past submissions do not establish current collection. - -The call is repeatable without business side effects. Unreachable management returns unavailable, not a fabricated latest state from Server cache. Authorized hosts can still read Sources already saved on Server. +tracking expresses enabled / stopping / stopped. Submission records show confirmed and reported persistence, not complete collection of a target. No new submission does not distinguish unprocessed resources, unchanged content, or absent data. Disconnected plugin instance Runtime is offline / unknown; unreachable management makes this query unavailable, while authorized direct reads of saved Sources remain possible. ## pc_source_target_set_enabled: stop or resume a target -**Parameters:** scope_id, target_id, and enabled. Hosts express stop/resume intent directly. - -```json -{ - "name": "pc_source_target_set_enabled", - "arguments": { - "scope_id": "scp_demo", - "target_id": "tgt_b", - "enabled": false - } -} -``` - -**Call-to-response flow:** +**Input:** scope_id, target_id, enabled. -1. Server checks scope.contribute for the target Scope and forwards the call through the management channel. -2. Management verifies target Scope ownership and, when resuming, the instance service range, then saves enabled while preserving target identity, existing Sources, and checkpoint. Setting the current state again creates no action. -3. Management returns control state through Server to the original MCP call. Runtime synchronizes target control state and applies it to collection. +**Flow:** Server checks scope.contribute → management verifies target Scope and, when resuming, instance serviceable scope → persists desired state → reply → returns target control state; plugin instance Runtime synchronizes and applies it. -The response confirms persisted intent, not completed execution. Resuming a target does not enable a disabled instance or rerun an ended apply. +The response confirms saved intent rather than completed execution. Target identity, Sources, and checkpoint remain. Resuming a target does not enable a disabled instance or rerun completed apply. ## pc_source_instances: discover instances or read instance control state -This method manages collect instances. Server-local write capabilities have no associated instance. - -capabilities starts from “what can be done”; instances starts from “which instances are available and how is this instance running.” It loads no code and starts no instance. - -**Two parameter combinations:** - -| View | Parameters | Purpose | -| --- | --- | --- | -| Business | scope_id, operation="add" | Find visible instances serving this Scope, PC permissions, and running state | -| Control | instance_id, view="control" | Read authorized control state for an instance without specifying Scope | - -```json -{ - "name": "pc_source_instances", - "arguments": {"scope_id": "scp_demo", "operation": "add"} -} -``` - -```json -{ - "name": "pc_source_instances", - "arguments": {"instance_id": "yuque-personal", "view": "control"} -} -``` - -**Call-to-response flow:** - -1. Server authenticates. Business view checks scope.read and determines current write permission; control view checks server.admin. -2. Server directly answers business view from synchronized public visibility, Scope ranges, and instance state without asking management. Catalog and connection handling match capabilities. -3. After server.admin authorization, Server forwards control queries to management without additional Scope permission requirements. Management reads persistent instance control state and latest execution reports, replies to Server, and Server returns to the host. Neither view reruns apply or accesses external accounts. - -**Host result:** business view returns instance IDs, capabilities, PC permissions, and running state. Control view returns instance_id, enabled, actual, availability, and latest update time, without business Config, target inputs, or Source results. - -Desired enabled=true does not guarantee actual=running. Normal apply completion can be actual=complete; unhandled errors can be failed, while outer Runtime remains online. Control view distinguishes desired and actual state; enabled alone must not be described as collecting. +| View and input | Call-to-return flow | +| --- | --- | +| Business: scope_id, operation=add | Server checks Scope permissions and directly returns visible instances, capabilities, writability, and state using capabilities catalog rules | +| Control: instance_id, view=control | Server checks server.admin, queries persistent instance control state and latest execution reports through management, then returns instance_id, enabled, actual, availability, and update time | -Calls are read-only and repeatable. Business view uses Server's synchronized public information; invalid management connectivity marks the custom collect portion unavailable. Old catalogs do not guarantee current permission or executability, and records are withheld when visibility cannot be confirmed. Control view requires reachable management. If only Runtime is offline, persistent control state remains readable but running availability is unknown/offline. Built-in and custom write need no plugin instances. +Control view requires no Scope and returns no Config, target input, or Sources. Persistent control state remains readable when only plugin instance Runtime is offline, with offline or unknown availability; unreachable management makes control queries unavailable. This method does not load code, access external accounts, or start apply. write has no instances. ## pc_source_instance_set_enabled: enable or disable instance business execution -This method controls apply / Context inside Runtime, affecting all Scopes and targets served by the instance. It is not a command to install, uninstall, or start a remote system process. - -**Parameters:** instance_id and enabled. Hosts express intent directly. +**Input:** instance_id, enabled. This affects every Scope and target served by the instance, not installation or remote system-process startup. -```json -{ - "name": "pc_source_instance_set_enabled", - "arguments": { - "instance_id": "yuque-personal", - "enabled": false - } -} -``` - -**Call-to-response flow:** - -1. Server authenticates and checks server.admin before forwarding instance control. An instance may serve multiple Scopes; one Scope's permission cannot control the entire instance. -2. Management checks instance existence and transition conditions, then transactionally sets desired state in processing order. Same-state requests return current results. Disable saves enabled=false and, if execution confirmation is outstanding, stopping. Enable requires the old run confirmed stopped, then saves enabled=true, creates a new run_id (business-run identity), and waits for Runtime execution. -3. Management replies with persistent control state; Server returns it to the host. Fields such as instance_id, enabled, and actual do not mean the system process was installed or business execution started successfully. -4. Runtime synchronizes and applies desired state. Disable blocks new batches, cancels and awaits apply and owned subtasks, and closes Context/business resources. It reports stopped only after completion; outer Runtime stays online. Enable creates a new Context, calls apply, and reports actual state. Hosts may use instances control view for confirmation; querying does not drive transitions. - -An offline Runtime permits saving disable intent but not reporting completed stopping. Cleanup failure or surviving old tasks prevents creating a second Context. Management disconnection prevents confirmation of control changes. Re-enabling processes only still-enabled targets; it does not resume individually stopped targets or reset their checkpoints. +**Flow:** Server checks server.admin → management checks transition conditions and persists desired state → reply → returns instance_id, enabled, actual → plugin instance Runtime synchronizes and executes. Section 5 defines cleanup and restart behavior. -Within one Runtime process, repeated enable while already enabled does not rerun apply, even after complete or failed. Rerun requires explicit disable, confirmed stopped, then explicit enable. After Runtime is shut down or uninstalled, management cannot remotely start its system process. See sections 2 and 5 for deployment. +Responses do not promise execution has started or stopped. Disablement can be saved while plugin instance Runtime is offline, but stopping cannot be confirmed; reenabling requires confirmed stopped business execution. Repeated enablement does not rerun apply even after complete or failed. Rerunning requires explicit disablement, confirmed stop, and enablement. Individually disabled targets remain disabled when the instance is reenabled. ## Operation retries -Repeated collect add reuses targets; repeated write reuses underlying Source rules; controls directly set desired state. Setting the same state creates no action, and different intents take effect in processing order. Communication timeout does not roll back saved data or control changes. Hosts may retry or read state according to each method's rules. +Discovery and queries are read-only. collect add reuses stable target identity; write reuses underlying Source rules. Controls set desired state: identical state causes no new action, and different intents take effect in processing order. Communication timeout does not undo persisted data or control state; hosts can repeat calls or read state under these rules. # 4. Plugin interfaces and data pipeline -This section is for plugin authors. write provides input models, Definition, and Projection implementations that Server can load. collect additionally provides apply and Connector, controls its own timing, and submits through Worker. Section 3 describes host calls; section 5 describes loading, process controls, and restart behavior. - ## Contracts provided by plugins -Plugin code can load from a local directory without publishing a wheel. write code and dependencies reside on the Server machine; collect code and dependencies reside on the Runtime machine. Both may reuse Definition and Projection implementations from one package without splitting packages. - -**Shared data contracts.** SourceDefinition declares its input model, Source model, name, and version. `resolve(value)` converts input to a Source with stable identity; `project(source)` on each implementation in `projections` produces its Projection. `definitions` exports these objects. Capability declarations contain name, version, description, kind, and input_schema and describe callable operations; Definitions describe how results are saved. One collect capability can submit several Source types, and several capabilities can reuse one Source type. +Plugins can load directly from local directories. write code and dependencies reside on the Server machine; collect code resides on the plugin instance Runtime machine. Both can reuse data definitions from the same package. -**write loading contract.** Each write capability explicitly associates a Definition name and version, with input_schema matching its input model. Server can validate input and call the associated Definition directly. Plugins export `definitions` and `capabilities` for Server to load. Capability names cannot override built-ins; identical name/version combinations must preserve input and operation semantics. +**Shared data contracts:** SourceDefinition declares input and Source models, name, and version. `resolve(value)` produces a stable Source identity; `project(source)` implementations in `projections` produce Projections. Plugins export implementation objects in `definitions` and declare capability name, version, description, kind, and input_schema in `capabilities`. Capabilities describe callable behavior; Definitions describe output data, without requiring a one-to-one type mapping. -The following **interface sketch** illustrates how Server obtains the Projection function by relating plugin exports to Server loading. Concrete class names, module paths, and supporting code may change during implementation; write association fields and deployment entry support remain to be implemented. Suppose a deployer prepares this importable local package: +**write loading:** Each write capability associates a Definition name and version, with input_schema matching its input model. Plugin modules export actual Definition and Projection objects. Server's generic loader imports and registers them through a trusted entry, so Server needs no type-specific imports. Adding a module requires code, dependencies, and entry configuration; uploading definition structure through registration alone cannot supply executable functions. Import does not start collection tasks. -```text -/srv/powercontext/custom_sources/tickets/ - __init__.py - ticket_definition.py -``` +**collect runtime contracts:** -`ticket_definition.py` implements TicketCapture, TicketDefinition, and TicketProjection. The key relationship is shown below, omitting the existing Source model and resolve implementation: - -```python -class TicketProjection: - # Also declare Projection name, version, input Source class, and output model. - def project(self, source): - return TextEvidence( - source_type="example-ticket", - source_id=source.name, - content=f"{source.capture.title}\n\n{source.capture.body}", - ) - - -class TicketDefinition: - name = "example-ticket" - version = "1" - input_class = TicketCapture - projections = (TicketProjection(),) - # resolve() produces TicketSource with stable identity; other members follow SourceDefinition. -``` - -The plugin's own `__init__.py` exports objects. Type-specific imports belong in the plugin: - -```python -from .ticket_definition import TicketCapture, TicketDefinition - -definitions = (TicketDefinition(),) -capabilities = ({ - "name": "ticket.write", - "version": "1", - "kind": "write", - "description": "Save ticket content supplied by the host", - "input_schema": TicketCapture.model_json_schema(), - "definition_name": "example-ticket", - "definition_version": "1", -},) -``` - -Server's configured entry points to this package, imported by a generic loader. This shows only the imported object relationship, not a complete loader: - -```python -module = importlib.import_module(entry) # For example tickets, from trusted deployment configuration. -registry = SourceDefinitionRegistry(module.definitions) -# registry.resolve / registry.project call these actual Python objects at runtime. -``` - -Server can execute TicketProjection.project because **it imported the local module containing that object**. Adding a type adds a module and entry configuration, not a hardcoded type-specific Server import. The deployer makes code available on Server's module search path and installs dependencies in that environment. The generic loader also resolves file-path entries. Implementations validate and transform current input without starting collection tasks during import. - -**Registering SourceDefinition and loading code are separate actions.** The existing registration API saves type structure, identity, and Projection contracts, not Python implementations. Server's write loader validates code, contracts, and conflicts and publishes capabilities only after successful registration. Registration alone cannot execute write. collect still registers Definitions with `server.admin` during deployment or upgrade. Daily Worker execution uses only Scope execution permissions, without registering each batch or temporarily acquiring admin credentials. Missing or mismatched Definitions produce explicit errors. - -**Additional collect runtime contracts:** - -| Export or object | Plugin definition | +| Export | Plugin responsibility | | --- | --- | -| `plugin_api_version` | Interface version used with Runtime, currently the string `"2"` | -| `Config` | Pydantic model defining and validating configuration needed by plugin logic, such as external credentials or credential references | -| `capabilities / definitions` | Nonempty collect capability declarations and SourceDefinition collection; capability input describes external locations or selection criteria | -| `connector_name / connector_version` | Stable Connector name and compatibility version for checkpoint ownership; cannot change randomly on restart | -| `identify_target(request, config)` | Derives stable resource identity and normalized collection parameters from input | -| `apply(ctx, config)` | Async business entry; creates clients, connects a Connector, and decides batches, waits, retries, and exit | -| `Connector.run(session)` | Processes and submits the batch, returning ConnectorRunCompletion (batch completion and next checkpoint) | +| plugin_api_version | Declares the interface version used with plugin instance Runtime | +| Config | Pydantic model for plugin business configuration, such as external credential references; not public to hosts | +| capabilities / definitions | Nonempty capabilities and data definitions; capability input describes external resources or selection criteria | +| connector_name / connector_version | Stable Connector name and compatibility version for checkpoint ownership | +| identify_target(request, config) | Derives stable resource identity and normalized parameters from capability, version, input, and instance configuration | +| apply(ctx, config) | Async entry; creates clients, connects a Connector, and decides loops, batches, waits, retries, and exit | +| Connector.run(session) | Reads and submits batch data, returning ConnectorRunCompletion: batch completion and next checkpoint | -collect Config remains with the instance. Plugins handle external credentials and permissions. Capability descriptions and input_schema expose only what hosts need to call them; catalogs, results, and ordinary logs do not expose secrets. Same-version collect capabilities must also preserve resource identity rules. - -**Identifying the same resource for collect:** Runtime invokes identify_target during target acceptance, independently of apply's collection loop. The request contains capability, capability_version, and input. The plugin returns: - -| Result | Meaning | -| --- | --- | -| target_key | Stable resource identity used by management to reuse a target | -| normalized_input | Complete normalized collection parameters, preserving all options that affect collection, for detecting configuration conflicts on the same target | +`identify_target` runs independently of collection and returns `target_key` and `normalized_input`. The former identifies a resource, such as a document ID after removing a navigation-only URL fragment. The latter preserves every collection-affecting option to detect configuration conflicts. This method does not check authorization or readability, which are handled during collection; insufficient input for identity produces an explicit failure. -For example, a plugin may remove URL fragments used only for page navigation and identify two addresses as the same target. identify_target resolves resource identity from input and instance configuration; content and readability are handled during actual collection. input_schema must require sufficient identifiers, and equivalence must not be promised when it cannot be established. +Capabilities cannot override built-ins. The same name/version preserves input and operation semantics; collect also preserves target identity rules. Administrators register Definitions during deployment or upgrade. Daily Worker execution uses only Scope permissions and fails on missing or mismatched definitions without registering every batch. ## collect: from apply to Source persistence -`ctx` is the PluginContext passed by Runtime to apply. It holds the Connector and business resources and exposes the SDK below. One Context connects exactly one Connector. Worker creates `session` per batch, providing its Scope, current checkpoint, and submission capability. - -One apply can execute multiple batches. **One `run_batch()` is one batch, fixed to one Scope and a selected set of content.** A collect target may continue producing Sources over many batches. SDK associates targets with actual submissions; hosts do not generate batch IDs. - -The diagram shows one batch's call order. apply and Connector on the left are plugin code; SDK and existing RemoteConnectorWorker are in the middle, and PC Server is on the right. Arrows indicate calls/returns, not process counts. It shows successful collect submission, and submit can be called repeatedly. binding identifies this instance's execution within a Scope; isolation is explained at the end of this section. +Context holds one Connector and business resources. Worker creates each batch's session with Scope, checkpoint, and submission capability. **One run_batch is one batch, fixed to one Scope and selected targets; one apply can execute many batches.** ```mermaid sequenceDiagram @@ -562,7 +281,7 @@ sequenceDiagram SDK->>S: session.submit / Source and Projection S-->>SDK: Actual submission receipt SDK-->>C: ConnectorItemOutcome - Note over SDK: Associate target and SourceRef for Runtime reporting + Note over SDK: Associate target and SourceRef for plugin instance Runtime reporting C-->>W: ConnectorRunCompletion W->>S: CAS checkpoint under completion and receipt rules W-->>SDK: Batch result @@ -570,46 +289,23 @@ sequenceDiagram Note over A: Plugin decides wait, next batch, or exit ``` -apply obtains work, selects entries for one Scope, and calls run_batch. Worker reads checkpoint and invokes Connector.run; Connector reads the external system and transforms data as needed, then submits through SDK. The plugin decides waiting and subsequent batches. - -| Step | collect interface | -| --- | --- | -| Obtain currently eligible work | `ctx.targets.list()` | -| Execute the selected batch | `ctx.run_batch(scope_id=..., targets=selected)` | -| Submit one item | `ctx.targets.submit(...)` | - -list returns a target snapshot for current processing: - -| Field | Meaning | -| --- | --- | -| target_id | Corresponding target persisted by management | -| scope_id, instance_id, capability, capability_version | Ownership and capability contract | -| input | Normalized collection parameters confirmed by identify_target | -| source_refs | Actual submission associations received by management, available for plugin reference; not proof of complete collection | +Plugins obtain synchronized target snapshots through `ctx.targets.list()` (identity, ownership, normalized input, and reported Source references), select targets in one Scope, and call `ctx.run_batch(scope_id=..., targets=selected)`. Batches execute sequentially within a Context without concurrent mutation of the batch snapshot. Managed execution supplies an explicit selection, with an explicit empty set for autonomous collection; the standalone entry retains `run_batch(scope_id=...)`. -After initial synchronization, ctx.targets.list / run_batch may use the last in-memory target/control snapshot while management is disconnected. Without an initial snapshot, they report unavailable rather than pretending there is no work. New targets appear after synchronization. - -Batches in one Context execute sequentially without overlap. Each selects targets from one Scope without concurrently modifying its snapshot. Managed mode explicitly passes the selected set; autonomous collection uses an explicit empty set, and omission does not mean all targets. Direct execution retains `run_batch(scope_id=...)`. - -submit accepts the current entry, session, batch-unique item_id, Definition name, and business value. SDK checks instance, Scope, and batch ownership, wraps `session.submit(...)`, and associates the actual receipt with that entry. Returned ConnectorItemOutcome describes the submission result; plugins must check acceptance. Processed data goes directly to PC Server rather than through management. - -Actual collect submission associations are reported to management for persistence. Appendix B shows full apply and Connector code. +Connector submits through `ctx.targets.submit(target, session, item_id, definition_name, value)`. SDK checks instance, Scope, and batch membership, wraps session.submit, and associates targets with Sources using actual receipts. item_id is unique within the batch; plugins check whether the returned ConnectorItemOutcome was accepted. Data goes directly to Server, not through management storage. ## collect receipts and batch completion -Plugins maintain loops, pagination, and retries and submit through `ctx.targets.submit(...)`. Submission receipts describe persistence results; ConnectorRunCompletion describes batch completion. An ongoing collection target may produce Sources across multiple batches. - -| Action | Meaning | +| Signal | Meaning | | --- | --- | -| submit receives a successful receipt | Server accepted one item; SDK associates its actual SourceRef with the collect target | -| Connector returns ConnectorRunCompletion | Declares batch completion and next checkpoint; Worker validates and processes progress under existing rules | -| collect submission report reaches management | Saves confirmed Source references and timestamps for optional host queries, without waiting for the whole target or other targets in that batch to finish | +| Successful submission receipt | Server accepted one item; SDK can associate and report its real SourceRef without waiting for other targets | +| ConnectorRunCompletion | Batch completion and next checkpoint, processed by Worker under existing receipt/completion rules | +| Management receives submission report | References and times are stored for host queries; continuous collection is not declared fully complete | -Without new submissions, the framework does not infer whether a target is unprocessed, unchanged, or empty. Plugins decide retries and continuation; unhandled exceptions end apply as described in section 5. Source saving, checkpoint updates, and reporting are separate: incomplete batches or lost reports do not undo saved Sources, and reports are not guaranteed to be restored after restart. +Plugins decide loops, pagination, and retries. Source persistence, checkpoint advancement, and reporting are separate actions; section 5 describes failure recovery. The framework does not infer business outcomes from the absence of new submissions. ## Checkpoint isolation, advancement, and new targets -ConnectorBinding describes which instance's Connector executes in which Scope. This design uses instance_id as binding_id. PC Server stores checkpoints by `(scope_id, binding_id)` and verifies connector_name / connector_version. ConnectorBinding differs from the permission system's role-granting Scope Binding. +ConnectorBinding identifies an instance's Connector execution within a Scope, using instance_id as binding_id. Server stores checkpoints by `(scope_id, binding_id)` and checks Connector name/version. This differs from the authorization system's Scope Binding. ```mermaid flowchart TB @@ -625,35 +321,21 @@ flowchart TB A --> P[Plugin-defined format may include per-target progress] ``` -Server stores every checkpoint in the diagram; plugins define the content format. Multiple targets or capabilities in the same instance and Scope share one checkpoint. The framework does not automatically create a cursor per target; plugins may record individual target progress inside it. Different instances or Scopes do not share checkpoints. - -Worker reads checkpoint at batch start, and Connector returns the next value based on business progress. Updates use CAS: save the new value only if Server still has the previously read value, preventing overwrites of concurrent progress. Plugins must not advance past unprocessed data. - -For example, A has been collected through time T when document B, older than T, is added. The plugin must not skip B using A's cursor. It recognizes the new target, initializes B's progress, and backfills it using a plugin-defined representation. Stopping/resuming targets does not automatically clear checkpoints; changing Connector versions cannot bypass compatibility checks on existing progress. +Plugins define checkpoint content. Targets and capabilities within one instance and Scope share a checkpoint, which can hold per-target progress. Worker reads the current value and uses CAS to advance according to Connector's returned business progress (update only while stored state still matches the value read for this batch), without skipping unprocessed resources. -After Runtime process restart, targets are resynchronized and Worker reads Server's current checkpoint. Plugins decide whether to continue or recollect; the framework does not restore individual steps from the previous batch. +For example, if A is collected through time T and newly added B predates T, the plugin initializes and backfills B rather than skipping it using A's cursor. Target controls do not clear checkpoints. Restart reads Server progress without restoring batch execution steps; upgrades cannot evade progress compatibility checks by changing versions. # 5. Lifecycle, restarts, and permissions -Runtime, instance, and target lifecycle in this section applies to collect; apply still controls collection timing. write implementations load and release with Server without separate instance controls. New implementations can be loaded explicitly, and identical repeat loads reuse registration. Existing code replacement takes effect through controlled Server restart. Saved entries are revalidated on startup; failed capabilities are not marked executable, while other valid capabilities remain usable. +write implementations load and release with Server. New implementations load explicitly, and matching repeated loads reuse registration. Replacing existing code requires a controlled restart. Startup revalidates entries; failed capabilities are unavailable while other valid capabilities remain usable. The instance lifecycle below applies to collect. -## Runtime connectivity and apply execution +## Plugin instance Runtime connectivity and apply execution -In managed background mode, each instance has one Runtime process. Runtime maintains management connectivity and synchronizes targets/control state. **Context is the Connector, clients, and business tasks owned by one apply run.** Business waits or sleep do not block Runtime from receiving management operations. - -Instances distinguish three things: - -| Information | Meaning | -| --- | --- | -| enabled | Desired control state stored by management; true does not mean apply is currently running | -| actual | Latest reported business state, such as running, normal completion (complete), failure (failed), stopping, or stopped | -| Connection availability | Whether Runtime can communicate; disconnected means offline / unknown, not that an old actual state proves ongoing execution | - -**Background Runtime remains online after apply ends.** It can receive controls but does not rerun apply because a target is added, connectivity recovers, or enable is repeated. Plugins that continuously handle new targets keep their own loop inside apply. +Plugin instance Runtime maintains management connectivity; Context owns one apply run's resources. enabled is desired state, while actual is last confirmed running / complete / failed / stopping / stopped. Connection failure is separately offline / unknown; old state cannot promise active collection. ```mermaid flowchart TD - S[Runtime starts and syncs instance state and targets] --> E{enabled?} + S[plugin instance Runtime starts and syncs instance state and targets] --> E{enabled?} E -->|No| I[Remain online waiting] E -->|Yes, old process exited| R[Create Context and run apply] R -->|Normal return| C[Clean up Context, report complete] @@ -663,458 +345,75 @@ flowchart TD R -->|Disable| D[Cancel and await tasks, close resources] I -->|Disable| D I -->|Enable from disabled, old run confirmed stopped| R - D -->|Cleanup complete| T[Confirm stopped, Runtime stays online] + D -->|Cleanup complete| T[Confirm stopped, plugin instance Runtime stays online] T -->|Explicit enable| R I -->|New target, reconnect, or enable while already enabled| I D -->|Cleanup failed| U[Report error, no second Context] ``` -Uninstalling the system service exits Runtime; starting it again requires a deployment operation on the execution machine. +After apply returns or fails, Context is cleaned up but plugin instance Runtime stays online. New targets, reconnection, or repeated enablement do not rerun apply. Plugins needing continuous target handling maintain their own loop without blocking management operations while waiting. Internal run identifiers distinguish reports for the current connection and run; hosts do not provide them. -To rerun an ended apply within the same process, explicitly disable, confirm stopped, then enable. Direct `pc-source run` exits when apply ends; direct and background execution cannot occupy the same instance identity simultaneously. - -Management saves desired state, latest actual state, and update time. Internal run_id distinguishes apply reports; only the current connection/run's messages are accepted so late reports cannot overwrite newer state. Hosts do not supply run_id, and Runtime does not use it for cross-process business recovery. +Rerunning within a process requires disablement, confirmed stopped state, and enablement. Process restart instead synchronizes and decides whether to create a Context from enabled. The direct execution entry exits when apply ends and cannot share instance identity with background mode. After a system service exits or is uninstalled, deployment tooling on that machine must restart it. ## Instance disablement and resource cleanup -To disable an instance, management saves enabled=false. Runtime blocks new batches, cancels and awaits apply and its subtasks, and closes Context. After cleanup, it reports stopped; outer Runtime remains online awaiting enable. - -Business subtasks must belong to apply, for example through TaskGroup. Resources registered with `ctx.enter_async_context` / `ctx.on_dispose` close with Context. Plugins cannot leave detached background tasks or swallow cancellation. If tasks remain or cleanup fails, do not report stopped or create a second Context. +On disablement, plugin instance Runtime blocks new batches, cancels and waits for apply and its child tasks, and closes Context. It reports stopped only after cleanup; the outer process stays online. Business tasks belong to apply, and resources register with Context for cleanup. Plugins must not leave detached tasks or swallow cancellation. Cleanup failure or surviving old tasks prevent stopped confirmation and creation of a second Context. -Instance disablement does not affect loaded write capabilities or undo saved Sources. An offline Runtime cannot be claimed stopped; unreachable management cannot confirm persistence of new control intent. +Disablement does not undo saved Sources or affect Server write capabilities. Disconnection does not mean stopped; unreachable management cannot confirm a new control state was saved. ## Disconnections, restarts, and data recovery -**Reconnection preserves process memory; process restart creates a new business run.** Server or management restart is not Runtime restart. The system service manager maintains processes, not Worker invocations. - -| Situation | Subsequent behavior | +| Situation | Behavior | | --- | --- | -| Runtime lacks an initial target snapshot | collect list / run_batch reports unavailable; plugin waits or exits. Process restarts also need targets first | -| Synchronized Runtime loses management | collect may start new batches using the last in-memory target/control state on plugin timing. Server still checks submission permissions; reports may be lost | -| Same Runtime process reconnects | Synchronize latest instance/target control before further processing. Apply synchronized control state; do not rerun active or ended apply | -| Runtime process restarts | In-memory batches and undelivered reports are lost. After synchronization, if enabled, create Context and execute apply even after previous complete / failed. Worker reads Server checkpoint | -| Only management-to-Server channel fails, while Runtime still connects to both | Host collect management is unavailable. Existing collection may continue under synchronized state. Built-ins and loaded custom write remain independently usable | - -Do not start another Runtime when the old process's exit cannot be confirmed. Heartbeat loss is not grounds for cross-machine takeover. apply and external side effects are not guaranteed exactly once; repeat submissions use underlying Source identity rules. - -Runtime retains startup files only, with execution information in memory. Section 2 defines storage ownership. Recovery uses management targets/control state and Server Sources/checkpoints. +| No initial target snapshot | list / run_batch reports unavailable; plugin decides to wait or exit, without treating failure as an empty target list | +| Plugin instance Runtime disconnected from management | Collection, including new batches, may continue with the last in-memory targets and controls; Server still checks submission permissions | +| Same plugin instance Runtime reconnects | Synchronize current controls before further processing; do not rerun active or ended apply | +| Plugin instance Runtime process restarts | Resynchronize targets/state and run new apply if enabled, even after previous complete / failed; Worker reads Server checkpoint, and undelivered reports are lost | +| Management disconnected from Server | Host collect management is unavailable; existing collection can continue, while built-ins and loaded write remain independent | +| Server unavailable | Submissions fail or wait for recovery; saved Sources remain, and write calls are not automatically recovered or replayed | -collect **Source saving, checkpoint update, and submission reporting** are three separate actions; section 4 describes normal processing. Failures are handled separately: +If exit of the previous process with the same identity cannot be established, another plugin instance Runtime must not start. Missing heartbeats do not authorize cross-machine takeover. The framework does not guarantee exactly-once apply or external effects. -| Failure point | Recovery and effect | -| --- | --- | -| Source saved before checkpoint update | Later collection may repeat; identical identity/data reuse storage. Plugins maintain stable Source identity and handle changed data according to their Definition | -| Checkpoint update response lost | Read actual Server progress next time. Do not claim this update confirmed, but actual successful submission receipts may still be reported | -| Source and checkpoint saved, report lost | Data and progress remain valid; management may lack submission records. Do not roll back checkpoint, and do not guarantee report repair after restart | - -Missing management records do not prove Server lacks Sources; authorized hosts can still read through Server data interfaces. collect continues on plugin timing without restoring previous batch steps. Server restart interrupts in-flight write. After restart implementations reload, but old calls are neither resumed nor replayed. Saved Sources remain; retries retain the same identity and data. +Recovery follows actual persisted data: a saved Source without an advanced checkpoint may be recollected using stable identity; a lost checkpoint-update response requires rereading it. Lost submission reports do not roll back Sources or checkpoints and are not guaranteed to be backfilled in management records. ## Adding instances, configuration changes, and upgrades -| Change | Handling | -| --- | --- | -| New instance | Prepare code/dependencies; deployment registers Definitions and instance and starts Runtime. Publish new capabilities without restarting existing instances | -| Add a Scope to a running instance | Grant its execution identity the appropriate Scope Binding and update service range. After Runtime sync, hosts can add targets in that Scope while preserving Context | -| Change business configuration or code | Disable and confirm stopped, stop old Runtime, then validate and update code, Config, and registration. Start the new Runtime and explicitly enable. External credential changes use the same configuration-update flow; plugins interpret their business meaning | +- **New instance:** Deploy code and dependencies, register Definitions and the instance, and start plugin instance Runtime without restarting others. +- **Expand Scopes:** Grant execution identity the relevant Scope Binding (Server role assignment) and update serviceable scope. After synchronization, targets can use the new Scope with the same Context. Passing scope_id alone grants no permission. +- **Reduce scope, change execution configuration, or upgrade collect:** Disable and confirm stopped, exit old plugin instance Runtime, validate and update code, Config, and registration, then explicitly enable after the new plugin instance Runtime validates. Credential changes use configuration updates, with business meaning interpreted by the plugin. -Scope Binding is Server's relation granting an identity a role in a Scope, not section 4's checkpoint ConnectorBinding. Passing a new scope_id grants no permissions and does not require a separate plugin instance per Scope. +Upgrades preserve Sources, targets, checkpoints, and received submission records. Validate compatibility of capability input and target identity, Definitions, Connector, and checkpoint formats. New code must handle retained targets, including disabled ones, without resetting progress to bypass checks. Uncertain compatibility keeps the instance disabled for explicit deployment handling, without automatic migration. New instances do not inherit old targets automatically. -Confirm old Runtime exit before updating collect code/configuration; retain Sources, targets, and checkpoints. For write, first handle in-flight Server calls, then load new code through controlled restart, preserving capability/Definition compatibility and Source identity without migrating old calls. collect upgrades check three compatibility dimensions: - -| Retained data or contract | Requirement | -| --- | --- | -| Target capability/version, input, and identity rules | New code can still handle registered targets, including stopped ones; a v2-only entry cannot take over v1 targets directly | -| SourceDefinition, ConnectorBinding, and checkpoint format | Remain compatible with existing Sources and progress; do not bypass checks by changing versions or resetting cursors | -| Targets and received submission records | Preserve records and Scope ownership under current PC permissions; do not require business reruns to recreate history | - -If compatibility cannot be established, remain disabled without deleting records or resetting progress. Deployers can supply compatible code or explicitly address incompatible targets and progress; the first version provides no automatic migration, and a new instance does not automatically inherit old targets. - -After validation, the deployer updates registration through configure while keeping it disabled, then explicitly enables the new Runtime after successful validation. Failure leaves it non-executable without creating a second Context. Ordinary repeated startup/loading only reuses identical configuration; conflicting configuration must fail explicitly rather than substitute for the update procedure. Appendix C lists management interfaces. +write updates first resolve in-flight calls, then restart and load while preserving capability and Definition contracts and existing Source identity. Ordinary repeated loading does not replace configuration updates. ## Permission responsibilities and operation conditions -**Reuse PC Server's existing Access authorization.** Principal is the user or service identity authenticated by Server. Hosts cannot claim another identity in input. - -| Component | Responsibility | -| --- | --- | -| PC Server | Authenticate hosts and check current Scope/Server permissions before forwarding management requests. Check write persistence under the host identity, every collect submission under its execution identity, and current read permission when returning data | -| Management service | Accept only Server management requests over authenticated channels; verify actual target Scope, instance service range, and execution conditions | -| Plugin | Interpret private configuration/external credentials and handle business permissions when accessing external systems; identify_target resolves identity only and does not authorize | +User authorization reuses Server Access. A Principal is the user or service identity authenticated by Server; management does not maintain a second user authorization system. -| Operation | PC Server permission | +| Operation | Server check | | --- | --- | -| Capability and business-instance discovery | scope.read; indicate whether scope.contribute is currently present | -| write and new/repeated collect | scope.contribute plus scope.read because responses contain data or existing Source references | -| Query collect targets/submissions or read Sources | scope.read | -| collect control view, target stop/resume | scope.contribute; only control fields returned, without additional scope.read | -| Load Server-local write; instance control view, loading, configuration, unload, or whole-instance enable/disable | server.admin; actual process operations still belong to deployment tools on the execution machine | -| Grant users/execution identities Scope permissions | Existing Access authorization rules | +| Capability and business instance discovery | scope.read, with scope.contribute indicated | +| write, new or repeated collect | scope.contribute and scope.read, since replies may contain data or Source references | +| Query collect results or read Sources | scope.read | +| Target control view and target controls | scope.contribute, returning control fields only | +| Load write; instance control view, loading, configuration, removal, or controls | server.admin; deployment tools perform actual process operations | +| Grant Scope roles | Existing Access authorization rules | -All scope.contribute holders can control that Scope's targets, not only creators; creator identity is informational. An instance may serve multiple Scopes, so whole-instance control requires server.admin, not one Scope's permission. Instance control responses contain no business Config, target inputs, or Source data. +Server forwards authorized operations and trusted identity. Management verifies actual resource Scope, instance serviceable scope, and execution conditions to prevent addressing another Scope's target through an authorized Scope. Authenticated plugin instance Runtime synchronizes and reports only its instance and cannot impersonate Server management requests. Plugins handle external authorization; identify_target does not authorize. -Server forwards the authorized operation together with Scope, target_id, or instance_id. Management verifies actual ownership to prevent using an authorized Scope to address another Scope's target; this adds no new authorization model. Management channels and instance connections still authenticate. Runtime can synchronize only its own targets and report its own state, not impersonate Server management commands. - -Permission changes/revocation occur only in PC Server Access. Current read permissions remain checked while write waits and before returning. Revoked read permission prevents returning results but does not imply an accepted save did not occur. Continuous collect uses its own PC execution identity; creator permission changes do not automatically disable existing targets. Normal controls stop targets, and Server checks execution permission on every submission. - -On external access refusal, the collect plugin decides to wait, retry, or end apply. Server returns write validation, resolve, project, or storage errors through the original call. +Any contributor in the Scope can control targets, regardless of creator; instance-wide controls require server.admin. collect uses the instance PC identity for continuous execution, checked on each submission. Creator permission changes do not automatically stop targets. write uses the current host identity, and returning data requires current read permission; denial does not undo already saved data. Control responses exclude business configuration, target input, and Source data. ## Credentials and initial authorization -AuthenticationProvider verifies Server credentials and maps them to Principal. Local deployment may create or reuse execution identities through a deployment-controlled Provider. Externally managed Servers keep their own Provider; administrators pre-provision identities where needed. Deployment does not overwrite authentication configuration or promise automatic credential issuance. - -| Credential | Purpose | Location | -| --- | --- | --- | -| Deployment | Load write, register Definitions, create Scopes, grant roles, and configure instances | Deployment secret storage, not distributed to Runtime | -| Host | Call Server as the host user/service identity | Host secure configuration | -| Management channel | Authenticate management to Server on its dedicated channel | Management service | -| PC execution | Per-instance Principal submits to authorized Scopes and handles checkpoints | Execution machine, without server.admin | -| Instance connection | Runtime proves instance identity to management for its own work/reports | Execution machine | -| External system | Plugin accesses Yuque, ticket systems, etc. | Plugin-managed storage/interpretation; Config may reference files, environment, or secret services | - -write loading requires admin permission, but daily writes use the current host identity without creating another execution identity or credential. collect deployment uses separate Bearer credentials mapped by Provider to explicit identities. Before collect business execution, deployment: - -1. Validates Server, deployment identity, permissions, and target Scope. Confirms actual host/execution Principals rather than treating credential-file existence as validation. -2. Registers SourceDefinition with deployment credentials and creates/reuses execution Scope Bindings through existing Access interfaces. Daily execution needs neither Definition registration nor server.admin. -3. Configures management channel identity, registers execution location and served Scopes, reuses Server Access for management authorization, and supplies Runtime only required credentials/references. -4. Checks actual execution identity and Scope permissions before allowing work in that Scope. An unauthorized instance may be registered but cannot accept that Scope's collection. Host MCP connectivity/authorization follows host integration procedures. - -These steps use existing `POST /v1/source-definitions/register`, `POST /v1/access/bindings/create`, `GET /v1/access/me`, and `POST /v1/access/check`, without another registration or OAuth system. - -Automatic local deployment generates internal configuration without bypassing authorization. Deployment credentials do not reach collect Runtime. Only administrators load trusted write code, while daily writes do not borrow deployment permissions. Credential values are excluded from command arguments, ordinary logs, and capability catalogs. Rotation uses Provider and secret references; invalid identities cannot remain in use. Cross-machine communication uses TLS or equivalent authenticated secure entry points. - -# Appendix A: usage scenarios—startup and adding resources dynamically - -These scenarios first show collect user operations and component interactions by deployment location, then write loading/calls. For local collect, Server startup scanning and `source load` prepare shared management and Runtime without separately running both `install` commands. For cross-machine deployment, deployers prepare full configuration, start management, complete registration/authorization, then start remote Runtime. Service installation commands here use Linux `systemd --user`; Server must already have authentication configured and PC Access enabled. - -Commands and configuration in this appendix are **usage illustrations** showing deployment locations, operation order, and component interactions. Final commands and arguments are documented in the user guide shipped with the implementation. Run shell commands on the indicated machine. JSON uses MCP `tools/call` parameters for section 3's **proposed interfaces**, sent by the host to Server. Replace example Scopes, paths, and document URLs with real values. - -## Scenario 1: start Codex, Server, and plugins on one machine - -**1. Prepare the instance directory on the Server machine.** Assume bundled code, directory `/srv/powercontext/source-plugins/yuque-personal`, and external credentials kept separately in a protected file. `powercontext.yml`: - -```yaml -id: yuque-personal -plugin: yuque -config: - token_file: /srv/powercontext/secrets/yuque.token - base_url: https://www.yuque.com - poll_interval: 60 -# Optional targets for first initialization. -targets: - - scope_id: scp_demo - capability: yuque.document - capability_version: "1" - input: - url: https://www.yuque.com/team/book/doc-a -``` - -The plugin defines `config`, including business parameters such as external credential references. Target Scopes must exist, and both initialization caller and execution identity need appropriate permissions. Initial targets use normal add rules; management becomes authoritative after initialization. Later file changes do not overwrite or supplement targets, and neither empty lists nor connection failures trigger reimport. Omit targets if Scopes are not ready; complete Scope authorization/service-range expansion below, then add through public MCP. - -**2. Configure the plugin directory and start Server in the background.** Keep existing database, authentication, and deployment settings in `pc.env`, adding: - -```bash -POWERCONTEXT_SERVER_SOURCE_PLUGINS_DIR=/srv/powercontext/source-plugins -``` - -From the PowerContext repository root: - -```bash -uv run powercontext service install --env-file ./pc.env -``` - -`service install` installs and starts Server's background service, then returns. Startup performs the following local deployment without separate `manager install` or `runtime install`: - -| Order | Components and interactions | -| --- | --- | -| 1 | Server deployment scans trusted directories, reads instance configuration, and prepares local service configuration and internal identity references | -| 2 | Start/reuse independent management; management initiates its authenticated Server channel | -| 3 | Validate plugin declarations, register Definition/instance with deployment permission, and confirm execution Scope permissions and served range | -| 4 | Start Runtime, connecting to management and Server; Runtime creates Context and executes apply, with capabilities and instance state synchronized to Server | -| 5 | If targets are not initialized, import file targets through normal add after instance readiness. Plugin obtains them through ctx.targets.list() and decides collection timing | - -Deployment permissions come from Server deployment configuration, not daily Runtime. Existing instance restarts do not reimport targets; management retains runtime additions and stop/resume state. Built-in and custom write need no management service when no collect instances exist. - -**3. Install host integration and connect to the same Server.** If Codex is installed but PowerContext integration is not, run from repository root: - -```bash -uv run powercontext setup codex --source . --server-url http://127.0.0.1:8000 -POWERCONTEXT_CODEX_SCOPE_ID='scp_demo' codex -``` - -Prepare host credentials through existing host authentication. The second command starts a new Codex session from this shell. Desktop hosts use their own integration configuration and establish a new session; they do not inherit another terminal's environment. Verify the session Scope through `resolve_scope_binding` and explicitly pass the same Scope to Source MCP calls below. - -Hosts access only Server `/mcp`, not management or Runtime directly. Closing Codex does not stop background plugins; adding a Source plugin does not require reinstalling host integration. - -## Scenario 2: collect a new document through a running instance - -Assume `yuque-personal` already collects A and needs document B added. - -**1. Codex creates B through an authorized Yuque tool and obtains its actual URL.** For example, with Yuque CLI: - -```bash -yuque create doc --namespace team/book --title 'Document B' --body-file /tmp/doc-b.md --yes --json -``` - -Public Source MCP neither creates Yuque documents nor monitors all external tool operations. - -**2. Codex discovers capabilities and instances.** - -```json -{ - "name": "pc_source_capabilities", - "arguments": {"scope_id": "scp_demo", "operation": "add"} -} -``` - -Server returns capabilities, versions, schemas, and instance state based on current PC permissions and synchronized catalog. Choose the returned `yuque.document` version and `yuque-personal` instance. - -**3. Codex adds B.** - -```json -{ - "name": "pc_source_add", - "arguments": { - "scope_id": "scp_demo", - "capability": "yuque.document", - "capability_version": "1", - "instance_id": "yuque-personal", - "input": {"url": "https://www.yuque.com/team/book/doc-b"} - } -} -``` - -Server checks current Scope permissions and forwards to management. Runtime invokes identify_target to confirm resource identity; management creates/reuses the target and returns target_id. The response establishes ongoing collection, not that B is saved. Repeating the same Scope, instance, capability version, and resource identity reuses the target without resuming stopped targets. - -**4. The running plugin obtains and processes B.** Runtime synchronizes targets; apply reads B through `ctx.targets.list()` and calls `run_batch()` on its own schedule. Worker executes Connector, which reads Yuque and submits directly to Server. SDK automatically associates actual receipts, and Runtime reports Source references/timestamps to management. - -**5. Call query/control tools only when the user needs results or wants to stop tracking.** Queries are not needed to sustain collection: - -```json -{ - "name": "pc_source_status", - "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b"} -} -``` - -source_refs and last_submitted_at describe confirmed and reported submissions, not complete collection of B. Stop B using the actual returned target_id: - -```json -{ - "name": "pc_source_target_set_enabled", - "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b", "enabled": false} -} -``` - -Any scope.contribute holder can stop/resume Scope targets, not only their creator; reading submissions requires scope.read. The response confirms saved intent; a disconnected Runtime may still collect from its last synchronized state. Resume uses the same target_id and enabled=true. Repeated add is not resume. - -## Scenario 3: add plugins, Scopes, and remote execution - -**Add a local custom plugin instance.** Prepare code, dependencies, and an independent instance configuration: - -```text -/srv/powercontext/source-plugins/tickets-team/ - powercontext.yml - plugin.py -``` - -```yaml -id: tickets-team -plugin: ./plugin.py -scope_ids: [scp_demo] -config: - api_base: https://tickets.example.org - token_file: /srv/powercontext/secrets/tickets.token -``` - -Business parameters must be declared by the plugin's Config. The instance ID is deployment-unique, and a local entry needs no published wheel. Once Server is running with the plugin directory configured, execute on the same machine using its `pc.env`: - -```bash -uv run powercontext source load --config /srv/powercontext/source-plugins/tickets-team/powercontext.yml --env-file ./pc.env -``` - -`load` uses protected local deployment credentials, prepares/reuses shared management, registers Definitions and the instance, configures execution Scope permissions, generates RuntimeConfig, and installs/starts Runtime. First loading also imports optional initial targets. It calls the service-installation logic used by the two `install` commands, so this scenario needs only `load`. Background execution continues after the command returns, without restarting Server or existing instances. Prepare plugin dependencies beforehand. - -Adding files alone while running does not start an instance. Repeat loading neither overwrites initialized targets nor restarts an instance with identical configuration. Hosts then call `pc_source_capabilities` again and use the new input_schema with `pc_source_add`, without modifying the MCP tool list. - -The three plugin configuration files have different purposes: - -- `powercontext.yml`: user-authored instance configuration with ID, entry, business config, served Scopes, and optional initial targets; read by `source load` or startup scanning. -- `source-manager.yml`: management deployment configuration (ManagerConfig), including Server address/credential file, management ID, listening address, and durable state directory; read by `manager install`. -- `plugin-runtime.yml`: full deployment configuration for one plugin process (RuntimeConfig), including plugin directory, business config, Server/management addresses, both connection credential files, and configuration digest; read by `runtime install`. An instance powercontext.yml cannot substitute for it. - -`pc.env` is Server environment configuration used by load for connection addresses and local deployment directories. - -**Add a Scope to the same instance.** An administrator grants the execution identity permissions through existing Access and expands the instance service range through `POST /v1/source-management/configure`. Hosts also need corresponding PC permissions. Add targets with the new scope_id without recreating Context or editing initialized file targets. - -**Manual cross-machine deployment.** For example, machine A runs Server and management, while B can reach the ticket system and runs `tickets-team`. Codex connects only to A's public MCP. A's `source load` does not install code or start processes on B. - -Prepare reachable addresses and connection credentials. With deployment permissions, register Definitions on Server, register instance ID, execution location, configuration digest, and served Scopes with management, and grant execution Scope permissions through PC Access. B's code, dependencies, and RuntimeConfig must match registration. If management has not started, install it first as below, register the instance, then install Runtime. - -```bash -# Machine A: manually deploy shared management; skip if it already exists. -uv run powercontext source manager install --config ./source-manager.yml - -# Machine B: after registration and authorization, start the prepared Runtime. -uv run powercontext source runtime install --config ./plugin-runtime.yml -``` - -These commands only install/start their background services; they do not install dependencies or automatically register Definitions/instances or authorize identities. Deployment generates or prepares RuntimeConfig from that information. Current source load orchestrates local deployment without remote configuration distribution or SSH execution. - -Management initiates requests to Server; Runtime initiates requests to management, Server, and external sources. Cross-machine connections need reachable secure endpoints, not loopback-only defaults. Management may run on a third machine with the same responsibilities; each Server has one shared management service. +AuthenticationProvider maps credentials to Principals. Local deployment can prepare identities through its deployment-controlled Provider. External Servers retain their Provider and may require administrator-provisioned identities; deployment does not replace authentication settings or promise automatic issuance. -**Custom write: load implementations, then save complete content directly.** For example, a host already has complete ticket content to save as an `example-ticket` Source: - -1. Prepare section 4's local `tickets` package and dependencies on Server's machine, importable from Server's Python environment. -2. Supply entry configuration and trigger Server loading with administrator permission. The configuration sketch is `entry: tickets`; loading validates/registers Definitions, saves entry references, and exposes `ticket.write / 1`. -3. The host connects to the same `/mcp`, discovers and submits below without an instance, target, or manager/runtime configuration: - -```json -{ - "name": "pc_source_capabilities", - "arguments": {"scope_id": "scp_demo", "operation": "add"} -} -``` - -After discovery confirms `ticket.write / 1` and its input model: - -```json -{ - "name": "pc_source_add", - "arguments": { - "scope_id": "scp_demo", - "capability": "ticket.write", - "capability_version": "1", - "input": { - "namespace": "team", - "project": "support", - "ticket_id": "T-100", - "title": "Login failure", - "body": "A user reports being unable to log in." - } - } -} -``` - -Server validates TicketCapture, calls TicketDefinition.resolve and TicketProjection.project, saves, and returns actual SourceRef through this call. Hosts read Source/Projection through the reference without status queries. This example's Definition derives stable identity from the complete ticket snapshot, so identical-content retries reuse Sources. Continuous ticket-change collection uses collect above; both paths may reuse the same Definition. - -# Appendix B: plugin collection pipeline reference - -This collect **interface sketch** relates identity confirmation, plugin loops, submissions, and checkpoints; concrete class names, module paths, and helper functions may change during implementation. Plugins implement `BusinessClient`, `DocumentDefinition`, `TemporaryReadError`, and Config external-credential fields. It is not a complete runnable client and does not replace SDK changes still to be implemented. - -Plugins declare `plugin_api_version = "2"`, Config, capabilities, definitions, and stable Connector name/version. Capabilities describe collect input/business contracts; identify_target confirms resource identity. - -For collect input `{"document_id":"A"}`, an identity entry can return: - -```python -async def identify_target(request, config): - # document_id is a nonempty business ID required by input_schema. - document_id = request.input["document_id"].strip() - if not document_id: - raise ValueError("document_id must not be blank") - return { - "target_key": document_id, - "normalized_input": {"document_id": document_id}, - } -``` - -This entry resolves resource identity. If there are additional collection options, normalized_input must preserve all parameters affecting collection. Omitting options must not equate different requirements. - -```python -import asyncio - -from powercontext.source_management import ManagementError -from powercontext.sources import ( - ConnectorRunCompletion, - ConnectorRunStatus, - ConnectorSubmissionStatus, -) - -plugin_api_version = "2" -connector_name = "document-reader" -connector_version = "1" -document_definition = DocumentDefinition() -definitions = (document_definition,) - - -class BusinessConnector: - name = connector_name - version = connector_version - source_definitions = frozenset({document_definition.name}) - - def __init__(self, ctx, client): - self.ctx = ctx - self.client = client - self.targets = () - - async def run(self, session): - batch_complete = True - for target in self.targets: - try: - document = await self.client.read(target.input) - if document is None: - # Confirmed absent content; network or permission errors must not become None. - continue - receipt = await self.ctx.targets.submit( - target, - session=session, - item_id=target.target_id, # One item per target in this example, unique within the batch. - definition_name=document_definition.name, - value=document, - ) - if receipt.status != ConnectorSubmissionStatus.ACCEPTED: - batch_complete = False - except TemporaryReadError: - # Retry in a later batch; do not swallow cancellation or other unhandled errors. - batch_complete = False - return ConnectorRunCompletion( - status=ConnectorRunStatus.COMPLETE if batch_complete else ConnectorRunStatus.INCOMPLETE, - checkpoint=session.checkpoint, # Full reads in this example, without advancing an incremental cursor. - ) - - -async def apply(ctx, config): - client = await ctx.enter_async_context(BusinessClient(config)) - connector = BusinessConnector(ctx, client) - ctx.connect(connector) - while True: - try: - targets = await ctx.targets.list() - for scope_id in sorted({target.scope_id for target in targets}): - selected = tuple(target for target in targets if target.scope_id == scope_id) - connector.targets = selected - await ctx.run_batch(scope_id=scope_id, targets=selected) - except ManagementError as error: - # Wait for an initial target snapshot; propagate other configuration/permission errors. - if error.code not in {"targets-unavailable", "management-unavailable"}: - raise - await asyncio.sleep(config.poll_interval) -``` - -SDK records actual submission associations automatically. Connector batch results serve the Worker/checkpoint pipeline; section 5 defines disconnection and restart behavior. - -Incremental/paginated plugins calculate checkpoints themselves and may maintain target-specific progress inside the instance/Scope checkpoint. New targets need backfill rather than being skipped because another target's cursor is newer. Repeated data must produce stable Source identity, with failure retries following existing saving rules. - -Section 5 defines full cleanup and stop behavior. write loading/calls are in section 4 and Appendix A. - -# Appendix C: management communication and instance configuration reference - -This appendix serves deployers and framework implementers. Hosts use public MCP; plugins use SDK to obtain targets and submit data. Paths and configuration fields below are **interface sketches**; connection directions, permissions, and state semantics remain governed by the main design. - -## Channels and identity - -Sections 2 and 3 define connection directions and request handoff; section 5 defines authentication and permission responsibilities. Management and Runtime each use their own credentials to connect upstream. - -| Interface | Purpose | +| Credential | Use and ownership | | --- | --- | -| Server `/v1/source-management/channel/open`, `catalog`, `poll`, `reply`, `close` | Authenticate management, synchronize catalog, receive/respond to messages. Once catalog is ready, Server answers discovery directly | -| Server `POST /v1/source-management/configure` | After server.admin checks, register or explicitly update instances through the management channel; does not upload arbitrary Python code | -| Management `POST /v1/source-management/instances/{instance_id}/sync` | Runtime authenticates with instance credentials and synchronizes its collect targets/control state; SDK handles pagination | -| Management `POST /v1/source-management/instances/{instance_id}/results` | Runtime reports actual Source references/timestamps for its instance. Duplicate reports reuse associations | - -On reconnection, management authenticates again and synchronizes its complete catalog. Server checks capability names, versions, kind, and input contracts for consistency with existing capabilities. Management message size, in-flight count, and waiting time are bounded. - -## Instance configuration and updates - -`registration` in `POST /v1/source-management/configure` is complete instance registration; `update=true` requests an explicit update. Section 2 defines initial target import; section 5 defines configuration updates and upgrades. - -Expanding an already authorized Scope range can preserve Context; shrinking the range or changing execution configuration follows the disable/update procedure. +| Deployment | Deployment tooling loads code, registers Definitions, grants permissions, and configures instances; not passed to plugin instance Runtime | +| Host | Stored in host security configuration to call Server | +| Management channel | Management authenticates to Server | +| PC execution | Per-instance identity for Source submission and checkpoints, without server.admin | +| Instance connection | plugin instance Runtime proves its own identity to management | +| External system | Stored and interpreted by plugins, optionally referenced in Config | + +Before business execution, deployment verifies actual identities, registers Definitions, grants Scope permissions, records instance scope and connection identities, and checks access using execution credentials. Existing identity and Access interfaces are reused. Instances may be registered without Scope authorization but cannot accept work for that Scope. write has no separate execution credentials and does not borrow deployment authority for daily saves. Credentials are excluded from command arguments, catalogs, and ordinary logs; cross-machine connections use TLS or equivalent security. diff --git a/docs/zh/rfcs/0000-source-plugin-system.md b/docs/zh/rfcs/0000-source-plugin-system.md index 737b0365b..459531b4f 100644 --- a/docs/zh/rfcs/0000-source-plugin-system.md +++ b/docs/zh/rfcs/0000-source-plugin-system.md @@ -29,16 +29,14 @@ description: 支持用户开发和使用自定义 Source 插件，扩展数据 # 1. 方案概览 -宿主通过 **PC Server 的同一个公共 MCP** 使用内置和自定义 Source 能力，两类操作采用不同执行方式： +宿主通过 **PC Server 的同一个公共 MCP** 使用内置和自定义 Source 能力： -- **write**：宿主已有完整内容，PC Server 使用已加载的 SourceDefinition 和 Projection 实现，同步保存并返回结果；不启动插件实例或管理服务。 -- **collect**：插件从外部系统持续读取资料，由一个共享的后台管理服务管理目标，每个插件实例在自己的 Runtime 后台进程中执行。默认在 Server 启动时发现指定目录中的实例，自动启动本机独立管理服务和 Runtime；多个宿主共享实例。远端 Runtime 也可接入同一个管理服务，管理服务亦可异机部署。 +- **write**：宿主提供完整内容，Server 执行已加载的 SourceDefinition 和 Projection，同步保存并返回。 +- **collect**：插件持续读取外部资料。一个独立后台插件管理服务保存采集目标，每个插件实例由独立的插件实例 Runtime 进程执行，可服务多个获授权 Scope。插件管理服务和插件实例 Runtime 均可异机部署，多个宿主共享实例。 -两条路径复用 SourceDefinition 与 Projection 契约，同一种 Source type 可以同时由宿主写入和插件采集产生。 +两条路径复用 SourceDefinition、Projection 和已有 Source 保存规则，同一种 Source type 可以同时由 write 和 collect 产生。框架提供加载、生命周期和数据管道；插件决定采集内容、节奏、分批、重试及退出。 -**框架提供 SourceDefinition（输出类型、身份与 Projection 契约）和 Source 保存管道；collect 插件决定采集什么、何时执行、何时提交以及何时退出。** collect 复用已有 Worker 与 checkpoint 管道，由插件调用批次执行。write 的校验和转换则在当前 Server 调用内执行。 - -一个管理服务只服务一个 Server。支持多实例、多 Scope、多执行机器；不做管理集群、自动迁移、通用任务调度或进程内 Python 代码热替换。本文规定目标设计；目录发现与自动启动的目标使用步骤见附录 A。 +一个插件管理服务对应一个 Server；不包含管理集群、自动迁移、通用任务调度或进程内 Python 代码热替换。默认本机部署由 Server 启动流程准备独立后台服务，安装或启动命令完成后退出，宿主退出不影响采集。 # 2. 架构、连接与存储 @@ -54,8 +52,8 @@ flowchart TB M -->|write| W W --> S end - G[独立管理服务] - subgraph RT[每个 collect 实例的 Runtime 进程] + G[独立插件管理服务] + subgraph RT[每个插件实例 Runtime 进程] R[目标同步与生命周期] A[用户 apply 和 Connector] C[PluginContext.run_batch] @@ -71,167 +69,92 @@ flowchart TB K -->|Source 提交和 checkpoint| S ``` -图中箭头表示业务调用或数据流向。write 的实现运行在 Server 进程内；collect 的 Context、Worker 和用户 Connector 均在对应 Runtime 进程内。管理服务交付目标与控制状态，不转存 Source 数据，也不调度采集批次。管理消息通过接收方主动发起的轮询响应交付，网络连接方向见下文。 +图中箭头表示业务调用和数据流向。write 实现位于 Server 进程，collect 的 Context、Worker 和 Connector 位于插件实例 Runtime 进程。 -| 组件 | 职责与运行位置 | +| 组件 | 职责 | | --- | --- | -| 公共 Source MCP | 位于 PC Server 的 `/mcp`；认证宿主、检查 Scope 权限，统一发现、提交、查询和启停接口 | -| PC Server 数据管道 | 加载自定义 write 的 Definition 与 Projection 代码，注册 Definition；保存 Source / Projection 和 checkpoint，提供数据读取。内置和自定义 write 直接使用 Server 内的保存管道 | -| 共享管理服务 | 独立后台进程；保存实例、采集目标和结果，执行 Server 已授权的管理请求，检查 Scope 归属与运行条件，向 Server 同步 collect 能力和实例状态 | -| Runtime | 每个 collect 实例进程的外层框架；实例由稳定的 instance_id 标识，代表一份账号、配置和执行环境，可服务多个获授权的 Scope。Runtime 加载插件、维护管理连接、同步目标与启停状态，管理业务运行的创建和结束 | -| PluginContext（下文简称 Context） | 一次业务运行的上下文对象，作为 ctx 传给 apply；持有 Connector 和业务资源，提供 SDK（插件调用框架的一组接口），包括 run_batch、提交和清理 | -| RemoteConnectorWorker | 已有执行管道；由 `run_batch()` 调用，建立 session、执行 Connector、处理提交回执及 checkpoint CAS | -| apply / Connector | 插件业务代码；apply 决定节奏，Connector.run(session) 读取或转换数据并提交 | -| 宿主专属插件 | 可选适配层，仅负责 Hook、Scope 默认值和上下文注入；不托管 Source 实例、不复制公共 MCP 的管理实现 | +| 公共 Source MCP | 位于 Server `/mcp`；认证宿主、检查 PC 权限，提供统一发现、提交、查询和启停入口 | +| PC Server 数据管道 | 加载 write 实现、注册 Definition，保存和读取 Source、Projection 与 checkpoint | +| 插件管理服务 | 保存插件实例、能力、目标、启停状态与收到的提交记录；交付管理请求，不调度采集或转存 Source | +| 插件实例 Runtime | 每个插件实例的外层框架；加载插件、维护管理连接、同步目标和控制状态，创建与结束业务运行 | +| PluginContext（Context） | 一次业务运行的上下文，以 ctx 传给 apply；管理 Connector、客户端和业务资源，提供 run_batch 等插件接口 | +| RemoteConnectorWorker | 已有批次管道；创建 session、调用 Connector.run、处理提交回执及 checkpoint CAS | +| apply / Connector | 插件业务代码；apply 决定节奏，Connector 读取、转换并提交本批数据 | -只使用内置能力或自定义 write 时只需 Server。使用 N 个 collect 实例时增加一个管理服务和 N 个实例进程；Server 的启动流程负责准备这些独立服务。系统服务托管后台进程，安装或启动命令完成后退出，关闭 Codex 不会停止已启动的采集。 +只使用 write 时仅需 Server；N 个插件实例增加一个插件管理服务和 N 个插件实例 Runtime 进程，由系统服务托管。 ## 启动加载与运行时新增实例 -**write：加载 Server 本地实现。** 部署者先在 Server 机器准备可信插件代码与依赖，并配置代码入口。Server 的通用加载器导入入口，读取其 `definitions` 和 write 能力声明，校验输入、Definition、Projection 与名称版本冲突，完成 SourceDefinition 注册后才公开能力。部署操作要求 `server.admin`；命令行只触发加载，真正导入代码的是 Server 进程。仅通过 API 注册 SourceDefinition 的结构与契约，并不会把 Python 函数传给 Server，不能据此宣称 write 已可执行。 - -成功加载的入口与必要配置引用由 Server 保存，重启时重新导入和校验。支持显式新增和一致配置的重复加载；配置不一致时报冲突，修改已有实现按第 5 节重启更新，不做进程内 Python 代码替换。接口和示例见第 4 节；此路径不创建 collect 实例、Runtime 或目标。 +**write：** 部署者在 Server 机器准备可信代码与依赖，并以 `server.admin` 配置加载入口。Server 通用加载器导入模块，校验 Definition、Projection、能力声明及名称版本冲突，注册成功后才公开能力。注册 API 只保存定义契约，不传输 Python 实现，不能替代代码加载。入口及必要配置引用持久保存，Server 重启时重新导入。 -**collect：加载实例并启动独立 Runtime。** 以下配置与部署流程用于 collect。 +**collect：** Server 启动时扫描一次通过环境变量指定的插件目录；每份 `powercontext.yml` 声明实例 `id`、代码入口、业务配置、凭据引用和可服务 Scope。id 即 instance_id，在同一 PC 部署内唯一且跨重启稳定。同一份本地代码可供多个实例使用，无需发布 wheel；配置进入可信目录表示部署者允许加载。 -部署者通过 Server 的 `source_plugins_dir` 指定可信插件目录。目录中的每个 `powercontext.yml` 声明一个实例：插件实例 ID、代码入口、插件业务配置和外部凭据引用。配置中的 `id` 对应运行时的 `instance_id`，例如 `yuque-personal`；它在同一 PC 部署内唯一，重启后保持不变，不是插件代码名称、Scope ID 或 Source ID。随包代码不会仅因“已安装”而执行；将实例配置放入该目录表示部署者允许加载它。同一份代码可由不同 ID 的配置复用，无需复制代码。 +部署流程准备或复用插件管理服务，校验插件、注册 Definition、登记实例并授权，再启动独立插件实例 Runtime。运行中新增实例需显式加载，不持续扫描目录，也不重启其他实例。重复加载同一 ID 和一致配置复用登记及进程，不重置启停状态；配置不同则报冲突，更新按第 5 节处理。 -`powercontext.yml` 可选配置初始 `targets`（每项包含 `scope_id`、能力及业务输入）；首次启动时，若该实例尚未初始化目标，部署流程按正常添加规则写入管理服务，并在实例记录中标记已初始化。此后以管理服务中的目标和启停状态为准，文件中的 `targets` 不再用于覆盖、补充目标或判定实例配置冲突；目标为空不重新导入，管理服务连接失败也不视为未初始化。 +配置可包含初始 `targets`，每项指定 Scope、能力及业务输入。实例首次初始化目标时按正常添加规则导入插件管理服务，并记录已初始化；此后以插件管理服务为准，文件不再覆盖或补充目标。目标为空或连接失败不触发重新导入，初始 targets 的变化不算实例配置冲突。 -Server 的部署流程在启动时扫描一次指定目录，也支持运行中显式加载实例；通过本机服务管理器准备独立管理服务和 Runtime，生成并保存所需连接配置。行为如下，具体操作见附录 A。 - -| 情况 | 行为 | -| --- | --- | -| 没有本地实例，也未启用管理服务 | 只运行 Server；之后显式加载首个本地实例时，按需启动管理服务 | -| 首次加载有效实例配置 | 按需启动一个管理服务，校验代码契约与配置、注册 Definition、登记实例，再启动业务；准备失败的实例显示原因，不阻塞内置功能 | -| Server 运行中新增实例 | 部署者准备好代码、依赖和完整的 `powercontext.yml` 后显式触发加载；只放入目录不会自动启动，已有实例不重启 | -| Server 重启或再次加载同一插件实例 ID | 按配置中的 id 复用已有登记和进程；配置一致时不重置启停意图；已有 Runtime 存活时不重复调用 apply，进程需要重启时按第 5 节启动规则处理；配置不同则报冲突，更新须走第 5 节流程 | -| 删除目录或所有实例停止 | 不把文件删除当作卸载，不删除目标、历史或管理数据库；已经启动的管理服务继续等待，实例停用按显式管理操作执行 | - -停止 Server 不隐式卸载独立服务或删除状态；数据提交在 Server 不可用时失败或等待恢复，仍遵守第 5 节的断线与重启规则。远端代码和 Runtime 由执行机器部署，本机加载不承担远程 shell 或代码分发。 +无插件实例时不必启动插件管理服务；单个实例准备失败不阻塞内置功能。删除目录、停止 Server 或停用全部实例不会隐式卸载独立服务或删除数据。远端机器自行部署代码与插件实例 Runtime，本机加载不负责远程 shell 或代码分发。 ## 组件通信与连接方向 -| 发起方 | 接收方 | 通信内容 | +| 发起连接的一方 | 接收方 | 内容 | | --- | --- | --- | -| 宿主 | PC Server | `/mcp`、现有数据读取等接口 | -| 管理服务 | PC Server | 建立管理通道、注册能力、获取管理请求并回传处理结果 | -| Runtime | 管理服务 | 实例认证、资料身份确认、采集目标同步、启停状态和采集结果报告 | -| Runtime | PC Server | Worker 提交 Source、读取和更新 checkpoint | -| 插件业务 | 外部系统 | 获取资料、验证外部业务权限 | +| 宿主 | Server | 公共 MCP 与数据读取 | +| 插件管理服务 | Server | 认证建连、同步能力目录、领取管理请求并回报结果 | +| 插件实例 Runtime | 插件管理服务 | 实例认证、资料身份确认、目标与控制同步、提交记录报告 | +| 插件实例 Runtime | Server | Worker 提交 Source、读取与更新 checkpoint | +| 插件业务 | 外部系统 | 获取资料，处理外部业务权限 | -当前管理通道使用**管理服务主动发起的 HTTP 轮询和回报**：管理服务从 Server 领取待处理消息，再向 Server 回传结果；具体路径见附录 C。因此，业务上是 Server 把请求交给管理服务，网络上却始终是管理服务访问 Server。Server 无需连接管理服务的监听地址；管理服务仍须能被 Runtime 访问。 +管理通道采用主动 HTTP 轮询：插件管理服务调用 Server 的 `channel/poll` 领取请求，处理后通过 `channel/reply` 回传；Server 用临时消息 ID 关联原 MCP 调用并返回。业务请求从 Server 流向插件管理服务，网络连接始终由插件管理服务发起。插件实例 Runtime 同样主动连接插件管理服务，插件管理服务须能被插件实例 Runtime 访问。 -连接建立后，管理服务把自己管理的完整 collect 能力清单发送给 Server，包括能力名称、版本、输入要求、对应实例、实例状态与发现所需的公开范围配置。这份清单称为能力目录；发生变化时管理服务主动同步。Server 将该目录与本地内置、已加载的 write 能力合并，结合当前调用者的 PC 权限直接回答能力和业务实例发现，不为每次发现再询问管理服务。collect 的添加、目标查询、控制视图和启停仍经管理通道处理；目录不代替实际操作的权限与执行条件检查。write 由 Server 本地处理。 +插件管理服务连接或重连后主动同步完整**能力目录**：能力名称、版本、输入要求、对应实例、状态与公开范围；变化时更新。Server 合并内置及已加载的 write 能力，结合当前 PC 权限直接回答发现请求。实际 collect 管理操作仍经通道处理，目录缓存不代替操作时的权限和运行条件检查。 ## 持久化数据与存储归属 -| 保存方 | 存储与内容 | -| --- | --- | -| 管理服务 | 自己维护 `/management.sqlite3`，无需独立数据库服务；保存实例、能力、collect 目标、启停状态与采集结果；用户授权统一保存在 PC Server | -| Runtime 所在机器 | 仅保留插件代码、启动配置、凭据引用等启动所需文件；目标列表、当前批次和提交回执保存在进程内存中 | -| PC Server | 已有数据存储保存最终 Source、Projection 与 checkpoint；另外保存成功加载的 write 代码入口和必要配置引用，供重启恢复。代码与依赖留在 Server 文件系统，导入后的实现对象留在内存；write 的重复提交沿用已有 Source 存储规则 | - -管理服务的 SQLite 按以下两张表组织： - -| 表 | 保存内容 | +| 保存方 | 权威数据与恢复依据 | | --- | --- | -| `records` | 部署绑定、实例、能力声明、collect（采集）目标及已收到的提交记录（Source 引用、提交时间）；Scope 归属、创建者记录与控制状态保存在对应记录中 | -| `follow_identity` | collect 的稳定资料身份与 target_id 的对应关系，用于重复添加时复用同一目标 | - -write 的输入和应答仅在当前同步调用中使用，最终 Source 与 Projection 由 Server 持久保存。 - -**管理服务保存“需要采什么”，PC Server 保存“已经采到什么”，Runtime 负责当前执行。** collect 的运行目标由管理服务持久保存，Runtime 启动和重连时同步；文件中的初始 targets 不作为运行状态副本。实例和目标的期望启停状态以管理服务为准，Source 与 checkpoint 以 PC Server 为准；最近收到的执行状态和采集结果由管理服务保存，供宿主查询。 +| 插件管理服务 | 自管 SQLite 保存实例、能力、collect 目标、期望启停状态及收到的提交记录；同时记录最近执行状态与确认时间 | +| PC Server | 已有存储保存 Source、Projection、checkpoint 和用户授权；另存 write 加载入口及配置引用，代码与依赖留在 Server 文件系统 | +| 插件实例 Runtime | 仅持久保留代码、启动配置和凭据引用；目标快照、批次与未送达报告放在内存，重启后重新同步 | -Runtime 进程退出时内存工作丢失；重启后若实例仍启用，就重新执行 apply，通过 Server 的 checkpoint 决定后续采集位置。插件可按业务需要自行持久保存数据。 +管理存储中的 `records` 保存部署绑定、实例、能力、目标及提交记录（Source 引用和时间）；`follow_identity` 保存 collect 稳定资料身份到 target_id 的映射，用于重复添加时复用目标。write 直接沿用 Source 存储规则，不保存独立受理记录。 -Source 提交和向管理服务报告结果是两个独立动作。Source 已保存但结果报告丢失时，数据仍在 Server，管理服务可能缺少对应的提交记录；框架不保证重启后补齐。管理服务和 Server 的数据库须持久保存，Runtime 只需保留启动文件。系统服务管理器负责进程启停，不保存采集业务状态。 +**插件管理服务保存“需要采什么”，Server 保存“已经采到什么”，插件实例 Runtime 负责执行。** Source 保存与提交记录报告是独立动作：报告丢失不撤销数据，插件管理服务也不保证重启后补齐。其结果视图不是 Server 全部数据的清单，两方的数据库均须持久保存。插件可按业务需要自行持久化数据。 # 3. 公共 MCP：宿主如何统一使用 -宿主只连接 PC Server 的 `/mcp`。公共 MCP 负责能力发现、Source 写入、采集目标管理和实例启停；宿主专属插件只提供连接、身份与 Scope 上下文，不直接连接管理服务或 Runtime。 - -本节描述公共 MCP 的**拟议接口**，按每个方法说明必要输入、从调用到返回的流程，以及结果和错误的含义。以下 JSON 是拟议请求与返回格式的示例，其中请求使用 MCP `tools/call` 参数结构；Scope、实例和目标 ID 均须替换为实际值。方法名和字段名可在实现评审中调整并同步本文，权限检查和行为语义仍按正文规定。 - ## 调用前需要理解的对象 | 对象 | 含义 | | --- | --- | -| capability | 可调用的业务能力，包含名称、版本、说明、input_schema（业务参数结构）和 kind。它不是 Source type；collect 能力可以生成多个 Source type，write 能力关联其输入对应的 Definition | -| kind | 能力声明的操作类型：write 接收宿主已有内容，同步保存并返回；collect 接收外部资料位置或选择条件，建立持续采集关系。操作类型由能力声明确定 | -| instance_id | 执行 collect 能力的插件实例 ID，例如 yuque-personal；代表账号、配置和执行环境，可服务多个获授权 Scope。write 不使用 instance_id | -| target_id | collect 目标的持久 ID，标识持续采集关系，用于查询、停止和恢复；不是 Source ID | -| SourceRef | 已保存 Source 的引用，包含 source_type 和 source_id；结合本次 scope_id，通过 Server 已有数据接口读取内容 | -| enabled | 目标或实例的期望启停状态；不等于执行端已完成启停 | -| capability_version | 能力输入与行为契约的版本，宿主使用发现返回的版本；与输出 SourceDefinition 的版本分开 | - -参数错误、权限不足或处理失败均通过原 MCP 调用返回错误；不执行尚未通过检查的后续步骤。 +| capability | 宿主可调用的能力，声明名称、版本、说明、input_schema（业务输入结构）和 kind；不是 Source type，一个 collect 能力可产生多个类型 | +| kind | write 接收完整内容并同步保存；collect 建立持续采集关系；由能力声明确定 | +| capability_version | 输入和操作契约的版本；collect 还包含资料身份规则，与输出 SourceDefinition 版本分开 | +| instance_id | collect 的执行实例，代表一份配置、外部凭据和执行环境，可服务多个 Scope；write 不使用实例 | +| target_id | 持久采集关系的 ID，用于查询和启停，不是 Source ID | +| SourceRef | 已保存 Source 的 source_type 与 source_id，结合 Scope 通过已有数据接口读取 | +| enabled / actual | 期望启停状态 / 插件实例 Runtime 最近确认的业务状态；均不能代替连接可用性判断 | -所有调用由 Server 认证宿主身份，并按操作检查 PC 权限；scope\_id 指定数据归属。collect 由管理服务核对目标所属 Scope、实例可服务范围与执行条件。外部系统的账号、凭据及业务权限由插件在实际访问时处理。 +宿主只调用 Server。Server 认证并检查 PC 权限后，直接处理 write 和目录发现，或经第 2 节的管理通道交付 collect 请求。插件管理服务核对资源所属 Scope 与执行条件，通过 reply 回传，Server 再结束原 MCP 调用；数据返回须通过当前读取权限检查。业务输入不能自行声明调用者身份，外部系统权限由插件实际访问时处理。各操作权限集中列在第 5 节。 -能力和业务实例发现直接由 Server 回答；需要管理服务处理的调用使用以下交接方式。 - -**管理请求的交接方式统一：** Server 将消息交给管理服务主动发起的 `channel/poll` 响应；管理服务处理后调用 Server 的 `channel/reply` 回传，Server 再结束对应的 MCP 调用。框架用临时 message\_id 匹配问答，宿主无需填写或保存。下文“Server → 管理服务”表示业务消息流向，不表示 Server 反向建立网络连接。Runtime 同样主动连接管理服务；通信细节见附录 C。 - -| 方法 | 宿主使用时机 | -| --- | --- | -| `pc_source_capabilities` | 了解能做什么、参数如何填写、哪些实例可以执行 | -| `pc_source_add` | 写入已有内容，或添加持续采集目标 | -| `pc_source_status` | 查询 collect 目标的采集结果或控制状态 | -| `pc_source_target_set_enabled` | 停止或恢复一个 collect 目标 | -| `pc_source_instances` | 查看可用实例，或读取实例控制状态 | -| `pc_source_instance_set_enabled` | 启用或停用整个实例的业务运行 | +以下说明拟议方法的关键参数、调用链路与返回含义。 ## pc_source_capabilities：发现能力与输入要求 -宿主不知道应选哪个能力、如何填写 input，或想确认可用实例时调用。发现只读管理信息，不加载新代码、不启动 apply，也不访问外部资料。 - -**调用参数：** scope\_id、operation；本节的业务发现使用 `operation="add"`。可用 capability、instance\_id 缩小查询范围。 - -```json -{ - "name": "pc_source_capabilities", - "arguments": {"scope_id": "scp_demo", "operation": "add"} -} -``` - -**从调用到返回：** +**输入：** scope_id、operation（业务发现为 `add`），可按 capability、instance_id 筛选。 -1. 宿主向 Server 发出调用。Server 检查当前 Scope 的读取权限，并取得调用者是否具有写入权限的判断；没有写入权限时，发现结果不能宣称可写。 -2. Server 读取内置能力、已加载的自定义 write 能力，以及管理服务主动同步的 collect 目录；根据公开范围、实例可服务 Scope 和当前调用者 PC 权限筛选结果。本次发现不再向管理服务发送查询。 -3. write 根据 Server 本地实现是否加载就绪标注能否执行；collect 根据目录中的实例状态及管理连接可用性标注。可见但离线或停用的实例仍可显示，并明确不可用。 -4. Server 将上述结果直接经原 MCP 调用返回宿主，不经过 channel/reply。返回只包含允许公开的信息，不暴露实例 Config 或外部凭据。 +**链路：** 宿主调用 → Server 检查 Scope 读取权限及是否可写 → 根据本地能力和已同步 collect 目录筛选公开范围、可服务 Scope → 直接返回能力名称、版本、kind、input_schema，以及权限、实例和可用性。 -**宿主获得什么：** 每个能力的名称、版本、kind、说明和 input\_schema，以及 PC 权限和可用性；collect 另包含可见实例、默认实例建议与运行状态。权限和运行状态分开表达：有写入权限不等于 Runtime 在线，在线也不等于 apply 正在处理工作，更不保证外部资料可访问。 +发现不访问外部资料，也不请求插件业务授权预检。可见但离线或停用的实例仍可显示，明确不可用；管理通道失效时不能用旧目录承诺 collect 可执行，无法确认可见范围时不暴露记录。没有插件管理服务不影响内置和已加载的 write 能力。 -宿主据此选择能力，再调用 `pc_source_add`。collect 还须明确传 instance\_id；默认实例仅为发现建议，不在重试时重新选择账号。write 不选择实例。能力版本也固定为本次选定版本。capability\_version 用于防止插件升级后以不兼容的输入规则解释原调用和持久 collect 目标；同一版本须保持输入、操作含义和资料身份规则兼容。 - -未配置管理服务时仍返回内置和已加载的自定义 write 能力。管理服务断线时明确说明 collect 部分暂不能接收新管理请求，不能将整次发现变成失败，也不能把缓存中的旧状态报为可执行。这里的不可用不表示 Runtime 已停止已有采集；发现结果只是调用时的状态，add 仍会重新检查。 +宿主用发现返回的契约填写 input；collect 显式选择 instance_id，默认实例仅作建议，重试不重新选择实例或版本。发现状态是当时快照，add 仍须检查。 ## pc_source_add：写入内容或添加采集目标 -**调用参数：** scope\_id、capability、capability\_version、input；collect 还须 instance\_id，write 不需要。input 按发现所得 input\_schema 填写，操作类型由 capability 的 kind 决定。 - -Server 对 add 检查当前 `scope.contribute` 和 `scope.read`。write 要求能力版本、对应 Definition 与 Projection 实现已在 Server 加载并通过校验。新建 collect 工作还要求能力与版本已登记、SourceDefinition 部署校验完成、Runtime 管理连接有效、实例 enabled=true 且最近确认 actual=running。只有外层 Runtime 在线、apply 已结束的实例不能接下新采集工作。 +**输入：** scope_id、capability、capability_version、input；collect 另须 instance_id。Server 检查 `scope.contribute` 与 `scope.read`，按能力 kind 选择路径。 ### write：原调用等待 Source 保存结果 -例如宿主已有一段正文，使用内置 `content.inline / 2` 保存： - -```json -{ - "name": "pc_source_add", - "arguments": { - "scope_id": "scp_demo", - "capability": "content.inline", - "capability_version": "2", - "input": {"source_id": "meeting-note-001", "text": "需要保存的正文"} - } -} -``` - -source\_id 是这份数据的稳定身份，重复提交时保持不变。自定义 write 使用同一工具，选择其公开能力和业务输入，不传 instance\_id。 - ```mermaid sequenceDiagram participant H as 宿主 @@ -248,300 +171,96 @@ sequenceDiagram S-->>H: SourceRef 与保存结果，或错误 ``` -**从调用到返回：** - -1. 宿主提交已有完整内容。Server 认证并检查 Scope 权限、能力版本和输入要求，找到该能力关联的、已加载的 Definition 实现。 -2. Server 使用 Definition 的输入模型校验 input，调用 `resolve()` 形成带稳定身份的 Source，再调用对应 Projection 的 `project()` 形成投影数据。自定义函数来自已加载的插件模块，无需修改 Server 代码添加某个类型的 import。 -3. Server 校验输出契约，沿已有 Source 写入规则保存 Source 和 Projection；同身份同数据复用，同身份不同数据报冲突。 -4. Server 取得真实保存结果，检查调用者当前读取权限，经原 MCP 调用返回 SourceRef；输入、转换或保存失败则返回错误。原调用等待实际结果，不先返回“已受理”，宿主无需主动查询。 +1. Server 检查能力版本和加载状态，按关联 Definition 的输入模型校验完整内容。 +2. 调用 `resolve()` 形成稳定身份的 Source，再调用 `project()` 生成 Projection，校验输出并沿已有管道保存。 +3. 取得真实保存回执后，经原 MCP 调用返回 SourceRef 与 saved / unchanged；校验、转换或保存失败返回错误。宿主不需要再查询处理结果。 -内置与自定义 write 都在 Server 内完成上述流程；自定义类型需要预先加载实现。write 接收完整内容，不承担远程资料的持续获取。 +内置与自定义 write 使用同一路径。同一 `(scope_id, source_type, source_id)`、同数据复用，不同数据冲突；不同 ID 是不同 Source。稳定身份由底层提交方负责：内置适配层提供，自定义插件按 Definition 生成；需要宿主提供资料 ID 时在 input_schema 中说明。 -**成功返回示例：** - -```json -{ - "kind": "write", - "result": { - "outcome": "saved", - "source_refs": [{"source_type": "content", "source_id": "meeting-note-001"}] - } -} -``` - -SourceRef 以实际保存回执为准。saved 表示所需写入已完成；unchanged 表示底层确认已有相同 Source。宿主使用 SourceRef 和现有 Server 数据接口读取正文。 - -**重复调用与异常：** write 复用底层 `(scope_id, source_type, source_id)` 规则：同身份、同数据复用；同身份、不同数据报冲突；不同 ID 保存为不同 Source。稳定身份由底层提交方负责，内置适配层提供，自定义插件按 Definition 生成；需要宿主提供资料 ID 时在 input\_schema 中说明。重复调用可以再次执行校验或转换，不承诺外部副作用只发生一次。 - -调用超时、连接中断或 Server 退出时，结果可能未确认，不能据此判断 Source 没有保存。Server 检测到调用失效后清理当前处理；已经开始的保存可能成功，重连不自动重放。宿主需要重试时保持相同身份和实际提交数据，不能重新生成随机 ID 或变化的元数据。已保存的数据不因应答丢失而回滚。 +超时或应答丢失不表示未保存，重试应保持相同身份和实际数据。Server 不恢复或重放中断调用；重复调用可能再次执行校验与转换，不承诺外部副作用只执行一次。 ### collect：先返回采集关系，插件随后持续处理 -例如把新建的语雀文档 B 加入已经运行的 yuque-personal 实例： - -```json -{ - "name": "pc_source_add", - "arguments": { - "scope_id": "scp_demo", - "capability": "yuque.document", - "capability_version": "1", - "instance_id": "yuque-personal", - "input": {"url": "https://www.yuque.com/team/book/doc-b"} - } -} -``` - -**从调用到返回：** +1. Server 检查权限和输入，经管理通道把请求及可信调用者身份交给插件管理服务。 +2. 插件管理服务通过插件实例 Runtime 调用插件 `identify_target`，取得稳定 `target_key` 和完整 `normalized_input`，含义见第 4 节。 +3. 按 **Scope + 实例 + 能力及版本 + target_key** 查找目标；同身份同配置复用，不同配置报冲突。新目标还须确认 Definition 部署校验通过、实例可服务该 Scope、管理连接有效、enabled=true 且业务正在运行，再持久保存目标与身份关联。 +4. 插件管理服务回传目标，Server 返回 target_id、enabled、tracking 及已有 Source 引用和最近提交时间。新目标尚无提交时，引用为空、时间为空；响应确认的是采集关系，不是外部资料已保存。 -1. Server 检查宿主身份、Scope 权限和输入，将调用及可信调用者身份经管理通道交给管理服务。 -2. 管理服务通过 Runtime 调用插件的 `identify_target(request, config)`，取得 target\_key（稳定资料标识）和 normalized\_input（包含采集选项的规范化输入）。该入口独立于 apply，根据输入与实例配置确认资料身份。 -3. 管理服务按“Scope + 实例 + 能力及版本 + target\_key”查找目标。同身份同配置复用，同身份不同配置返回 target-config-conflict。新目标须通过实例可服务 Scope 与就绪检查，再将目标和身份索引一并持久保存；唯一约束处理并发重复。创建者仅作记录，不参与目标身份或管理授权判断。 -4. 管理服务通过 reply 返回目标。Server 在当前读取权限允许时，经原 MCP 调用把 target\_id、跟踪状态和已有采集结果返回宿主。到这里确认的是采集关系已建立，不是外部资料已保存。 +返回后，插件实例 Runtime 同步目标，现有 apply 从 `ctx.targets.list()` 取得新增资料，按插件节奏调用 run_batch。Worker 执行 Connector，Source 直接保存到 Server；SDK 关联真实提交回执，插件实例 Runtime 向插件管理服务报告引用与时间。宿主是否查询进展由宿主决定，查询不驱动采集。 -**首次创建的返回示例：** - -```json -{ - "kind": "collect", - "target_id": "tgt_b", - "enabled": true, - "tracking": "enabled", - "source_refs": [], - "last_submitted_at": null -} -``` - -**返回之后如何采集：** Runtime 从管理服务同步目标，现有 apply 在后续 `ctx.targets.list()` 中取得 B，自行决定分批和节奏。Worker 执行 Connector，插件使用自己的凭据读取语雀并提交；数据直接存到 Server。SDK 根据实际成功提交回执关联目标与 Source，由 Runtime 向管理服务报告 Source 引用和提交时间，管理服务收到后持久保存。这些记录表示已收到的提交。Worker 仍按已有规则处理批次与 checkpoint。宿主需要了解进展时调用 `pc_source_status`；查询既不触发采集，也不是维持跟踪的必要动作。 - -新增 B 不重启实例，也不重建文档 A 的业务客户端。对已有目标的重复 add 不重置进度、不额外采集、不恢复 stopping/stopped 目标；调用始终需要当前 PC 权限。重复身份确认需要 Runtime 可达，但复用已有目标不要求 apply 正在运行；无法确认身份时明确报错，不猜测新建。已有 target\_id 的查询或停止仍可由管理服务处理。 - -受理响应丢失时，宿主可用原 Scope、实例、能力版本和业务 input 再次调用，取得原目标。同一 Scope 内，不同用户添加相同实例、能力版本及资料身份且配置一致时复用同一目标；不同 Scope、实例或能力版本不合并。停止目标仍占原身份，本版不提供目标硬删除或自动合并历史重复记录。 +重复 add 不重置进度、不触发额外采集、不恢复已停止目标。同 Scope 不同用户符合上述身份与配置时复用目标；不同 Scope、实例或版本不合并。身份确认要求插件实例 Runtime 可达，复用已有目标不要求 apply 正在运行；无法确认时返回错误，不猜测新建。停止目标保留身份，本方案不提供目标硬删除。 ## pc_source_status：查询采集结果或控制状态 -此方法查询管理服务已保存的 collect 目标和提交记录。 - -**调用参数：** scope\_id、target\_id；默认读取采集状态，`view="control"` 只读取启停字段，供后续控制操作使用。 +**输入：** scope_id、target_id；默认查询采集状态，`view="control"` 查询启停字段。 -```json -{ - "name": "pc_source_status", - "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b"} -} -``` - -**从调用到返回：** - -1. Server 认证宿主。默认视图检查当前 scope.read；control 视图检查 scope.contribute。 -2. Server 完成相应 Scope 权限检查后，经管理通道转交目标查询。管理服务核对目标实际所属 Scope 与请求 Scope 一致；不一致时拒绝，不返回其他 Scope 的目标信息。 -3. 管理服务读取自己的目标记录和已经收到的采集结果，不调用 identify\_target、不要求 Runtime 在线、不等待新的 Worker 批次。 -4. 管理服务经 reply 返回，Server 将允许读取的结果交给原 MCP 调用。采集结果返回仍须当前读取权限；control 响应只包含有权查看的控制字段。 - -**宿主获得什么：** +**链路：** Server 检查对应视图权限 → 插件管理服务核对目标实际 Scope → 读取持久目标及已收到的提交记录 → reply → Server 返回宿主。查询不调用 identify_target、不等待新批次，也不要求插件实例 Runtime 在线。 -| 视图 | 返回内容与含义 | +| 视图 | 返回 | | --- | --- | -| 默认 | target_id、enabled、tracking、source_refs、last_submitted_at，以及最近确认时间和实例可用性。source_refs 是管理服务已收到的实际提交关联，last_submitted_at 是这些记录中的最近提交时间；没有记录时分别为空列表和 null | -| control | target_id、enabled、tracking；不包含 input、SourceRef 或业务错误正文。具有 scope.contribute 的调用者可读取控制状态，不额外要求 scope.read | +| 默认 | target_id、enabled、tracking、source_refs、last_submitted_at、最近确认时间及实例可用性 | +| control | target_id、enabled、tracking；不含输入、Source 引用或业务错误正文 | -tracking 表示采集关系的 enabled / stopping / stopped。source\_refs 和 last\_submitted\_at 仅反映已经确认并报告的提交事实，不表示目标已全部采完；没有新提交时，不区分尚未处理、暂无资料或内容未变化。提交报告可能丢失，因此此视图不是 Server 全部数据的清单。实例失联时保留已有提交记录，但可用性显示 offline / unknown，不能用历史提交表示此刻仍在采集。 - -该调用可以重复执行且无业务副作用。管理服务不可达时返回不可用，不由 Server 缓存伪造最新状态；这不影响有权限的宿主直接读取 Server 已保存的 Source。 +tracking 表示 enabled / stopping / stopped。提交记录只表示已确认并报告的保存事实，不表示目标全部采完；无新提交也不区分尚未处理、没有变化或暂无资料。插件实例 Runtime 失联显示 offline / unknown；插件管理服务不可达则查询返回不可用，仍可按权限直接读取 Server 中的 Source。 ## pc_source_target_set_enabled：停止或恢复一个目标 -**调用参数：** scope\_id、target\_id、enabled。宿主直接表达停止或恢复意图。 - -```json -{ - "name": "pc_source_target_set_enabled", - "arguments": { - "scope_id": "scp_demo", - "target_id": "tgt_b", - "enabled": false - } -} -``` - -**从调用到返回：** +**输入：** scope_id、target_id、enabled。 -1. Server 检查目标 Scope 的 scope.contribute，经管理通道转交调用。 -2. 管理服务核对目标所属 Scope，恢复时检查实例可服务范围，再保存 enabled，保留目标身份、已有 Source 与 checkpoint；重复设置当前状态不产生新动作。 -3. 管理服务返回控制状态，Server 经原 MCP 调用返回宿主。Runtime 同步后按目标启停状态处理采集。 +**链路：** Server 检查 scope.contribute → 插件管理服务核对目标 Scope，恢复时核对实例可服务范围 → 保存期望状态 → reply → 返回目标控制状态；插件实例 Runtime 同步后处理。 -返回表示启停意图已保存，不代表执行端已完成操作。恢复目标不会自动启用已停用的实例，也不会重新调用已经结束的 apply。 +应答表示意图已保存，不承诺执行端已完成；保留目标身份、Source 与 checkpoint。恢复目标不会启用已停用的实例或重新运行已结束的 apply。 ## pc_source_instances：发现实例或读取实例控制状态 -本方法管理 collect 实例；Server 内的 write 能力没有对应实例。 - -capabilities 以“能做什么”为入口；instances 以“哪些实例可用、某实例当前怎样运行”为入口。此方法不加载代码或启动实例。 - -**调用参数有两种组合：** - -| 视图 | 参数 | 用途 | -| --- | --- | --- | -| 业务视图 | scope_id、operation="add" | 查找能服务当前 Scope 的可见实例、PC 权限与运行状态 | -| 控制视图 | instance_id、view="control" | 读取有权管理的某个实例的控制状态；不要求指定 Scope | - -```json -{ - "name": "pc_source_instances", - "arguments": {"scope_id": "scp_demo", "operation": "add"} -} -``` - -```json -{ - "name": "pc_source_instances", - "arguments": {"instance_id": "yuque-personal", "view": "control"} -} -``` - -**从调用到返回：** - -1. Server 认证宿主。业务视图检查 scope.read，并取得当前 Scope 写权限判断；控制视图检查 server.admin。 -2. 业务视图由 Server 根据已同步目录中的公开范围、可服务 Scope 和实例状态直接返回，不再询问管理服务；目录与连接可用性的处理同 capabilities。 -3. Server 通过 server.admin 检查后，将控制视图查询经管理通道交给管理服务，不额外要求 Scope 权限。管理服务读取指定实例的持久控制状态与最近执行报告，经 reply 返回 Server，再交给宿主。两种视图都不重新调用 apply 或访问外部账号。 - -**宿主获得什么：** 业务视图返回实例 ID、可用能力、PC 权限及运行状态；控制视图返回 instance\_id、enabled、actual、可用性和最近更新时间，不包含业务 Config、目标输入或 Source 结果。 - -期望 enabled=true 不保证 actual=running；apply 正常结束可以是 actual=complete，未处理异常可以是 failed。Runtime 外层仍可能在线。控制视图用于了解期望状态与实际运行状态的区别，不能因“已启用”就宣称正在采集。 +| 视图与输入 | 从调用到返回 | +| --- | --- | +| 业务：scope_id、operation=add | Server 检查 Scope 权限，按 capabilities 的目录规则直接返回可见实例、能力、可写性与运行状态 | +| 控制：instance_id、view=control | Server 检查 server.admin，经管理通道读取实例的持久控制状态与最近执行报告，再返回 instance_id、enabled、actual、可用性与更新时间 | -调用只读，可重复执行。业务视图按 Server 已同步的公开信息回答，管理连接失效时明确标注自定义部分不可用，不用旧目录保证当前权限或可执行性；无法确认可见范围时不暴露记录。控制视图依赖管理服务可达；仅 Runtime 失联时仍可读取持久控制状态，但运行可用性须标明未知或离线。内置与自定义 write 能力都不需要插件实例。 +控制视图不要求 Scope，也不返回 Config、目标输入或 Source。仅插件实例 Runtime 离线时仍可读取控制状态，但须标注离线或未知；插件管理服务不可达则控制查询不可用。本方法不加载代码、访问外部账号或启动 apply；write 没有实例。 ## pc_source_instance_set_enabled：启停整个实例的业务运行 -此方法控制 Runtime 内的 apply / Context，影响该实例服务的所有 Scope 和目标；它不是安装、卸载或启动远端系统进程的命令。 - -**调用参数：** instance\_id、enabled。宿主直接表达启用或停用意图。 +**输入：** instance_id、enabled。影响此实例所有 Scope 和目标，不负责安装或远程启动系统进程。 -```json -{ - "name": "pc_source_instance_set_enabled", - "arguments": { - "instance_id": "yuque-personal", - "enabled": false - } -} -``` - -**从调用到返回：** - -1. Server 认证宿主并检查 server.admin，再将实例控制调用交给管理服务；实例可能服务多个 Scope，单个 Scope 的权限不能用于启停整个实例。 -2. 管理服务检查实例存在及切换条件，在事务内按实际处理顺序设置期望状态。同状态返回当前结果；停用保存 enabled=false，并在仍需执行端确认时标为 stopping；启用要求旧业务运行已经确认 stopped，再保存 enabled=true、生成新的 run\_id（一次业务运行的身份）并等待 Runtime 执行。 -3. 管理服务通过 reply 将持久控制状态交回 Server，再返回宿主。响应包含 instance\_id、enabled、actual 等控制字段，不表示系统进程已经安装或业务已经启动成功。 -4. Runtime 同步后执行期望状态。停用阻止新批次，取消并等待 apply 及所属子任务，关闭 Context 和业务资源；全部完成后报告 stopped，外层 Runtime 继续在线。启用创建新的 Context 并调用 apply，报告实际运行状态。宿主需要确认时调用 instances 的 control 视图；查询不驱动启停。 - -仅 Runtime 失联时可以保存停用意图，但不能报告已停止；清理失败或旧任务未退出时不启动第二份 Context。管理服务失联时不能确认控制变更。重新启用后只处理仍 enabled 的目标，不恢复已经单独停止的目标，不重置目标 checkpoint。 +**链路：** Server 检查 server.admin → 插件管理服务核对切换条件并保存期望状态 → reply → 返回 instance_id、enabled、actual → 插件实例 Runtime 同步并执行，第 5 节规定清理与重启行为。 -同一 Runtime 进程内，已 enabled 时重复启用不会再调用 apply，即使本次 apply 已 complete 或 failed。需要重新运行时明确停用，确认 stopped，再显式启用。关闭或卸载整个 Runtime 后，管理服务不能远程替它启动系统进程；部署操作见第 2、5 节。 +响应不承诺业务已启停。插件实例 Runtime 失联时可以保存停用意图，不能确认已停止；重新启用须确认旧业务 stopped。已启用时重复启用不重跑 apply，即使它已经 complete 或 failed；再次运行须显式停用、确认停止，再启用。单独停用的目标不会随实例重新启用而恢复。 ## 操作重试 -collect 重复 add 复用目标，write 重复提交复用底层 Source 规则，启停直接设置期望状态；相同状态不产生新动作，不同意图按实际处理顺序生效。通信超时不表示已保存的数据或控制变更被撤销，宿主可按各方法的规则重试或读取当前状态。 +发现和查询只读；collect add 按稳定目标身份复用，write 沿用底层 Source 规则。启停直接设置期望状态，相同状态不产生新动作，不同意图按实际处理顺序生效。通信超时不撤销已保存数据或控制状态，宿主可按这些规则重复调用或读取当前状态。 # 4. 插件接口与数据管道 -本节面向插件作者。write 提供 Server 可以加载的输入模型、Definition 与 Projection 实现；collect 在这些数据契约之外提供 apply 和 Connector，自行决定采集节奏，通过 Worker 提交。宿主调用方式见第 3 节，加载、进程启停和重启行为见第 5 节。 - ## 插件需要提供的契约 -插件代码可以从本地目录加载，无需发布 wheel。write 的代码与依赖部署在 Server 机器；collect 的代码与依赖部署在 Runtime 机器。两者可以复用同一个代码包中的 Definition 和 Projection，无需强制拆成两个包。 - -**共同的数据契约。** SourceDefinition 声明输入模型、Source 模型、名称和版本；`resolve(value)` 将输入转换成具有稳定身份的 Source，`projections` 中各实现的 `project(source)` 生成相应 Projection。`definitions` 导出这些实现对象。能力声明包含 name、version、description、kind 和 input\_schema，描述宿主能调用什么；Definition 描述结果如何保存。collect 的一个能力可以提交多种 Source，多个能力也可以复用同一 Source type。 +插件可直接加载本地目录。write 代码与依赖部署在 Server 机器，collect 部署在插件实例 Runtime 机器，可以复用同一代码包的数据定义。 -**write 的加载契约。** 每项 write 能力明确关联一个 Definition 的名称与版本，input\_schema 对应它的输入模型；Server 可以据此直接校验 input 并调用该 Definition。插件导出 `definitions` 和 `capabilities` 供 Server 加载。能力名称不得覆盖内置能力，同名同版本须保持输入和操作含义兼容。 +**共同数据契约：** SourceDefinition 声明输入模型、Source 模型、名称及版本；`resolve(value)` 生成稳定身份的 Source，`projections` 中的 `project(source)` 生成 Projection。插件通过 `definitions` 导出实现对象，通过 `capabilities` 声明能力名称、版本、说明、kind 和 input_schema。能力描述可调用行为，Definition 描述输出数据；二者不是一一对应的类型体系。 -下面通过**接口示意**展示“Server 如何得到 Projection 函数”。示例说明插件导出对象与 Server 加载的关系，具体类名、模块路径和辅助代码可随实现调整；write 能力关联字段和部署入口尚待实现。假设部署者准备一个可导入的本地包： +**write 加载：** 每个 write 能力关联一个 Definition 的名称与版本，input_schema 对应其输入模型。插件模块导出实际 Definition 和 Projection 对象，Server 通用加载器按受信任入口导入并登记，因此无需在 Server 代码中增加类型专属 import。新增模块只需准备代码、依赖和入口配置，不能仅通过注册 API 上传定义结构来获得可执行函数。导入不启动采集任务。 -```text -/srv/powercontext/custom_sources/tickets/ - __init__.py - ticket_definition.py -``` +**collect 运行契约：** -`ticket_definition.py` 实现 TicketCapture、TicketDefinition 和 TicketProjection。其中关键关系如下（省略已有 Source 模型和 resolve 实现）： - -```python -class TicketProjection: - # 同时声明 Projection 名称、版本、输入 Source 类与输出模型。 - def project(self, source): - return TextEvidence( - source_type="example-ticket", - source_id=source.name, - content=f"{source.capture.title}\n\n{source.capture.body}", - ) - - -class TicketDefinition: - name = "example-ticket" - version = "1" - input_class = TicketCapture - projections = (TicketProjection(),) - # resolve() 生成稳定身份的 TicketSource；其余成员遵循 SourceDefinition。 -``` - -插件自己的 `__init__.py` 导出对象，类型专属 import 写在插件中： - -```python -from .ticket_definition import TicketCapture, TicketDefinition - -definitions = (TicketDefinition(),) -capabilities = ({ - "name": "ticket.write", - "version": "1", - "kind": "write", - "description": "保存宿主提供的工单内容", - "input_schema": TicketCapture.model_json_schema(), - "definition_name": "example-ticket", - "definition_version": "1", -},) -``` - -Server 配置入口指向这个包，由通用加载器导入。以下只展示导入对象关系，不是完整加载器实现： - -```python -module = importlib.import_module(entry) # 例如 tickets；来自受信任部署配置。 -registry = SourceDefinitionRegistry(module.definitions) -# 运行时 registry.resolve / registry.project 调用这些实际 Python 对象。 -``` - -因此，Server 能执行 TicketProjection.project，是因为**它已导入包含该对象的本地模块**。新增类型只增加模块与入口配置，不在 Server 中硬编码类型专属 import。部署者须将代码加入 Server 的模块搜索路径，并准备相同环境中的依赖；文件路径入口也由通用加载器解析。实现只校验和转换当前输入，不在导入时启动采集任务。 - -**注册 SourceDefinition 与加载代码是两个动作。** 现有注册 API 保存类型结构、身份和 Projection 契约，不传输 Python 实现。Server 的 write 加载流程校验代码、契约与冲突，并在注册成功后才发布能力；仅有注册记录不能执行 write。collect 仍在部署或升级阶段使用 `server.admin` 注册 Definition，日常 Worker 只使用 Scope 执行权限，不每批注册或临时取得管理员凭据；Definition 缺失或不匹配时明确报错。 - -**collect 额外提供的运行契约：** - -| 导出项或对象 | 插件需要定义的内容 | +| 导出项 | 插件职责 | | --- | --- | -| `plugin_api_version` | 与 Runtime 对接的插件接口版本，当前为字符串 `"2"` | -| `Config` | Pydantic 配置模型，用于定义和校验插件自身逻辑所需配置，例如访问外部系统的凭据或凭据引用 | -| `capabilities / definitions` | 非空的 collect 能力声明与 SourceDefinition 集合；能力输入描述外部资料位置或选择条件 | -| `connector_name / connector_version` | Connector 的稳定名称与兼容版本，用于核对 checkpoint 归属，不能随重启随机变化 | -| `identify_target(request, config)` | 用输入确定稳定资料身份和规范化采集参数 | -| `apply(ctx, config)` | 异步业务入口；建立客户端、connect Connector，自行决定分批、等待、重试和退出 | -| `Connector.run(session)` | 处理并提交本批数据，返回 ConnectorRunCompletion（本批是否完成及下一 checkpoint） | +| plugin_api_version | 声明与插件实例 Runtime 对接的接口版本 | +| Config | Pydantic 模型，校验插件自身逻辑配置，例如外部凭据引用；不公开给宿主 | +| capabilities / definitions | 非空能力与数据定义；能力输入描述外部资料位置或选择条件 | +| connector_name / connector_version | 稳定 Connector 名称与兼容版本，用于核对 checkpoint 归属 | +| identify_target(request, config) | 按能力、版本、输入与实例配置确定稳定资料身份和规范化参数 | +| apply(ctx, config) | 异步入口；建立客户端、connect Connector，决定循环、分批、等待、重试与退出 | +| Connector.run(session) | 读取并提交本批数据，返回 ConnectorRunCompletion：本批完成情况与下一 checkpoint | -collect 的 Config 留在实例侧，外部凭据与业务权限由插件自行处理。能力描述和 input\_schema 只公开宿主调用所需信息；目录、结果和普通日志不暴露秘密。同版本 collect 能力还须保持资料身份规则兼容。 - -**collect 如何识别同一份资料：** Runtime 在受理目标时调用 identify\_target，独立于 apply 的采集循环。request 包含 capability、capability\_version 和 input；插件返回： - -| 返回项 | 含义 | -| --- | --- | -| target_key | 稳定资料标识，管理服务据此复用同一目标 | -| normalized_input | 规范化后的完整采集参数，保留全部影响采集的选项，用于检查同一目标是否发生配置冲突 | +`identify_target` 独立于采集循环，返回 `target_key` 和 `normalized_input`。前者识别同一资料，例如去掉 URL 页面定位锚点后的文档 ID；后者保留全部影响采集的参数，用于识别配置冲突。它不判断授权或资料是否可读，这些在实际采集时处理；输入不足以确认身份时明确失败。 -例如，插件可以去掉文档 URL 中仅用于页面定位的锚点，将两个地址识别为同一目标。identify\_target 根据输入和实例配置确认资料身份，文档内容与可读性在实际采集时处理。input\_schema 应要求足够的资料标识，无法确认相同时，不提前承诺合并。 +能力名称不能覆盖内置能力，同名同版本保持输入和操作语义兼容；collect 还须保持目标身份规则兼容。Definition 在部署或升级阶段由管理员注册，日常 Worker 只使用 Scope 执行权限，缺失或不匹配时报错，不每批注册。 ## collect：从 apply 到 Source 保存 -`ctx` 是 Runtime 传给 apply 的 PluginContext，持有 Connector 和业务资源，并提供下表中的 SDK 接口。一个 Context 只 connect 一个 Connector。`session` 由 Worker 为每批创建，提供本批 Scope、当前 checkpoint 和提交能力。 - -一次 apply 可以执行多个批次；**一次 `run_batch()` 就是一批，固定一个 Scope 和一组待处理内容。** 同一 collect 目标可以在多个批次中持续产生 Source，SDK 关联目标与实际提交，宿主不生成批次 ID。 - -下图展示一次批次调用的顺序。左侧 apply 和 Connector 都是插件代码，中间是 SDK 与已有 RemoteConnectorWorker，右侧是 PC Server；箭头表示调用或返回，不表示独立进程数量。图中展示 collect 的成功提交路径，submit 可以重复调用。binding 表示本实例在某个 Scope 的执行归属，隔离规则见本节末尾。 +Context 持有一个 Connector 和业务资源；Worker 为每批建立 session，提供 Scope、checkpoint 与提交能力。**一次 run_batch 是一批，固定一个 Scope 和一组所选目标；一次 apply 可执行多批。** ```mermaid sequenceDiagram @@ -562,7 +281,7 @@ sequenceDiagram SDK->>S: session.submit / Source 与 Projection S-->>SDK: 真实提交回执 SDK-->>C: ConnectorItemOutcome - Note over SDK: 关联 target 与 SourceRef，供 Runtime 报告 + Note over SDK: 关联 target 与 SourceRef，供插件实例 Runtime 报告 C-->>W: ConnectorRunCompletion W->>S: 按完成与回执规则 CAS 更新 checkpoint W-->>SDK: 批次结果 @@ -570,46 +289,23 @@ sequenceDiagram Note over A: 插件决定等待、下一批或退出 ``` -apply 先取得待处理内容，选择同一 Scope 的条目，再调用 run\_batch。Worker 读取 checkpoint，调用 Connector.run；Connector 按业务需要读取外部系统并转换数据，再通过 SDK 提交。等待和下一批由插件决定。 - -| 步骤 | collect 接口 | -| --- | --- | -| 获取当前可处理内容 | `ctx.targets.list()` | -| 执行所选批次 | `ctx.run_batch(scope_id=..., targets=selected)` | -| 提交一项数据 | `ctx.targets.submit(...)` | - -list 返回的是本次处理使用的目标快照，字段含义如下： - -| 字段 | 含义 | -| --- | --- | -| target_id | 对应管理服务持久保存的目标 | -| scope_id、instance_id、capability、capability_version | 归属与能力契约 | -| input | identify_target 确认后的规范化采集参数 | -| source_refs | 管理服务已收到的真实提交关联，供插件按需参考；不代表目标已采完 | +插件通过 `ctx.targets.list()` 取得已同步的目标快照（目标身份、归属、规范化输入及已报告的 Source 引用），选择同 Scope 的目标，调用 `ctx.run_batch(scope_id=..., targets=selected)`。同一 Context 顺序执行批次，批内快照不并发改写。管理模式显式传所选集合，自主采集用显式空集合；独立入口保留 `run_batch(scope_id=...)`。 -collect 首次同步成功后，ctx.targets.list / run\_batch 可在管理服务断线期间继续使用内存中最后同步的目标与启停状态；没有初始快照时明确报不可用，不用空列表冒充没有工作。新增目标在同步后的快照中出现。 - -同一 Context 的批次顺序执行，不重叠；一批选择同一 Scope 的 targets，不并发改写本批快照。管理模式必须显式传入所选集合；自主采集使用显式空集合，遗漏参数不表示执行所有目标。独立运行入口保留 `run_batch(scope_id=...)`。 - -submit 接收当前条目、session、本批唯一的 item\_id、Definition 名称和业务 value。SDK 校验实例、Scope 与批次归属，包装 `session.submit(...)`，并把实际提交回执关联到该条目。返回的 ConnectorItemOutcome 表示这一项提交的处理结果，插件必须检查是否被接受。处理后的数据直接进入 PC Server，不经管理服务转存。 - -collect 的实际提交关联报告给管理服务保存。完整 apply 与 Connector 代码见附录 B。 +Connector 用 `ctx.targets.submit(target, session, item_id, definition_name, value)` 提交。SDK 校验实例、Scope 与批次归属，包装 session.submit，并用真实回执关联目标和 Source。item_id 在批内唯一，插件检查返回的 ConnectorItemOutcome 是否接受；数据直接保存到 Server，不由插件管理服务转存。 ## collect 的提交回执与批次完成 -插件自行维护循环、分页和重试，通过 `ctx.targets.submit(...)` 提交数据。提交回执表示数据保存结果，ConnectorRunCompletion 表示本批完成情况；持续采集目标可以跨多个批次产生 Source。 - -| 动作 | 含义 | +| 信号 | 含义 | | --- | --- | -| submit 收到成功回执 | 一项数据已由 Server 接受；SDK 将真实 SourceRef 关联到对应 collect 目标 | -| Connector 返回 ConnectorRunCompletion | 声明本批是否完成及下一 checkpoint；Worker 按已有规则核对并处理进度 | -| collect 提交记录报告管理服务 | 保存已确认的 Source 引用与提交时间，供宿主按需查看；不等待该目标“全部采完”，也不要求同批其他目标完成 | +| 成功提交回执 | Server 已接受一项数据，SDK 可以关联真实 SourceRef 并报告，不等待其他目标完成 | +| ConnectorRunCompletion | 本批是否完成及下一 checkpoint，Worker 按已有回执和完成规则处理 | +| 插件管理服务收到提交报告 | 已保存引用与时间，可供宿主查询；不表示持续采集目标已全部处理完 | -没有新提交时，框架不推断目标是尚未处理、没有变化还是暂无资料。失败后是否重试、何时继续仍由插件决定；未处理异常按第 5 节结束当前 apply。Source 保存、checkpoint 更新和向管理服务报告提交记录是独立动作：批次未完成或报告丢失不撤销已保存 Source，报告也不保证重启后补齐。 +循环、分页和失败重试由插件决定。Source 保存、checkpoint 更新和报告是独立动作，故障恢复见第 5 节；框架不根据“没有新提交”推断业务处理结果。 ## checkpoint 的隔离、推进与新增目标 -ConnectorBinding 表示“哪个实例的 Connector 在哪个 Scope 执行”。本方案用 instance\_id 作为 binding\_id，PC Server 按 `(scope_id, binding_id)` 保存 checkpoint，并核对 connector\_name / connector\_version。这里的 ConnectorBinding 不同于权限系统中授予角色的 Scope Binding。 +ConnectorBinding 表示实例的 Connector 在某个 Scope 的执行归属，采用 instance_id 作为 binding_id。Server 按 `(scope_id, binding_id)` 保存 checkpoint，并核对 Connector 名称与版本；它不同于权限系统的 Scope Binding。 ```mermaid flowchart TB @@ -625,35 +321,21 @@ flowchart TB A --> P[内容格式由插件定义，可包含逐目标进度] ``` -图中的每个 checkpoint 由 Server 保存，内容格式由插件定义。同实例、同 Scope 的多个目标或能力共用一份 checkpoint，框架不会为每个 target 自动创建独立游标；插件可以在其中记录各目标的进度。不同实例或不同 Scope 的 checkpoint 互不共享。 - -Worker 在批次开始时读取 checkpoint，Connector 根据实际业务进度返回下一值；需要更新时，Worker 使用 CAS（仅当 Server 中仍是本次读取的旧值才写入新值），避免覆盖并发进度。插件不能在尚有未处理资料时推进游标而跳过它们。 - -例如文档 A 已采集至时间 T，后来加入一篇早于 T 的文档 B。插件不能直接用 A 的时间游标跳过 B，应识别新增目标，为 B 初始化进度并补采；具体记录方式由插件定义。停止或恢复目标不自动清空 checkpoint，Connector 升级也不能靠换版本号绕过已有进度的兼容性检查。 +checkpoint 内容由插件定义，同实例同 Scope 的目标和能力共用一份，插件可在其中维护逐目标进度。Worker 读取当前值，按 Connector 返回的业务进度使用 CAS 更新（仅当存储中仍是本批读取的旧值才更新），不能越过尚未处理的资料。 -Runtime 进程重启后重新同步 targets，由 Worker 读取 Server 当前 checkpoint，插件决定继续或重新采集；框架不恢复上次批次的具体执行步骤。 +例如 A 已采至时间 T，新加入的 B 早于 T，插件应为 B 初始化进度并补采，不能直接沿用 A 的时间游标跳过它。目标启停不清空 checkpoint，重启后从 Server 读取进度继续，不恢复上次批次的执行步骤；升级不能靠更换版本绕过进度兼容检查。 # 5. 生命周期、重启与权限 -本节的 Runtime、实例与目标生命周期用于 collect，采集节奏仍由 apply 决定。write 实现随 Server 加载和释放，没有独立实例启停：新增实现可显式加载，一致配置重复加载复用已有登记；已有代码替换通过受控重启 Server 生效。启动时重新校验已保存入口，加载失败的能力不标为可执行，其他有效能力照常使用。 +write 随 Server 加载和释放，新增实现可显式加载，一致配置重复加载复用登记；替换已有代码通过受控重启生效。重启重新校验入口，失败能力不可执行，其他有效能力照常使用。以下实例生命周期用于 collect。 -## Runtime 在线与 apply 运行 +## 插件实例 Runtime 在线与 apply 运行 -后台管理模式下，每个实例有一个 Runtime 进程。Runtime 维护管理连接、同步目标和启停状态；其中的 **Context 是一次 apply 运行所持有的 Connector、客户端和业务任务的集合**。业务等待或 sleep 不阻塞 Runtime 接收管理操作。 - -实例需要区分三件事： - -| 信息 | 含义 | -| --- | --- | -| enabled | 管理服务保存的期望启停状态；为 true 不代表 apply 此刻一定在运行 | -| actual | Runtime 最近报告的业务状态，例如正在运行、正常结束（complete）、异常结束（failed）、正在停止（stopping）或已停止（stopped） | -| 连接可用性 | Runtime 是否仍可通信；失联显示 offline / unknown，不能用旧的 actual 宣称业务仍在执行 | - -**apply 结束后，后台 Runtime 仍保持在线。** 它可以接收控制操作，但不会因新增目标、通信重连或重复启用而自动再次调用 apply。持续处理新增目标的插件，需要在 apply 内保持自己的采集循环。 +插件实例 Runtime 维护管理连接，Context 持有一次 apply 的业务资源。enabled 是期望状态，actual 是最近确认的 running / complete / failed / stopping / stopped；连接失效另标 offline / unknown，不能用旧状态保证正在采集。 ```mermaid flowchart TD - S[Runtime 启动并同步实例状态与目标] --> E{enabled?} + S[插件实例 Runtime 启动并同步实例状态与目标] --> E{enabled?} E -->|否| I[保持在线等待] E -->|是，旧进程已退出| R[创建 Context 并执行 apply] R -->|正常返回| C[清理 Context，报告 complete] @@ -663,458 +345,75 @@ flowchart TD R -->|停用| D[取消并等待任务，关闭资源] I -->|停用| D I -->|从 disabled 启用，旧业务已确认停止| R - D -->|清理完成| T[确认 stopped，Runtime 仍在线] + D -->|清理完成| T[确认 stopped，插件实例 Runtime 仍在线] T -->|显式启用| R I -->|新增目标、重连或已 enabled 时重复启用| I D -->|清理失败| U[报告错误，不创建第二份 Context] ``` -卸载系统服务会使 Runtime 进程退出；再次启动需要在执行机器执行部署操作。 +apply 返回或异常结束后清理 Context，插件实例 Runtime 仍在线；新增目标、重连或重复启用不重新调用 apply。需要持续处理新增目标的插件自行保持循环，等待不阻塞管理操作。内部运行标识只用于区分当前连接和本次运行的报告，宿主无需提供。 -若要在同一进程中重新执行已经结束的 apply，显式停用、确认 stopped，再启用。直接运行的 `pc-source run` 则在 apply 结束后退出；它与后台模式不能同时占用同一实例身份。 - -管理服务保存期望状态、最近实际状态及更新时间。框架用内部运行标识（run\_id）区分每次 apply 的报告，只接收当前连接和当前运行的消息，避免迟到报告覆盖新状态。宿主无需提供该标识，Runtime 也不用它跨进程恢复业务。 +同一进程再次运行须停用、确认 stopped，再启用；进程重启则在同步后按 enabled 决定是否新建 Context。直接运行入口在 apply 结束后退出，不能与后台模式同时占用同一实例。系统服务卸载或退出后，须由执行机器的部署工具重新启动。 ## 实例停用与资源清理 -停用实例时，管理服务保存 enabled=false；Runtime 阻止新批次，取消并等待 apply 及其子任务，关闭 Context。清理完成后报告 stopped，外层 Runtime 保持在线，等待再次启用。 - -业务子任务必须归属于 apply，例如使用 TaskGroup；资源通过 `ctx.enter_async_context` / `ctx.on_dispose` 登记，随 Context 关闭。插件不能留下脱离生命周期的后台任务，也不能吞掉取消异常。任务未结束或资源清理失败时，不报告 stopped，不创建第二份 Context。 +插件实例 Runtime 收到停用状态后阻止新批次，取消并等待 apply 及所属子任务，关闭 Context；完成后报告 stopped，外层保持在线。业务子任务归属 apply，资源通过 Context 登记清理，不能留下脱离生命周期的任务或吞掉取消异常。清理失败或旧任务未退出时不确认 stopped，也不创建第二份 Context。 -实例停用不影响 Server 已加载的 write 能力，也不撤销已经保存的 Source。Runtime 失联时不能宣称业务已经停止；管理服务不可达时不能确认新的启停意图已保存。 +停用不撤销已保存 Source，也不影响 Server 的 write 能力。失联不等于停止；插件管理服务不可达时不能确认新的控制状态已保存。 ## 断线、重启与数据恢复 -**连接重连保留当前进程内存，进程重启则创建新的业务运行。** Server 或管理服务重启不等于 Runtime 重启。系统服务管理器负责进程存活，不负责调用 Worker。 - -| 情况 | 后续行为 | +| 情况 | 行为 | | --- | --- | -| Runtime 尚未取得初始目标快照 | collect 的 list / run_batch 报不可用，插件决定等待或退出；进程重启后也须先取得目标 | -| 已同步的 Runtime 与管理服务断线 | collect 可使用内存中最后同步的目标和启停状态开启新批次，节奏仍由插件决定；提交继续受 Server 权限检查，提交报告可能丢失 | -| 同一 Runtime 进程重连 | 先同步最新实例和目标启停状态，再允许后续处理；按同步后的启停状态处理，不重跑仍在运行或已经结束的 apply | -| Runtime 进程重启 | 内存中的批次和未送达报告丢失；重新同步后，实例仍启用就创建 Context、执行 apply，即使上次状态为 complete / failed；Worker 读取 Server checkpoint | -| 仅管理服务与 Server 的通道中断，Runtime 仍连接两者 | 宿主的 collect 管理操作不可用；已有 collect 可在当前同步状态允许下继续。内置功能与已加载的自定义 write 独立可用 | - -同身份旧进程是否退出无法确认时，不启动另一份 Runtime；心跳丢失不构成跨机器接管的依据。框架不保证 apply 或外部副作用只执行一次，重复提交沿用底层 Source 身份规则。 - -Runtime 只保留启动所需文件，当前执行信息在内存中；持久存储归属见第 2 节。恢复依赖管理服务中的目标与启停状态，以及 Server 中的 Source 和 checkpoint。 +| 尚无初始目标快照 | list / run_batch 报不可用，插件决定等待或退出，不用空列表冒充无目标 | +| 插件实例 Runtime 与插件管理服务断线 | 按内存中最后同步的目标和启停状态继续采集，允许新批次；提交仍受 Server 权限检查 | +| 同一插件实例 Runtime 重连 | 先同步最新控制状态，再继续处理；不重跑当前或已结束的 apply | +| 插件实例 Runtime 进程重启 | 重新同步目标与状态，enabled 时执行新的 apply，即使上次 complete / failed；Worker 读取 Server checkpoint，未送达报告丢失 | +| 插件管理服务与 Server 断线 | 宿主 collect 管理操作不可用，已有采集可继续；内置和已加载 write 独立可用 | +| Server 不可用 | 数据提交失败或等待恢复；已保存 Source 保留，write 调用不自动恢复或重放 | -collect 的 **Source 保存、checkpoint 更新、提交记录报告**是三个独立动作，正常处理关系见第 4 节。故障时分别处理： +同身份旧进程是否退出无法确认时，不启动另一份插件实例 Runtime；心跳丢失不是跨机器接管依据。框架不保证 apply 或外部副作用只执行一次。 -| 故障位置 | 恢复依据与影响 | -| --- | --- | -| Source 已保存，checkpoint 尚未更新 | 下次可能重新采集；同身份同数据由底层复用。插件须保持 Source 身份稳定，数据变化按 Definition 的规则处理 | -| checkpoint 更新响应丢失 | 下次读取 Server 中实际保存的进度；不能声称本次进度已确认，但已有成功提交回执仍可报告 | -| Source 和 checkpoint 已保存，提交报告丢失 | 数据和进度有效；管理服务可能缺少对应提交记录，不回退 checkpoint，也不保证重启后补齐报告 | - -因此，管理服务没有提交记录，不代表 Server 没有 Source；有权限的宿主仍可通过 Server 数据接口读取。collect 按插件节奏继续，不恢复上次批次的具体执行步骤。Server 重启会中断进行中的 write；重启后重新加载实现，不恢复或重放旧调用。已保存 Source 不受影响，重试仍沿用相同 Source 身份与数据。 +恢复以各存储中的真实数据为准：Source 已保存而 checkpoint 未更新时可能重新采集，沿用稳定身份避免重复；checkpoint 更新响应丢失时重新读取；提交报告丢失不回退 Source 或 checkpoint，也不保证补齐管理记录。 ## 新增实例、修改配置与升级 -| 变更 | 处理方式 | -| --- | --- | -| 新增实例 | 准备代码和依赖，由部署流程注册 Definition、登记实例并启动 Runtime；发布新能力，不重启已有实例 | -| 给运行实例增加 Scope | 部署者授予执行身份相应 Scope Binding，并更新实例可服务范围；Runtime 同步后，宿主可添加该 Scope 的目标，原 Context 保留 | -| 修改业务配置或升级代码 | 停用并确认 stopped，停止旧 Runtime，再校验和更新代码、Config 与登记；启动新 Runtime 后显式启用。外部凭据变更按同一配置更新流程处理，其业务含义由插件解释 | +- **新增实例：** 部署代码与依赖，注册 Definition、登记并启动插件实例 Runtime，不重启其他实例。 +- **扩展 Scope：** 授予执行身份相应 Scope Binding（Server 中的角色授权关系），更新实例可服务范围；同步后可添加新 Scope 目标，保留原 Context。传入 scope_id 本身不授予权限。 +- **缩减范围、修改执行配置或升级 collect：** 停用并确认停止，退出旧插件实例 Runtime，校验和更新代码、Config 与登记；新插件实例 Runtime 校验成功后显式启用。凭据变更按配置更新处理，业务含义由插件解释。 -Scope Binding 是 Server 权限系统中授予身份某个 Scope 角色的关系，不是第 4 节用于 checkpoint 的 ConnectorBinding。仅提交新的 scope\_id 不会授予权限，也不要求按 Scope 新建插件实例。 +升级保留 Source、目标、checkpoint 与已收到的提交记录，核对能力输入和目标身份、Definition、Connector 与 checkpoint 格式的兼容性。新代码须能处理保留的目标，包括停用目标；不能通过重置进度绕过检查。无法确认兼容时保持停用，由部署者明确处理，不自动迁移；新实例不自动继承旧目标。 -collect 代码或配置更新前须确认旧 Runtime 已退出；已有 Source、目标和 checkpoint 保留。write 实现更新则先处置 Server 在途调用，再通过受控重启加载新代码；须保持能力与 Definition 契约兼容，不迁移旧调用或改变已有 Source 身份。collect 升级需核对三类兼容性： - -| 保留的数据或契约 | 要求 | -| --- | --- | -| collect 目标的 capability/version、输入与身份规则 | 新代码仍能处理已登记的目标，包括已停止目标；只支持 v2 的入口不能直接接管 v1 目标 | -| SourceDefinition、ConnectorBinding 与 checkpoint 格式 | 兼容已有 Source 和进度，不通过更换版本或重置游标绕过检查 | -| 目标与已收到的提交记录 | 保留原记录和 Scope 归属，访问统一检查当前 PC 权限，不要求重新执行业务来重建历史 | - -无法确认兼容时，保持停用，不删除记录或重置进度。部署者可准备兼容代码，或明确处理不兼容的目标与进度；第一版不提供自动迁移，新实例也不会自动继承旧目标。 - -通过检查后，部署者用 configure 接口更新登记，仍保持 disabled；新 Runtime 校验成功后再显式启用。失败保持不可执行，不并行启动第二份 Context。普通重复启动或加载只复用一致配置，配置冲突须明确报错，不能代替更新流程；管理接口见附录 C。 +write 更新先处置在途调用，再重启加载，保留能力与 Definition 契约及已有 Source 身份。普通重复加载不代替配置更新。 ## 权限职责与操作条件 -**用户授权统一复用 PC Server 的 Access 体系。** Principal 是 Server 认证得到的用户或服务身份，宿主不能在业务 input 中自行声明它。 +用户授权统一复用 Server 的 Access 体系。Principal 是 Server 认证出的用户或服务身份；插件管理服务不另建用户授权系统。 -| 负责方 | 职责 | +| 操作 | Server 检查 | | --- | --- | -| PC Server | 认证宿主并检查当前 Scope 或 Server 权限，授权通过后才转交管理请求；write 按宿主身份检查保存权限，collect 每次提交检查实例执行身份；返回数据时检查当前读取权限 | -| 管理服务 | 只接受认证通道中的 Server 管理请求，核对目标实际所属 Scope、实例可服务范围和运行条件，再执行操作 | -| 插件 | 解释自身配置与外部凭据，在访问外部系统时处理业务权限；identify_target 只确认资料身份，不参与授权 | - -| 操作 | PC Server 检查的权限 | -| --- | --- | -| 能力与业务实例发现 | scope.read，并在结果中说明当前是否具备 scope.contribute | -| write、新建或重复 collect | scope.contribute；本接口应答包含数据或已有 Source 引用，因此同时检查 scope.read | -| 查询 collect 目标及提交结果、读取 Source | scope.read | -| collect control 视图、停止或恢复目标 | scope.contribute；仅返回启停字段，不额外要求 scope.read | -| 加载 Server 本地 write 实现；实例 control 视图、加载、配置、卸载或启停整个实例 | server.admin；实际进程操作仍由执行机器的部署工具完成 | -| 为用户或执行身份授予 Scope 权限 | 沿用现有 Access 接口的授权规则 | - -同一 Scope 内，具有 scope.contribute 的用户都可以控制该 Scope 的采集目标，不限于创建者；创建者只作记录。实例可能服务多个 Scope，因此控制整个实例需要 server.admin，不能以某个 Scope 的权限代替。实例控制响应不含业务 Config、目标输入或 Source 数据。 +| 能力与业务实例发现 | scope.read，并标明是否有 scope.contribute | +| write、新建或重复 collect | scope.contribute 与 scope.read，因应答可包含数据或 Source 引用 | +| 查询 collect 结果、读取 Source | scope.read | +| 目标 control 视图、目标启停 | scope.contribute，仅返回控制字段 | +| 加载 write；实例控制视图、加载、配置、卸载或启停 | server.admin；实际进程操作由部署工具执行 | +| 授予 Scope 角色 | 沿用现有 Access 授权规则 | -Server 将已授权操作与请求中的 Scope、target\_id 或 instance\_id 一并传给管理服务。管理服务必须核对实际资源归属，防止用有权限的 Scope 指向另一 Scope 的目标；这个检查不增加新的授权模型。实例接入与管理通道仍需认证，Runtime 只能同步自身目标和报告自身状态，不能冒充 Server 下发管理操作。 +Server 转交已授权操作及可信身份；插件管理服务核对资源实际 Scope、实例可服务范围和运行条件，避免以有权限的 Scope 操作其他 Scope 的目标。插件实例 Runtime 认证后仅同步和报告自身实例，不能冒充 Server 管理请求。插件处理外部系统权限，identify_target 不承担授权。 -修改或撤销用户权限只操作 PC Server 的 Access 体系。write 等待期间和最终返回前仍检查当前读取权限；权限已撤销就拒绝返回结果，但不声称已接受的写入未发生。collect 的持续执行使用实例自身的 PC 执行身份，创建者权限变化不自动停用既有目标；停止目标使用正常控制操作，实例提交权限则由 Server 每次检查。 - -collect 遇到外部系统拒绝访问时，由插件决定等待、重试或结束 apply。write 的输入校验、resolve、project 或保存失败由 Server 经原调用返回错误。 +同 Scope 的贡献者均可控制目标，不限创建者；跨 Scope 的实例控制须 server.admin。collect 持续执行使用实例 PC 身份，每次提交由 Server 检查；创建者权限变化不自动停止目标。write 使用当前宿主身份，返回结果前检查当前读取权限；拒绝返回不撤销已经保存的数据。控制响应不含业务配置、目标输入或 Source 数据。 ## 凭据与初始化授权 -AuthenticationProvider 是 Server 验证凭据并映射 Principal 的组件。本机部署流程可用部署控制的 Provider 创建或复用执行身份；连接外部管理的 Server 时，沿用它的 Provider，必要时由管理员预置身份，不覆盖认证配置或承诺自动签发凭据。 - -| 凭据 | 用途 | 保存位置 | -| --- | --- | --- | -| 部署凭据 | 加载 write 实现、注册 Definition、创建 Scope、授予角色与配置实例 | 部署工具的秘密存储，不下发 Runtime | -| 宿主凭据 | 以宿主用户或服务身份调用 Server | 宿主安全配置 | -| 管理通道凭据 | 管理服务向 Server 认证并建立专用通道 | 管理服务 | -| PC 执行凭据 | 每实例独立 Principal，向获授权的 Scope 提交 Source、处理 checkpoint | 执行机器，不保留 server.admin | -| 实例连接凭据 | Runtime 向管理服务证明实例身份，同步本实例工作与报告 | 执行机器 | -| 外部系统凭据 | 插件访问语雀、工单等业务系统 | 插件自行保存和解释，可在 Config 中引用文件、环境或秘密服务 | - -write 的部署加载需要管理员权限，日常保存使用当前宿主身份，不另外创建 PC 执行身份或发放执行凭据。collect 部署使用独立 Bearer 凭据，经 Provider 映射到明确身份。启动 collect 业务前，部署流程完成： - -1. 验证 Server、部署标识、部署权限和目标 Scope；实际确认宿主及执行凭据对应的 Principal，不以文件存在代替验证。 -2. 使用部署凭据注册 SourceDefinition，并通过现有 Access 接口创建或复用执行身份的 Scope Binding；日常执行无需注册 Definition 或取得 server.admin。 -3. 配置管理通道身份，登记实例执行位置与可服务 Scope；管理操作授权复用 Server Access，Runtime 只取得自身所需凭据或引用。 -4. 用执行凭据检查实际身份和 Scope 权限，通过后才允许处理该 Scope。没有 Scope 授权的实例可以登记，但不能受理该 Scope 的采集。宿主 MCP 的连接与授权沿用宿主集成流程。 - -上述步骤使用现有的 `POST /v1/source-definitions/register`、`POST /v1/access/bindings/create`、`GET /v1/access/me` 和 `POST /v1/access/check`，不另建用户注册或 OAuth 系统。 - -本机自动部署生成内部连接配置，不跳过授权。部署凭据不下发 collect Runtime；Server 只允许管理员加载受信任的 write 代码，日常 write 不借用部署权限保存数据；凭据值不写入命令参数、普通日志或能力目录。轮换沿用 Provider 与秘密引用，失效身份不能继续使用；跨机通信使用 TLS 或等效的已认证安全入口。 +AuthenticationProvider 将凭据映射为 Principal。本机部署可由部署控制的 Provider 准备身份；外部 Server 沿用其 Provider，必要时由管理员预置，不覆盖认证配置或承诺自动签发。 -# 附录 A：使用场景——启动与动态加入资料 - -以下先按部署位置展示 collect 的用户操作与组件交互，随后补充 write 的加载与调用。本机 collect 部署时，Server 启动扫描和 `source load` 会准备共享管理服务及 Runtime，无须再手动执行两个 `install`；跨机器部署时，部署者准备完整部署配置，启动共享管理服务，完成登记和授权后再启动远端 Runtime。本文中的服务安装命令使用 Linux `systemd --user`；PC Server 须已配置认证并启用 PC Access。 - -本附录的命令和配置为**使用示意**，说明部署位置、操作顺序和组件交互；最终命令与参数以随实现发布的用户指南为准。Shell 命令在标明的机器执行；JSON 为第 3 节**拟议接口**的 MCP `tools/call` 参数，由宿主发送到 Server。示例 Scope、路径和文档 URL 必须替换为实际值。 - -## 场景一：在同一台机器启动 Codex、Server 和插件 - -**1\. 在 Server 所在机器准备实例目录。** 假设代码随包提供，目录为 `/srv/powercontext/source-plugins/yuque-personal`，外部凭据单独保存在受保护文件中。`powercontext.yml` 如下： - -```yaml -id: yuque-personal -plugin: yuque -config: - token_file: /srv/powercontext/secrets/yuque.token - base_url: https://www.yuque.com - poll_interval: 60 -# 可选：首次初始化的采集目标。 -targets: - - scope_id: scp_demo - capability: yuque.document - capability_version: "1" - input: - url: https://www.yuque.com/team/book/doc-a -``` - -`config` 由插件定义，包含外部凭据引用等业务参数；targets 中的 Scope 必须存在，且初始化调用方及实例执行身份具备相应权限。初始目标按正常添加规则写入管理服务，初始化后以管理服务为准；后续改文件不会覆盖或补充目标，空目标列表或连接失败也不会触发重新导入。尚未准备好 Scope 时可省略 targets；之后先完成下文的 Scope 授权与实例范围扩展，再通过公共 MCP 添加。 - -**2\. 配置插件目录并启动 Server 后台服务。** 复用已有 `pc.env` 中的数据库、认证与部署配置，增加目录配置： - -```bash -POWERCONTEXT_SERVER_SOURCE_PLUGINS_DIR=/srv/powercontext/source-plugins -``` - -在 PowerContext 仓库根目录执行： - -```bash -uv run powercontext service install --env-file ./pc.env -``` - -`service install` 安装并启动 Server 后台服务后返回终端；Server 启动时执行以下本机部署流程，无须另行执行 `manager install` 或 `runtime install`： - -| 顺序 | 组件与交互 | +| 凭据 | 用途与归属 | | --- | --- | -| 1 | Server 部署流程扫描可信目录，读取实例配置，准备本机服务配置与内部身份引用 | -| 2 | 启动或复用独立管理服务；管理服务主动连接 Server，建立认证后的管理通道 | -| 3 | 部署流程校验插件声明，使用部署权限注册 Definition、登记实例，并确认执行身份的 Scope 权限与实例可服务范围 | -| 4 | 启动 Runtime，连接管理服务和 Server；Runtime 创建 Context、执行 apply，能力与实例状态同步到 Server | -| 5 | 若实例尚未初始化目标，部署流程在实例就绪后按正常添加规则导入文件中的 targets；插件从 ctx.targets.list() 取得目标，自行决定何时采集 | - -部署权限通过 Server 部署配置准备，不交给日常 Runtime。已有实例重启不会重新导入 targets；运行中添加、停止或恢复的目标状态由管理服务保留。没有 collect 实例时，内置与自定义 write 能力无需启动管理服务。 - -**3\. 安装宿主集成并连接同一个 Server。** Codex 已安装，但 PowerContext 集成尚未安装时，在仓库根目录执行： - -```bash -uv run powercontext setup codex --source . --server-url http://127.0.0.1:8000 -POWERCONTEXT_CODEX_SCOPE_ID='scp_demo' codex -``` - -按现有宿主认证流程准备宿主凭据；第二条命令适用于从当前 Shell 启动的新 Codex 会话。桌面宿主应使用自己的集成配置并重新建立会话，不会继承另一终端的环境变量。通过 `resolve_scope_binding` 核对会话 Scope，并为下面的 Source MCP 调用显式传入同一 Scope。 - -宿主只访问 Server `/mcp`，不直接连接管理服务或 Runtime。插件后台进程不会随 Codex 关闭而停止；新增 Source 插件不要求重新安装宿主集成。 - -## 场景二：新建文档，交给已运行实例采集 - -假设 `yuque-personal` 已经采集文档 A，需要加入文档 B。 - -**1\. Codex 使用已授权的语雀工具创建 B，取得真实 URL。** 例如语雀 CLI： - -```bash -yuque create doc --namespace team/book --title '文档 B' --body-file /tmp/doc-b.md --yes --json -``` - -公共 Source MCP 不创建语雀文档，也不监听所有外部工具操作。 - -**2\. Codex 发现能力和可用实例。** - -```json -{ - "name": "pc_source_capabilities", - "arguments": {"scope_id": "scp_demo", "operation": "add"} -} -``` - -Server 根据当前 PC 权限和已同步目录返回能力、版本、输入 schema 与实例状态。选择发现返回的 `yuque.document` 版本和 `yuque-personal` 实例。 - -**3\. Codex 添加 B。** - -```json -{ - "name": "pc_source_add", - "arguments": { - "scope_id": "scp_demo", - "capability": "yuque.document", - "capability_version": "1", - "instance_id": "yuque-personal", - "input": {"url": "https://www.yuque.com/team/book/doc-b"} - } -} -``` - -Server 检查当前 Scope 权限，再经管理通道转交管理服务；Runtime 调用插件的 identify\_target 确认资料身份，管理服务创建或复用目标并返回 target\_id。响应表示已建立持续采集关系，不表示 B 已保存；相同 Scope、实例、能力版本及资料身份的重复添加复用目标，不恢复已停止目标。 - -**4\. 已运行的插件取得并处理 B。** Runtime 同步 targets，apply 从 `ctx.targets.list()` 取得 B，按自身节奏调用 `run_batch()`；Worker 执行 Connector，插件读取语雀并提交，Source 直接保存到 Server。SDK 自动关联真实提交回执，由 Runtime 向管理服务报告 Source 引用和提交时间。 - -**5\. 用户需要查看结果或停止跟踪时，再调用相应工具。** 查询不是维持采集的必要动作： - -```json -{ - "name": "pc_source_status", - "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b"} -} -``` - -返回的 source\_refs 与 last\_submitted\_at 只表示已确认并报告的提交，不表示 B 已全部采完。使用实际返回的 target\_id 停止 B： - -```json -{ - "name": "pc_source_target_set_enabled", - "arguments": {"scope_id": "scp_demo", "target_id": "tgt_b", "enabled": false} -} -``` - -有 scope.contribute 的用户可以停止或恢复该 Scope 的目标，不限创建者；读取提交结果需要 scope.read。返回表示启停意图已保存；断线中的 Runtime 可能仍按最后同步状态采集。恢复使用相同 target\_id 和 enabled=true，重复 add 不代替恢复。 - -## 场景三：新增插件、Scope 与异机运行 - -**新增本地自定义插件实例。** 部署者准备代码、依赖和一份独立的实例配置，示例目录如下： - -```text -/srv/powercontext/source-plugins/tickets-team/ - powercontext.yml - plugin.py -``` - -```yaml -id: tickets-team -plugin: ./plugin.py -scope_ids: [scp_demo] -config: - api_base: https://tickets.example.org - token_file: /srv/powercontext/secrets/tickets.token -``` - -其中业务参数须由这个插件的 Config 声明。实例 ID 在部署内唯一，入口使用本地路径，无需发布 wheel。Server 已配置上述插件目录并启动后，在同一台机器使用其 `pc.env` 执行： - -```bash -uv run powercontext source load --config /srv/powercontext/source-plugins/tickets-team/powercontext.yml --env-file ./pc.env -``` - -`load` 使用本机受保护的部署凭据，准备或复用共享管理服务，注册 Definition、登记实例、配置执行身份的 Scope 权限，生成 RuntimeConfig 并安装启动 Runtime；首次加载时还会导入可选的初始 targets。它内部调用两个 `install` 所用的服务安装逻辑，因此本场景只需执行 `load`。命令返回后后台进程继续运行，不重启 Server 或已有实例；插件依赖须事先准备。 - -运行中只放入目录不会自动启动，重复加载不覆盖已初始化 targets，也不会重启配置相同的实例。加载后宿主重新调用 `pc_source_capabilities`，按新能力的 `input_schema` 调用 `pc_source_add`，无需修改 MCP 工具清单。 - -三个插件配置文件的用途不同： - -- `powercontext.yml`：用户编写的实例配置，声明实例 ID、插件入口、业务配置、可服务 Scope 和可选的初始 targets，供 `source load` 或 Server 启动扫描读取。 -- `source-manager.yml`：管理服务的部署配置（ManagerConfig），包含 Server 地址及凭据文件、管理服务 ID、监听地址和持久数据目录，供 `manager install` 读取。 -- `plugin-runtime.yml`：单个插件进程的完整部署配置（RuntimeConfig），包含插件目录、业务配置、Server 与管理服务地址、两套连接凭据文件及配置摘要，供 `runtime install` 读取，不能用实例 `powercontext.yml` 替代。 - -`pc.env` 是 Server 的环境配置，供 `load` 确定连接地址与本地部署目录等信息。 - -**给同一实例增加 Scope。** 管理员通过已有 Access 接口授予执行身份所需权限，并经 `POST /v1/source-management/configure` 扩展实例可服务范围；宿主也需要相应 PC 权限。随后添加携带新 scope\_id 的目标即可，不重建 Context，不修改已初始化文件中的 targets。 - -**跨机器手动部署。** 例如机器 A 运行 Server 和共享管理服务，机器 B 能访问工单系统并运行 `tickets-team`；Codex 只连接机器 A 的公共 MCP。机器 A 上的 `source load` 不会到机器 B 安装代码或启动进程。 - -部署者先准备机器间的连接凭据和可访问地址；使用部署权限向 Server 注册 Definition，向管理服务登记实例的 ID、执行位置、配置摘要及可服务 Scope，并通过 PC Access 授予执行身份相应 Scope 权限。机器 B 的插件代码、依赖和 RuntimeConfig 必须与登记信息一致。管理服务尚未启动时，先完成下面的管理服务安装，再登记实例，最后安装 Runtime。 - -```bash -# 机器 A：手动部署共享管理服务；已有管理服务时跳过。 -uv run powercontext source manager install --config ./source-manager.yml - -# 机器 B：登记和授权完成后，启动已准备好的插件 Runtime。 -uv run powercontext source runtime install --config ./plugin-runtime.yml -``` - -这两条命令只安装并启动对应的后台服务，不安装插件依赖，也不自动完成 Definition 注册、实例登记或授权。RuntimeConfig 由部署阶段根据上述信息生成或准备；当前 `source load` 只编排本机部署，不提供远端配置分发或 SSH 执行。 - -管理服务主动访问 Server；Runtime 主动访问管理服务、Server 和外部数据源。跨机器连接需要可访问的安全入口，不能直接使用只监听本机的默认地址。管理服务也可部署在第三台机器，职责不变；每个 Server 只对接一个共享管理服务。 - -**自定义 write：加载实现后直接保存内容。** 例如宿主已经获得工单完整内容，只需保存为 `example-ticket` Source： - -1. 部署者在 Server 机器准备第 4 节的 `tickets` 本地包及依赖，确保 Server 的 Python 环境能够导入它。 -2. 部署者提供入口配置并以管理员权限触发 Server 加载。配置契约示意为 `entry: tickets`；加载流程校验并注册 Definition，保存入口引用，公开 `ticket.write / 1`。 -3. 宿主连接同一个 `/mcp`，执行下面的发现和提交，不创建实例或 target，不准备 manager / runtime 配置： - -```json -{ - "name": "pc_source_capabilities", - "arguments": {"scope_id": "scp_demo", "operation": "add"} -} -``` - -确认发现返回 `ticket.write / 1` 及对应输入模型后： - -```json -{ - "name": "pc_source_add", - "arguments": { - "scope_id": "scp_demo", - "capability": "ticket.write", - "capability_version": "1", - "input": { - "namespace": "team", - "project": "support", - "ticket_id": "T-100", - "title": "登录失败", - "body": "用户反馈无法登录。" - } - } -} -``` - -Server 校验 TicketCapture，调用 TicketDefinition.resolve 和 TicketProjection.project，保存后沿本次调用返回真实 SourceRef。宿主使用该引用读取 Source 或 Projection；无须调用状态查询。此示例的 Definition 根据完整工单快照生成稳定身份，相同内容重试复用已有 Source。若需要持续获取工单变化，则使用前述 collect 场景；两者可以复用同一个 Definition。 - -# 附录 B：插件采集链路参考 - -下面是 collect 的**接口示意**，展示资料身份确认、插件自循环、提交与 checkpoint 的关系；具体类名、模块路径和辅助函数可随实现调整。`BusinessClient`、`DocumentDefinition`、`TemporaryReadError` 和 Config 的外部凭据字段由插件实现；它不是可直接运行的完整客户端，也不能替代当前代码尚未完成的 SDK 调整。 - -插件声明 `plugin_api_version = "2"`、Config、capabilities、definitions 和稳定 Connector 名称与版本。capabilities 描述 collect 的输入与业务契约，并提供 identify\_target 确认资料身份。 - -例如一种 collect 输入为 `{"document_id":"A"}`，其身份入口可写为以下返回值示意： - -```python -async def identify_target(request, config): - # document_id 为 input_schema 要求的非空业务 ID。 - document_id = request.input["document_id"].strip() - if not document_id: - raise ValueError("document_id must not be blank") - return { - "target_key": document_id, - "normalized_input": {"document_id": document_id}, - } -``` - -该入口用于确认资料身份。实际能力有其他采集选项时，normalized\_input 须保留全部影响采集的参数；不能通过省略参数把不同要求当成同一配置。 - -```python -import asyncio - -from powercontext.source_management import ManagementError -from powercontext.sources import ( - ConnectorRunCompletion, - ConnectorRunStatus, - ConnectorSubmissionStatus, -) - -plugin_api_version = "2" -connector_name = "document-reader" -connector_version = "1" -document_definition = DocumentDefinition() -definitions = (document_definition,) - - -class BusinessConnector: - name = connector_name - version = connector_version - source_definitions = frozenset({document_definition.name}) - - def __init__(self, ctx, client): - self.ctx = ctx - self.client = client - self.targets = () - - async def run(self, session): - batch_complete = True - for target in self.targets: - try: - document = await self.client.read(target.input) - if document is None: - # 已确认没有资料；网络或权限错误不能伪装成 None。 - continue - receipt = await self.ctx.targets.submit( - target, - session=session, - item_id=target.target_id, # 本例每个目标提交一项，批内唯一。 - definition_name=document_definition.name, - value=document, - ) - if receipt.status != ConnectorSubmissionStatus.ACCEPTED: - batch_complete = False - except TemporaryReadError: - # 此插件选择后续批次重试，不吞掉取消或其他未处理异常。 - batch_complete = False - return ConnectorRunCompletion( - status=ConnectorRunStatus.COMPLETE if batch_complete else ConnectorRunStatus.INCOMPLETE, - checkpoint=session.checkpoint, # 本例全量读取，不推进增量游标。 - ) - - -async def apply(ctx, config): - client = await ctx.enter_async_context(BusinessClient(config)) - connector = BusinessConnector(ctx, client) - ctx.connect(connector) - while True: - try: - targets = await ctx.targets.list() - for scope_id in sorted({target.scope_id for target in targets}): - selected = tuple(target for target in targets if target.scope_id == scope_id) - connector.targets = selected - await ctx.run_batch(scope_id=scope_id, targets=selected) - except ManagementError as error: - # 尚无初始目标快照时等待；其他配置、权限等错误继续抛出。 - if error.code not in {"targets-unavailable", "management-unavailable"}: - raise - await asyncio.sleep(config.poll_interval) -``` - -SDK 自动记录真实提交关联，Connector 返回的批次结果用于 Worker 与 checkpoint 管线；断线和重启行为见第 5 节。 - -增量或分页插件自行计算 checkpoint，可在同一实例、同一 Scope 的 checkpoint 中维护各目标进度；新增目标必须补采，不能因旧目标的游标较新而跳过。插件提交同一数据须生成稳定的 Source 身份，异常重试沿用已有 Source 保存规则。 - -完整 collect 资源清理及停止规则见第 5 节。write 的实现加载与调用见第 4 节和附录 A。 - -# 附录 C：管理通信与实例配置参考 - -本附录供部署者与框架实现者查阅；宿主使用公共 MCP，插件通过 SDK 获取目标和提交。下列路径与配置字段属于**接口示意**，调用方向、权限和状态语义仍遵循正文约定。 - -## 通道与身份 - -连接方向与请求交接见第 2、3 节，身份认证和权限职责见第 5 节。管理服务和 Runtime 分别使用自己的凭据连接上游。 - -| 接口 | 用途 | -| --- | --- | -| Server `/v1/source-management/channel/open`、`catalog`、`poll`、`reply`、`close` | 管理服务认证建连、同步能力目录、领取消息并应答；目录就绪后，发现由 Server 直接回答 | -| Server `POST /v1/source-management/configure` | Server 检查 server.admin 后登记或明确更新实例；经管理通道执行，不上传任意 Python 代码 | -| 管理服务 `POST /v1/source-management/instances/{instance_id}/sync` | Runtime 使用实例凭据同步自身 collect 目标和控制状态；分页由 SDK 处理 | -| 管理服务 `POST /v1/source-management/instances/{instance_id}/results` | Runtime 报告本实例实际提交的 Source 引用与时间，重复报告复用相同关联 | - -重连后，管理服务重新认证并同步完整能力目录；Server 校验能力名称、版本、kind 和输入契约与已有能力的一致性。管理通道的消息大小、在途数量与等待时间有界。 - -## 实例配置与更新 - -`POST /v1/source-management/configure` 的 registration 表示完整实例登记，`update=true` 表示显式更新。初始 targets 的导入规则见第 2 节；配置更新和升级流程见第 5 节。 - -仅扩展已获授权的 Scope 范围可以保留原 Context；缩减范围或改变执行配置按停用更新流程处理。 +| 部署凭据 | 部署工具加载代码、注册 Definition、授权与登记，不下发插件实例 Runtime | +| 宿主凭据 | 保存在宿主安全配置，用于调用 Server | +| 管理通道凭据 | 插件管理服务向 Server 认证 | +| PC 执行凭据 | 每实例独立身份，提交 Source 和处理 checkpoint，不持有 server.admin | +| 实例连接凭据 | 插件实例 Runtime 向插件管理服务证明自身身份 | +| 外部系统凭据 | 插件保存和解释，可通过 Config 引用 | + +部署启动业务前确认实际身份、完成 Definition 注册和 Scope 授权、登记实例范围及连接身份，再用执行凭据检查权限。复用已有身份与 Access 接口；无 Scope 授权可登记实例，但不能受理该 Scope 工作。write 不另发执行凭据，日常保存不借用部署权限。凭据不进入命令参数、能力目录或普通日志；跨机使用 TLS 或等效安全入口。 From ec6799317a8be42e4ab238fb0e1b0e8686287f69 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=94=98=E9=9C=96?= Date: Thu, 15 Oct 2026 14:26:50 +0800 Subject: [PATCH 3/5] docs(rfc): assign source plugin RFC number 2012 --- ...000-source-plugin-system.md => 2012-source-plugin-system.md} | 2 +- ...000-source-plugin-system.md => 2012-source-plugin-system.md} | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) rename docs/en/rfcs/{0000-source-plugin-system.md => 2012-source-plugin-system.md} (99%) rename docs/zh/rfcs/{0000-source-plugin-system.md => 2012-source-plugin-system.md} (99%) diff --git a/docs/en/rfcs/0000-source-plugin-system.md b/docs/en/rfcs/2012-source-plugin-system.md similarity index 99% rename from docs/en/rfcs/0000-source-plugin-system.md rename to docs/en/rfcs/2012-source-plugin-system.md index 58ee9b7c2..86f99d770 100644 --- a/docs/en/rfcs/0000-source-plugin-system.md +++ b/docs/en/rfcs/2012-source-plugin-system.md @@ -5,7 +5,7 @@ description: Develop and use custom Source plugins to extend data types, Project - Proposal Name: `source_plugin_framework` - Start Date: 2026-10-14 -- RFC PR: Not assigned +- RFC PR: [oceanbase/powercontext#2012](https://github.com/oceanbase/powercontext/pull/2012) # Summary diff --git a/docs/zh/rfcs/0000-source-plugin-system.md b/docs/zh/rfcs/2012-source-plugin-system.md similarity index 99% rename from docs/zh/rfcs/0000-source-plugin-system.md rename to docs/zh/rfcs/2012-source-plugin-system.md index 459531b4f..2dccc3442 100644 --- a/docs/zh/rfcs/0000-source-plugin-system.md +++ b/docs/zh/rfcs/2012-source-plugin-system.md @@ -5,7 +5,7 @@ description: 支持用户开发和使用自定义 Source 插件，扩展数据 - Proposal Name: `source_plugin_framework` - Start Date: 2026-10-14 -- RFC PR: 尚未分配 +- RFC PR: [oceanbase/powercontext#2012](https://github.com/oceanbase/powercontext/pull/2012) # 概要 From 1c7ed6fa1347e1fdad1f2c50f407a78da4aaa58e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=94=98=E9=9C=96?= Date: Thu, 15 Oct 2026 16:37:49 +0800 Subject: [PATCH 4/5] docs(rfc): clarify source plugin deployment and simplify discovery --- docs/en/rfcs/2012-source-plugin-system.md | 36 +++++++++++++---------- docs/zh/rfcs/2012-source-plugin-system.md | 36 +++++++++++++---------- 2 files changed, 40 insertions(+), 32 deletions(-) diff --git a/docs/en/rfcs/2012-source-plugin-system.md b/docs/en/rfcs/2012-source-plugin-system.md index 86f99d770..88b02235f 100644 --- a/docs/en/rfcs/2012-source-plugin-system.md +++ b/docs/en/rfcs/2012-source-plugin-system.md @@ -81,7 +81,7 @@ Arrows show business calls and data flow. write implementations run in Server; c | RemoteConnectorWorker | Existing batch pipeline; creates sessions, calls Connector.run, and processes receipts and checkpoint CAS | | apply / Connector | Plugin business code; apply decides timing, while Connector reads, transforms, and submits batch data | -write alone requires only Server. N plugin instances add one plugin management service and N plugin instance Runtime processes, hosted by the system service manager. +write alone requires only Server. N plugin instances add one plugin management service and N plugin instance Runtime processes. Plugin instance Runtimes run as ordinary background processes by default; users explicitly opt into system service management. ## Startup loading and adding instances at runtime @@ -89,11 +89,13 @@ write alone requires only Server. N plugin instances add one plugin management s **collect:** Server scans the plugin directory specified through an environment variable once at startup. Each `powercontext.yml` declares an instance `id`, code entry, business configuration, credential references, and serviceable Scopes. id is instance_id, unique within a PC deployment and stable across restarts. Several instances can use the same local code without publishing a wheel; placing configuration in the trusted directory authorizes loading. -Deployment prepares or reuses the plugin management service, validates the plugin, registers Definitions and the instance, grants permissions, and starts an independent plugin instance Runtime. Adding an instance while running requires explicit loading, without continuous directory scanning or restarting other instances. Repeated loading of the same ID and matching configuration reuses registration and processes without resetting desired state. Conflicting configuration is rejected; updates follow section 5. +Users run `source load` on the plugin execution machine to load instance configuration and connect to a local or remote Server and plugin management service. The command validates the plugin, uses deployment credentials to register Definitions and the instance and grant required Scope permissions, prepares Runtime credentials and startup configuration, and starts an independent Runtime on that machine. Existing permissions are reused; identities for an external Server are provisioned through its authentication mechanism. Users do not run a separate registration script or manually submit credential digests. Local automatic deployment can prepare management as needed; cross-machine deployment connects to services that have already been prepared. -Configuration may contain initial `targets`, each specifying Scope, capability, and business input. First target initialization imports them using normal add rules and records initialization. Management is authoritative thereafter; the file no longer replaces or adds targets. Empty targets or connection failure do not trigger reimport, and changes to initial targets do not constitute instance configuration conflicts. +Server startup scanning reuses the same loading flow. Adding an instance while running explicitly invokes `source load`, without continuous directory scanning or restarting other instances. Repeated loading of the same ID and matching configuration reuses registration and processes without resetting desired state. Conflicting configuration is rejected; updates follow section 5. -No plugin instances means management need not start. Failure to prepare one instance does not block built-ins. Removing a directory, stopping Server, or disabling all instances does not implicitly uninstall independent services or delete data. Remote machines deploy their own code and plugin instance Runtimes; local loading does not distribute code or invoke remote shells. +Configuration may contain initial `targets`, each specifying Scope, capability, and business input. First target initialization imports them using normal add rules and records initialization. Management is authoritative thereafter; the file no longer replaces or adds targets. Empty targets or connection failure do not trigger reimport. + +No plugin instances means management need not start. Failure to prepare one instance does not block built-ins. Removing a directory, stopping Server, or disabling all instances does not implicitly uninstall independent services or delete data. After preparing code and dependencies on a remote execution machine, users run `source load` there. The command starts Runtime only on the machine where it runs; it does not distribute code or invoke remote shells. ## Component communication and connection directions @@ -111,10 +113,12 @@ On connection or reconnection, management proactively synchronizes the complete ## Persistent data and storage ownership +The framework's new independent persistence management is confined to the plugin management service. PC Server reuses existing data storage, and plugin instance Runtime retains only startup files; neither adds a separate plugin business-state store. write entries and required configuration references are retained as deployment configuration. + | Owner | Authoritative data and recovery basis | | --- | --- | | Plugin management service | Its own SQLite stores instances, capabilities, collect targets, desired state, and received submission records, plus last reported execution state and confirmation time | -| PC Server | Existing storage holds Sources, Projections, checkpoints, and user authorization; additionally stores write entries and configuration references, while code and dependencies remain on its filesystem | +| PC Server | Existing storage holds Sources, Projections, checkpoints, and user authorization; write entries and configuration references belong to deployment configuration, while code and dependencies remain on its filesystem | | Plugin instance Runtime | Persists only code, startup configuration, and credential references; target snapshots, batches, and undelivered reports remain in memory and are resynchronized after restart | Management's `records` stores deployment bindings, instances, capabilities, targets, and submission records (Source references and times). `follow_identity` maps stable collect resource identities to target_id for repeat-add reuse. write uses existing Source storage rules without a separate acceptance ledger. @@ -141,7 +145,7 @@ The following describes proposed methods, essential parameters, call flows, and ## pc_source_capabilities: discover capabilities and input requirements -**Input:** scope_id and operation (`add` for business discovery), optionally filtered by capability or instance_id. +**Input:** scope_id, optionally filtered by capability or instance_id. **Flow:** Host calls → Server checks Scope read permission and whether writing is permitted → filters local capabilities and the synchronized collect catalog by visibility and serviceable Scope → directly returns names, versions, kind, input_schema, permissions, instances, and availability. @@ -194,7 +198,7 @@ Repeated add does not reset progress, trigger extra collection, or resume a stop **Input:** scope_id and target_id; the default view reads collection status, while `view="control"` reads control fields. -**Flow:** Server checks the view's permissions → management verifies the target's actual Scope → reads persistent target and received submission records → reply → Server returns to the host. Queries do not call identify_target, wait for a new batch, or require online plugin instance Runtime. +**Flow:** Server checks the view's permissions → management verifies the target's actual Scope → reads persistent target and received submission records → reply → Server returns to the host. | View | Return | | --- | --- | @@ -215,10 +219,10 @@ The response confirms saved intent rather than completed execution. Target ident | View and input | Call-to-return flow | | --- | --- | -| Business: scope_id, operation=add | Server checks Scope permissions and directly returns visible instances, capabilities, writability, and state using capabilities catalog rules | +| Business: scope_id | Server checks Scope permissions and directly returns visible instances, capabilities, writability, and state using capabilities catalog rules | | Control: instance_id, view=control | Server checks server.admin, queries persistent instance control state and latest execution reports through management, then returns instance_id, enabled, actual, availability, and update time | -Control view requires no Scope and returns no Config, target input, or Sources. Persistent control state remains readable when only plugin instance Runtime is offline, with offline or unknown availability; unreachable management makes control queries unavailable. This method does not load code, access external accounts, or start apply. write has no instances. +Control view requires no Scope and returns no Config, target input, or Sources. Persistent control state remains readable when only plugin instance Runtime is offline, with offline or unknown availability; unreachable management makes control queries unavailable. ## pc_source_instance_set_enabled: enable or disable instance business execution @@ -240,7 +244,7 @@ Plugins can load directly from local directories. write code and dependencies re **Shared data contracts:** SourceDefinition declares input and Source models, name, and version. `resolve(value)` produces a stable Source identity; `project(source)` implementations in `projections` produce Projections. Plugins export implementation objects in `definitions` and declare capability name, version, description, kind, and input_schema in `capabilities`. Capabilities describe callable behavior; Definitions describe output data, without requiring a one-to-one type mapping. -**write loading:** Each write capability associates a Definition name and version, with input_schema matching its input model. Plugin modules export actual Definition and Projection objects. Server's generic loader imports and registers them through a trusted entry, so Server needs no type-specific imports. Adding a module requires code, dependencies, and entry configuration; uploading definition structure through registration alone cannot supply executable functions. Import does not start collection tasks. +**write loading:** Each write capability associates a Definition name and version, with input_schema matching its input model. Plugin modules export actual Definition and Projection objects. Server's generic loader imports and registers them through a trusted entry. Adding a module requires code, dependencies, and entry configuration; uploading definition structure through registration alone cannot supply executable functions. **collect runtime contracts:** @@ -351,7 +355,7 @@ flowchart TD D -->|Cleanup failed| U[Report error, no second Context] ``` -After apply returns or fails, Context is cleaned up but plugin instance Runtime stays online. New targets, reconnection, or repeated enablement do not rerun apply. Plugins needing continuous target handling maintain their own loop without blocking management operations while waiting. Internal run identifiers distinguish reports for the current connection and run; hosts do not provide them. +After apply returns or fails, Context is cleaned up but plugin instance Runtime stays online. New targets, reconnection, or repeated enablement do not rerun apply. Plugins needing continuous target handling maintain their own loop without blocking management operations while waiting. Rerunning within a process requires disablement, confirmed stopped state, and enablement. Process restart instead synchronizes and decides whether to create a Context from enabled. The direct execution entry exits when apply ends and cannot share instance identity with background mode. After a system service exits or is uninstalled, deployment tooling on that machine must restart it. @@ -372,7 +376,7 @@ Disablement does not undo saved Sources or affect Server write capabilities. Dis | Management disconnected from Server | Host collect management is unavailable; existing collection can continue, while built-ins and loaded write remain independent | | Server unavailable | Submissions fail or wait for recovery; saved Sources remain, and write calls are not automatically recovered or replayed | -If exit of the previous process with the same identity cannot be established, another plugin instance Runtime must not start. Missing heartbeats do not authorize cross-machine takeover. The framework does not guarantee exactly-once apply or external effects. +Each plugin instance runs only one Runtime. Stop the old Runtime before redeploying. Recovery follows actual persisted data: a saved Source without an advanced checkpoint may be recollected using stable identity; a lost checkpoint-update response requires rereading it. Lost submission reports do not roll back Sources or checkpoints and are not guaranteed to be backfilled in management records. @@ -382,7 +386,7 @@ Recovery follows actual persisted data: a saved Source without an advanced check - **Expand Scopes:** Grant execution identity the relevant Scope Binding (Server role assignment) and update serviceable scope. After synchronization, targets can use the new Scope with the same Context. Passing scope_id alone grants no permission. - **Reduce scope, change execution configuration, or upgrade collect:** Disable and confirm stopped, exit old plugin instance Runtime, validate and update code, Config, and registration, then explicitly enable after the new plugin instance Runtime validates. Credential changes use configuration updates, with business meaning interpreted by the plugin. -Upgrades preserve Sources, targets, checkpoints, and received submission records. Validate compatibility of capability input and target identity, Definitions, Connector, and checkpoint formats. New code must handle retained targets, including disabled ones, without resetting progress to bypass checks. Uncertain compatibility keeps the instance disabled for explicit deployment handling, without automatic migration. New instances do not inherit old targets automatically. +Upgrades preserve Sources, targets, checkpoints, and received submission records. Validate compatibility of capability input and target identity, Definitions, Connector, and checkpoint formats. New code must handle retained targets, including disabled ones. Uncertain compatibility keeps the instance disabled for explicit deployment handling, without automatic migration. New instances do not inherit old targets automatically. write updates first resolve in-flight calls, then restart and load while preserving capability and Definition contracts and existing Source identity. Ordinary repeated loading does not replace configuration updates. @@ -401,7 +405,7 @@ User authorization reuses Server Access. A Principal is the user or service iden Server forwards authorized operations and trusted identity. Management verifies actual resource Scope, instance serviceable scope, and execution conditions to prevent addressing another Scope's target through an authorized Scope. Authenticated plugin instance Runtime synchronizes and reports only its instance and cannot impersonate Server management requests. Plugins handle external authorization; identify_target does not authorize. -Any contributor in the Scope can control targets, regardless of creator; instance-wide controls require server.admin. collect uses the instance PC identity for continuous execution, checked on each submission. Creator permission changes do not automatically stop targets. write uses the current host identity, and returning data requires current read permission; denial does not undo already saved data. Control responses exclude business configuration, target input, and Source data. +Any contributor in the Scope can control targets, regardless of creator; instance-wide controls require server.admin. collect uses the instance PC identity for continuous execution, checked on each submission. write uses the current host identity; denial does not undo already saved data. Control responses exclude business configuration, target input, and Source data. ## Credentials and initial authorization @@ -409,11 +413,11 @@ AuthenticationProvider maps credentials to Principals. Local deployment can prep | Credential | Use and ownership | | --- | --- | -| Deployment | Deployment tooling loads code, registers Definitions, grants permissions, and configures instances; not passed to plugin instance Runtime | +| Deployment | The startup loading flow uses these credentials to load code, register Definitions, grant permissions, and configure instances; not passed to plugin instance Runtime | | Host | Stored in host security configuration to call Server | | Management channel | Management authenticates to Server | | PC execution | Per-instance identity for Source submission and checkpoints, without server.admin | | Instance connection | plugin instance Runtime proves its own identity to management | | External system | Stored and interpreted by plugins, optionally referenced in Config | -Before business execution, deployment verifies actual identities, registers Definitions, grants Scope permissions, records instance scope and connection identities, and checks access using execution credentials. Existing identity and Access interfaces are reused. Instances may be registered without Scope authorization but cannot accept work for that Scope. write has no separate execution credentials and does not borrow deployment authority for daily saves. Credentials are excluded from command arguments, catalogs, and ordinary logs; cross-machine connections use TLS or equivalent security. +Before business execution, the startup loading flow verifies actual identities, registers Definitions, grants Scope permissions, records instance scope and connection identities, and checks access using execution credentials. Server checks deployment authority before forwarding instance registration to management, which stores the instance connection credential digest. Management authenticates Runtime connections directly without consulting Server on each connection. Existing identity and Access interfaces are reused. Instances may be registered without Scope authorization but cannot accept work for that Scope. write has no separate execution credentials and does not borrow deployment authority for daily saves. Credentials are excluded from command arguments, catalogs, and ordinary logs; cross-machine connections use TLS or equivalent security. diff --git a/docs/zh/rfcs/2012-source-plugin-system.md b/docs/zh/rfcs/2012-source-plugin-system.md index 2dccc3442..167430ee8 100644 --- a/docs/zh/rfcs/2012-source-plugin-system.md +++ b/docs/zh/rfcs/2012-source-plugin-system.md @@ -81,7 +81,7 @@ flowchart TB | RemoteConnectorWorker | 已有批次管道；创建 session、调用 Connector.run、处理提交回执及 checkpoint CAS | | apply / Connector | 插件业务代码；apply 决定节奏，Connector 读取、转换并提交本批数据 | -只使用 write 时仅需 Server；N 个插件实例增加一个插件管理服务和 N 个插件实例 Runtime 进程，由系统服务托管。 +只使用 write 时仅需 Server；N 个插件实例增加一个插件管理服务和 N 个插件实例 Runtime 进程。插件实例 Runtime 默认作为普通后台进程运行，系统服务托管由用户显式选择。 ## 启动加载与运行时新增实例 @@ -89,11 +89,13 @@ flowchart TB **collect：** Server 启动时扫描一次通过环境变量指定的插件目录；每份 `powercontext.yml` 声明实例 `id`、代码入口、业务配置、凭据引用和可服务 Scope。id 即 instance_id，在同一 PC 部署内唯一且跨重启稳定。同一份本地代码可供多个实例使用，无需发布 wheel；配置进入可信目录表示部署者允许加载。 -部署流程准备或复用插件管理服务，校验插件、注册 Definition、登记实例并授权，再启动独立插件实例 Runtime。运行中新增实例需显式加载，不持续扫描目录，也不重启其他实例。重复加载同一 ID 和一致配置复用登记及进程，不重置启停状态；配置不同则报冲突，更新按第 5 节处理。 +用户在插件执行机器通过 `source load` 加载实例配置，连接本机或远端 Server 和插件管理服务。命令校验插件，使用部署凭据注册 Definition、登记实例并完成所需 Scope 授权，准备 Runtime 凭据和启动配置，再启动本机独立插件实例 Runtime；已有权限会复用，外部 Server 的身份由其认证机制准备。用户无须另行执行登记脚本或手工提交凭据摘要。本机自动部署可按需准备插件管理服务；跨机器部署连接已准备好的服务。 -配置可包含初始 `targets`，每项指定 Scope、能力及业务输入。实例首次初始化目标时按正常添加规则导入插件管理服务，并记录已初始化；此后以插件管理服务为准，文件不再覆盖或补充目标。目标为空或连接失败不触发重新导入，初始 targets 的变化不算实例配置冲突。 +Server 启动扫描复用同一加载流程；运行中新增实例显式执行 `source load`，不持续扫描目录，也不重启其他实例。重复加载同一 ID 和一致配置复用登记及进程，不重置启停状态；配置不同则报冲突，更新按第 5 节处理。 -无插件实例时不必启动插件管理服务；单个实例准备失败不阻塞内置功能。删除目录、停止 Server 或停用全部实例不会隐式卸载独立服务或删除数据。远端机器自行部署代码与插件实例 Runtime，本机加载不负责远程 shell 或代码分发。 +配置可包含初始 `targets`，每项指定 Scope、能力及业务输入。实例首次初始化目标时按正常添加规则导入插件管理服务，并记录已初始化；此后以插件管理服务为准，文件不再覆盖或补充目标。目标为空或连接失败不触发重新导入。 + +无插件实例时不必启动插件管理服务；单个实例准备失败不阻塞内置功能。删除目录、停止 Server 或停用全部实例不会隐式卸载独立服务或删除数据。远端机器准备代码和依赖后，在该机器执行 `source load`；命令只启动执行机器上的 Runtime，不负责远程 shell 或代码分发。 ## 组件通信与连接方向 @@ -111,10 +113,12 @@ flowchart TB ## 持久化数据与存储归属 +插件框架新增的独立持久化管理集中在插件管理服务。PC Server 复用已有数据存储，插件实例 Runtime 仅保留启动所需文件，两者不另建插件业务状态存储；write 加载入口及必要配置引用作为部署配置保留。 + | 保存方 | 权威数据与恢复依据 | | --- | --- | | 插件管理服务 | 自管 SQLite 保存实例、能力、collect 目标、期望启停状态及收到的提交记录；同时记录最近执行状态与确认时间 | -| PC Server | 已有存储保存 Source、Projection、checkpoint 和用户授权；另存 write 加载入口及配置引用，代码与依赖留在 Server 文件系统 | +| PC Server | 已有存储保存 Source、Projection、checkpoint 和用户授权；write 加载入口及配置引用属于部署配置，代码与依赖留在 Server 文件系统 | | 插件实例 Runtime | 仅持久保留代码、启动配置和凭据引用；目标快照、批次与未送达报告放在内存，重启后重新同步 | 管理存储中的 `records` 保存部署绑定、实例、能力、目标及提交记录（Source 引用和时间）；`follow_identity` 保存 collect 稳定资料身份到 target_id 的映射，用于重复添加时复用目标。write 直接沿用 Source 存储规则，不保存独立受理记录。 @@ -141,7 +145,7 @@ flowchart TB ## pc_source_capabilities：发现能力与输入要求 -**输入：** scope_id、operation（业务发现为 `add`），可按 capability、instance_id 筛选。 +**输入：** scope_id，可按 capability、instance_id 筛选。 **链路：** 宿主调用 → Server 检查 Scope 读取权限及是否可写 → 根据本地能力和已同步 collect 目录筛选公开范围、可服务 Scope → 直接返回能力名称、版本、kind、input_schema，以及权限、实例和可用性。 @@ -194,7 +198,7 @@ sequenceDiagram **输入：** scope_id、target_id；默认查询采集状态，`view="control"` 查询启停字段。 -**链路：** Server 检查对应视图权限 → 插件管理服务核对目标实际 Scope → 读取持久目标及已收到的提交记录 → reply → Server 返回宿主。查询不调用 identify_target、不等待新批次，也不要求插件实例 Runtime 在线。 +**链路：** Server 检查对应视图权限 → 插件管理服务核对目标实际 Scope → 读取持久目标及已收到的提交记录 → reply → Server 返回宿主。 | 视图 | 返回 | | --- | --- | @@ -215,10 +219,10 @@ tracking 表示 enabled / stopping / stopped。提交记录只表示已确认并 | 视图与输入 | 从调用到返回 | | --- | --- | -| 业务：scope_id、operation=add | Server 检查 Scope 权限，按 capabilities 的目录规则直接返回可见实例、能力、可写性与运行状态 | +| 业务：scope_id | Server 检查 Scope 权限，按 capabilities 的目录规则直接返回可见实例、能力、可写性与运行状态 | | 控制：instance_id、view=control | Server 检查 server.admin，经管理通道读取实例的持久控制状态与最近执行报告，再返回 instance_id、enabled、actual、可用性与更新时间 | -控制视图不要求 Scope，也不返回 Config、目标输入或 Source。仅插件实例 Runtime 离线时仍可读取控制状态，但须标注离线或未知；插件管理服务不可达则控制查询不可用。本方法不加载代码、访问外部账号或启动 apply；write 没有实例。 +控制视图不要求 Scope，也不返回 Config、目标输入或 Source。仅插件实例 Runtime 离线时仍可读取控制状态，但须标注离线或未知；插件管理服务不可达则控制查询不可用。 ## pc_source_instance_set_enabled：启停整个实例的业务运行 @@ -240,7 +244,7 @@ tracking 表示 enabled / stopping / stopped。提交记录只表示已确认并 **共同数据契约：** SourceDefinition 声明输入模型、Source 模型、名称及版本；`resolve(value)` 生成稳定身份的 Source，`projections` 中的 `project(source)` 生成 Projection。插件通过 `definitions` 导出实现对象，通过 `capabilities` 声明能力名称、版本、说明、kind 和 input_schema。能力描述可调用行为，Definition 描述输出数据；二者不是一一对应的类型体系。 -**write 加载：** 每个 write 能力关联一个 Definition 的名称与版本，input_schema 对应其输入模型。插件模块导出实际 Definition 和 Projection 对象，Server 通用加载器按受信任入口导入并登记，因此无需在 Server 代码中增加类型专属 import。新增模块只需准备代码、依赖和入口配置，不能仅通过注册 API 上传定义结构来获得可执行函数。导入不启动采集任务。 +**write 加载：** 每个 write 能力关联一个 Definition 的名称与版本，input_schema 对应其输入模型。插件模块导出实际 Definition 和 Projection 对象，Server 通用加载器按受信任入口导入并登记。新增模块只需准备代码、依赖和入口配置，不能仅通过注册 API 上传定义结构来获得可执行函数。 **collect 运行契约：** @@ -351,7 +355,7 @@ flowchart TD D -->|清理失败| U[报告错误，不创建第二份 Context] ``` -apply 返回或异常结束后清理 Context，插件实例 Runtime 仍在线；新增目标、重连或重复启用不重新调用 apply。需要持续处理新增目标的插件自行保持循环，等待不阻塞管理操作。内部运行标识只用于区分当前连接和本次运行的报告，宿主无需提供。 +apply 返回或异常结束后清理 Context，插件实例 Runtime 仍在线；新增目标、重连或重复启用不重新调用 apply。需要持续处理新增目标的插件自行保持循环，等待不阻塞管理操作。 同一进程再次运行须停用、确认 stopped，再启用；进程重启则在同步后按 enabled 决定是否新建 Context。直接运行入口在 apply 结束后退出，不能与后台模式同时占用同一实例。系统服务卸载或退出后，须由执行机器的部署工具重新启动。 @@ -372,7 +376,7 @@ apply 返回或异常结束后清理 Context，插件实例 Runtime 仍在线； | 插件管理服务与 Server 断线 | 宿主 collect 管理操作不可用，已有采集可继续；内置和已加载 write 独立可用 | | Server 不可用 | 数据提交失败或等待恢复；已保存 Source 保留，write 调用不自动恢复或重放 | -同身份旧进程是否退出无法确认时，不启动另一份插件实例 Runtime；心跳丢失不是跨机器接管依据。框架不保证 apply 或外部副作用只执行一次。 +同一插件实例只运行一个 Runtime。重新部署前，先停止旧 Runtime。 恢复以各存储中的真实数据为准：Source 已保存而 checkpoint 未更新时可能重新采集，沿用稳定身份避免重复；checkpoint 更新响应丢失时重新读取；提交报告丢失不回退 Source 或 checkpoint，也不保证补齐管理记录。 @@ -382,7 +386,7 @@ apply 返回或异常结束后清理 Context，插件实例 Runtime 仍在线； - **扩展 Scope：** 授予执行身份相应 Scope Binding（Server 中的角色授权关系），更新实例可服务范围；同步后可添加新 Scope 目标，保留原 Context。传入 scope_id 本身不授予权限。 - **缩减范围、修改执行配置或升级 collect：** 停用并确认停止，退出旧插件实例 Runtime，校验和更新代码、Config 与登记；新插件实例 Runtime 校验成功后显式启用。凭据变更按配置更新处理，业务含义由插件解释。 -升级保留 Source、目标、checkpoint 与已收到的提交记录，核对能力输入和目标身份、Definition、Connector 与 checkpoint 格式的兼容性。新代码须能处理保留的目标，包括停用目标；不能通过重置进度绕过检查。无法确认兼容时保持停用，由部署者明确处理，不自动迁移；新实例不自动继承旧目标。 +升级保留 Source、目标、checkpoint 与已收到的提交记录，核对能力输入和目标身份、Definition、Connector 与 checkpoint 格式的兼容性。新代码须能处理保留的目标，包括停用目标。无法确认兼容时保持停用，由部署者明确处理，不自动迁移；新实例不自动继承旧目标。 write 更新先处置在途调用，再重启加载，保留能力与 Definition 契约及已有 Source 身份。普通重复加载不代替配置更新。 @@ -401,7 +405,7 @@ write 更新先处置在途调用，再重启加载，保留能力与 Definition Server 转交已授权操作及可信身份；插件管理服务核对资源实际 Scope、实例可服务范围和运行条件，避免以有权限的 Scope 操作其他 Scope 的目标。插件实例 Runtime 认证后仅同步和报告自身实例，不能冒充 Server 管理请求。插件处理外部系统权限，identify_target 不承担授权。 -同 Scope 的贡献者均可控制目标，不限创建者；跨 Scope 的实例控制须 server.admin。collect 持续执行使用实例 PC 身份，每次提交由 Server 检查；创建者权限变化不自动停止目标。write 使用当前宿主身份，返回结果前检查当前读取权限；拒绝返回不撤销已经保存的数据。控制响应不含业务配置、目标输入或 Source 数据。 +同 Scope 的贡献者均可控制目标，不限创建者；跨 Scope 的实例控制须 server.admin。collect 持续执行使用实例 PC 身份，每次提交由 Server 检查。write 使用当前宿主身份；拒绝返回不撤销已经保存的数据。控制响应不含业务配置、目标输入或 Source 数据。 ## 凭据与初始化授权 @@ -409,11 +413,11 @@ AuthenticationProvider 将凭据映射为 Principal。本机部署可由部署 | 凭据 | 用途与归属 | | --- | --- | -| 部署凭据 | 部署工具加载代码、注册 Definition、授权与登记，不下发插件实例 Runtime | +| 部署凭据 | 启动加载流程用于加载代码、注册 Definition、授权与登记，不下发插件实例 Runtime | | 宿主凭据 | 保存在宿主安全配置，用于调用 Server | | 管理通道凭据 | 插件管理服务向 Server 认证 | | PC 执行凭据 | 每实例独立身份，提交 Source 和处理 checkpoint，不持有 server.admin | | 实例连接凭据 | 插件实例 Runtime 向插件管理服务证明自身身份 | | 外部系统凭据 | 插件保存和解释，可通过 Config 引用 | -部署启动业务前确认实际身份、完成 Definition 注册和 Scope 授权、登记实例范围及连接身份，再用执行凭据检查权限。复用已有身份与 Access 接口；无 Scope 授权可登记实例，但不能受理该 Scope 工作。write 不另发执行凭据，日常保存不借用部署权限。凭据不进入命令参数、能力目录或普通日志；跨机使用 TLS 或等效安全入口。 +启动加载流程在业务运行前确认实际身份、完成 Definition 注册和 Scope 授权、登记实例范围及连接身份，再用执行凭据检查权限。实例登记经 Server 校验部署权限后传给插件管理服务，后者保存实例连接凭据摘要；Runtime 连接时由插件管理服务直接校验，无须逐次请求 Server。复用已有身份与 Access 接口；无 Scope 授权可登记实例，但不能受理该 Scope 工作。write 不另发执行凭据，日常保存不借用部署权限。凭据不进入命令参数、能力目录或普通日志；跨机使用 TLS 或等效安全入口。 From 102f2d1d66bda364e7f3eb845da8256df4edb378 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=94=98=E9=9C=96?= Date: Fri, 16 Oct 2026 16:20:11 +0800 Subject: [PATCH 5/5] docs(rfc): document source plugin tradeoffs and trust boundaries --- docs/en/rfcs/2012-source-plugin-system.md | 51 +++++++++++++++++++++++ docs/zh/rfcs/2012-source-plugin-system.md | 51 +++++++++++++++++++++++ 2 files changed, 102 insertions(+) diff --git a/docs/en/rfcs/2012-source-plugin-system.md b/docs/en/rfcs/2012-source-plugin-system.md index 88b02235f..cd1525fd6 100644 --- a/docs/en/rfcs/2012-source-plugin-system.md +++ b/docs/en/rfcs/2012-source-plugin-system.md @@ -421,3 +421,54 @@ AuthenticationProvider maps credentials to Principals. Local deployment can prep | External system | Stored and interpreted by plugins, optionally referenced in Config | Before business execution, the startup loading flow verifies actual identities, registers Definitions, grants Scope permissions, records instance scope and connection identities, and checks access using execution credentials. Server checks deployment authority before forwarding instance registration to management, which stores the instance connection credential digest. Management authenticates Runtime connections directly without consulting Server on each connection. Existing identity and Access interfaces are reused. Instances may be registered without Scope authorization but cannot accept work for that Scope. write has no separate execution credentials and does not borrow deployment authority for daily saves. Credentials are excluded from command arguments, catalogs, and ordinary logs; cross-machine connections use TLS or equivalent security. + +# 6. Drawbacks + +**The main cost is greater framework complexity than the initial independent-plugin-process design.** That design handled loading, background start/stop, and the data pipeline, leaving target management to plugins. This proposal adds a common capability catalog, persistent targets and control state, and synchronization, authentication, and disconnection handling between management and Runtime. Developers maintain more protocols, deployers manage additional components and storage, and troubleshooting spans more steps. + +These costs provide consistent discovery and management of built-in and custom Sources across hosts, reducing per-plugin integration. Target and lifecycle management behind that interface needs an explicit owner; this proposal assigns it to the framework. An independent management service is an architectural choice, not the only way to provide a common interface; section 7 compares alternatives. Users needing only autonomous collection with fixed configuration also bear this general management cost. + +**The first release executes only plugin code trusted by the deployer; it provides no sandbox for untrusted code, automatic signature verification, or automatic capability reduction.** The first release provides no plugin distribution system. Plugins supplied with the PowerContext repository and their dependencies are covered by project code review, testing, and release processes. User-supplied plugins remain loadable; deployers review their provenance and dependencies. Administrator approval to load code does not prove it safe. PC Access restricts data operations through standard interfaces, not arbitrary Python code's system access. + +| Path | Execution identity and impact | +| --- | --- | +| write | Standard saves use the host's PC identity, but plugin code runs with the Server process's operating-system permissions and access to its resources. Blocking, resource exhaustion, or process-level failure may affect the entire Server | +| collect | Worker uses the instance PC identity for authorized Scopes; code runs with the Runtime process's operating-system permissions. Separate processes reduce direct process-failure propagation, but shared-machine resource contention and excessive submissions can still affect other components | + +Deployers needing stronger isolation can use separate system users, containers, or machines; these deployment measures are not a framework sandbox guarantee. Process separation does not guarantee high availability or resource quotas. + +# 7. Rationale and alternatives + +**write reduces execution components while accepting the risk of trusted code running inside Server.** When the host already has complete content, a Server-local implementation can validate, transform, and save it without a separately managed execution process. External execution can also return synchronously, so synchronous semantics do not require execution inside Server. This path reuses data contracts and persistence rules but extends #1400's boundary where custom implementations execute in Worker. + +**collect provides a common management interface.** The initial independent-plugin-process design already supported one background process per directory, local run / stop, multiple Scopes, and plugin-defined target updates through files, configuration services, or events; the process exited when apply ended. Its limitation was that each plugin defined its target format and update interface, requiring hosts to integrate with plugins individually. + +For example, a Yuque plugin already collecting document A should accept document B created by a host through the same instance. The initial design could do this, but the host needed to understand the plugin's target interface. This proposal requires a consistent entry point for built-in and custom Sources and shared management capabilities across hosts. The framework therefore accepts and persists targets and controls, while plugins retain control of collection timing. + +| collect alternative | Benefits | Costs or limitations | +| --- | --- | --- | +| Add only discovery and loading to existing Worker | Few components; reuses collection and submission pipelines | Plugins or external systems still provide dynamic targets and common management | +| Initial design: one background process per directory, plugin-managed targets | Background start/stop, resource cleanup, and multiple Scopes; plugins can refresh targets dynamically | No common target interface across plugins; no outer Runtime receives controls after apply ends | +| Place collect management inside Server | One fewer service and less management communication; remote Runtimes remain possible | Collection management and data service share a process lifecycle | +| Independent plugin management service and Runtime | Multiple instances share a management entry point; management deploys independently | Additional service, storage, and communication costs | + +A common MCP does not inherently require a separate management process. This proposal also requires one service shared by multiple instances, deployable on any network-reachable machine and separate from the PC Server process. It therefore selects an independent service, still serving only one Server. write execution placement and collect management structure are independent choices. The lighter independent-process design remains suitable when only autonomous plugin collection is needed. + +# 8. Prior art + +[#1400: Source Definition and Connector contracts](1400_source_definition_and_observation_model.md) provides data definitions, Projections, remote registration and submission, receipts, and checkpoint rules. Its remote ingestion boundary executes custom implementations in Worker while Server validates and persists results; definition registration does not transfer executable code. collect reuses this pipeline and adds capability discovery and target management; write adds Server-local implementation loading. User authorization continues to reuse existing Access. Plugin management does not replace these data and authorization contracts. + +# 9. Unresolved questions + +Implementers must propose and validate the following in implementation PRs, with maintainer confirmation before the first release. Changes to this RFC's public semantics require a design update first. + +| Question | Established boundary and remaining decisions | +| --- | --- | +| Plugin API compatibility support | The first release loads only explicitly supported plugin_api_version values and rejects unsupported versions at load time. It provides no multi-version compatibility layer or commitment to support periods for older versions. Repository-supplied plugins are maintained alongside the framework. Implementation PRs must list and validate supported versions and rejection behavior; Definition and checkpoint compatibility rules still apply | +| Server resource protection | Scope permissions and process separation do not provide resource quotas. Input-size, concurrency, and submission-load limits and their failure behavior need to be specified and validated. This proposal does not yet promise an upper bound on a plugin's resource impact on Server | + +Section 6 defines plugin maintenance and trust boundaries for the first release. Plugin distribution and automatic signature and supply-chain verification are outside this proposal’s scope. + +# 10. Future possibilities + +Future work may evaluate isolated write execution, sandboxes for untrusted code, plugin signatures and dependency verification, and per-instance resource quotas. These are not first-release commitments. Changes to execution placement, the trust model, or public invocation semantics require separate design review. diff --git a/docs/zh/rfcs/2012-source-plugin-system.md b/docs/zh/rfcs/2012-source-plugin-system.md index 167430ee8..bee085d9f 100644 --- a/docs/zh/rfcs/2012-source-plugin-system.md +++ b/docs/zh/rfcs/2012-source-plugin-system.md @@ -421,3 +421,54 @@ AuthenticationProvider 将凭据映射为 Principal。本机部署可由部署 | 外部系统凭据 | 插件保存和解释，可通过 Config 引用 | 启动加载流程在业务运行前确认实际身份、完成 Definition 注册和 Scope 授权、登记实例范围及连接身份，再用执行凭据检查权限。实例登记经 Server 校验部署权限后传给插件管理服务，后者保存实例连接凭据摘要；Runtime 连接时由插件管理服务直接校验，无须逐次请求 Server。复用已有身份与 Access 接口；无 Scope 授权可登记实例，但不能受理该 Scope 工作。write 不另发执行凭据，日常保存不借用部署权限。凭据不进入命令参数、能力目录或普通日志；跨机使用 TLS 或等效安全入口。 + +# 6. 缺点 + +**主要代价是框架比最初的独立插件进程方案更复杂。** 原方案只负责加载、后台启停和数据管道，目标管理由插件自行实现；本方案增加公共能力目录、持久目标与控制状态，以及插件管理服务和 Runtime 之间的同步、认证和断线处理。开发者需要维护更多协议，部署者需要管理额外组件与存储，排查问题也涉及更多环节。 + +这些成本用于换取不同宿主对内置与自定义 Source 的统一发现和管理，减少逐插件对接。统一入口背后的目标与生命周期管理必须有明确的承担方；本方案将其纳入框架。独立管理服务是具体的架构选择，并非统一入口的唯一实现方式，替代方案见第 7 节。仅需固定配置、自主采集的用户，也会承担这部分通用管理成本。 + +**第一版只执行部署者信任的插件代码，不提供不可信代码沙箱、自动签名验证或自动能力降级。** 第一版不提供插件分发体系；随 PowerContext 仓库提供的插件及其依赖纳入项目的代码审查、测试与发布流程。用户自行提供的插件仍可加载，其代码来源与依赖由部署者审核。管理员允许加载不等于框架证明代码安全；PC Access 限制标准接口上的数据操作，不能限制任意 Python 代码的系统访问。 + +| 路径 | 执行身份与影响范围 | +| --- | --- | +| write | 标准保存按宿主 PC 身份授权，但插件代码使用 Server 进程的操作系统权限，可访问该进程可用的资源。阻塞、资源耗尽或进程级故障可能影响整个 Server | +| collect | Worker 使用获授权 Scope 的实例 PC 身份，代码使用 Runtime 进程的操作系统权限。独立进程减少直接的进程故障传播，但同机资源竞争和过量提交仍可能影响其他组件 | + +需要更强隔离时，部署者可使用独立系统用户、容器或机器；这些部署措施不构成框架提供的沙箱保证，进程分离也不保证高可用或资源配额。 + +# 7. 设计理由与替代方案 + +**write 的选择是减少执行组件，同时接受 Server 内执行可信代码的风险。** 宿主已有完整内容时，部署 Server 本地实现即可完成校验、转换与保存，无需另行管理执行进程。外部进程执行后同样可以同步返回，因此同步语义并不要求 Server 内执行。此路径复用数据契约和保存规则，但扩展了 #1400 中自定义实现由 Worker 执行的边界。 + +**collect 的选择是统一管理入口。** 最初的独立插件进程方案已支持一目录一后台进程、本地 run / stop、多 Scope，以及插件自行读取目标文件、配置服务或事件来新增目标；apply 结束则进程退出。其不足是目标格式与更新入口由各插件约定，宿主需要逐插件对接。 + +例如语雀插件已采集文档 A，宿主新建 B 后希望加入同一实例。原方案可以实现，但宿主须了解插件自己的目标接口。本提案要求内置与自定义 Source 使用一致入口，不同宿主复用公共管理能力，因此将目标接收、持久保存与启停纳入框架，采集节奏仍交给插件。 + +| collect 方案 | 优点 | 代价或限制 | +| --- | --- | --- | +| 仅为已有 Worker 增加发现与加载 | 组件少，复用采集和提交管道 | 动态目标与统一管理仍需插件或外部系统提供 | +| 最初方案：一目录一后台进程，插件自行管理目标 | 已提供后台启停、资源清理与多 Scope，插件可动态刷新目标 | 缺少跨插件的公共目标接口；apply 结束后没有继续接收控制的外层 Runtime | +| 将 collect 管理放入 Server | 少一个服务，减少管理通信，也能接入远端 Runtime | 采集管理与数据服务共用进程生命周期 | +| 独立插件管理服务与 Runtime | 多实例共享管理入口，管理服务可独立部署 | 增加服务、存储和通信成本 | + +统一 MCP 不必然要求独立管理进程。本提案另有部署要求：单一管理服务供多个实例共享，可位于任意网络可达的机器，与 PC Server 分进程；因此选择独立服务，一个管理服务仍只对应一个 Server。write 的执行位置与 collect 的管理结构是两项独立取舍。仅需插件自主采集时，较轻的独立进程方案仍然适用。 + +# 8. 参考设计 + +[#1400：Source Definition 与 Connector 契约](1400_source_definition_and_observation_model.md) 提供数据定义、Projection、远程注册与提交、回执和 checkpoint 规则；其远程接入边界由 Worker 执行自定义实现，Server 校验并保存结果，定义注册不传输可执行代码。collect 复用这套管道，新增能力发现和目标管理；write 增加 Server 本地实现加载。用户授权继续复用现有 Access。本提案不以插件管理协议替换这些数据与权限契约。 + +# 9. 未决问题 + +以下事项由实现者在实现 PR 中提出并验证，维护者在首次发布前确认；改变本 RFC 的公共语义须先更新设计。 + +| 问题 | 已确定的边界与待决定事项 | +| --- | --- | +| 插件 API 兼容支持 | 第一版只加载框架明确支持的 plugin_api_version，不支持则在加载时拒绝；不建设多版本兼容层，也不承诺旧版本支持周期。随仓库提供的插件随框架同步维护，具体支持版本及拒绝行为在实现 PR 中列明并验证；Definition 和 checkpoint 的兼容规则仍须遵守 | +| Server 资源保护 | Scope 权限和进程分离不提供资源配额。需确定输入大小、并发和提交负载的限制及超限行为，并验证相应保护；当前不承诺插件对 Server 的资源影响上限 | + +第一版的插件维护与信任边界见第 6 节；插件分发、自动签名与供应链验证不在本次范围内。 + +# 10. 后续可能性 + +后续可评估隔离执行 write、支持不可信代码的沙箱、插件签名与依赖验证，以及按实例设置资源配额。它们不属于第一版承诺；若改变执行位置、信任模型或公共调用语义，应单独设计评审。