feat(router): add Agent-based Router orchestration

[m199369309]

Summary

• add Supervisor-managed Token Router nodes, deployments, assignments, and reconciliation
• add the standalone xinference-router-agent service with runtime process and tokenizer asset lifecycle management
• expose Agent/orchestration APIs, lifecycle state, desired/observed generations, and runtime health metadata
• add Router-specific structured logging and process resource metrics
• add focused API, core, and runtime tests

Dependency

Depends on #5376
Depends on #5377

This PR intentionally targets main; maintainers can merge it after the dependencies above are merged.

Validation

• pre-commit run --files <all changed files>
• focused orchestration/asset/agent/API suite: 41 passed
• expanded focused suite reached 186 passed, 1 skipped; the process was interrupted after the repository's known async API teardown hang
• post-format targeted suite reached 58 passed; the process was interrupted after the same teardown hang

Please review now if convenient, but merge only after the dependency PRs have landed.

[gemini-code-assist]

Code Review

This pull request introduces an independent, token-aware router for DeepSeek-V4 on Xinference, including a Supervisor orchestration controller, a desired-state scheduler, a runtime registry, and an Agent service for managing local router processes. It also adds example systemd service configurations, documentation, and comprehensive unit tests. The review feedback highlights several critical robustness improvements, such as preventing connection leaks in the chat completion proxy, ensuring process termination during Agent shutdown even if status reporting fails, and adding defensive guards against null values in capabilities and message tokenization.

[qinxuye]

LGTM