Plugin Installation via Dashboard & `simplens.config.yaml` with Centralized Plugin Registry

[Adhish-Krishna]

Plugin Installation via Dashboard & simplens.config.yaml with Centralized Plugin Registry

Summary

Implement a centralized plugin management system that allows plugins to be installed and managed either through the Dashboard UI or via a simplens.config.yaml configuration file.

The API Service will own the source of truth for installed plugins using a centralized database collection. Any plugin installation, update, enable/disable, or removal should publish an event that is consumed by all API Service instances and Notification Processor instances, allowing every running instance to update its in-memory plugin registry without requiring a restart.

Motivation

Currently plugin installation is local and not synchronized across multiple service instances.

This feature enables:

• Centralized plugin management
• Horizontal scaling
• Dynamic plugin updates
• Zero-restart plugin synchronization
• Infrastructure as Code using simplens.config.yaml

Functional Requirements

Plugin Installation

Support plugin installation through:

• Dashboard UI
• simplens.config.yaml

Centralized Plugin Registry

Create a centralized database collection that stores:

• Plugin name
• Version
• Configuration
• Status (Enabled/Disabled)
• Installation metadata

This collection acts as the single source of truth.

Event-Driven Synchronization

Whenever a plugin is:

• Installed
• Updated
• Enabled
• Disabled
• Removed

The API Service should publish an event.

All API Service instances and Notification Processor instances must subscribe to these events and refresh their in-memory plugin registry.

Configuration Synchronization

Support bootstrapping and synchronizing plugins using simplens.config.yaml.

The configuration file should be reconciled with the centralized plugin registry so that the desired state defined in the configuration is reflected across the deployment.

Dynamic Plugin Loading

Services should load or unload plugins dynamically whenever possible without requiring a restart.

Technical Tasks

• Design the centralized Plugin Registry schema.
• Implement Plugin Management APIs.
• Implement plugin installation and management through the Dashboard.
• Add support for parsing and reconciling simplens.config.yaml.
• Design plugin lifecycle events.
• Publish plugin lifecycle events from the API Service.
• Subscribe to plugin events in all API Service instances.
• Subscribe to plugin events in all Notification Processor instances.
• Implement automatic in-memory plugin registry refresh.
• Handle plugin version conflicts and duplicate installations.
• Add logging and observability for plugin synchronization.

Acceptance Criteria

• Plugins can be installed and managed from the Dashboard.
• Plugins can be declared using simplens.config.yaml.
• Plugin metadata is stored in a centralized database.
• All API Service instances automatically receive plugin updates.
• All Notification Processor instances automatically receive plugin updates.
• Plugin lifecycle changes are propagated through events.
• No manual restart is required for plugin synchronization.
• All running instances remain synchronized with the centralized plugin registry.