[Feature]: HoloRouter serve multiple concurrent Holodeck instances on a single ESX host?

[chinthub]

Description

We run a shared Holodeck lab on one physical ESX host (dual AMD EPYC 7K62,
96C/192T, 1 TB RAM). Three engineers need their own nested VCF environments
on it, sized well within the host's capacity.

The documentation says multiple VCF deployments on a single ESX host of
sufficient capacity are supported, that each environment is completely
self-contained, and that each configuration file maps to a single deployment
with a new config created per deployment. What it does not state is how the
HoloRouter behaves when more than one of those deployments exists at once.

Reading HoloDeckDNSConfig.psm1 and networkmanager.psm1 (Holodeck 5.2.x), our
understanding was that Set-HoloRouter regenerates the dnsmasq and FRR
configuration on the HoloRouter as global objects applied with kubectl apply,
which replaces rather than merges. If that is correct, running Set-HoloRouter
for a second InstanceID would discard the first instance's host inventory,
DNS entries and routing.

Three questions:

1. Is one HoloRouter per concurrent Holodeck instance required, or can a
   single HoloRouter serve several instances on one host?
2. If several are supported, does Set-HoloRouter for a second InstanceID
   preserve the first instance's dnsmasq and FRR state, or replace it?
3. Did this change in Holodeck 9.1 with the move to Technitium DNS?

If multiple HoloRouters on one host is the supported answer, guidance on
avoiding VLAN, CIDR and uplink conflicts between them would be very welcome.

Environment: Holodeck 9.1, single stand-alone ESX host, single-site,
VCF 5.2 / 9.0 / 9.1 nested deployments planned.

Use case / Motivation

We run a shared Holodeck lab on one physical ESX host - dual AMD EPYC 7K62,
96C/192T, 1 TB RAM, NVMe storage - used by three engineers for VCF study and
customer-scenario rehearsal. Each engineer wants an independent nested VCF
environment, sized within the host's capacity, across VCF 5.2, 9.0 and 9.1.

Without a documented answer we have to choose between two bad options: run
strictly one instance at a time and leave two people idle through 5 to 12 hour
builds, or run concurrent instances and risk one engineer's Set-HoloRouter
silently destroying another's estate. The failure mode, if real, is silent - the
first instance keeps running and only breaks later when something needs DNS or
routing, so the person who loses the work is not the person who caused it.

This is a common shape for partner and customer labs: one large host, several
engineers, independent environments. A clear statement in the documentation
would settle it for everyone in that position.

Proposed solution

Add a short section to the HoloRouter documentation stating the supported
concurrency model - HoloRouter to instances, one-to-one or one-to-many - and
what Set-HoloRouter does to existing state when invoked for a further
InstanceID. If one HoloRouter per concurrent instance is the answer, a brief
note on avoiding VLAN, CIDR and uplink conflicts between routers on the same
host would complete it.

Additional context

Environment: Holodeck 9.1, single stand-alone ESX host, single site, nested VCF
5.2 / 9.0 / 9.1 planned. Source reading was against the 5.2.x toolkit modules;
not re-verified against the 9.1 HoloRouter appliance.

[dhruv-tyagi-broadcom]

Is one HoloRouter per concurrent Holodeck instance required, or can a single HoloRouter serve several instances on one host?

One HoloRouter per Holodeck instance is required as the networking set up on Holorouter is for one specific nested VCF instance.

If several are supported, does Set-HoloRouter for a second InstanceID preserve the first instance's dnsmasq and FRR state, or replace it?

Since each instance is on its own HoloRouter, their networking is setup independently from each other from scracth.

Did this change in Holodeck 9.1 with the move to Technitium DNS?

Only change in Holodeck 9.1 in this regard is we replaced dnsmasq with technitium.

For running multiple instances on the same physical ESX host, we provide optionality to use "VLANRangeStart" parameter in the New-HoloDeckInstance command. This ensures each Holodeck instance is running on its own set of VLANs and each instance can operate without overlapping VLANs.

If you have NSX deployed on your physical environment, you can create NSX segments for each holorouter to keep traffic for each Holodeck instance on its own NSX segment.

[chinthub]

Thanks a lot for the detailed answers.