docs: add Cilium CNI integration example and documentation (#121)

[Dasmat13]

Fixes #121

Summary

Adds documentation and an example manifest for integrating Cilium CNI with the Node Readiness Controller.

Key Details

• Taint Key Alignment: Details configuring Cilium's --agent-not-ready-taint-key to match readiness.k8s.io/network-not-ready.
• NodeReadinessRule: Demonstrates using NetworkUnavailable=False for bootstrap network readiness.
• Continuous Health: Explains differences between bootstrap NetworkUnavailable status and continuous runtime health probes.

Changes

• examples/cni-readiness/cilium-network-readiness-rule.yaml: Manifest for Cilium network readiness rule.
• docs/book/src/examples/cni-readiness.md: Added Cilium CNI integration guide.
• examples/cni-readiness/README.md: Added Cilium note.

[netlify]

✅ Deploy Preview for node-readiness-controller ready!

Name	Link
🔨 Latest commit	a5a9d64
🔍 Latest deploy log	https://app.netlify.com/projects/node-readiness-controller/deploys/6a746ee86d6f7700096f6781
😎 Deploy Preview	https://deploy-preview-364--node-readiness-controller.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[kubernetes-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Dasmat13
Once this PR has been reviewed and has the lgtm label, please assign dchen1107 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kubernetes-prow]

Hi @Dasmat13. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[Dasmat13]

Hi @haircommander @tallclair, quick ping on this PR. Could you take a look and /ok-to-test when you get a chance? Thanks

[ajaysundark]

cc @arnab-logs

could you take a look?

[arnab-logs]

Thanks for picking this up @Dasmat13 I went through yor PR thoroughly had a few thoughts

The issue's suggested resolution was to pick a single owner for the readiness taint either NRC or Cilium, not both, since two independent controllers managing the same taint risks an overlap

This PR renames Cilium's taint key to the prefix --agent-not-ready-taint-key=readiness.k8s.io/network-not-ready & adds an NRC NodeReadinessRule targeting that same key via NetworkUnavailable. But asper Cilium's own docs, --agent-not-ready-taint-key only changes the name of the taint Cilium manages; Cilium's agent still adds and removes it itself independent of NRC

So in the common case Cilium will remove the taint on its own before NRC's rule could do anything i.e. nrc rule ends up watching a taint someone else already cleared.

Also, I found 2 documented cases where trusting its own internal readiness signal alone caused problems (and where NRC could be of help)

• NetworkUnavailable is set to false too early cilium/cilium#10762
• Cilium removes agent-not-ready taint too early cilium/cilium#29405

I think a way to work with it would be either:

1. Make cilium as authoritative i.e. keep the renamed flag (with this we get the standardized prefix for autoscaler tooling which was also the core motivation in Interaction between Cilium bootstrap taint and Node Readiness Controller #121) and drop the NodeReadinessRule for this specific taint since it is nit doing anything Cilium isn't already doing by itself
2. Give NRC the authority to remove taints i.e. we don't touch Cilium's flag at all, leave it at its default node.cilium.io/agent-not-ready. Instead the docs needs to explain how to get the node to start already tainted with readiness.k8s.io/network-not-ready (e.g. via --register-with-taints at node creation) and then let the NodeReadinessRule from this PR remove that taint based on NetworkUnavailable.

cc: @ajaysundark since this touches a bit of a design decision, wanted to get your read on which direction makes sense here