Name
Kubernetes and HAMi Deployment at NB Bank
Organization
Bank of Ningbo Co., Ltd. (“NB Bank”), DaoCloud
Team(s)
Bank of Ningbo Co., Ltd. (“NB Bank”) is a leading regional city commercial bank with a strong presence in local inclusive finance, retail banking, corporate banking, and intelligent risk management. The bank continues to advance its digital and intelligent transformation. In recent years, NB Bank has steadily deployed AI capabilities including private deployment of financial large language models, scenario-specific fine-tuning, and intelligent inference services. It has built a diverse portfolio of model assets encompassing large-scale general-purpose models, long-context business models, lightweight question-answering models, and financial-domain-specific models. At the same time, the bank supports multiple types of heterogeneous computing hardware, including general-purpose GPUs and domestic AI accelerators, and has gradually established a scalable, operationalized in-house AI ecosystem.
Compared with large banks, city commercial banks typically focus their AI strategies on lightweight implementation, steady iteration, cost reduction, and efficiency improvement rather than the accumulation of very large clusters. They place greater emphasis on reusing existing computing resources, standardizing model-service governance, and accurately matching technology to business scenarios. This cloud-native AI initiative is a core project in NB Bank’s effort to standardize and consolidate its AI infrastructure.
Overview and Goals
Built on the open source Kubernetes foundation, NB Bank developed a lightweight cloud-native AI support system tailored to a city commercial bank, centered on two core capabilities: heterogeneous computing-resource governance with HAMi and inference-service traffic governance with Higress. The system directly addresses production challenges including fragmented heterogeneous computing deployment, uneven resource utilization, mismatched multi-model invocation, and rigid scheduling across multi-chip inference clusters.
This initiative moves away from a capital-intensive, large-scale expansion model. Instead, it uses open source cloud-native technologies to unlock the value of existing computing resources and model assets, enabling lightweight, standardized, and efficient implementation of financial AI scenarios. It provides a practical, replicable benchmark aligned with industry realities for regional city commercial banks and other small and mid-sized banks seeking to scale AI services through the open source ecosystem.
Projects
- Kubernetes: A Unified Cloud-Native Foundation for Financial AI
- HAMi: The Core of Unified Heterogeneous Computing-Resource Governance
- Higress: A Governance Gateway Designed for Financial AI Inference Services
- Kueue: A Technology Reserve for Future Evolution
Planned Evolution
- Continue optimizing HAMi scheduling strategies for heterogeneous computing resources, deepen unified adaptation across domestic accelerators and general-purpose GPUs, and further improve fine-grained computing-resource utilization;
- Continue evolving Higress AI inference scheduling, optimize multi-model routing strategies and weighted traffic scheduling across clusters, and improve intelligent scheduling accuracy for complex business scenarios;
- Strengthen integration between the large-model service platform and the two core scheduling systems to enable intelligent end-to-end scheduling of computing resources, models, and services;
- Introduce Kueue batch-workload queue scheduling in a lightweight manner and, when computing resources are shared across more departments and batch training reaches greater scale, gradually deploy queue-priority and quota-management capabilities;
- Continue documenting implementation experience with heterogeneous AI clusters in a city commercial bank, contribute lessons back to the open source community, and promote scenario-specific optimization of HAMi and Higress for small and mid-sized financial institutions.
Name
Kubernetes and HAMi Deployment at NB Bank
Organization
Bank of Ningbo Co., Ltd. (“NB Bank”), DaoCloud
Team(s)
Bank of Ningbo Co., Ltd. (“NB Bank”) is a leading regional city commercial bank with a strong presence in local inclusive finance, retail banking, corporate banking, and intelligent risk management. The bank continues to advance its digital and intelligent transformation. In recent years, NB Bank has steadily deployed AI capabilities including private deployment of financial large language models, scenario-specific fine-tuning, and intelligent inference services. It has built a diverse portfolio of model assets encompassing large-scale general-purpose models, long-context business models, lightweight question-answering models, and financial-domain-specific models. At the same time, the bank supports multiple types of heterogeneous computing hardware, including general-purpose GPUs and domestic AI accelerators, and has gradually established a scalable, operationalized in-house AI ecosystem.
Compared with large banks, city commercial banks typically focus their AI strategies on lightweight implementation, steady iteration, cost reduction, and efficiency improvement rather than the accumulation of very large clusters. They place greater emphasis on reusing existing computing resources, standardizing model-service governance, and accurately matching technology to business scenarios. This cloud-native AI initiative is a core project in NB Bank’s effort to standardize and consolidate its AI infrastructure.
Overview and Goals
Built on the open source Kubernetes foundation, NB Bank developed a lightweight cloud-native AI support system tailored to a city commercial bank, centered on two core capabilities: heterogeneous computing-resource governance with HAMi and inference-service traffic governance with Higress. The system directly addresses production challenges including fragmented heterogeneous computing deployment, uneven resource utilization, mismatched multi-model invocation, and rigid scheduling across multi-chip inference clusters.
This initiative moves away from a capital-intensive, large-scale expansion model. Instead, it uses open source cloud-native technologies to unlock the value of existing computing resources and model assets, enabling lightweight, standardized, and efficient implementation of financial AI scenarios. It provides a practical, replicable benchmark aligned with industry realities for regional city commercial banks and other small and mid-sized banks seeking to scale AI services through the open source ecosystem.
Projects
Planned Evolution