The diversification of high-performance computing (HPC) applications and the development of heterogeneous workflow applications involving HPC, artificial intelligence and machine learning, and in situ analysis are challenging the standard architecture of HPC clusters. Modern HPC workflows increasingly rely on cloud-based services and applications, which are often not supported in HPC environments. Cloud computing has historically been the preferred platform for running services, leading to the development of a large ecosystem of software for cloud environments. In addition, it is increasingly supporting HPC applications by offering dedicated hardware and job scheduling via standard HPC tools or cloud tools. The convergence of the needs of both communities has sparked growing interest in an architecture that combines the advantages of both platforms and is capable of supporting both HPC and cloud applications within the same environment, without compromising performance, security, or usability. This type of approach is generally referred to as converged infrastructure computing. This thesis explores several approaches to converging computing to enable the deployment of cloud-based applications in HPC clusters. We study two approaches based on the on-demand deployment of Kubernetes clusters on HPC resources, offering different compromises in terms of flexibility for resource management and supported Kubernetes features.
Our main contribution lies in the design and implementation of a secure container network that ensures user isolation and provides the flexibility and security necessary for deploying service-type applications, while remaining transparent to end users. We implemented this network in the pcocc container engine used in production at the CEA and measured no significant impact of the container network on MPI application performance. This innovative approach demonstrates that network virtualization can increase the flexibility of HPC clusters and enable the deployment of cloud-based applications without impacting the performance of traditional HPC applications.