Kubernetes has become the de facto standard for orchestrating cloud-native workloads across clusters of machines, both on-premises and in the cloud. As organizations scale their infrastructure and diversify their workloads (including AI/ML, IoT, security functions and latency-sensitive applications), the need for intelligent, policy-driven placement and rescheduling of pods becomes critical. Standard Kubernetes scheduling is powerful but generic, and lacks the ability to optimise for custom objectives such as cost, energy efficiency, resource utilisation, or application-specific constraints. kubeDelphi fills this gap and drive the evolution of Kubernetes towards:

• Cost optimization, reducing cloud expenditure by placing workloads on the most cost-effective resources.
• Resource efficiency, maximizing the use of available CPU, memory, and specialized hardware (e.g., GPUs).
• Performance and QoS, ensuring latency-sensitive applications meet their SLAs.

kubeDelphi is an extension of Kubernetes based on an advanced workload scheduling system, which is designed to optimize workload placement and rescheduling in cloud-native environments.

Framework overview

kubeDelphi enables cloud operators and developers to (i) experiment with custom placement algorithms, (ii) seamlessly integrate them into the Kubernetes scheduling cycle, and (iii) automate workload rescheduling in response to changing cluster conditions, workload status, or business objectives. These capabilities are delivered through three main components:

kubectl Plugin

• Purpose: to extend the standard kubectl CLI with new commands for advanced cluster management tasks.
• Current functionality: different implementations of this component are available, enabling the rescheduling of pods based on updated placement decisions, improving energy efficiency, optimizing the usage of GPU devices. Others can be added in the future.
• Role in the framework: acts as an orchestrator, gathering cluster state, invoking Placement Algorithm, and updating Kubernetes resources.

Placement Algorithm

• Purpose: to implement the logic for optimal or heuristic placement of workload, considering objectives and constraints related to cost, resource usage, energy efficiency or other custom constraints.
• Current functionalities: in parallel with the kubectl Plugin component, a number of algorithms have been developed to reduce costs, improve energy efficiency, optimize GPU utilization, and balance security with performance requirements.
• Role in framework : Placement Algorithm is decoupled from the rest of the system via a gRPC interface, allowing it to be written in any language. Its role is to provide placement decisions according to the current status of the infrastructure and of the workload to be deployed.

Kubernetes Scheduler Plugin

• Purpose: to integrate with the Kubernetes scheduler to enforce placement decisions made by the algorithms.
• Current functionalities: the kubeDelphi scheduling plugin reads placement scores generated by the Placement Algorithm and influences pod scheduling accordingly.
• Role in framework: extends the Kubernetes Scheduling Framework adding a new plugin able to take into account the placement decisions made by the placement algorithm in use.