Run A JobSet
This document explains how you can use Kueue’s scheduling and resource management functionality when running JobSet Operator JobSet.
This guide is for batch users that have a basic understanding of Kueue. For more information, see Kueue’s overview.
Before you begin
-
Check Administer cluster quotas for details on the initial Kueue setup.
-
See JobSet Installation for installation and configuration details of JobSet Operator.
JobSet definition
When running JobSets on Kueue, take into consideration the following aspects:
a. Queue selection
The target local queue should be specified in the metadata.labels section of the JobSet configuration.
metadata:
labels:
kueue.x-k8s.io/queue-name: user-queue
b. Configure the resource needs
The resource needs of the workload can be configured in the spec.replicatedJobs. Should also be taken into account that number of replicas, parallelism and completions affect the resource calculations.
- replicas: 1
template:
spec:
completions: 2
parallelism: 2
template:
spec:
containers:
- resources:
requests:
cpu: 1
c. Jobs prioritisation
The first PriorityClassName of spec.replicatedJobs that is not empty will be used as the priority.
- template:
spec:
template:
spec:
priorityClassName: high-priority
Example JobSet
The JobSet looks like the following:
# jobset-sample.yaml
apiVersion: jobset.x-k8s.io/v1alpha2
kind: JobSet
metadata:
generateName: sleep-job-
labels:
kueue.x-k8s.io/queue-name: user-queue
spec:
network:
enableDNSHostnames: false
subdomain: some-subdomain
replicatedJobs:
- name: workers
replicas: 2
template:
spec:
parallelism: 4
completions: 4
backoffLimit: 0
template:
spec:
containers:
- name: sleep
image: busybox
resources:
requests:
cpu: 1
memory: "200Mi"
command:
- sleep
args:
- 100s
- name: driver
template:
spec:
parallelism: 1
completions: 1
backoffLimit: 0
template:
spec:
containers:
- name: sleep
image: busybox
resources:
requests:
cpu: 2
memory: "200Mi"
command:
- sleep
args:
- 100s
Note
The samejobset-sample.yaml file from single cluster environment can be used in a MultiKueue environment.
In that setup, the spec.managedBy field will be set to kueue.x-k8s.io/multikueue
automatically, if not specified, as long as the kueue.x-k8s.io/queue-name annotation
is specified and the corresponding Cluster Queue uses the Multi Kueue admission check.
You can run this JobSet with the following commands:
# To monitor the queue and admission of the jobs, you can run this example multiple times:
kubectl create -f jobset-sample.yaml
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.