From c08b8f59723b57d7815feae034123e59b62bb4a3 Mon Sep 17 00:00:00 2001 From: Fred Rolland Date: Thu, 7 Mar 2024 12:06:26 +0200 Subject: [PATCH] Add kubernetes tuning page Signed-off-by: Fred Rolland --- docs/kubernetes-perfomance.rst | 38 +++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/docs/kubernetes-perfomance.rst b/docs/kubernetes-perfomance.rst index 64fd1d0..1eee8e7 100644 --- a/docs/kubernetes-perfomance.rst +++ b/docs/kubernetes-perfomance.rst @@ -19,4 +19,40 @@ ***************************** Kubernetes Performance Tuning -***************************** \ No newline at end of file +***************************** + +This section provides a configuration example for Kubernetes performance tuning for SR-IOV. + +The machine in this example includes the following CPUs: + +.. code-block:: + + numactl --hardware + available: 2 nodes (0-1) + node 0 cpus: 0 1 2 3 4 5 12 13 14 15 16 17 + node 0 size: 31990 MB + node 0 free: 25314 MB + node 1 cpus: 6 7 8 9 10 11 18 19 20 21 22 23 + node 1 size: 32237 MB + node 1 free: 27135 MB + node distances: + node 0 1 + 0: 10 21 + 1: 21 10 + +3 CPUs are reserved on NUMA, and 1 for the system and for Kubernetes. Edit `/var/lib/kubelet/config.yaml`: + +.. code-block:: yaml + + cpuManagerPolicy: static + reservedSystemCPUs: 6-8 + topologyManagerPolicy: single-numa-node + +Using `isolcpus` kernel boot command-line isolates the CPUs from the kernel scheduler. This will ensure that a user-space process will not be scheduled by the kernel. In this example, CPUs 0-5 and 9-23 are isolated. + +.. code-block:: + + isolcpus=0,1,2,3,4,5,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24 + +.. warning:: + In case of changes in the `reservedSystemCPUs` or in the `cpuManagerPolicy` the `/var/lib/kubelet/cpu_manager_state` should be deleted, and the `kubelet` should be restarted.