Profile applicability: Level 1 - Worker Node
This recommendation stresses the importance of setting timeouts on streaming connections
to the Kubelet to mitigate risks associated with Denial-of-Service (DoS) attacks,
inactive connections, and depletion of ephemeral ports. Properly configuring these
timeouts ensures the secure and efficient operation of network connections.
Impact
Long-lived connections could be interrupted.
Audit
Audit method 1:
NoteKubelets can be configured via a configuration file or command line arguments. Command
line arguments take precedence. Check both command line arguments and configuration
file entries when auditing Kubelet configurations.
|
-
SSH into each node and run the following command to view details of the active Kubelet process:
ps -ef | grep kubelet
-
If the command line for the process includes the argument
--streaming-connection-idle-timeout
, verify that it is not set to 0. If the--streaming-connection-idle-timeout
argument is not present, refer instead to the--config
argument that specifies the location of the Kubelet config file:cat /etc/kubernetes/kubelet/kubelet-config.json
-
Open the Kubelet config file:
sudo less /path/to/kubelet-config.json
-
Verify that the
streamingConnectionIdleTimeout
argument is not set to 0.
Audit method 2:
If using the API configz endpoint, consider searching for the status of
"streamingConnectionIdleTimeout": "4h0m0s"
by extracting the live configuration from the nodes running Kubelet.-
Set the local proxy port and the following variables and provide proxy port number and node name:
export HOSTNAME_PORT=localhost:8001 export NODE_NAME=my-node-name curl -sSL "http://${HOSTNAME_PORT}/api/v1/nodes/${NODE_NAME}/proxy/configz"
Remediation
Method 1:
-
If modifying the Kubelet config file, edit the
/etc/kubernetes/kubelet/kubelet-config.json
file:{ "streamingConnectionIdleTimeout": "4h0m0s" }
-
You should ensure that the kubelet service file
/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf
does not specify a--streaming-connection-idle-timeout
argument because it would override the Kubelet config file. -
Based on your system, restart the kubelet service and check status:
systemctl daemon-reload systemctl restart kubelet.service systemctl status kubelet -l
Method 2:
-
If using executable arguments, edit the kubelet service file
/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf
on each worker node and add the below parameter at the end of the KUBELET_ARGS variable string:--streaming-connection-idle-timeout=4h0m0s
-
Based on your system, restart the kubelet service and check status:
systemctl daemon-reload systemctl restart kubelet.service systemctl status kubelet -l
Method 3:
-
If using the API configz endpoint, consider searching for the status of
"streamingConnectionIdleTimeout"
by extracting the live configuration from the nodes running kubelet.kubectl proxy --port=8001 & export HOSTNAME_PORT=localhost:8001 export NODE_NAME=ip-192.168.31.226.ec2.internal curl -sSL "http://${HOSTNAME_PORT}/api/v1/nodes/${NODE_NAME}/proxy/configz"
-
Based on your system, restart the kubelet service and check status:
systemctl daemon-reload systemctl restart kubelet.service systemctl status kubelet -l