{"id":4828,"date":"2024-07-17T16:44:46","date_gmt":"2024-07-17T08:44:46","guid":{"rendered":"https:\/\/www.ookangzheng.com\/?p=4828"},"modified":"2024-07-17T17:40:18","modified_gmt":"2024-07-17T09:40:18","slug":"turn-off-taint-on-k0s-kubernetes","status":"publish","type":"post","link":"https:\/\/www.ookangzheng.com\/turn-off-taint-on-k0s-kubernetes\/","title":{"rendered":"Turn off taint on k0s kubernetes"},"content":{"rendered":"\n<p>Here is the error when I run Nginx on <code>k0s <\/code><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u276f kubectl get pods -A\nNAMESPACE     NAME                              READY   STATUS    RESTARTS   AGE\ndefault       k0s-nginx-576c6b7b6-2kn87         0\/1     Pending   0          8m53s\nkube-system   coredns-85c69f454c-zwl4t          1\/1     Running   0          9d\nkube-system   konnectivity-agent-whswd          1\/1     Running   0          9d\nkube-system   kube-proxy-t998n                  1\/1     Running   0          9d\nkube-system   kube-router-pb2fd                 1\/1     Running   0          9d\nkube-system   metrics-server-5cd4986bbc-sjpz7   1\/1     Running   0          9d\n\u276f kubectl describe pod k0s-nginx-576c6b7b6-2kn87\nName:             k0s-nginx-576c6b7b6-2kn87\nNamespace:        default\nPriority:         0\nService Account:  default\nNode:             &lt;none&gt;\nLabels:           app=nginx\n                  pod-template-hash=576c6b7b6\nAnnotations:      &lt;none&gt;\nStatus:           Pending\nIP:\nIPs:              &lt;none&gt;\nControlled By:    ReplicaSet\/k0s-nginx-576c6b7b6\nContainers:\n  nginx:\n    Image:        nginx:latest\n    Port:         80\/TCP\n    Host Port:    0\/TCP\n    Environment:  &lt;none&gt;\n    Mounts:\n      \/var\/run\/secrets\/kubernetes.io\/serviceaccount from kube-api-access-t58ld (ro)\nConditions:\n  Type           Status\n  PodScheduled   False\nVolumes:\n  kube-api-access-t58ld:\n    Type:                    Projected (a volume that contains injected data from multiple sources)\n    TokenExpirationSeconds:  3607\n    ConfigMapName:           kube-root-ca.crt\n    ConfigMapOptional:       &lt;nil&gt;\n    DownwardAPI:             true\nQoS Class:                   BestEffort\nNode-Selectors:              &lt;none&gt;\nTolerations:                 node.kubernetes.io\/not-ready:NoExecute op=Exists for 300s\n                             node.kubernetes.io\/unreachable:NoExecute op=Exists for 300s\nEvents:\n  Type     Reason            Age    From               Message\n  ----     ------            ----   ----               -------\n  Warning  FailedScheduling  9m10s  default-scheduler  0\/1 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io\/master: }. preemption: 0\/1 nodes are available: 1 Preemption is not helpful for scheduling.\n  Warning  FailedScheduling  3m52s  default-scheduler  0\/1 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io\/master: }. preemption: 0\/1 nodes are available: 1 Preemption is not helpful for scheduling<\/code><\/pre>\n\n\n\n<p>When I played with <code>k0s<\/code> on bare metal server, I realized that k0s control plane assigned <code>node-role.kubernetes.io\/master:NoExecute<\/code> taint automatically. Therefore I have to disable it.<\/p>\n\n\n\n<p>Here is the simple Nginx yaml for testing<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl apply -f - &lt;&lt;EOF\napiVersion: apps\/v1\nkind: Deployment\nmetadata:\n  name: k0s-nginx\nspec:\n  selector:\n    matchLabels:\n      app: nginx\n  replicas: 1\n  template:\n    metadata:\n      labels:\n        app: nginx\n    spec:\n      containers:\n      - name: nginx\n        image: nginx:latest\n        ports:\n        - containerPort: 80\nEOF<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Solution<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Manually taint your master node<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl taint nodes \"xxx.xxx.com\" node-role.kubernetes.io\/master:NoSchedule<\/code><\/pre>\n\n\n\n<p>After you apply the taint, drain the master node so the pods scheduled on it shift to the worker node.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl drain \"xxx.xxx.com\" --ignore-daemonsets --delete-emptydir-data\n\n> node\/xxx.xxx.com already cordoned\nWarning: ignoring DaemonSet-managed Pods: kube-system\/konnectivity-agent-84mfn, kube-system\/kube-proxy-fzztz, kube-system\/kube-router-8mczs\nevicting pod kube-system\/metrics-server-5cd4986bbc-sv4bj\nevicting pod kube-system\/coredns-85c69f454c-zk9x5\nevicting pod default\/k0s-nginx-576c6b7b6-wmms5\npod\/k0s-nginx-576c6b7b6-wmms5 evicted\npod\/metrics-server-5cd4986bbc-sv4bj evicted\npod\/coredns-85c69f454c-zk9x5 evicted\nnode\/xxx.xxx.com drained<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Solution 2 – Reset the control plane<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code># Stop the k0s\nk0s stop\n\n# Reset the k0s\nk0s reset\n\n# Start the k0s\nk0s install controller --enable-worker --no-taints -c k0s.yaml<\/code><\/pre>\n\n\n\n<p>If you dont have <code>k0s.yaml<\/code>, you have to export it first.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>apt-get install -y kubectl\ncurl -sSLf https:\/\/get.k0s.sh | sudo sh\nk0s start\nk0s default-config &gt; k0s.yaml\nk0s stop &amp;&amp; k0s reset\n\n# Print control plane credential\ncat \/var\/lib\/k0s\/pki\/admin.conf<\/code><\/pre>\n\n\n\n<p>References:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><a href=\"https:\/\/docs.k0sproject.io\/stable\/worker-node-config\/?h=taint#taints\">https:\/\/docs.k0sproject.io\/stable\/worker-node-config\/?h=taint#taints<\/a><\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Here is the error when I run Nginx on k0s When I played with k0s on bare metal server, I&hellip;<\/p>\n","protected":false},"author":1,"featured_media":4831,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[37,32],"tags":[],"class_list":["post-4828","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-linux","category-technology"],"_links":{"self":[{"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/posts\/4828","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/comments?post=4828"}],"version-history":[{"count":4,"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/posts\/4828\/revisions"}],"predecessor-version":[{"id":4835,"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/posts\/4828\/revisions\/4835"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/media\/4831"}],"wp:attachment":[{"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/media?parent=4828"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/categories?post=4828"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ookangzheng.com\/wp-json\/wp\/v2\/tags?post=4828"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}