kubernetes 1.12.1 高可用安装之部署kubelet

二进制包准备 

1.将软件包从master01复制到各node节点中去。

[root@master01 ~]# scp kubernetes/server/bin/{kubelet,kube-proxy} node01:/opt/kubernetes/bin/
[root@master01 ~]# scp kubernetes/server/bin/{kubelet,kube-proxy} node02:/opt/kubernetes/bin/
[root@master01 ~]# scp kubernetes/server/bin/{kubelet,kube-proxy} node03:/opt/kubernetes/bin/

2.创建角色绑定

[root@master01 ~]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap
clusterrolebinding "kubelet-bootstrap" created

3.创建 kubelet bootstrapping kubeconfig 文件 设置集群参数

[root@master01 ~]# kubectl config set-cluster kubernetes \
   --certificate-authority=/opt/kubernetes/ssl/ca.pem \
   --embed-certs=true \
   --server=https://10.80.4.200:6443 \
   --kubeconfig=bootstrap.kubeconfig
Cluster "kubernetes" set.

设置客户端认证参数

[root@master01 ~]# kubectl config set-credentials kubelet-bootstrap \
   --token=cdacf2b5563c36ebbb15edd7d46fc857 \
   --kubeconfig=bootstrap.kubeconfig\
User "kubelet-bootstrap" set.

设置上下文参数

[root@master01 ~]# kubectl config set-context default \
   --cluster=kubernetes \
   --user=kubelet-bootstrap \
   --kubeconfig=bootstrap.kubeconfig
Context "default" created.

选择默认上下文

[root@master01 ~]# kubectl config use-context default --kubeconfig=bootstrap.kubeconfig
Switched to context "default".
[root@master01 ~]# scp bootstrap.kubeconfig node01:/opt/kubernetes/cfg/
[root@master01 ~]# scp bootstrap.kubeconfig node02:/opt/kubernetes/cfg/
[root@master01 ~]# scp bootstrap.kubeconfig node03:/opt/kubernetes/cfg/

部署kubelet 

1.设置CNI支持

[root@node01 ~]# mkdir -p /etc/cni/net.d
[root@node01 ~]# vim /etc/cni/net.d/10-default.conf
{
        "name": "flannel",
        "type": "flannel",
        "delegate": {
            "bridge": "docker0",
            "isDefaultGateway": true,
            "mtu": 1400
        }
}

[root@node01 ~]# scp /etc/cni/net.d/10-default.conf node02:/etc/cni/net.d/
[root@node01 ~]# scp /etc/cni/net.d/10-default.conf node03:/etc/cni/net.d/

2.创建kubelet目录

[root@node01 ~]# mkdir /var/lib/kubelet

3.创建kubelet服务配置

[root@node01 ~]# vim /usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
ExecStart=/opt/kubernetes/bin/kubelet \
  --address=10.80.4.203 \
  --hostname-override=10.80.4.203 \
  --pod-infra-container-image=mirrorgooglecontainers/pause-amd64:3.0 \
  --experimental-bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \
  --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \
  --cert-dir=/opt/kubernetes/ssl \
  --network-plugin=cni \
  --cni-conf-dir=/etc/cni/net.d \
  --cni-bin-dir=/opt/kubernetes/bin/cni \
  --cluster-dns=10.1.0.2 \
  --cluster-domain=cluster.local. \
  --hairpin-mode hairpin-veth \
  --allow-privileged=true \
  --fail-swap-on=false \
  --logtostderr=true \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log
Restart=on-failure
RestartSec=5
[root@node01 ~]# scp /usr/lib/systemd/system/kubelet.service node02:/usr/lib/systemd/system #注意修改节点信息
[root@node01 ~]# scp /usr/lib/systemd/system/kubelet.service node03:/usr/lib/systemd/system #注意修改节点信息

4.启动Kubelet

[root@node01 ~]# systemctl daemon-reload
[root@node01 ~]# systemctl enable kubelet
[root@node01 ~]# systemctl start kubelet

5.查看服务状态

[root@node01]# systemctl status kubelet

6.查看csr请求 注意是在linux-node1上执行。

[root@master01 ~]# kubectl get csr
NAME                                                   AGE     REQUESTOR           CONDITION
node-csr-4_Q4dHo68CyM4H-gwS3t-o5JXzf-BW-sBw_7vpEIdss   2m41s   kubelet-bootstrap   Pending
node-csr-P_F1Lz8tX_C99vyC3mjp5voaINwO1PnZlo_aynrGHhw   3m6s    kubelet-bootstrap   Pending
node-csr-flAmT1K0Pe6KTw9U5guw0-zUhf5Djt0gLQfb72Grfjw   2m47s   kubelet-bootstrap   Pending

7.批准kubelet 的 TLS 证书请求

[root@master01 ~]# kubectl get csr|grep 'Pending' | awk 'NR>0{print $1}'| xargs kubectl certificate approve

执行完毕后，查看节点状态已经是Ready的状态了

[root@master01 ~]# kubectl get node 

NAME          STATUS   ROLES    AGE   VERSION

10.80.4.203   Ready    <none>   26s   v1.12.1

10.80.4.204   Ready    <none>   26s   v1.12.1

10.80.4.205   Ready    <none>   26s   v1.12.1

部署Kubernetes Proxy

1.配置kube-proxy使用LVS

[root@node01 ~]# yum install -y ipvsadm ipset conntrack

[root@node02 ~]# yum install -y ipvsadm ipset conntrack

[root@node03 ~]# yum install -y ipvsadm ipset conntrack

2.创建 kube-proxy 证书请求

[root@master01 ~]# cd ssl
[root@master01 ~]# vim kube-proxy-csr.json
{
  "CN": "system:kube-proxy",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "ShenZheng",
      "L": "ShenZheng",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

3.生成证书

[root@master01 ~]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
   -ca-key=/opt/kubernetes/ssl/ca-key.pem \
   -config=/opt/kubernetes/ssl/ca-config.json \
   -profile=kubernetes  kube-proxy-csr.json | cfssljson -bare kube-proxy

4.分发证书到所有Node节点

[root@master01 ssl]# cp kube-proxy*.pem /opt/kubernetes/ssl/
[root@master01 ssl]# scp kube-proxy*.pem node01:/opt/kubernetes/ssl/
[root@master01 ssl]# scp kube-proxy*.pem node02:/opt/kubernetes/ssl/
[root@master01 ssl]# scp kube-proxy*.pem node03:/opt/kubernetes/ssl/

5.创建kube-proxy配置文件

[root@master01 ~]# kubectl config set-cluster kubernetes \
   --certificate-authority=/opt/kubernetes/ssl/ca.pem \
   --embed-certs=true \
   --server=https://10.80.4.200:6443 \
   --kubeconfig=kube-proxy.kubeconfig
Cluster "kubernetes" set.

[root@master01 ~]# kubectl config set-credentials kube-proxy \
   --client-certificate=/opt/kubernetes/ssl/kube-proxy.pem \
   --client-key=/opt/kubernetes/ssl/kube-proxy-key.pem \
   --embed-certs=true \
   --kubeconfig=kube-proxy.kubeconfig
User "kube-proxy" set.

[root@master01 ~]# kubectl config set-context default \
   --cluster=kubernetes \
   --user=kube-proxy \
   --kubeconfig=kube-proxy.kubeconfig
Context "default" created.

[root@mastre01 ~]# kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
Switched to context "default".

6.分发kubeconfig配置文件

[root@master01 ~]# scp kube-proxy.kubeconfig node01:/opt/kubernetes/cfg/
[root@master01 ~]# scp kube-proxy.kubeconfig node02:/opt/kubernetes/cfg/
[root@master01 ~]# scp kube-proxy.kubeconfig node03:/opt/kubernetes/cfg/

7.创建kube-proxy服务配置

[root@node01 ~]# mkdir /var/lib/kube-proxy

[root@node01 ~]# vim /usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
WorkingDirectory=/var/lib/kube-proxy
ExecStart=/opt/kubernetes/bin/kube-proxy \
  --bind-address=10.80.4.203 \
  --hostname-override=10.80.4.203 \
  --kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig \
--masquerade-all \
  --feature-gates=SupportIPVSProxyMode=true \
  --proxy-mode=ipvs \
  --ipvs-min-sync-period=5s \
  --ipvs-sync-period=5s \
  --ipvs-scheduler=rr \
  --logtostderr=true \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log

Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
[root@node01 ~]# scp /usr/lib/systemd/system/kube-proxy.service node02:/usr/lib/systemd/system/ #注意要修改节点信息
[root@node01 ~]# scp /usr/lib/systemd/system/kube-proxy.service node03:/usr/lib/systemd/system/ #注意要修改节点信息
8.启动Kubernetes Proxy
[root@node01 ~]# systemctl daemon-reload
[root@node01 ~]# systemctl enable kube-proxy
[root@node01 ~]# systemctl start kube-proxy

9.查看服务状态 查看kube-proxy服务状态

[root@node01 ~]# systemctl status kube-proxy

检查LVS状态
[root@node01 ~]#  ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.1.0.1:443 rr
  -> 10.80.4.201:6443             Masq    1      0          0         
  -> 10.80.4.202:6443             Masq    1      0          0        

如果你在三台实验机器都安装了kubelet和proxy服务，使用下面的命令可以检查状态：

[root@master01 ~]# kubectl get node
NAME          STATUS   ROLES    AGE   VERSION
10.80.4.203   Ready    <none>   17m   v1.12.1
10.80.4.204   Ready    <none>   17m   v1.12.1
10.80.4.205   Ready    <none>   17m   v1.12.1