一、集群信息
1. 节点规划
部署k8s集群的节点按照用途可以划分为如下2类角色:
-
master:集群的master节点,集群的初始化节点,基础配置不低于2C4G -
slave:集群的slave节点,可以多台,基础配置不低于2C4G
本例为了演示slave节点的添加,会部署一台master+2台slave,节点规划如下:
主机名 | 节点ip | 角色 | 部署组件 |
---|---|---|---|
k8s-master | 192.168.2.14 | master | etcd, kube-apiserver, kube-controller-manager, kubectl, kubeadm, kubelet, kube-proxy, flannel |
k8s-slave1 | 192.168.2.15 | slave | kubectl, kubelet, kube-proxy, flannel |
k8s-slave2 | 192.168.2.16 | slave | kubectl, kubelet, kube-proxy, flannel |
2. 组件版本
组件 | 版本 | 说明 |
---|---|---|
CentOS | 7.6.1810 | |
Kernel | Linux 3.10.0-1062.9.1.el7.x86_64 | |
etcd | 3.4.13-0 | 使用容器方式部署,默认数据挂载到本地路径 |
coredns | 1.7.0 | |
kubeadm | v1.20.0 | |
kubectl | v1.20.0 | |
kubelet | v1.20.0 | |
kube-proxy | v1.20.0 | |
pause | 3.2 |
二、安装前准备工作
1. 设置hosts解析
操作节点:所有节点(k8s-master,k8s-slave
)均需执行
-
修改hostnamehostname必须只能包含小写字母、数字、”,”、”-“,且开头结尾必须是小写字母或数字
# 在master节点
$ hostnamectl set-hostname k8s-master #设置master节点的hostname
# 在slave-1节点
$ hostnamectl set-hostname k8s-slave1 #设置slave1节点的hostname
# 在slave-2节点
$ hostnamectl set-hostname k8s-slave2 #设置slave2节点的hostname
-
添加hosts解析
$ cat >>/etc/hosts<<EOF
192.168.2.14 k8s-master
192.168.2.15 k8s-slave1
192.168.2.16 k8s-slave2
EOF
2. 调整系统配置
操作节点:所有的master和slave节点(k8s-master,k8s-slave
)需要执行
本章下述操作均以k8s-master为例,其他节点均是相同的操作(ip和hostname的值换成对应机器的真实值)
-
设置安全组开放端口
如果节点间无安全组限制(内网机器间可以任意访问),可以忽略,否则,至少保证如下端口可通:k8s-master节点:TCP:6443,2379,2380,60080,60081UDP协议端口全部打开 k8s-slave节点:UDP协议端口全部打开
-
设置iptables
iptables -P FORWARD ACCEPT
-
关闭swap
swapoff -a
# 防止开机自动挂载 swap 分区
sed -i '/ swap / s/^(.*)$/#1/g' /etc/fstab
-
关闭selinux和防火墙
sed -ri 's#(SELINUX=).*#1disabled#' /etc/selinux/config
setenforce 0
systemctl disable firewalld && systemctl stop firewalld
-
修改内核参数
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward=1
vm.max_map_count=262144
EOF
modprobe br_netfilter
sysctl -p /etc/sysctl.d/k8s.conf
-
设置yum源
$ curl -o /etc/yum.repos.d/Centos-7.repo http://mirrors.aliyun.com/repo/Centos-7.repo
$ curl -o /etc/yum.repos.d/Docker-ce.repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
$ cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
$ yum clean all && yum makecache
3. 安装docker
操作节点:所有节点
## 查看所有的可用版本
$ yum list docker-ce --showduplicates | sort -r
##安装旧版本 yum install docker-ce-cli-18.09.9-3.el7 docker-ce-18.09.9-3.el7
## 安装源里最新版本
$ yum install docker-ce
## 配置docker加速
$ mkdir -p /etc/docker
vi /etc/docker/daemon.json
{
"insecure-registries": [
"192.168.2.14:5000"
],
"registry-mirrors" : [
"https://8xpk5wnt.mirror.aliyuncs.com"
]
}
## 启动docker
$ systemctl enable docker && systemctl start docker
三、部署kubernetes
1. 安装 kubeadm, kubelet 和 kubectl
操作节点:所有的master和slave节点(k8s-master,k8s-slave
) 需要执行
$ yum install -y kubelet-1.20.0 kubeadm-1.20.0 kubectl-1.20.0 --disableexcludes=kubernetes
## 查看kubeadm 版本
$ kubeadm version
## 设置kubelet开机启动
$ systemctl enable kubelet
2. 初始化配置文件
操作节点:只在master节点(k8s-master
)执行
$ kubeadm config print init-defaults > kubeadm.yaml
$ cat kubeadm.yaml
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.2.14 # apiserver地址,因为单master,所以配置master的节点内网IP
bindPort: 6443
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: k8s-master # 默认读取当前master节点的hostname
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers # 修改成阿里镜像源
kind: ClusterConfiguration
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16 # Pod 网段,flannel插件需要使用这个网段
serviceSubnet: 10.96.0.0/12
scheduler: {}
对于上面的资源清单的文档比较杂,要想完整了解上面的资源对象对应的属性,可以查看对应的 godoc 文档,地址。
3. 提前下载镜像
操作节点:只在master节点(k8s-master
)执行
# 查看需要使用的镜像列表,若无问题,将得到如下列表
$ kubeadm config images list --config kubeadm.yaml
registry.aliyuncs.com/google_containers/kube-apiserver:v1.20.0
registry.aliyuncs.com/google_containers/kube-controller-manager:v1.20.0
registry.aliyuncs.com/google_containers/kube-scheduler:v1.20.0
registry.aliyuncs.com/google_containers/kube-proxy:v1.20.0
registry.aliyuncs.com/google_containers/pause:3.2
registry.aliyuncs.com/google_containers/etcd:3.4.13-0
registry.aliyuncs.com/google_containers/coredns:1.7.0
# 提前下载镜像到本地
$ kubeadm config images pull --config kubeadm.yaml
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.20.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.20.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.20.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.20.0
[config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.2
[config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.4.13-0
[config/images] Pulled registry.aliyuncs.com/google_containers/coredns:1.7.0
重要更新:如果出现不可用的情况,请使用如下方式来代替:
-
还原kubeadm.yaml的imageRepository
...
imageRepository: k8s.gcr.io
...
## 查看使用的镜像源
kubeadm config images list --config kubeadm.yaml
k8s.gcr.io/kube-apiserver:v1.20.0
k8s.gcr.io/kube-controller-manager:v1.20.0
k8s.gcr.io/kube-scheduler:v1.20.0
k8s.gcr.io/kube-proxy:v1.20.0
k8s.gcr.io/pause:3.2
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns:1.7.0 -
使用docker hub中的镜像源来下载,注意上述列表中要加上处理器架构,通常我们使用的虚拟机都是amd64
$ docker pull mirrorgooglecontainers/kube-scheduler-amd64:v1.20.0
$ docker pull mirrorgooglecontainers/etcd-amd64:3.4.13-0
...
4. 初始化master节点
操作节点:只在master节点(k8s-master
)执行
kubeadm init --config kubeadm.yaml
若初始化成功后,最后会提示如下信息:
...
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.2.14:6443 --token abcdef.0123456789abcdef
--discovery-token-ca-cert-hash sha256:1c4305f032f4bf534f628c32f5039084f4b103c922ff71b12a5f0f98d1ca9a4f
接下来按照上述提示信息操作,配置kubectl客户端的认证
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
⚠️注意:此时使用 kubectl get nodes查看节点应该处于notReady状态,因为还未配置网络插件
若执行初始化过程中出错,根据错误信息调整后,执行kubeadm reset后再次执行init操作即可
5. 添加slave节点到集群中
操作节点:所有的slave节点(k8s-slave
)需要执行 在每台slave节点,执行如下命令,该命令是在kubeadm init成功后提示信息中打印出来的,需要替换成实际init后打印出的命令。
kubeadm join 192.168.2.14:6443 --token abcdef.0123456789abcdef
--discovery-token-ca-cert-hash sha256:1c4305f032f4bf534f628c32f5039084f4b103c922ff71b12a5f0f98d1ca9a4f
6. 安装flannel插件
操作节点:只在master节点(k8s-master
)执行
-
下载flannel的yaml文件
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
-
修改配置,指定网卡名称,大概在文件的190行,添加一行配置:
$ vi kube-flannel.yml
...
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.11.0-amd64
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
- --iface=eth0 # 如果机器存在多网卡的话,指定内网网卡的名称,默认不指定的话会找第一块网
resources:
requests:
cpu: "100m"
...
-
执行安装flannel网络插件
# 先拉取镜像,此过程国内速度比较慢
# $ docker pull quay.io/coreos/flannel:v0.11.0-amd64 这步可以忽略
# 执行flannel安装
$ kubectl apply -f kube-flannel.yml
7. 设置master节点是否可调度(可选)
操作节点:k8s-master
默认部署成功后,master节点无法调度业务pod,如需设置master节点也可以参与pod的调度,需执行:
$ kubectl taint node k8s-master node-role.kubernetes.io/master:NoSchedule-
8. 验证集群
操作节点:在master节点(k8s-master
)执行
$ kubectl get nodes #观察集群节点是否全部Ready
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 22h v1.20.0
k8s-slave Ready <none> 22h v1.20.0
如果没有Ready $ kubectl get -n kube-system pod
[root@k8s-master k8s-install]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7f89b7bc75-dwr4j 0/1 pending 0 17h
coredns-7f89b7bc75-swn5l 0/1 pending 0 17h
etcd-k8s-master 1/1 Running 0 17h
kube-apiserver-k8s-master 1/1 Running 0 17h
kube-controller-manager-k8s-master 1/1 Running 2 17h
kube-proxy-cdgql 1/1 Running 0 17h
kube-proxy-cts6g 1/1 Running 0 16h
kube-proxy-xgwkf 1/1 Running 0 17h
kube-scheduler-k8s-master 1/1 Running 2 17h
在slave上查看 $ systemctl status kubelet -l 报以下错误
runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
可以查看有没有该文件,没有就创建
$ cat /etc/cni/net.d/10-flannel.conflist
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
$ systemctl daemon-reload
$ systemctl restart kubelet
还不行就直接删除该节点,重新加入
创建测试Nginx服务
$ kubectl run test-nginx --image=nginx:alpine
查看pod是否创建成功,并访问pod ip测试是否可用
$ kubectl get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
test-nginx-5bd8859b98-5nnnw 1/1 Running 0 9s 10.244.1.2 k8s-slave1 <none> <none>
$ curl 10.244.1.2
...
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
9. k8s从私有仓库拉取镜像部署myblog
-环境介绍
#myblog的两个镜像(myblog:v1、MySQL:5.7)已经上传至私有镜像仓库registry:v2
$ curl http://192.168.2.14:5000/v2/_catalog
{"repositories":["myblog","mysql"]}
#创建myblog的yaml文件
$ cat myblog.yaml
apiVersion: v1
kind: Pod
metadata:
name: myblog
namespace: zxc
labels:
component: myblog
spec:
containers:
- name: myblog
image: 192.168.2.14:5000/myblog:v1
env:
- name: MYSQL_HOST # 指定root用户的用户名
value: "127.0.0.1"
- name: MYSQL_PASSWD
value: "123456"
ports:
- containerPort: 8002
- name: mysql
image: 192.168.2.14:5000/mysql:5.7
ports:
- containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
value: "123456"
- name: MYSQL_DATABASE
value: "myblog"
## 创建namespace, namespace是逻辑上的资源池
$ kubectl create namespace zxc
## 使用指定文件创建Pod
$ kubectl create -f myblog.yaml
## 查看pod,可以简写po
## 所有的操作都需要指定namespace,如果是在default命名空间下,则可以省略
$ kubectl -n zxc get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
myblog 2/2 pending 0 3m 10.244.1.9 k8s-slave1
## 看见状态为pending,查看详细内容和事件
$ kubectl describe -n zxc pod myblog
##错误信息
Failed to pull image: rpc error: code = Unknown desc = failed to pull and unpack image. dial tcp: lookup read: connection refused
## 原因:kubectl apply -f web.yaml 时,不能正确的获取 docker login 的认证信息,手工创建了一个secret,使用 secret 来指定 pull 权限:
harbor注册
k8s设置参考
## 创建 secret,此处在harbor上新建了一个用户 manager:Harbor123
#方法一:在命令行上提供凭证
kubectl create secret docker-registry registry-harbor --namespace=zxc
--docker-server=harbor.unchch.com --docker-username=manager
--docker-password=Harbor123 --docker-email=g1003287899@gmail.com
## 查看创建的 secret
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-rqf8z kubernetes.io/service-account-token 3 9d
registry-harbor kubernetes.io/dockerconfigjson 1 3h24m
$kubectl describe secrets registry-harbor
Name: registry-harbor
Namespace: default
Labels: <none>
Annotations: <none>
Type: kubernetes.io/dockerconfigjson
Data
====
.dockerconfigjson: 141 bytes
## 在 yaml 文件对应的部分指定secret:
spec:
containers:
- name: myblog
image: 192.168.2.14:5000/myblog:v1
env:
- name: MYSQL_HOST # 指定root用户的用户名
value: "127.0.0.1"
- name: MYSQL_PASSWD
value: "123456"
ports:
- containerPort: 8002
imagePullSecrets:
-name: registry-harbor
#方法二:创建一个基于现有凭证的 Kubernetes
集群使用 kubernetes.io/dockerconfigjson 类型的 Secret 来通过镜像仓库的身份验证,进而提取私有镜像。
如果你已经运行了 docker login 命令,你可以复制该镜像仓库的凭证到 Kubernetes:
kubectl create secret generic regcred
--from-file=.dockerconfigjson=.docker/config.json
--type=kubernetes.io/dockerconfigjson
再次部署,一切正常。
10. 部署dashboard
-
部署服务
# 推荐使用下面这种方式
$ wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta5/aio/deploy/recommended.yaml
$ vi recommended.yaml
# 修改Service为NodePort类型
......
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
ports:
- port: 443
targetPort: 8443
selector:
k8s-app: kubernetes-dashboard
type: NodePort # 加上type=NodePort变成NodePort类型的服务
......
-
查看访问地址,本例为30133端口
kubectl -n kubernetes-dashboard get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.105.62.124 <none> 8000/TCP 31m
kubernetes-dashboard NodePort 10.103.74.46 <none> 443:30133/TCP 31m
-
使用浏览器访问 https://62.234.133.177:30133,其中62.234.133.177为master节点的外网ip地址,chrome目前由于安全限制,测试访问不了,使用firefox可以进行访问。
-
创建ServiceAccount进行访问
$ vi admin.conf
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: admin
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: admin
namespace: kubernetes-dashboard
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin
namespace: kubernetes-dashboard
$ kubectl -n kubernetes-dashboard get secret |grep admin-token
admin-token-fqdpf kubernetes.io/service-account-token 3 7m17s
# 使用该命令拿到token,然后粘贴到
$ kubectl -n kubernetes-dashboard get secret admin-token-fqdpf -o jsonpath={.data.token}|base64 -d
eyJhbGciOiJSUzI1NiIsImtpZCI6Ik1rb2xHWHMwbWFPMjJaRzhleGRqaExnVi1BLVNRc2txaEhETmVpRzlDeDQifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi10b2tlbi1mcWRwZiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJhZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjYyNWMxNjJlLTQ1ZG...
11. 清理环境
如果你的集群安装过程中遇到了其他问题,我们可以使用下面的命令来进行重置:
$ kubeadm reset
$ ifconfig cni0 down && ip link delete cni0
$ ifconfig flannel.1 down && ip link delete flannel.1
$ rm -rf /var/lib/cni/
原文始发于微信公众号(运维库):Kubernetes 安装-单Master
版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 举报,一经查实,本站将立刻删除。
文章由极客之音整理,本文链接:https://www.bmabk.com/index.php/post/64267.html