使用sealos工具安装kubernetes集群

目录

• 一、安装前准备

  • 1.1、主机规划

  • 1.2、修改主机名

  • 1.3、关闭防火墙

  • 1.4、关闭SELinux

  • 1.5、关闭swap

  • 1.6、设置时间同步

• 二、部署k8s集群

  • 2.1、带calico网络插件安装

  • 2.2、不带网络插件安装

• 三、更换网络插件

正文

一、安装前准备

1.1、主机规划

1.2、修改主机名

按主机规划设备各主机的主机名，并在 /etc/hosts 文件中添加解析配置

#修改主机名
hostnamectl set-hostname k8s-master-1
<p>#修改/etc/hosts，添加以下配置
vim /etc/hosts
192.168.80.7    k8s-master-1
192.168.80.17   k8s-master-2
192.168.80.27   k8s-master-3
192.168.80.37   k8s-node-1

1.3、关闭防火墙

# 停止
systemctl stop firewalld.service</p>
<h1 id="禁用">禁用</h1>
<p>systemctl disable firewalld.service

1.4、关闭SELinux

setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/sysconfig/selinux

1.5、关闭swap

swapoff -a

1.6、设置时间同步

# 设置时区
timedatectl set-timezone Asia/Shanghai</p>
<h1 id="同步时间">同步时间</h1>
<p>yum install -y ntpdate
ntpdate time1.aliyun.com

二、部署k8s集群

2.1、带calico网络插件安装

(1) 下载并安装sealos，sealos是个golang的二进制工具，直接下载拷贝到bin目录即可, release页面也可下载，当时latest版本为v3.3.9，目前最新版本为v4.1.6，安装k8s集群的方式已不同，请参考官网，v3.3.9-rc11下载地址为：https://github.com/labring/sealos/releases/download/v3.3.9-rc.11/sealos_3.3.9-rc.11_linux_amd64.tar.gz

wget -c https://sealyun.oss-cn-beijing.aliyuncs.com/latest/sealos && chmod +x sealos && mv sealos /usr/bin

(2) 下载离线资源包（请自行下载）

#新建软件包存放目录
mkdir -pv /usr/local/soft/package
#下载软件包
kube1.18.14.tar.gz

(3) 安装集群

sealos init --passwd '123456' <br />
--master 192.168.80.7  --master 192.168.80.17  --master 192.168.80.27 <br />
--node 192.168.80.37  <br />
--pkg-url /usr/local/soft/package/kube1.18.14.tar.gz <br />
--version v1.18.14

参数含义

(4) 等待安装完成即可

(5) 其它命令

#增加master
sealos join --master 192.168.80.47 --master 192.168.80.57
sealos join --master 192.168.80.47-192.168.80.57  # 或者多个连续IP</p>
<p>#增加node
sealos join --node 192.168.80.47 --master 192.168.80.57
sealos join --node 192.168.80.47-192.168.80.57  # 或者多个连续IP</p>
<p>#删除指定master节点
sealos clean --master 192.168.80.47 --master 192.168.80.57
sealos clean --master 192.168.80.47-192.168.80.57  # 或者多个连续IP</p>
<p>#删除指定node节点
sealos clean --node 192.168.80.47 --node 192.168.80.57
sealos clean --node 192.168.80.47-192.168.80.57  # 或者多个连续IP</p>
<p>#清理集群
sealos clean --all</p>
<p>#备份集群
sealos etcd save

2.2、不带网络插件安装

(1) 下载并安装sealos，sealos是个golang的二进制工具，直接下载拷贝到bin目录即可, release页面也可下载

wget -c https://sealyun.oss-cn-beijing.aliyuncs.com/latest/sealos && chmod +x sealos && mv sealos /usr/bin

(2) 下载离线资源包（请自行下载）

#新建软件包存放目录
mkdir -pv /usr/local/soft/package
#下载软件包
kube1.18.14.tar.gz

(3) 安装集群

sealos init --passwd '123456' <br />
--master 192.168.80.7  --master 192.168.80.17  --master 192.168.80.27 <br />
--node 192.168.80.37  <br />
--without-cni <br />
--pkg-url /usr/local/soft/package/kube1.18.14.tar.gz <br />
--version v1.18.14</p>
<p>#使用flannel网络插件时，可以添加 --podcidr 10.244.0.0/16 参数，后续就不用改kube-flannel.ywl文件中的网段，而直接使用 kubectl apply -f kube-flannel.ywl 即可

(4) 下载cni网络插件工具

#下载
wget -c https://github.com/containernetworking/plugins/releases/download/v0.9.1/cni-plugins-linux-amd64-v0.9.1.tgz</p>
<p>#创建目录
mkdir -pv /opt/cni/bin</p>
<p>#解压
tar -xf cni-plugins-linux-amd64-v0.9.1.tgz -C /opt/cni/bin/

(5) 安装flannel插件

wget -c https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml</p>
<p>#修改kube-flannel.yml文件中的网段，如果和pod的网段不在同一网段，在pod内可能ping不通外网
vim kube-flannel.yml
#找到以下内容，将Network的值改为100.64.0.0/10，sealos默认安装的podcidr为100.64.0.0/24
net-conf.json: |
{
"Network": "10.244.0.0/16",	#修改此处
"Backend": {
"Type": "vxlan"
}
}</p>
<p>kubectl apply -f kube-flannel.yml

kube-flannel.yml 文件内容如下：

---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp.flannel.unprivileged
annotations:
seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
privileged: false
volumes:</p>
<ul>
<li>configMap</li>
<li>secret</li>
<li>emptyDir</li>
<li>hostPath
allowedHostPaths:</li>
<li>pathPrefix: "/etc/cni/net.d"</li>
<li>pathPrefix: "/etc/kube-flannel"</li>
<li>pathPrefix: "/run/flannel"
readOnlyRootFilesystem: false</li>
</ul>
<h1 id="Users and groups">Users and groups</h1>
<p>runAsUser:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
fsGroup:
rule: RunAsAny</p>
<h1 id="Privilege Escalation">Privilege Escalation</h1>
<p>allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false</p>
<h1 id="Capabilities">Capabilities</h1>
<p>allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
defaultAddCapabilities: []
requiredDropCapabilities: []</p>
<h1 id="Host namespaces">Host namespaces</h1>
<p>hostPID: false
hostIPC: false
hostNetwork: true
hostPorts:</p>
<ul>
<li>min: 0
max: 65535</li>
</ul>
<h1 id="SELinux">SELinux</h1><h2 id="seLinux:
# SELinux is unused in CaaSP
rule: 'RunAsAny'">seLinux:
# SELinux is unused in CaaSP
rule: 'RunAsAny'</h2>
<p>kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: flannel
rules:</p>
<ul>
<li>apiGroups: ['extensions']
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames: ['psp.flannel.unprivileged']</li>
<li>apiGroups:
<ul>
<li>""
resources:</li>
<li>pods
verbs:</li>
<li>get</li>
</ul>
</li>
<li>apiGroups:
<ul>
<li>""
resources:</li>
<li>nodes
verbs:</li>
<li>list</li>
<li>watch</li>
</ul>
</li>
<li>apiGroups:
<ul>
<li>""
resources:</li>
<li>nodes/status
verbs:</li>
<li>patch</li>
</ul>
</li>
</ul>
<hr />
<p>kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flannel
subjects:</p>
<ul>
<li>kind: ServiceAccount
name: flannel
namespace: kube-system</li>
</ul>
<hr />
<h2 id="apiVersion: v1
kind: ServiceAccount
metadata:
name: flannel
namespace: kube-system">apiVersion: v1
kind: ServiceAccount
metadata:
name: flannel
namespace: kube-system</h2><h2 id="kind: ConfigMap
apiVersion: v1
metadata:
name: kube-flannel-cfg
namespace: kube-system
labels:
tier: node
app: flannel
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "100.64.0.0/10",
"Backend": {
"Type": "vxlan"
}
}">kind: ConfigMap
apiVersion: v1
metadata:
name: kube-flannel-cfg
namespace: kube-system
labels:
tier: node
app: flannel
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "100.64.0.0/10",
"Backend": {
"Type": "vxlan"
}
}</h2>
<p>apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: In
values:
- linux
hostNetwork: true
priorityClassName: system-node-critical
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.14.0
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.14.0
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN", "NET_RAW"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg

(6) 下载并导入镜像

​ 因国内网络原因，quay.io镜像可能拉取不到，可以用中科大镜像，然后修改tag即可，如果是内网，建议在有网的主机上下载，然后上传导入。

#拉取镜像
docker pull quay.mirrors.ustc.edu.cn/coreos/flannel:v0.14.0</p>
<p>#打标签
docker tag  quay.mirrors.ustc.edu.cn/coreos/flannel:v0.14.0 quay.io/coreos/flannel:v0.14.0</p>
<p>#导出
docker save quay.io/coreos/flannel:v0.14.0 | gzip > flannel-014.tgz</p>
<p>#导入
docker load -i flannel-014.tgz

三、更换网络插件

​ sealos默认使用的calico插件，有些云平台可能不支持，导致主节点上的NodePort无法telnet，也无法访问。刚可能需要更换网络插件为flannel。

(1) 清空iptables规则

iptables -F &&  iptables -X &&  iptables -F -t nat &&  iptables -X -t nat

(2) 停用tunl0虚拟网卡

ip link set tunl0 down

(3) 删除calico的一些文件

rm -f /etc/cni/net.d/*
rm -rf /run/calico/

(4) 替换cni网络插件工具

#下载
wget -c https://github.com/containernetworking/plugins/releases/download/v0.9.1/cni-plugins-linux-amd64-v0.9.1.tgz</p>
<p>#备份
mkdir /root/cni-bak
mv /opt/cni/bin/* cni-bak/</p>
<p>#解压
tar -xf cni-plugins-linux-amd64-v0.9.1.tgz -C /opt/cni/bin/

(5) 安装flannel插件

wget -c https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml</p>
<p>kubectl apply -f kube-flannel.yml

(6) 重启docker和kubelet

systemctl restart docker
systemctl restart kubelet

(7) 修改iptables FORWARD 链中访问规则（如有需要）

iptables -P FORWARD ACCEPT</p>
<p>#或开启内核数据包转发参数
net.ipv4.ip_forward = 1