拓冰网站优化 
1. 为什么选择二进制部署Kubernetes在Rocky Linux上部署Kubernetes集群时大多数人会直接使用kubeadm等工具但二进制部署才是真正理解Kubernetes组件交互的绝佳方式。我最近在生产环境用二进制方式部署了Kubernetes v1.33集群整个过程踩了不少坑也积累了一些实战经验。二进制部署最大的优势在于可控性。你可以精确控制每个组件的版本、启动参数和部署位置。这对于需要定制化Kubernetes的企业环境特别有用。比如我们曾经遇到需要修改kube-apiserver的审计策略二进制部署就能轻松实现而kubeadm方式则需要折腾半天。提示二进制部署适合对Kubernetes有较深理解的中高级用户如果是初学者建议先用kubeadm熟悉基本概念。2. 环境准备与系统配置2.1 Rocky Linux基础配置我选择Rocky Linux 9作为操作系统这是RHEL的完美替代品特别适合企业级部署。以下是必须完成的系统配置网络配置# 设置静态IP以ens192为例 nmcli con mod ens192 ipv4.addresses 192.168.1.100/24 nmcli con mod ens192 ipv4.gateway 192.168.1.1 nmcli con mod ens192 ipv4.dns 8.8.8.8 8.8.4.4 nmcli con mod ens192 ipv4.method manual nmcli con up ens192主机名与hosts配置hostnamectl set-hostname k8s-master01 echo 192.168.1.100 k8s-master01 /etc/hosts echo 192.168.1.101 k8s-node01 /etc/hosts关闭防火墙和SELinux生产环境需谨慎systemctl stop firewalld systemctl disable firewalld setenforce 0 sed -i s/SELINUXenforcing/SELINUXdisabled/g /etc/selinux/config2.2 内核参数优化Kubernetes对Linux内核有特殊要求必须调整以下参数cat /etc/sysctl.d/k8s.conf EOF net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 net.ipv4.ip_forward 1 vm.swappiness 0 EOF sysctl --system注意如果遇到bridge-nf-call报错需要先加载br_netfilter模块modprobe br_netfilter3. 下载和安装Kubernetes组件3.1 获取二进制文件从官方GitHub下载v1.33版本的二进制文件wget https://dl.k8s.io/v1.33.0/kubernetes-server-linux-amd64.tar.gz tar -zxvf kubernetes-server-linux-amd64.tar.gz cd kubernetes/server/bin关键组件包括kube-apiserverkube-controller-managerkube-schedulerkubeletkube-proxykubectl3.2 组件部署结构我推荐以下目录结构/opt/kubernetes/ ├── bin/ # 二进制文件 ├── cfg/ # 配置文件 ├── ssl/ # 证书文件 ├── logs/ # 日志文件 └── systemd/ # systemd服务文件部署命令示例mkdir -p /opt/kubernetes/{bin,cfg,ssl,logs,systemd} cp kube-apiserver kube-controller-manager kube-scheduler kubectl /opt/kubernetes/bin/4. 证书体系搭建4.1 自签CA证书使用cfssl工具创建CA证书cat ca-config.json EOF { signing: { default: { expiry: 87600h }, profiles: { kubernetes: { expiry: 87600h, usages: [ signing, key encipherment, server auth, client auth ] } } } } EOF cat ca-csr.json EOF { CN: kubernetes, key: { algo: rsa, size: 2048 }, names: [ { C: CN, L: Beijing, O: k8s, OU: System } ] } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca4.2 签发API Server证书创建证书签名请求cat server-csr.json EOF { CN: kubernetes, hosts: [ 127.0.0.1, 192.168.1.100, 10.96.0.1, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster, kubernetes.default.svc.cluster.local ], key: { algo: rsa, size: 2048 }, names: [ { C: CN, L: Beijing, O: k8s, OU: System } ] } EOF cfssl gencert -caca.pem -ca-keyca-key.pem -configca-config.json -profilekubernetes server-csr.json | cfssljson -bare server5. 核心组件配置与启动5.1 kube-apiserver配置创建配置文件/opt/kubernetes/cfg/kube-apiserver.confcat /opt/kubernetes/cfg/kube-apiserver.conf EOF KUBE_APISERVER_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/opt/kubernetes/logs \\ --etcd-servershttps://192.168.1.100:2379 \\ --bind-address192.168.1.100 \\ --secure-port6443 \\ --advertise-address192.168.1.100 \\ --allow-privilegedtrue \\ --service-cluster-ip-range10.96.0.0/12 \\ --enable-admission-pluginsNamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota \\ --authorization-modeRBAC,Node \\ --enable-bootstrap-token-authtrue \\ --token-auth-file/opt/kubernetes/cfg/token.csv \\ --service-node-port-range30000-32767 \\ --tls-cert-file/opt/kubernetes/ssl/server.pem \\ --tls-private-key-file/opt/kubernetes/ssl/server-key.pem \\ --client-ca-file/opt/kubernetes/ssl/ca.pem \\ --service-account-key-file/opt/kubernetes/ssl/ca-key.pem \\ --service-account-signing-key-file/opt/kubernetes/ssl/ca-key.pem \\ --service-account-issuerhttps://kubernetes.default.svc.cluster.local \\ --etcd-cafile/opt/etcd/ssl/ca.pem \\ --etcd-certfile/opt/etcd/ssl/server.pem \\ --etcd-keyfile/opt/etcd/ssl/server-key.pem EOF创建systemd服务文件cat /usr/lib/systemd/system/kube-apiserver.service EOF [Unit] DescriptionKubernetes API Server Documentationhttps://github.com/kubernetes/kubernetes [Service] EnvironmentFile-/opt/kubernetes/cfg/kube-apiserver.conf ExecStart/opt/kubernetes/bin/kube-apiserver \$KUBE_APISERVER_OPTS Restarton-failure [Install] WantedBymulti-user.target EOF启动服务systemctl daemon-reload systemctl enable kube-apiserver systemctl start kube-apiserver5.2 kube-controller-manager配置创建配置文件/opt/kubernetes/cfg/kube-controller-manager.confcat /opt/kubernetes/cfg/kube-controller-manager.conf EOF KUBE_CONTROLLER_MANAGER_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/opt/kubernetes/logs \\ --leader-electtrue \\ --master127.0.0.1:8080 \\ --bind-address127.0.0.1 \\ --allocate-node-cidrstrue \\ --cluster-cidr10.244.0.0/16 \\ --service-cluster-ip-range10.96.0.0/12 \\ --cluster-signing-cert-file/opt/kubernetes/ssl/ca.pem \\ --cluster-signing-key-file/opt/kubernetes/ssl/ca-key.pem \\ --root-ca-file/opt/kubernetes/ssl/ca.pem \\ --service-account-private-key-file/opt/kubernetes/ssl/ca-key.pem \\ --controllers*,bootstrapsigner,tokencleaner EOF6. 节点加入与网络插件部署6.1 部署kubeletworker节点需要单独部署kubelet配置文件示例cat /opt/kubernetes/cfg/kubelet.conf EOF KUBELET_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/opt/kubernetes/logs \\ --hostname-overridek8s-node01 \\ --network-plugincni \\ --kubeconfig/opt/kubernetes/cfg/kubelet.kubeconfig \\ --bootstrap-kubeconfig/opt/kubernetes/cfg/bootstrap.kubeconfig \\ --config/opt/kubernetes/cfg/kubelet-config.yml \\ --cert-dir/opt/kubernetes/ssl \\ --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.7 EOF6.2 部署Flannel网络下载并配置Flannelwget https://github.com/flannel-io/flannel/releases/download/v0.22.0/flanneld-amd64 mv flanneld-amd64 /opt/kubernetes/bin/flanneld chmod x /opt/kubernetes/bin/flanneld创建Flannel配置文件cat /opt/kubernetes/cfg/flanneld.conf EOF FLANNEL_OPTIONS--etcd-endpointshttps://192.168.1.100:2379 \\ --etcd-cafile/opt/etcd/ssl/ca.pem \\ --etcd-certfile/opt/etcd/ssl/server.pem \\ --etcd-keyfile/opt/etcd/ssl/server-key.pem EOF7. 验证与排错7.1 集群状态检查使用kubectl检查集群状态kubectl get cs kubectl get nodes kubectl get pods -n kube-system7.2 常见问题排查API Server无法启动检查端口6443是否被占用netstat -tlnp | grep 6443查看日志journalctl -u kube-apiserver -fNode节点NotReady检查kubelet日志journalctl -u kubelet -f确认网络插件是否正常运行证书过期问题检查证书有效期openssl x509 -in /opt/kubernetes/ssl/server.pem -noout -dates更新证书后需要重启所有相关服务经验分享在二进制部署过程中90%的问题都与证书配置有关。建议使用统一的证书管理工具并做好证书过期监控。 Rocky Linux上二进制部署Kubernetes集群实战指南
1. 为什么选择二进制部署Kubernetes在Rocky Linux上部署Kubernetes集群时大多数人会直接使用kubeadm等工具但二进制部署才是真正理解Kubernetes组件交互的绝佳方式。我最近在生产环境用二进制方式部署了Kubernetes v1.33集群整个过程踩了不少坑也积累了一些实战经验。二进制部署最大的优势在于可控性。你可以精确控制每个组件的版本、启动参数和部署位置。这对于需要定制化Kubernetes的企业环境特别有用。比如我们曾经遇到需要修改kube-apiserver的审计策略二进制部署就能轻松实现而kubeadm方式则需要折腾半天。提示二进制部署适合对Kubernetes有较深理解的中高级用户如果是初学者建议先用kubeadm熟悉基本概念。2. 环境准备与系统配置2.1 Rocky Linux基础配置我选择Rocky Linux 9作为操作系统这是RHEL的完美替代品特别适合企业级部署。以下是必须完成的系统配置网络配置# 设置静态IP以ens192为例 nmcli con mod ens192 ipv4.addresses 192.168.1.100/24 nmcli con mod ens192 ipv4.gateway 192.168.1.1 nmcli con mod ens192 ipv4.dns 8.8.8.8 8.8.4.4 nmcli con mod ens192 ipv4.method manual nmcli con up ens192主机名与hosts配置hostnamectl set-hostname k8s-master01 echo 192.168.1.100 k8s-master01 /etc/hosts echo 192.168.1.101 k8s-node01 /etc/hosts关闭防火墙和SELinux生产环境需谨慎systemctl stop firewalld systemctl disable firewalld setenforce 0 sed -i s/SELINUXenforcing/SELINUXdisabled/g /etc/selinux/config2.2 内核参数优化Kubernetes对Linux内核有特殊要求必须调整以下参数cat /etc/sysctl.d/k8s.conf EOF net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 net.ipv4.ip_forward 1 vm.swappiness 0 EOF sysctl --system注意如果遇到bridge-nf-call报错需要先加载br_netfilter模块modprobe br_netfilter3. 下载和安装Kubernetes组件3.1 获取二进制文件从官方GitHub下载v1.33版本的二进制文件wget https://dl.k8s.io/v1.33.0/kubernetes-server-linux-amd64.tar.gz tar -zxvf kubernetes-server-linux-amd64.tar.gz cd kubernetes/server/bin关键组件包括kube-apiserverkube-controller-managerkube-schedulerkubeletkube-proxykubectl3.2 组件部署结构我推荐以下目录结构/opt/kubernetes/ ├── bin/ # 二进制文件 ├── cfg/ # 配置文件 ├── ssl/ # 证书文件 ├── logs/ # 日志文件 └── systemd/ # systemd服务文件部署命令示例mkdir -p /opt/kubernetes/{bin,cfg,ssl,logs,systemd} cp kube-apiserver kube-controller-manager kube-scheduler kubectl /opt/kubernetes/bin/4. 证书体系搭建4.1 自签CA证书使用cfssl工具创建CA证书cat ca-config.json EOF { signing: { default: { expiry: 87600h }, profiles: { kubernetes: { expiry: 87600h, usages: [ signing, key encipherment, server auth, client auth ] } } } } EOF cat ca-csr.json EOF { CN: kubernetes, key: { algo: rsa, size: 2048 }, names: [ { C: CN, L: Beijing, O: k8s, OU: System } ] } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca4.2 签发API Server证书创建证书签名请求cat server-csr.json EOF { CN: kubernetes, hosts: [ 127.0.0.1, 192.168.1.100, 10.96.0.1, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster, kubernetes.default.svc.cluster.local ], key: { algo: rsa, size: 2048 }, names: [ { C: CN, L: Beijing, O: k8s, OU: System } ] } EOF cfssl gencert -caca.pem -ca-keyca-key.pem -configca-config.json -profilekubernetes server-csr.json | cfssljson -bare server5. 核心组件配置与启动5.1 kube-apiserver配置创建配置文件/opt/kubernetes/cfg/kube-apiserver.confcat /opt/kubernetes/cfg/kube-apiserver.conf EOF KUBE_APISERVER_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/opt/kubernetes/logs \\ --etcd-servershttps://192.168.1.100:2379 \\ --bind-address192.168.1.100 \\ --secure-port6443 \\ --advertise-address192.168.1.100 \\ --allow-privilegedtrue \\ --service-cluster-ip-range10.96.0.0/12 \\ --enable-admission-pluginsNamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota \\ --authorization-modeRBAC,Node \\ --enable-bootstrap-token-authtrue \\ --token-auth-file/opt/kubernetes/cfg/token.csv \\ --service-node-port-range30000-32767 \\ --tls-cert-file/opt/kubernetes/ssl/server.pem \\ --tls-private-key-file/opt/kubernetes/ssl/server-key.pem \\ --client-ca-file/opt/kubernetes/ssl/ca.pem \\ --service-account-key-file/opt/kubernetes/ssl/ca-key.pem \\ --service-account-signing-key-file/opt/kubernetes/ssl/ca-key.pem \\ --service-account-issuerhttps://kubernetes.default.svc.cluster.local \\ --etcd-cafile/opt/etcd/ssl/ca.pem \\ --etcd-certfile/opt/etcd/ssl/server.pem \\ --etcd-keyfile/opt/etcd/ssl/server-key.pem EOF创建systemd服务文件cat /usr/lib/systemd/system/kube-apiserver.service EOF [Unit] DescriptionKubernetes API Server Documentationhttps://github.com/kubernetes/kubernetes [Service] EnvironmentFile-/opt/kubernetes/cfg/kube-apiserver.conf ExecStart/opt/kubernetes/bin/kube-apiserver \$KUBE_APISERVER_OPTS Restarton-failure [Install] WantedBymulti-user.target EOF启动服务systemctl daemon-reload systemctl enable kube-apiserver systemctl start kube-apiserver5.2 kube-controller-manager配置创建配置文件/opt/kubernetes/cfg/kube-controller-manager.confcat /opt/kubernetes/cfg/kube-controller-manager.conf EOF KUBE_CONTROLLER_MANAGER_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/opt/kubernetes/logs \\ --leader-electtrue \\ --master127.0.0.1:8080 \\ --bind-address127.0.0.1 \\ --allocate-node-cidrstrue \\ --cluster-cidr10.244.0.0/16 \\ --service-cluster-ip-range10.96.0.0/12 \\ --cluster-signing-cert-file/opt/kubernetes/ssl/ca.pem \\ --cluster-signing-key-file/opt/kubernetes/ssl/ca-key.pem \\ --root-ca-file/opt/kubernetes/ssl/ca.pem \\ --service-account-private-key-file/opt/kubernetes/ssl/ca-key.pem \\ --controllers*,bootstrapsigner,tokencleaner EOF6. 节点加入与网络插件部署6.1 部署kubeletworker节点需要单独部署kubelet配置文件示例cat /opt/kubernetes/cfg/kubelet.conf EOF KUBELET_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/opt/kubernetes/logs \\ --hostname-overridek8s-node01 \\ --network-plugincni \\ --kubeconfig/opt/kubernetes/cfg/kubelet.kubeconfig \\ --bootstrap-kubeconfig/opt/kubernetes/cfg/bootstrap.kubeconfig \\ --config/opt/kubernetes/cfg/kubelet-config.yml \\ --cert-dir/opt/kubernetes/ssl \\ --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.7 EOF6.2 部署Flannel网络下载并配置Flannelwget https://github.com/flannel-io/flannel/releases/download/v0.22.0/flanneld-amd64 mv flanneld-amd64 /opt/kubernetes/bin/flanneld chmod x /opt/kubernetes/bin/flanneld创建Flannel配置文件cat /opt/kubernetes/cfg/flanneld.conf EOF FLANNEL_OPTIONS--etcd-endpointshttps://192.168.1.100:2379 \\ --etcd-cafile/opt/etcd/ssl/ca.pem \\ --etcd-certfile/opt/etcd/ssl/server.pem \\ --etcd-keyfile/opt/etcd/ssl/server-key.pem EOF7. 验证与排错7.1 集群状态检查使用kubectl检查集群状态kubectl get cs kubectl get nodes kubectl get pods -n kube-system7.2 常见问题排查API Server无法启动检查端口6443是否被占用netstat -tlnp | grep 6443查看日志journalctl -u kube-apiserver -fNode节点NotReady检查kubelet日志journalctl -u kubelet -f确认网络插件是否正常运行证书过期问题检查证书有效期openssl x509 -in /opt/kubernetes/ssl/server.pem -noout -dates更新证书后需要重启所有相关服务经验分享在二进制部署过程中90%的问题都与证书配置有关。建议使用统一的证书管理工具并做好证书过期监控。