仓库源文，站点原文

layout: post title: Gentoo 上安裝 kubernetes date: 2020-12-31 20:01 +0800

categories: [k8s, gentoo]

• 使用 external etcd cluster w/o TLS cert
• 3 master node HA
• 3 worker node
• OpenRC
• containerd

prepare

ntpdate clock.stdtime.gov.tw
emerge --sync

etcd

emerge --ask dev-db/etcd

修改檔案

/etc/conf.d/etcd

PEER0="k8s-node1"
PEER0_IP="10.168.100.101"
PEER1="k8s-node2"
PEER1_IP="10.168.100.102"
PEER2="k8s-node3"
PEER2_IP="10.168.100.103"

MY_HOST="${PEER1}"
MY_IP="${PEER1_IP}"

ETCD_OPTS="--name ${MY_HOST} --initial-advertise-peer-urls http://${MY_IP}:2380 \
  --listen-peer-urls http://${MY_IP}:2380 \
  --listen-client-urls http://${MY_IP}:2379,http://127.0.0.1:2379 \
  --advertise-client-urls http://${MY_IP}:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster ${PEER0}=http://${PEER0_IP}:2380,${PEER1}=http://${PEER1_IP}:2380,${PEER2}=http://${PEER2_IP}:2380 \
  --initial-cluster-state new"

start service

rc-update add etcd default rc-service etcd start

testing

etcd put foo bar

Load Balancer

emerge keepalived haproxy
rc-update add keepalived default
rc-update add haproxy default

其餘方式如下，省略

https://github.com/kubernetes/kubeadm/blob/master/docs/ha-considerations.md#options-for-software-load-balancing

k8s

安裝

emerge --unmerge docker
emerge --ask cri-tools containerd
emerge --ask ebtables ethtool
emerge --ask sys-cluster/kubeadm kubectl kubelet
emerge net-misc/socat conntrack-tools

rc-update add kubelet default
rc-update add containerd default
rc-service containerd start

系統設定

也可以寫進開機script

networking

modprobe br_netfilter
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables

swap

/etc/fstab 砍掉 swap 分區

sysctl -w vm.swappiness=0
swapoff -a

edit hostname

vim /etc/conf.d/hostname

add ip to dns server

我有內部DNS 懶惰也可以加 /etc/hosts

加入 kubelet service 的參數

/etc/conf.d/kubelet

command_args="--container-runtime=remote \
  --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
  --image-service-endpoint=unix:///run/containerd/containerd.sock \
  --runtime-request-timeout=3m \
  --kubeconfig=/etc/kubernetes/kubelet.conf \
  --config=/var/lib/kubelet/config.yaml \
  --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf"

using crictl

加進 .profile 檔

export CONTAINER_RUNTIME_ENDPOINT=unix:///run/containerd/containerd.sock

external ETCD cluster 設定法

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/#set-up-the-first-control-plane-node

照著新增

kubeadm init --config=kubeadm-config.yaml ...

my config

apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 10.168.100.101
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
  name: k8s-node1
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
  timeoutForControlPlane: 4m0s
  extraArgs:
    advertise-address: 10.168.100.100
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: "10.168.100.100:6443"
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  external:
    endpoints:
    - http://10.168.100.101:2379
    - http://10.168.100.102:2379
    - http://10.168.100.103:2379
imageRepository: k8s.gcr.io
kubernetesVersion: v1.20.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.197.0.0/16
scheduler: {}
---

加入其他 node

kubeadm join ...

找回加入的token

kubeadm token create --print-join-command

kubeadm certs certificate-key

command example

kubeadm join 10.168.100.100:6443 --token dmjuyu.ym3ly49ntcokc8hj \
   --discovery-token-ca-cert-hash sha256:70729369809845fc5f98b006b8f84e5e4cef7341c1899d74b9052b62b1757227
   --control-plane \
   --certificate-key b9d8bd1a612f9fafff08a5067da125d231612d1a7ab6fe098a62068a6996d2d3

新增 metrics server

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

在 container args 加上

        --kubelet-insecure-tls