[TOC]
0x00 前言导读
Q: 什么是 Web UI (Dashboard) ?
答: Kubernetes Dashboard 是一个通用的、基于web的Kubernetes集群UI。它允许用户管理在集群中运行的应用程序并对它们进行故障排除，以及管理集群本身。
Q: 为什么要使用 Dashboard?
答: 您可以使用 Dashboard 来概述集群上运行的应用程序，以及创建或修改单个Kubernetes资源（例如Deployments，Jobs，DaemonSets等）例如，您可以使用部署向导来扩展部署，启动滚动更新，重新启动Pod或部署新应用程序。
并且仪表板还提供有关集群中Kubernetes资源状态以及可能发生的任何错误的信息。
附录：
K8s官网介绍: https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
项目地址: https://github.com/kubernetes/dashboard
0x01 安装部署
(1) 环境准备
Kubernetes 环境: 集群环境实现业务高可用以及快速扩容缩
1 | ~$ kubectl get nodes |
(2) 安装流程
描述: 我们可以通过原生的dashboardyaml资源清单文件或者helm的方式进行安装
官方 安装方式
安装参考: https://github.com/kubernetes/dashboard/blob/master/docs/user/installation.md
Step 1.要部署仪表板请执行以下命令：
1
$ wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.1.0/aio/deploy/recommended.yaml -O dashboard-v2.1.0.yaml
Step 2.默认情况下会生成自签名证书并将其存储在内存中,而如果您想使用自定义证书请按照以下步骤操作
PS: 访问Dashboard应使用有效证书来建立安全的HTTPS连接(可以使用公共信任的证书颁发机构如Let's Encrypt生成它们,或者Cert-Manager可以自动颁发和自动更新它们)
1 | # (1) 自定义证书必须存储在 kubernetes-dashboard-certs 与 Kubernetes 仪表板创建的名称空间中的secret中。 |
Step 3.仪表板仅通过HTTP公开不使用证书方式部署（不推荐）
1
kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.1/aio/deploy/alternative.yaml
Step 4.权限修改
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27# (1) kubernetes-dashboard 管理员角色添加修改
cat > dashboard-role-admin.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
name: kubernetes-dashboard
namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kubernetes-dashboard
# 权限来源
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
# 绑定对象
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
EOF
kubectl apply -f dashboard-role-admin.yaml
# (2) dashboard 资源清单构建
kubectl create -f dashboard-v2.1.0.yamlStep 5.集群网络访问端口修改
1
2
3
4
5
6
7
8
9
10
11
12# (1) 修改 SVC 访问方式为 NodePort (30443)
~/k8s/dashboard$ kubectl edit svc -n kubernetes-dashboard
# service/dashboard-metrics-scraper skipped
# service/kubernetes-dashboard edited
# (2) kubernetes-dashboard NodePort 为 10.96.167.225:443 => 30443
~$ kubectl get svc -n kubernetes-dashboard
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# dashboard-metrics-scraper ClusterIP 10.108.185.162 <none> 8000/TCP 14h
# kubernetes-dashboard NodePort 10.96.167.225 <none> 443:30443/TCP 14h
# (3) 代理访问 kubectl -n kubernetes-dashboardport-forward kubernetes-dashboard 443:443Step 6.Dashboard 认证 Token 获取
1
2
3
4
5
6
7
8
9
10
11
12
13$ kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | grep kubernetes-dashboard-token | cut -d " " -f 1)
Name: kubernetes-dashboard-token-mssqb
Namespace: kubernetes-dashboard
Labels: <none>
Annotations: kubernetes.io/service-account.name: kubernetes-dashboard
kubernetes.io/service-account.uid: 71b738ab-0f07-4e2c-99f9-0236cddd9bb4
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1066 bytes
namespace: 20 bytes
token: eyJhbG.....Step 7.访问 https://192.168.11.107:30443/#/login 进入 Kubernetes Dashboard 登陆页面选择 Token 认证登陆即可
WeiyiGeek.Kubernetes Dashboard
参考地址: https://github.com/kubernetes/dashboard/blob/master/README.md
Helm 安装方式
操作流程: 此处对于helm安装不再累述，二进制安装没有什么好说的;1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24$ helm repo add k8s-dashboard https://kubernetes.github.io/dashboard
# "k8s-dashboard" has been added to your repositories
$ helm pull k8s-dashboard/kubernetes-dashboard --untar
~/K8s/Day10/dashboard$ tar -xzvf kubernetes-dashboard-3.0.0.tgz
~/K8s/Day10/dashboard$ ls kubernetes-dashboard
# charts Chart.yaml README.md requirements.lock requirements.yaml templates values.yaml
~/K8s/Day10/dashboard$ helm install kubernetes-dashboard kubernetes-dashboard/ --namespace kube-system
# NAME: kubernetes-dashboard
# LAST DEPLOYED: Sun Dec 6 21:45:22 2020
# NAMESPACE: kube-system
# STATUS: deployed
# REVISION: 1
# TEST SUITE: None
# NOTES:
# *********************************************************************************
# *** PLEASE BE PATIENT: kubernetes-dashboard may take a few minutes to install ***
# ********************************************************************************
# Get the Kubernetes Dashboard URL by running:
# export POD_NAME=$(kubectl get pods -n kube-system -l "app.kubernetes.io/name=kubernetes-dashboard,app.kubernetes.io/instance=kubernetes-dashboard" -o jsonpath="{.items[0].metadata.name}") # kubernetes-dashboard-879457794-kxvcr
# echo https://127.0.0.1:8443/
# kubectl -n kube-system port-forward $POD_NAME 8443:8443 # 端口转发
查看结果:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15~/K8s/Day10/dashboard$ helm list -n kube-system
# NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
# kubernetes-dashboard kube-system 1 2020-12-06 11:44:44.821856156 +0800 CST deployed kubernetes-dashboard-3.0.0 2.0.4
~/K8s/Day10/dashboard$ helm history kubernetes-dashboard -n kube-system
# REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
# 1 Sun Dec 6 21:45:22 2020 deployed kubernetes-dashboard-3.0.0 2.0.4 Install complete
# 查看 Dashboard Pod 信息以及标签
~/K8s/Day10/dashboard$ kubectl get pod -n kube-system -o wide --show-labels | grep "kubernetes-dashboard-879457794-kxvcr"
# kubernetes-dashboard-879457794-kxvcr 1/1 Running 0 11m 10.244.2.55 k8s-node-5 app.kubernetes.io/component=kubernetes-dashboard,app.kubernetes.io/instance=kubernetes-dashboard,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=kubernetes-dashboard,app.kubernetes.io/version=2.0.4,helm.sh/chart=kubernetes-dashboard-3.0.0,pod-template-hash=879457794
# 查看 Dashboard 的 SVC
~/K8s/Day10/dashboard$ kubectl get svc -n kube-system -o wide | grep "kubernetes-dashboard"
# kubernetes-dashboard ClusterIP 10.104.18.192 <none> 443/TCP 13m app.kubernetes.io/component=kubernetes-dashboard,app.kubernetes.io/instance=kubernetes-dashboard,app.kubernetes.io/name=kubernetes-dashboard
以NodePort的方式进行访问:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40~/K8s/Day10/dashboard$ kubectl edit svc -n kube-system kubernetes-dashboard
service/kubernetes-dashboard edited
# 目的: 将通过集群IP:443访问的模式变成节点IP:30443进行访问
apiVersion: v1
kind: Service
metadata:
annotations:
meta.helm.sh/release-name: kubernetes-dashboard
meta.helm.sh/release-namespace: kube-system
creationTimestamp: "2020-12-06T13:45:22Z"
labels:
app.kubernetes.io/component: kubernetes-dashboard
app.kubernetes.io/instance: kubernetes-dashboard
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: kubernetes-dashboard
app.kubernetes.io/version: 2.0.4
helm.sh/chart: kubernetes-dashboard-3.0.0
kubernetes.io/cluster-service: "true"
name: kubernetes-dashboard
namespace: kube-system
resourceVersion: "6111082"
selfLink: /api/v1/namespaces/kube-system/services/kubernetes-dashboard
uid: 51025b69-7c65-4ac0-a8f2-93a243a33e7d
spec:
clusterIP: 10.104.18.192
ports:
- name: https
port: 443
protocol: TCP
targetPort: https
nodePort: 30443 # 修改点
selector:
app.kubernetes.io/component: kubernetes-dashboard
app.kubernetes.io/instance: kubernetes-dashboard
app.kubernetes.io/name: kubernetes-dashboard
sessionAffinity: None
type: NodePort # 修改点 修改 ClusterIP 为 NodePort
status:
loadBalancer: {}
简单使用:
- 1) 认证的 Token 查看
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17~/K8s/Day10/dashboard$ kubectl -n kube-system get secret | grep kubernetes-dashboard-token
# kubernetes-dashboard-token-6nrqk kubernetes.io/service-account-token 3 19m
kubectl describe secret kubernetes-dashboard-token-6nrqk -n kube-system
# Name: kubernetes-dashboard-token-6nrqk
# Namespace: kube-system
# Labels: <none>
# Annotations: kubernetes.io/service-account.name: kubernetes-dashboard
# kubernetes.io/service-account.uid: a1685c3b-247e-4802-9d2e-28d5f48e432a
# Type: kubernetes.io/service-account-token
# Data
# ====
# ca.crt: 1066 bytes
# namespace: 11 bytes
# token: .......
WeiyiGeek.Kubernetes-仪表盘
PS : 需要注意令牌过期时间，登录账户将自动退出；
- 2) 登录Dashboard仪表盘控制台，可以看相关资源控制器下面的所属资源
WeiyiGeek.Dashboard仪表盘主页
参考地址：https://artifacthub.io/packages/helm/k8s-dashboard/kubernetes-dashboard?modal=install
安装部署 v2.5.1 版本
描述: 当前时间节点【2022年5月13日 16:50:07】，相对比于前面的 kubernetes-dashboard 版本，当前安装可能会有一定差异。
步骤 01.从Github中拉取dashboard部署资源清单，当前最新版本v2.5.11
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22# 下载部署
wget -L https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.1/aio/deploy/recommended.yaml
kubectl apply -f recommended.yaml
grep "image:" recommended.yaml
# image: kubernetesui/dashboard:v2.5.1
# image: kubernetesui/metrics-scraper:v1.0.7
# 或者一条命令搞定部署
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.1/aio/deploy/recommended.yaml
# serviceaccount/kubernetes-dashboard created
# service/kubernetes-dashboard created
# secret/kubernetes-dashboard-certs created
# secret/kubernetes-dashboard-csrf created
# secret/kubernetes-dashboard-key-holder created
# configmap/kubernetes-dashboard-settings created
# role.rbac.authorization.k8s.io/kubernetes-dashboard created
# clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
# rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
# clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
# deployment.apps/kubernetes-dashboard created
# service/dashboard-metrics-scraper created
# deployment.apps/dashboard-metrics-scraper created
步骤 02.查看部署的dashboard相关资源是否正常。
1 | $ kubectl get deploy,svc -n kubernetes-dashboard -o wide |
步骤 03.默认仪表板部署包含运行所需的最小RBAC权限集，而要想使用dashboard操作集群中的资源，通常我们还需要自定义创建kubernetes-dashboard管理员角色。
权限控制参考地址: https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/README.md
1 | # 创建后最小权限的Token(只能操作kubernetes-dashboard名称空间下的资源) |
WeiyiGeek.Dashboard默认两种认证方式
Kubernetes Dashboard 支持几种不同的用户身份验证方式：
- Authorization header
- Bearer Token (默认)
- Username/password
- Kubeconfig file (默认)
温馨提示: 此处使用Bearer Token方式, 为了方便演示我们向 Dashboard 的服务帐户授予管理员权限 (Admin privileges), 而在生产环境中通常不建议如此操作, 而是指定一个或者多个名称空间下的资源进行操作。
1 | tee rbac-dashboard-admin.yaml <<'EOF' |
步骤 04.获取 sa 创建的 dashboard-admin 用户的 secrets 名称并获取认证 token ，用于上述搭建的dashboard 认证使用。1
2
3
4
5kubectl get sa -n kubernetes-dashboard dashboard-admin -o yaml | grep "\- name" | awk '{print $3}'
# dashboard-admin-token-crh7v
kubectl describe secrets -n kubernetes-dashboard dashboard-admin-token-crh7v | grep "^token:" | awk '{print $2}'
# 获取到认证Token
eyJhbGciOiJSUzI1NiIsImtpZCI6IkJXdm1YSGNSQ3VFSEU3V0FTRlJKcU10bWxzUDZPY3lfU0lJOGJjNGgzRXMifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tY3JoN3YiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNDY3ODEwMDMtM2MzNi00NWE1LTliYTQtNDY3MTQ0OWE2N2E0Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmVybmV0ZXMtZGFzaGJvYXJkOmRhc2hib2FyZC1hZG1pbiJ9.X10AzWBxaHObYGoOqjfw3IYkhn8L5E7najdGSeLavb94LX5BY8_rCGizkWgNgNyvUe39NRP8r8YBU5sy9F2K-kN9_5cxUX125cj1drLDmgPJ-L-1m9-fs-luKnkDLRE5ENS_dgv7xsFfhtN7s9prgdqLw8dIrhshHVwflM_VOXW5D26QR6izy2AgPNGz9cRh6x2znrD-dpUNHO1enzvGzlWj7YhaOUFl310V93hh6EEc57gAwmDQM4nWP44KiaAiaW1cnC38Xs9CbWYxjsfxd3lObWShOd3knFk5PUVSBHo0opEv3HQ_-gwu6NGV6pLMY52p_JO1ECPSDnblVbVtPQ
步骤 05.利用上述 Token 进行登陆Kubernetes-dashboard的UI。 WeiyiGeek.拥有管理员权限的dashboard
(3) MetricServer
Q: MetricServer 是什么？
答: 它是kubernetes集群资源使用情况的聚合器，收集数据给kubernetes集群内使用，如 kubectl,hpa,scheduler等。
Kubernetes 推荐使用 metrics-server , 因为 heapster (https:/github.com/kubernetes/heapster) 已经DEPRECATED ,并从 Kubernetes 1.12开始将从 Kubernetes 各种安装脚本中移除,
PS : 如果采用官方的安装dashboard的方式则默认将Metric Server进行安装使用，而采用helm安装dashboard时候默认是将metrics-server禁用的需要手动启用;
helm 安装方式
下面我们使用Helm部署Dashboard时也可以利用第三方依赖进行安装metrics-server,只需要修改一个小小的注释1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70# (1) 启用Metrics-server以及插件
$ ~/K8s/Day10/dashboard/kubernetes-dashboard$ vim +200 values.yaml
## Enable this is you don't already have metrics-server enabled on your cluster and
## want to use it with dashboard metrics-scraper
## refs:
## - https://hub.helm.sh/charts/stable/metrics-server
## - https://github.com/kubernetes-sigs/metrics-server
metrics-server:
enabled: true
## Example for additional args
args:
- --logtostderr
- --kubelet-preferred-address-types=InternalIP
- --kubelet-insecure-tls
# (2) 此时如果需要更新部署时需要将SVC改回集群IP通信
~/K8s/Day10/dashboard$ kubectl edit svc -n kube-system kubernetes-dashboard
service/kubernetes-dashboard edited
# (3) 更新指定的RELEASE_NAME
~/K8s/Day10/dashboard$ helm upgrade kubernetes-dashboard kubernetes-dashboard/ -n kube-system
# (4) 查看与dashboard相关的Pod信息
~/K8s/Day10/dashboard$ kubectl get pod -n kube-system -o wide | grep "kubernetes-dashboard"
# kubernetes-dashboard-879457794-kxvcr 1/1 Running 0 23h 10.244.2.55 k8s-node-5
# kubernetes-dashboard-metrics-server-7bc85c65bc-vrxft 0/1 ImagePullBackOff # 关键点 0 10m 10.244.2.57 k8s-node-5
# (5) 发现镜像拉取失败下面我们手动查看并下载该镜像
~/K8s/Day10/dashboard$ kubectl describe pod kubernetes-dashboard-metrics-server-7bc85c65bc-vrxft -n kube-system kubernetes-dashboard
# GFW 没办法，解决利用阿里云镜像站进行手动下载然后上传到node-5节点中（或者在安全前选择修改该k8s.gcr.io镜像源）
Warning Failed 10m (x4 over 12m) kubelet Failed to pull image "k8s.gcr.io/metrics-server-amd64:v0.3.6": rpc error: code = Unknown desc = Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
~/K8s/Day10/dashboard$ docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6
~/K8s/Day10/dashboard$ docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6 k8s.gcr.io/metrics-server-amd64:v0.3.6
~/K8s/Day10/dashboard$ docker save k8s.gcr.io/metrics-server-amd64:v0.3.6 -o metrics-server-amd64.tar
~/K8s/Day10/dashboard$ scp -P 20211 metrics-server-amd64.tar weiyigeek@10.10.107.215:~
# metrics-server-amd64.tar 100% 39MB 187.5MB/s 00:00
~/K8s/Day10/dashboard$ ssh -p 20211 weiyigeek@10.10.107.215 "docker load -i metrics-server-amd64.tar"
# Loaded image: k8s.gcr.io/metrics-server-amd64:v0.3.6
# (6) 此时可以看见metrics-server已经成功安装了
~/K8s/Day10/dashboard$ kubectl get pod -n kube-system -o wide | grep "kubernetes-dashboard-metrics-server-7bc85c65bc-vrxft"
# kubernetes-dashboard-metrics-server-7bc85c65bc-vrxft 1/1 Running 0 27m 10.244.2.57 k8s-node-5 <none> <none>
# (7) 验证安装的 metrics-server 它获取到关于集群节点基本的指标信息:
~$ kubectl top pod
# NAME CPU(cores) MEMORY(bytes)
# dashboard-create-696f45d5db-fj5dg 0m 2Mi
~$ kubectl top node
# NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
# ubuntu 194m 2% 1377Mi 17%
# k8s-node-4 32m 1% 1351Mi 35%
# k8s-node-5 26m 1% 1134Mi 29%
# (8) 采用port-forward转发的方式访问我们创建的应用，此处访问master节点的端口还是30443端口->Pod暴露的8443端口之中
export POD_NAME=$(kubectl get pods -n kube-system -l "app.kubernetes.io/name=kubernetes-dashboard,app.kubernetes.io/instance=kubernetes-dashboard" -o jsonpath="{.items[0].metadata.name}")
echo https://127.0.0.1:30443/
~/K8s/Day10/dashboard$ kubectl -n kube-system port-forward --address 10.10.107.202 $POD_NAME 30443:8443
# Forwarding from 10.10.107.202:30443 -> 8443
# Handling connection for 30443
# 新开一个Terminal获取访问的token
~/K8s/Day10/dashboard$ kubectl describe secrets -n kube-system kubernetes-dashboard-token-6nrqk
token: eyJhbGciOiJSUzI1Ni................IsImtpZCI6IkNsknTWtKLBDUk-Q
# (9) 补充附录
~$ helm get all -n kube-system kubernetes-dashboard # 查看实际执行的资源清单
~$ helm uninstall kubernetes-dashboard -n kube-system # 卸载 Helm 安装的 kubernetes-dashboard
release "kubernetes-dashboard" uninstalled
WeiyiGeek.K8s-metrics-server
(4) 配置扩展
1.配置 Kubernetes-dashboard 以支持 http 方式访问
描述: 当前默认安装配置的 Kubernetes-dashboard 都是启用了https, 然而在当我们环境中存在ingress时，可能会有需要将其通过虚拟主机进行暴露时，此时将会在ingress端进行设置证书而不是在 Kubernetes-dashboard Pod中设置证书。
步骤 01.打开下载的Kubernetes-dashboard资源清单文件或者使用kubelet edit命令编辑已部署的资源清单,首先配置 kubernetesui/dashboard:v2.5.1 镜像的启动参数，主要是--enable-insecure-login与--insecure-port=8080参数。1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26$ kubectl edit deployments.apps -n kubernetes-dashboard kubernetes-dashboard
args:
# - --auto-generate-certificates
- --namespace=kubernetes-dashboard
- --enable-insecure-login
- --insecure-port=8080
# Pod 端口暴露
ports:
- name: https
containerPort: 8443
protocol: TCP
- name: http
containerPort: 8080
protocol: TCP
# Pod 健康检查
livenessProbe:
# httpGet:
# scheme: HTTPS
# path: /
# port: 8443
httpGet:
scheme: HTTP
path: /
port: 8080
步骤 02.配置 kubernetes-dashboard 的 Service 资源管理器
1 | $ kubectl edit svc -n kubernetes-dashboard kubernetes-dashboard |
步骤 03.服务验证以及部署ingress转发规则URL设置，最后浏览器访问如下URL(devops.weiyigeek.top/dashboard/)即可。1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40$ curl 11.19.103.247:8080
$ tee kubernetes-dashboard-ingress.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
ingressclass.kubernetes.io/is-default-class: "true"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "75"
nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
nginx.ingress.kubernetes.io/rewrite-target: /$2
labels:
app: devops-weiyigeek
name: devops-weiyigeek
namespace: kubernetes-dashboard
spec:
ingressClassName: nginx
rules:
- host: devops.weiyigeek.top
http:
paths:
- backend:
service:
name: kubernetes-dashboard
port:
number: 8080
path: /dashboard(/|$)(.*)
pathType: ImplementationSpecific
tls:
- hosts:
- devops.weiyigeek.top
secretName: devops-weiyigeek-top
EOF
# 部署 ingress 规则
$ kubectl apply -f kubernetes-dashboard-ingress.yaml
$ kubectl get ingress -n kubernetes-dashboard devops-weiyigeek
NAME CLASS HOSTS ADDRESS PORTS AGE
devops-weiyigeek nginx devops.weiyigeek.top 11.19.12.210 80, 443 3h52m
温馨提示: 在前面部署完成后, 我们便可可以通过https://devops.weiyigeek.top/dashboard/带https + 域名方式访问kubernetes-dashboard了。
0x03 使用实践
(1) Dashboard-小试牛刀之简单初识
- Step 1.右上角点击
+进行创建Deployment管理的Pod，按图所示输入应用名称和容器镜像名称，其次是Services资源控制器设置为内部的Internal；
WeiyiGeek.创建Deployment管理的Pod
Step 2.点击左边Workloads中的Deployments子菜单查看创建的Deployments资源
1
2
3~$ kubectl get pod -o wide --show-labels
# NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
# dashboard-create-696f45d5db-fj5dg 1/1 Running 0 4m23s 10.244.2.56 k8s-node-5 <none> <none> k8s-app=dashboard-create,pod-template-hash=696f45d5db
WeiyiGeek.Deployments
Step 3.同样点击Service中的Services子菜单将可以看见我们创建的Service相关资源信息
1
2
3~$ kubectl get svc -o wide --show-labels
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR LABELS
# dashboard-create ClusterIP 10.102.184.126 <none> 80/TCP 4m53s k8s-app=dashboard-create k8s-app=dashboard-create
WeiyiGeek.Service
- Step 4.#验证 集群IP地址访问 以及 Pod地址访问效果一致
1
2
3~$ curl http://10.102.184.126/host.html && curl http://10.244.2.56/host.html
# Hostname: dashboard-create-696f45d5db-fj5dg ,Image Version: 3.0, Nginx Version: 1.19.4
# Hostname: dashboard-create-696f45d5db-fj5dg ,Image Version: 3.0, Nginx Version: 1.19.4
PS : 总结可以看出使用Kubernetes-Kuboard是可以非常简单的创建我们指定的应用到kubernetes之中;
(2) Dashboard-利用rbac机制限制指定用户针对指定名称空间中的资源进行UI管理。
描述: 有时可能我们会遇到如下场景, 在进行持续CI/CD后，开发人员可能会需要查看部署应用的启动日志,如果都是我们运维人员手动去截图发给他们, 那这样的效率简直是在浪费生命，所有为了节约时间同时保证防止开发人员误操作集群, 此时我们只赋予其指定名称空间下的某些资源浏览权限即可.
在 Kubernetes 集群中我们可以使用 rbac 授权机制, 做用户角色权限分离，可以指定那些资源，我们可以进行那些操作，然后把该角色赋予给指定的用户，最好利用该用户的Token进行登陆Kubernetes-Dashborad界面进行相应管理。
步骤 01.创建一个服务用户此处我们可以采用两种方式创建资源清单或者命令行。1
2
3
4
5
6
7
8
9
10
11# 方式1
kubectl create serviceaccount -n devtest devtest-ns-viewonly
# 方式2
tee > devtest-ns-viewonly-sa.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
name: devtest-ns-viewonly
namespace: devtest
EOF
步骤 02.准备名称为dashboard-viewonly角色相关资源权限操作的资源清单。
1 | tee > dashboard-namespace-viewonly.yaml <<'EOF' |
步骤 03.绑定 dashboard-viewonly 角色给 ServiceAccount 的 devtest-ns-viewonly 用户.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17tee dashboard-viewonly-RoleBinding<<'EOF'
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: devtest-ns-viewonly
namespace: devtest
roleRef:
kind: Role
name: dashboard-viewonly
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: devtest-ns-viewonly
EOF
# 或者一条命令搞定
kubectl create rolebinding -n devtest devtest-ns-viewonly --role=devtest:dashboard-viewonly --serviceaccount=devtest-ns-viewonly
温馨提示: ClusterRole 与 ClusterRoleBinding 均不支持指定名称空间。
步骤 04.查看 devtest-ns-viewonly 用户存在 secrets 中的认证Token。1
kubectl describe secrets -n devtest devtest-ns-viewonly-token-gxgps | grep "^token:" | awk '{print $2}'
步骤 05.使用获取到的Token访问登陆，我们搭建的kubernetes-dashboard Web UI，此处使用浏览器访问(https://devops.weiyigeek.top/dashboard/#/workloads?namespace=devtest)，可以看到该使用Token认证的用户只能访问devtest名称空间下的特定资源。
WeiyiGeek.认证用户只能访问devtest名称空间下的特定资源
0x04 入坑与出坑
问题1.pods is forbidden: User "system:serviceaccount:kube-system:namespace-controller" cannot create resource clusterroles” in API group “rbac.authorization.k8s.io” at the cluster scope
问题原因:
- 1.API组中用户不能在默认命名空间创建Pod，也就是说使用原token认证登录的用户是无权操作
- 2.其次是采用Helm创建的时候只是将
kubernetes-dashboard-metrics与集群角色绑定1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19# 绑定的角色
~/K8s/Day10/dashboard/kubernetes-dashboard$ kubectl get ClusterRoleBinding -n kube-system | grep "kubernetes-dashboard"
kubernetes-dashboard-metrics ClusterRole/kubernetes-dashboard-metrics
# 查看集群所有权限
~/K8s/Day10/dashboard/kubernetes-dashboard$ kubectl get clusterrole
# 权限非常有限
~/K8s/Day10/dashboard/kubernetes-dashboard$ kubectl get clusterrole kubernetes-dashboard-metrics -o yaml
rules:
- apiGroups:
- metrics.k8s.io
resources:
- pods
- nodes
verbs:
- get
- list
- watch
解决方法:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49# 1.创建kubernetes-dashboard管理员角色
cat > k8s-admin.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
name: kubernetes-dashboard
namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
# 绑定对象
metadata:
name: kubernetes-dashboard
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
# 权限来源
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
EOF
kubectl create -f k8s-admin.yaml
# 3.验证查看 ClusterRoleBinding 资源
~/K8s/Day10/dashboard/$ kubectl get ClusterRoleBinding -n kube-system | grep "kubernetes-dashboard"
# NAME ROLE AGE
# kubernetes-dashboard ClusterRole/cluster-admin 17m
# kubernetes-dashboard-metrics ClusterRole/kubernetes-dashboard-metrics 70m
~/K8s/Day10/dashboard/$ kubectl describe ClusterRoleBinding -n kube-system kubernetes-dashboard
# Name: kubernetes-dashboard
# Labels: <none>
# Annotations: <none>
# Role:
# Kind: ClusterRole
# Name: cluster-admin
# Subjects:
# Kind Name Namespace
# ---- ---- ---------
# ServiceAccount kubernetes-dashboard kube-system
# 2.获取dashboard管理员角色token
kubectl describe secret kubernetes-dashboard-token-7z6zm -n kube-system
# 3.使用第二步第12行的token登陆kubernetes-dashboard web界面即可
PS : 在使用Helm创建Kubenertes-Dashboard时候已创建了ServiceAccount资源，所以只需要创建ClusterRoleBinding资源即可;
参考地址: https://blog.csdn.net/qq_38900565/article/details/100729686
问题2.采用Helm安装metric-server时镜像有误导致Pod状态ImagePullBackOff
错误信息:1
Warning Failed 10m (x4 over 12m) kubelet Failed to pull image "k8s.gcr.io/metrics-server-amd64:v0.3.6": rpc error: code = Unknown desc = Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
解决办法：
- 1.利用阿里云的K8s镜像站拉取metrics-server-amd64:v0.3.6镜像然后进行改名，随后上传到metrics-server运行的节点之上
- 2.在进行更新时候指定或者说修改配置文件中的
image.repository;1
2
3~/K8s/Day10/dashboard$ grep "k8s.gcr.io" kubernetes-dashboard/charts/metrics-server/*
# kubernetes-dashboard/charts/metrics-server/values.yaml: repository: k8s.gcr.io/metrics-server-amd64
~/K8s/Day10/dashboard$ sed -i "s#k8s.gcr.io#registry.cn-hangzhou.aliyuncs.com/google_containers#g" kubernetes-dashboard/charts/metrics-server/values.yaml:
