[TOC]
0x00 前言简述 描述： 到目前为止我们了解kubernetes常用的三种暴露服务的方式：LoadBlancer Service、 NodePort Service、Ingress
LoadBlancer Service 是kubernetes结合云平台的组件，如国外的GCE，AWS，国内阿里云等等。使用它项使用的底层云平台申请创建负载均衡器来实现，对使用云平台的集群比较方便，但有局限，费用高。
NodePort Service 是kubernetes在每个节点上暴露一个相同的端口(默认:30000-32000)但是由于安全和易用方面（服务多了就乱了还有端口冲突问题）且对主机安全性存在一定风险（内网环境，问题不大），所以实际使用并不多，当然对于小规模的集群服务，还是比较不错的。
Ingress 是 kubernetes 中对外暴露服务的一种方式，它是使用Nginx进行反代应用来实现，其特点是安全以及方便统一管理等。
Tips : 此外externalIPs也可以使各类service对外提供服务，但是当集群服务很多的时候，NodePort方式最大的缺点是会占用很多集群机器的端口；LB方式最大的缺点则是每个service一个LB又有点浪费和麻烦，并且需要k8s之外的支持； 而ingress则只需要一个NodePort或者一个LB就可以满足所有service对外服务的需求(便于管理)。
Ingress 基础介绍 Q: 什么是负载均衡Ingress? Nginx、Haproxy 等负载均衡器。
[TOC]
0x00 前言简述 描述： 到目前为止我们了解kubernetes常用的三种暴露服务的方式：LoadBlancer Service、 NodePort Service、Ingress
LoadBlancer Service 是kubernetes结合云平台的组件，如国外的GCE，AWS，国内阿里云等等。使用它项使用的底层云平台申请创建负载均衡器来实现，对使用云平台的集群比较方便，但有局限，费用高。
NodePort Service 是kubernetes在每个节点上暴露一个相同的端口(默认:30000-32000)但是由于安全和易用方面（服务多了就乱了还有端口冲突问题）且对主机安全性存在一定风险（内网环境，问题不大），所以实际使用并不多，当然对于小规模的集群服务，还是比较不错的。
Ingress 是 kubernetes 中对外暴露服务的一种方式，它是使用Nginx进行反代应用来实现，其特点是安全以及方便统一管理等。
Tips : 此外externalIPs也可以使各类service对外提供服务，但是当集群服务很多的时候，NodePort方式最大的缺点是会占用很多集群机器的端口；LB方式最大的缺点则是每个service一个LB又有点浪费和麻烦，并且需要k8s之外的支持； 而ingress则只需要一个NodePort或者一个LB就可以满足所有service对外服务的需求(便于管理)。
Ingress 基础介绍 Q: 什么是负载均衡Ingress? Nginx、Haproxy 等负载均衡器。
1 2 3 4 5 internet | [ Ingress ] --|-----|-- [ Services ]
Tips : 对于k8s传统的svc来说它仅支持4层代理, 如果需要做七层代理就需要使用k8s官方在1.11中推出了ingress api接口，通过ingress达到7层代理的效果我对于ingress来说必须要绑定一个域名因为它是基于7层代理的
Tips : ingress调度的是后端的service而不是pod.
NGINX 配置
NGINX 模型与构建 Ingresses、Services、Endpoints、Secrets 和 Configmaps 以生成反映集群状态的时间点配置文件.
描述: 建立模型是一项昂贵的操作，因此必须使用同步循环。通过使用工作队列，可以不丢失更改并删除使用sync.Mutex来强制同步循环的单次执行，此外还可以在同步循环的开始和结束之间创建一个时间窗口，允许我们丢弃不必要的更新。
Tips: NGINX 配置的最终表示是从Go 模板生成的，使用新模型作为模板所需变量的输入。
Q: ingress-nginx解决了生产环境中哪些问题?
1）动态配置服务：如果按照传统方式，当新增加一个服务时，我们可能需要在流量入口加一个反向代理指向我们新的服务，而使用ingress，只需要配置好ingress，当服务启动时，会自动注册到ingress当中，不需要额外的操作。
2）减少不必要的Port暴露（安全，端口容易管理）: 我们知道部署k8s时，是需要关闭防火墙的，主要原因是k8s的很多服务会以nodeport方式映射出去，这样对于宿主机来说是非常的不安全的，而ingress可以避免这个问题，只需要将ingress自身服务映射出去，就可代理后端所有的服务，则后端服务不需要映射出去。
Q: ingress-Nginx和ingress-Nginx-Controller的区别?
ingress-Nginx: 是每个服务自己创建的ingress,就是nginx的转发规则,生成Nginx的配置文件
ingress-Nginx-Controller: 相当于Nginx的服务,监听API Server,根据用户编写的ingress-nginx规则(ingress.yaml文件),动态的去更改Nginx服务的配置文件,并且reload使其生效,此过程是自动化的通过lua实现
Ingress 实现原理 描述: Ingress 实际上是通过服务发现的功能进行实现，通过它来提供路由信息的刷新, Ingress controller可以理解为一个监视器，不断监听 kube-apiserver 实时感知service、Pod的变化，其再结合Ingress的配置，更新反向代理负载均衡器，达到服务发现的作用。
Ingress 公开了从集群外部到集群内服务的 HTTP 和 HTTPS 路由，流量路由由 Ingress 资源上定义的规则控制,可以将 Ingress 配置为服务提供外部可访问的 URL、负载均衡流量、终止 SSL/TLS，以及提供基于名称的虚拟主机访问。
Tips: Ingress 控制器通常负责通过负载均衡器来实现 Ingress，尽管它也可以配置边缘路由器或其他前端来帮助处理流量，并且ingress不暴露任何端口或协议。
组件组成 Ingress-nginx一般由三个组件组成反向代理负载均衡器 , Ingress Controller, Ingress
1）反向代理负载均衡器：通常以service的port方式运行，接收并按照ingress定义的规则进行转发，常用的有nginx，Haproxy，Traefik等，本文中使用的就是nginx。
2）ingress-nginx-Controller： 监听APIServer，根据用户编写的ingress规则（编写ingress的yaml文件），动态地去更改nginx服务的配置文件，并且reload重载使其生效，此过程是自动化的（通过lua脚本来实现）。
3）Ingress：将nginx的配置抽象成一个Ingress对象，当用户每添加一个新的服务，只需要编写一个新的ingress的yaml文件即可。
工作流程
1）ingress controller通过和kubernetes api交互，动态的去感知集群中ingress规则变化。
2）然后读取它，按照自定义的规则，规则就是写明了那个域名对应哪个service，生成一段nginx配置。
3）在写到nginx-ingress-controller的pod里，这个Ingress controller的pod里运行着一个Nginx服务，控制器会把生成的nginx配置写入/etc/nginx.conf文件中。
4）最后 reload 一下使配置生效，以此达到分配和动态更新问题。
注意：写入 nginx.conf 的不是service的地址，而是service backend 的 pod 的地址，避免在 service 在增加一层负载均衡转发
weiyigeek.top-工作流程
Tips : 目前可以提供Ingress controller有很多比如traefik、nginx-ingress、Kubernetes Ingress Cpmtrper for Kong、HAProxy Ingress controller等。
Tips : 目前常见的负载均衡有Ingress-Nginx和Ingress-Traefik等。
Tips: 如果多个 Ingress 为同一主机定义了路径，则 Ingress 控制器会合并这些定义。
Tips: 入口控制器第一次启动时，两个作业创建了准入 Webhook 使用的 SSL 证书。因此，在可以创建和验证 Ingress 定义之前，会有最多两分钟的初始延迟。
Tips: admission webhook 需要Kubernetes API服务器和入口控制器之间的连接,请保证防火墙允许8443端口通信。
名词解析复习
0x01 Ingress 安装配置 描述: Ingress-Nginx本质上是创建了一个Nginx的Node pod，只不过这个无需手写Nginx的配置文件而是通过自动生成的方式实现;
帮助参考 https://kubernetes.github.io/ingress-nginx/ https://github.com/kubernetes/ingress-nginx https://github.com/kubernetes/ingress-nginx/releases
版本支持说明 : https://github.com/kubernetes/ingress-nginx/#support-versions-table
1 2 3 4 5 6 Ingress-nginx version | k8s supported version| Alpine Version | Nginx Version v1.0.0-alpha.2 1.22, 1.21, 1.20, 1.19 3.13.5 1.20.1 v1.0.0-alpha.1 1.21, 1.20, 1.19 3.13.5 1.20.1 v0.48.1 1.21, 1.20, 1.19 3.13.5 1.20.1 v0.47.0 1.21, 1.20, 1.19 3.13.5 1.20.1 v0.46.0 1.21, 1.20, 1.19 3.13.2 1.19.6
配置指南 描述: 默认配置监视来自所有命名空间的Ingress 对象。要更改此行为，请使用标志--watch-namespace将范围限制为特定命名空间。
1 2 3 4 kubectl wait --namespace ingress-nginx \ --for =condition=ready pod \ --selector=app.kubernetes.io/component=controller \ --timeout=120s
定制NGINX有三种方式: 参考地址(https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/ )
ConfigMap: 使用ConfigMap在NGINX中设置全局配置。
Annotations: 如果您需要特定入口规则的特定配置，请使用此选项。
Custom template: 当需要更具体的设置（如打开文件缓存）时，将侦听选项调整为rcvbuf或当无法通过ConfigMap更改配置时。
sysctl 调优 (1) 积压队列设置net.core.somaxconn从128到32768, (2) 临时端口设置net.ipv4.ip_local_port_range从32768 60999到1024 65000
1 2 kubectl patch deployment -n ingress-nginx nginx-ingress-controller \ --patch="$(curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/docs/examples/customization/sysctl/patch.json) "
安装部署 Ingress-Nginx 基础环境部署流程
1 wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.47.0/deploy/static/provider/baremetal/deploy.yaml
1 2 3 4 5 6 7 8 9 ~/K8s/Day7/demo1$ cat deploy.yaml | grep "image:" | sort | uniq docker pull pollyduan/ingress-nginx-controller:v0.47.0 docker pull docker.io/jettech/kube-webhook-certgen:v1.5.0
1 2 3 4 5 ~/K8s/Day7/demo1$ docker login harbor.weiyigeek.top ~/K8s/Day7/demo1$ docker tag docker.io/jettech/kube-webhook-certgen:v1.5.0 harbor.weiyigeek.top/test /kube-webhook-certgen:v1.5.0 ~/K8s/Day7/demo1$ docker tag pollyduan/ingress-nginx-controller:v0.41.0 harbor.weiyigeek.top/test /ingress-nginx-controller:v0.41.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 sed -i "s#docker.io/jettech/kube-webhook-certgen:v1.5.0#harbor.weiyigeek.top/test/kube-webhook-certgen:v1.5.0#g" deploy.yaml sed -i "s#k8s.gcr.io/ingress-nginx/controller:v0.47.0@sha256:1f4f402b9c14f3ae92b11ada1dfe9893a88f0faeb0b2f4b903e2c67a0c3bf0de#harbor.weiyigeek.top/test/ingress-nginx-controller:v0.41.0#g" deploy.yaml ~/K8s/Day7/demo1$ kubectl apply -f deploy.yaml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 $ kubectl get ns | grep "ingress" $ kubectl get svc -n ingress-nginx $ kubectl get deployment -n ingress-nginx -o wide $ kubectl get rs -n ingress-nginx -o wide $ kubectl get pod -n ingress-nginx -o wide --show-labels
1 2 3 4 5 6 7 8 9 10 11 12 13 ~/K8s/Day7/demo1$ kubectl describe -n ingress-nginx pod ingress-nginx-admission-patch-kzx2v
基础示例 官方示例参考: https://kubernetes.github.io/ingress-nginx/examples/
示例1.Ingress 常规使用方案 Ingress HTTP 代理访问资源清单的编写
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 cat > dep-svc-nginx-http-v1.yaml<<'EOF' # Deployment 控制器 apiVersion: apps/v1 kind: Deployment metadata: name: nginx-dm spec: replicas: 2 selector: matchLabels: name: nginx-ingress-v1 template: metadata: labels: name: nginx-ingress-v1 spec: containers: - name: nginx image: harbor.weiyigeek.top/test/nginx:v1.0 imagePullPolicy: IfNotPresent ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: nginx-ingress-v1-svc spec: ports: - port: 80 targetPort: 80 protocol: TCP selector: name: nginx-ingress-v1 EOF cat > ingress-nginx-http-v1.yaml<<'EOF' # Ingress 扩展 # Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: nginx-ingress-http spec: rules: - host: web.weiyigeek.top http: paths: - pathType: Prefix path: / backend: service: name: nginx-ingress-v1-svc port: number: 80 EOF
操作流程:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 ~/K8s/Day7/demo2$ kubectl create -f dep-svc-nginx-http-v1.yaml ~/K8s/Day7/demo2$ kubectl create -f ingress-nginx-http-v1.yaml $ kubectl get svc -n ingress-nginx ~/K8s/Day7/demo2$ kubectl get ingress ~/K8s/Day7/demo2$ kubectl get pod -o wide 10.10.107.214 web.weiyigeek.top ~/K8s/Day7/demo2$ curl web.weiyigeek.top:32268/host.html ~/K8s/Day7/demo2$ curl web.weiyigeek.top:32268/host.html /K8s/Day7/demo2$ sudo ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.17.0.1:30499 rr -> 10.244.1.124:443 Masq 1 0 0 TCP 172.17.0.1:32268 rr -> 10.244.1.124:80 Masq 1 0 0 TCP 172.18.0.1:30499 rr -> 10.244.1.124:443 Masq 1 0 0 TCP 172.18.0.1:32268 rr -> 10.244.1.124:80 Masq 1 0 0 TCP 172.18.0.1:32306 rr -> 10.244.1.107:80 Masq 1 0 0 -> 10.244.1.108:80 Masq 1 0 0 -> 10.244.1.109:80 Masq 1 0 0 TCP 10.10.107.202:30499 rr -> 10.244.1.124:443 Masq 1 0 0 TCP 10.10.107.202:32268 rr -> 10.244.1.124:80 TCP 10.96.183.29:80 rr -> 10.244.1.124:80 Masq 1 0 0 TCP 10.96.183.29:443 rr -> 10.244.1.124:443 Masq 1 0 0 TCP 10.98.154.140:443 rr -> 10.244.1.124:8443 Masq 1 0 0 TCP 10.102.158.47:80 rr -> 10.244.1.125:80 Masq 1 0 0 -> 10.244.1.126:80 Masq 1 0 0
示例2.Ingress HTTPS 代理访问 描述: TLS 证书配置参考地址(https://kubernetes.github.io/ingress-nginx/user-guide/tls/),注意创建自建证书（主：浏览器不认可的）。默认得Ingress-nginx配置了SSL会自动跳转到https网页。
1 2 3 nginx.ingress.kubernetes.io/ssl-redirect nginx.ingress.kubernetes.io/force-ssl-redirect
资源清单示例:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 cat > dep-ingress-nginx-https-v3.yaml<<'EOF' apiVersion: apps/v1 kind: Deployment metadata: name: ingress-nginx-https spec: replicas: 2 selector: matchLabels: name: ingress-nginx-https-v3 template: metadata: labels: name: ingress-nginx-https-v3 spec: containers: - name: ingress-nginx-https-v3 image: harbor.weiyigeek.top/test/nginx:v3.0 imagePullPolicy: IfNotPresent ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: ingress-nginx-https-svc spec: ports: - port: 80 targetPort: 80 protocol: TCP selector: name: ingress-nginx-https-v3 EOF cat > ingress-nginx-https-v3.yaml<<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-nginx-https spec: tls: - hosts: - www3.weiyigeek.top secretName: tls-secret rules: - host: www3.weiyigeek.top http: paths: - pathType: Prefix path: / backend: service: name: ingress-nginx-https-svc port: number: 80 EOF
操作流程：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 ~/K8s/Day7/demo2$ kubectl create -f dep-ingress-nginx-https-v3.yaml openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=www3.weiyigeek.top/O=www3.weiyigeek.top" kubectl create secret tls tls-secret --key tls.key --cert tls.crt kubectl create -f ingress-nginx-https-v3.yaml ~/K8s/Day7/demo2$ kubectl get ingress ~/K8s/Day7/demo2$ kubectl get svc -n ingress-nginx 无法访问 Hostname: ingress-nginx-https-55bb4cd6fd-4rmpm ,Image Version: 3.0, Nginx Version: 1.19.4
weiyigeek.top-ingress-nginx-https
PS : 实际上我们通过Ingress资源清单添加的域名绑定主机时被写入到 ingress-nginx-controller Pod 中的nginx.conf 文件之中，这才能让我们通过自定义域名的方式访问虚拟主机;
1 2 3 4 5 6 7 8 9 10 11 12 ~/K8s/Day7/demo2$ kubectl exec -n ingress-nginx ingress-nginx-controller-849d675ffc-6rxlm -it -- /bin/sh /etc/nginx $ ls /etc/nginx $ cat nginx.conf | grep "www3.weiyigeek.top" -n
Ingress-Nginx-Https配置示例:1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 # 下面可以帮助我们认是ingress-nginx实现https是如何进行配置的; ## start server www3.weiyigeek.top server { server_name www3.weiyigeek.top ; listen 80 ; listen 443 ssl http2 ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } location / { set $namespace "default"; set $ingress_name "ingress-nginx-https"; set $service_name "ingress-nginx-https-svc"; set $service_port "80"; set $location_path "/"; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, use_port_in_redirects = false, path_type = "Prefix", }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "default-ingress-nginx-https-svc-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 5s; proxy_send_timeout 60s; proxy_read_timeout 60s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; proxy_pass http://upstream_balancer; proxy_redirect off; } } ## end server www3.weiyigeek.top
示例3.Ingress Rewrite 重写重定向访问 描述: 以下是Ingress-Nginx 重写annotations的相关属性
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 名称 描述 值类型 nginx.ingress.kubernetes.io/use-regex nginx.ingress.kubernetes.io/rewrite-target nginx.ingress.kubernetes.io/app-root nginx.ingress.kubernetes.io/ssl-redirect nginx.ingress.kubernetes.io/force-ssl-redirect annotations: kubernetes.io/ingress.class: "nginx" nginx.ingress.kubernetes.io/use-regex: "true" nginx.ingress.kubernetes.io/proxy-connect-timeout: "600" nginx.ingress.kubernetes.io/proxy-send-timeout: "600" nginx.ingress.kubernetes.io/proxy-read-timeout: "600" nginx.ingress.kubernetes.io/proxy-body-size: "10m"
Ingress Rewrite 资源清单示例:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 cat > ingress-nginx-http-rewrite-https-v3.yaml<<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-nginx-http-rewrite-https annotations: nginx.ingress.kubernetes.io/rewrite-target: https://www3.weiyigeek.top:30499 spec: rules: - host: go.weiyigeek.top http: paths: - pathType: Prefix path: / backend: service: name: ingress-nginx-https-svc port: number: 80 EOF
操作流程:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 ~/K8s/Day7/demo2$ kubectl create -f ingress-nginx-http-rewrite-https-v3.yaml ~/K8s/Day7/demo2$ kubectl get ingress ~/K8s/Day7/demo2$ kubectl get svc -n ingress-nginx ~/K8s/Day7/demo2$ curl -I http://go.weiyigeek.top:32268
补充说明: Ingress-nginx 的前后端分离(Rewrite)演示
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 tee ingress-rewrite.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-rewrite namespace: demo annotations: nginx.ingress.kubernetes.io/rewrite-target: /$2 spec: rules: - host: rewrite.weiyigeek.top http: paths: - path: /weiyigeek(/|$)(.*) pathType: ImplementationSpecific backend: service: name: backend-svc port: number: 8080 EOF
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 $ kubectl create -f ingress-rewrite.yaml $ kubectl describe ing -n demo ingress-rewrite $ curl http://10.10.107.221 -H 'host: rewrite.weiyigeek.top' -k $ curl -v http://10.10.107.221/weiyigeek/ -H 'host: rewrite.weiyigeek.top' -k hostname-web-backend-1-WeiyiGeek-Tomcat $ curl http://10.10.107.221/weiyigeek/test -H 'host: rewrite.weiyigeek.top' -k
示例4.Ingress VirtualHost 虚拟主机访问 描述: 我们知道Ingress-Nginx是基于Nginx的所以其也支持虚拟主机进行绑定访问，实现以不同的域名访问同一个web界面;
Ingress 资源清单示例:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 cat > ingress-nginx-virtual-host.yaml<<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-nginx-virtual-host annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: defaultBackend: service: name: nginx-ingress-v1-svc port: number: 80 rules: - host: host.weiyigeek.top http: paths: - path: / pathType: Prefix backend: service: name: nginx-ingress-v1-svc port: number: 80 - path: /v3 pathType: Prefix backend: service: name: ingress-nginx-https-svc port: number: 80 - host: ingress.test2.com http: paths: - path: / backend: serviceName: httpd-svc servicePort: 80 EOF cat > ingress-nginx-virtual-host.yaml<<'EOF' apiVersion: extensions/v1beta1 kind: Ingress metadata: name: path-ingress spec: rules: - host: host.weiyigeek.top http: paths: - path: /v1 backend: serviceName: nginx-ingress-v1-svc servicePort: 80 - path: /v2 backend: serviceName: ingress-nginx-https-svc servicePort: 80 - host: ingress.weiyigeek.top http: paths: - path: / backend: serviceName: httpd-svc servicePort: 80 EOF
操作流程：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ~/K8s/Day7/demo2$ kubectl explain --api-version=networking.k8s.io/v1 ingress.spec.rules.http.paths kubectl explain --api-version=networking.k8s.io/v1beta1 ingress.spec.rules.http.paths ~/K8s/Day7/demo2$ kubectl create -f ingress-nginx-virtual-host.yaml ~/K8s/Day7/demo2$ kubectl get ingress | grep "virtual" 10.10.107.214 host.weiyigeek.top 10.10.107.214 ingress.weiyigeek.top $ kubectl describe ingress-nginx-virtual-host
示例5.Ingress BasicAuth（基础认证）功能 描述:因为Ingress的实现方案采用的是Nginx的软件，所以Nginx的相关特性Ingress都支持, 下面我们利用 Nginx 的 BasicAuth 实现基础认证演示
Ingress 资源清单:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 cat > ingress-nginx-auth.yaml<<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-nginx-auth annotations: nginx.ingress.kubernetes.io/auth-type: basic nginx.ingress.kubernetes.io/auth-secret: basic-auth nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - WeiyiGeek' spec: rules: - host: auth.weiyigeek.top http: paths: - path: / pathType: Prefix backend: service: name: nginx-ingress-v1-svc port: number: 80 defaultBackend: service: name: nginx-ingress-v1-svc port: number: 80 EOF
操作流程:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 $ apt search htpasswd $ sudo apt install apache2-utils $ htpasswd -c auth weiyigeek $ kubectl create secret generic basic-auth --from-file=auth weiyigeek@ubuntu:~/K8s/Day7/demo2$ kubectl create -f ingress-nginx-auth.yaml weiyigeek@ubuntu:~/K8s/Day7/demo2$ kubectl get ingress weiyigeek@ubuntu:~/K8s/Day7/demo2$ kubectl describe ingress ingress-nginx-auth ~/K8s/Day7/demo2$ curl -I http://auth.weiyigeek.top:32268
weiyigeek.top-WWW-Authenticate
示例6.Ingress Redirect 域名重定向功能 描述: 我们可以利用Ingress中的域名重定向(Redirect)注解来实现访问重定向。
1 2 3 annotations: nginx.ingress.kubernetes.io/permanent-redirect: 'https://blog.weiyigeek.top'
Ingress 规则配置清单示例:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 tee Ingress-Redirect.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-redirect namespace: demo annotations: nginx.ingress.kubernetes.io/permanent-redirect: 'https://blog.weiyigeek.top' spec: rules: - host: redirect.weiyigeek.top http: paths: - path: / pathType: Prefix backend: service: name: front-svc port: number: 80 defaultBackend: service: name: front-svc port: number: 80 EOF
部署资源清单以及查看
1 2 3 4 5 6 7 8 9 10 11 12 kubectl create -f Ingress-Redirect.yaml kubectl get ing -n demo ingress-redirect && kubectl describe ing -n demo ingress-redirect .... curl -v http://10.10.107.221 -H 'host: redirect.weiyigeek.top' -k curl -v http://10.10.107.221 -H 'host: r.weiyigeek.top' -k
weiyigeek.top-redirect
示例7.Ingress 黑白名单访问限制 描述: 某些内部系统可能只允许某些IP地址访问时，可利用 Ingress-nginx 的黑白名单，其实现的方式有如下两个。
Annotations(注解) ：只对指定的ingress生效nginx.ingress.kubernetes.io/whitelist-source-range
ConfigMap(配置文件) ：全局生效
Tips: 建议黑名单可以使用ConfigMap去配置，白名单建议使用Annotations去配置。
Step 1.添加白名单的方式可以直接写annotation也可以配置在ConfigMap中。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 tee ingress-whilte-annotation.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-whilte namespace: demo annotations: nginx.ingress.kubernetes.io/service-weight: '1' nginx.ingress.kubernetes.io/whitelist-source-range: 10.20.172.103,10.10.107.220 spec: rules: - host: whilte.weiyigeek.top http: paths: - path: / pathType: ImplementationSpecific backend: service: name: front-svc port: number: 80 EOF tee ingress-whilte-ConfigMap.yaml <<'EOF' apiVersion: v1 kind: ConfigMap metadata: name: ingress-whilte namespace: demo data: whitelist-source-range: 10.10.107.0/24 EOF $ kubectl create -f ingress-whilte-annotation.yaml $ kubectl get ingress -n demo ingress-whilte $ kubectl describe ingress -n demo ingress-whilte 10.10.107.220 $ curl http://10.10.107.221/ -H 'host: whilte.weiyigeek.top' -k -i 10.10.107.221 $ curl http://10.10.107.221/ -H 'host: whilte.weiyigeek.top' -k -i
Step 2.黑名单的方式也可以在annotations与ConfigMap中配置。nginx.ingress.kubernetes.io/server-snippet: |-则可以加入自定义片段。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 tee ingress-black-annotation.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-black namespace: demo annotations: nginx.ingress.kubernetes.io/server-snippet: |- deny 10.10.107.220; deny 10.20.172.103; allow all; spec: rules: - host: black.weiyigeek.top http: paths: - path: / pathType: ImplementationSpecific backend: service: name: backend-svc port: number: 8080 status: loadBalancer: {} EOF tee ingress-black-ConfigMap.yaml <<'EOF' apiVersion: v1 kind: ConfigMap metadata: name: ingress-black namespace: demo data: whitelist-source-range: 10.20.172.0/24 block-cidrs: 10.10.107.220 EOF kubectl apply -f ingress-black-annotation.yaml kubectl get ing -n demo ingress-black && kubectl describe ing -n demo ingress-black curl http://10.10.107.221/ -H 'host: black.weiyigeek.top' -k -i curl http://10.10.107.221/ -H 'host: black.weiyigeek.top' -k -i
描述: Ingress-nginx 的 匹配请求头，可以采用nginx.ingress.kubernetes.io/server-snippet注解。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 tee ingress-header-request.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-header namespace: demo annotations: nginx.ingress.kubernetes.io/server-snippet: |- set $agentflag 0; if ($http_user_agent ~* "(Firefox)" ){ set $agentflag 1; } if ($agentflag = 1) { return 301 https://www.weiyigeek.top; } spec: rules: - host: header.weiyigeek.top http: paths: - path: / pathType: ImplementationSpecific backend: service: name: backend-svc port: number: 8080 status: loadBalancer: {} EOF kubectl create -f ingress-header-request.yaml kubectl get ing -n demo ingress-header && kubectl describe ing -n demo ingress-header $ curl http://10.10.107.221/ -H 'host: header.weiyigeek.top' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0' -k -i $ curl http://10.10.107.221/ -H 'host: header.weiyigeek.top' -H 'User-Agent: Mozilla/5.0' -k -i
示例9.Ingress 灰度金丝雀发布 描述: 在某些情况下，您可能希望通过向与生产服务不同的服务发送少量请求来“金丝雀”一组新的更改。金丝雀注解使 Ingress 规范能够根据应用的规则充当路由请求的替代服务,在nginx.ingress.kubernetes.io/canary: "true"设置后可以启用以下用于配置金丝雀的注释：。
Nginx Annotations 支持以下 4 种 Canary 规则:
1 2 3 4 5 * nginx.ingress.kubernetes.io/canary-weight：基于服务权重的流量切分，适用于蓝绿部署，权重范围 0 - 100 按百分比将请求路由到 Canary Ingress 中指定的服务。权重为 0 意味着该金丝雀规则不会向 Canary 入口的服务发送任何请求。权重为 100 意味着所有请求都将被发送到 Canary 入口。 * nginx.ingress.kubernetes.io/canary-by-header：基于 Request Header 的流量切分，适用于灰度发布以及 A/B 测试。当 Request Header 设置为 always时，请求将会被一直发送到 Canary 版本；当 Request Header 设置为 never时，请求不会被发送到 Canary 入口；对于任何其他 Header 值，将忽略 Header，并通过优先级将请求与其他金丝雀规则进行优先级的比较。 * nginx.ingress.kubernetes.io/canary-by-header-value：要匹配的 Request Header 的值，用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务。当 Request Header 设置为此值时，它将被路由到 Canary 入口。该规则允许用户自定义 Request Header 的值，必须与上一个 annotation (即：canary-by-header）一起使用。 * nginx.ingress.kubernetes.io/canary-by-header-pattern：这与canary-by-header-valuePCRE 正则表达式匹配的工作方式相同。请注意，canary-by-header-value设置此注释时将被忽略。当给定的 Regex 在请求处理过程中导致错误时，该请求将被视为不匹配。 * nginx.ingress.kubernetes.io/canary-by-cookie：基于 Cookie 的流量切分，适用于灰度发布与 A/B 测试。用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务的cookie。当 cookie 值设置为 always时，它将被路由到 Canary 入口；当 cookie 值设置为 never时，请求不会被发送到 Canary 入口；对于任何其他值，将忽略 cookie 并将请求与其他金丝雀规则进行优先级的比较。 定义两个版本的代码。
Tips : 金丝雀规则是按优先顺序计算的。优先级如下：canary by header->canary by cookie->canary weight
Tips : 注意当您将入口标记为 Canary 时，除nginx.ingress.kubernetes.io/load-balanceand之外的所有其他非 Canary 注释都将被忽略（从相应的主入口继承）nginx.ingress.kubernetes.io/upstream-hash-by。`
资源配置清单:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 tee ingress-canary-v1.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-canary-v1 namespace: demo annotations: nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-by-header: canary nginx.ingress.kubernetes.io/canary-by-header-value: v1 nginx.ingress.kubernetes.io/canary-weight: "50" spec: rules: - host: canary.weiyigeek.top http: paths: - path: / pathType: ImplementationSpecific backend: service: name: front-svc port: number: 80 EOF tee ingress-canary-v2.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-canary-v2 namespace: demo annotations: nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-by-header: canary nginx.ingress.kubernetes.io/canary-by-header-value: v2 nginx.ingress.kubernetes.io/canary-weight: "50" spec: rules: - host: canary.weiyigeek.top http: paths: - path: / pathType: ImplementationSpecific backend: service: name: backend-svc port: number: 8080 EOF ~/k8s/ingress/demo1 ~/k8s/ingress/demo1 ~/k8s/ingress/demo1 for i in {1..3};do curl http://10.10.107.221/ -H 'host: canary.weiyigeek.top' -i;done for i in {1..3};do curl http://10.10.107.221/ -H 'host: canary.weiyigeek.top' -H 'weiyigeek: v2' -i;done
示例10.Ingress 相关配置补充
描述: 可以修改 deploy.yaml 中指定 nginx-ingress-controller 运行的参数，在此处添加- --default-backend-service=名称空间/SVC名称参数
1 2 3 4 5 6 $ grep -A 3 -n "/nginx-ingress" deploy.yaml 332: - /nginx-ingress-controller 333- - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller 334- - --election-id=ingress-controller-leader 335- - --ingress-class=nginx - --default-backend-service=demo/error-svc
(2) 连接和传输速率限制
1 2 3 4 5 6 7 * nginx.ingress.kubernetes.io/limit-connections：单个IP地址允许的并发连接数。超出此限制时，将返回503错误。 * nginx.ingress.kubernetes.io/limit-rps：每秒从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数，默认乘数为5。当客户端超过此限制时，将 返回limit-req-status-code默认值： 503 。 * nginx.ingress.kubernetes.io/limit-rpm：每分钟从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数，默认乘数为5。当客户端超过此限制时，将 返回limit-req-status-code默认值： 503 。 * nginx.ingress.kubernetes.io/limit-burst-multiplier：突发大小限制速率的倍数。默认的脉冲串乘数为5，此注释将覆盖默认的乘数。当客户端超过此限制时，将 返回limit-req-status-code默认值： 503 。 * nginx.ingress.kubernetes.io/limit-rate-after：最初的千字节数，在此之后，对给定连接的响应的进一步传输将受到速率的限制。必须在启用代理缓冲的情况下使用此功能。 * nginx.ingress.kubernetes.io/limit-rate：每秒允许发送到给定连接的千字节数。零值禁用速率限制。必须在启用代理缓冲的情况下使用此功能。 * nginx.ingress.kubernetes.io/limit-whitelist：客户端IP源范围要从速率限制中排除。该值是逗号分隔的CIDR列表。
示例说明:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 tee ingress-limit.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-limit namespace: demo annotations: nginx.ingress.kubernetes.io/limit-rate: "100K" nginx.ingress.kubernetes.io/limit-whitelist: "10.10.107.220" nginx.ingress.kubernetes.io/limit-rps: 1 nginx.ingress.kubernetes.io/limit-rpm: 30 spec: rules: - host: limit.weiyigeek.top http: paths: - path: / pathType: ImplementationSpecific backend: service: name: backend-svc port: number: 8080 status: loadBalancer: {} EOF
(3) ConfigMap在Ingress Controller中实战
Tips : 通过 Ingress ConfigMaps 查看详细的配置参数。
Ingress 规则存储配置清单：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 tee nginx-config.yaml <<'EOF' kind: ConfigMap apiVersion: v1 metadata: name: ingress-nginx-controller namespace: ingress-nginx labels: app: ingress-nginx data: proxy-connect-timeout: "15" EOF --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx
nginx-ingress-controller 使用 Configmap 作为配置存在控制器:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 vim +328 deploy.yaml containers: - name: controller image: pollyduan/ingress-nginx-controller:v0.47.0 imagePullPolicy: IfNotPresent lifecycle: preStop: exec : command : - /wait -shutdown args: - /nginx-ingress-controller - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller - --election-id=ingress-controller-leader - --ingress-class=nginx - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller - --validating-webhook=:8443 - --validating-webhook-certificate=/usr/local /certificates/cert - --validating-webhook-key=/usr/local /certificates/key
查看ingress-nginx名称空间中的ConfigMap配置
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 kubectl apply -f nginx-config.yaml kubectl get cm -n ingress-nginx kubectl describe cm -n ingress-nginx ingress-nginx-controller
验证是否在ingress-nginx-controller中配置生效:
1 2 3 4 5 6 7 8 kubectl get pod -n ingress-nginx -l app.kubernetes.io/component=controller root@k8s-master-1:~/k8s/ingress/deployment bash-5.1$ grep "proxy_connect_timeout" nginx.conf proxy_connect_timeout 15s;
企业实战 1.利用Ingress访问后端的Nginx Web和Tomcat后端综合实践。 实验目标:
1) 在腾讯云创建test.weiyigeek.top以及demo.weiyigeek.top证书以供Ingress使用。
2) 分别在kubernetes集群中创建Nginx Web与Tomcat 后端应用。
3) 分别使用Ingress控制器创建以SSL访问/为NginxWeb而/tomcatTomcat 后端需要BasicAuth（基础认证）访问功能
详细流程:
weiyigeek.top-SSL证书申请
Step 2.分别在kubernetes集群中创建Nginx Web与Tomcat 后端应用以及创建名称空间。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 tee sts-svc-nginx-tomcat.yaml<<'EOF' apiVersion: v1 kind: Namespace metadata: name: demo --- apiVersion: v1 kind: Service metadata: name: front-svc namespace: demo labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: selector: app: nginx-web ports: - port: 80 targetPort: 80 protocol: TCP --- apiVersion: v1 kind: Service metadata: name: backend-svc namespace: demo labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: selector: app: tomcat-web ports: - port: 8080 targetPort: 8080 protocol: TCP sessionAffinity: None --- apiVersion: apps/v1 kind: StatefulSet metadata: name: web-front namespace: demo spec: replicas: 2 serviceName: front-svc selector: matchLabels: app: nginx-web template: metadata: labels: app: nginx-web spec: initContainers: - name: init-html image: busybox:latest imagePullPolicy: IfNotPresent command: ['sh', '-c' , "echo hostname-${HOSTNAME}-${MSG} > /usr/share/nginx/html/index.html" ] env: - name: MSG value: "WeiyiGeek-Nginx" volumeMounts: - name: web mountPath: "/usr/share/nginx/html" securityContext: privileged: true containers: - name: nginx image: nginx:latest imagePullPolicy: IfNotPresent env: - name: TZ value: "Asia/Shanghai" ports: - name: nginx containerPort: 80 volumeMounts: - name: web mountPath: "/usr/share/nginx/html" volumes: - name: web emptyDir: {} --- apiVersion: apps/v1 kind: StatefulSet metadata: name: web-backend namespace: demo spec: replicas: 2 serviceName: backend-svc selector: matchLabels: app: tomcat-web template: metadata: labels: app: tomcat-web spec: initContainers: - name: init-html image: busybox:latest imagePullPolicy: IfNotPresent command: ['sh', '-c' , "echo hostname-${HOSTNAME}-${MSG} > /usr/local/tomcat/webapps/ROOT/index.html" ] env: - name: MSG value: WeiyiGeek-Tomcat volumeMounts: - name: web mountPath: "/usr/local/tomcat/webapps/ROOT/" securityContext: privileged: true containers: - name: nginx image: "tomcat:8.5.69-jdk8-openjdk-slim-buster" imagePullPolicy: IfNotPresent env: - name: TZ value: "Asia/Shanghai" ports: - name: nginx containerPort: 8080 volumeMounts: - name: web mountPath: "/usr/local/tomcat/webapps/ROOT/" volumes: - name: web emptyDir: {} EOF kubectl create -f sts-svc-nginx-tomcat.yaml
Step 3.验证SVC服务负载均衡和请求查看创建的应用
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 $ kubectl get svc -n demo -o wide $ kubectl get sts -n demo -o wide $ kubectl get pod -n demo -o wide --show-labels $ nslookup front-svc.demo.svc.cluster.local. 10.96.0.10 $ nslookup backend-svc.demo.svc.cluster.local. 10.96.0.10 ~/k8s/ingress/demo ~/k8s/ingress/demo ~/k8s/ingress/demo ~/k8s/ingress/demo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 $ ls demo.weiyigeek.top.zip test.weiyigeek.top.zip $ unzip demo.weiyigeek.top.zip -d demo.weiyigeek.top $ unzip test.weiyigeek.top.zip -d test.weiyigeek.top $ cd /tmp/demo.weiyigeek.top/Nginx kubectl -n demo create secret tls demoweiyigeektop --key 2_demo.weiyigeek.top.key --cert 1_demo.weiyigeek.top_bundle.crt $ cd /tmp/test.weiyigeek.top/Nginx kubectl -n demo create secret tls testweiyigeektop --key 2_test.weiyigeek.top.key --cert 1_test.weiyigeek.top_bundle.crt
Step 4.创建Ingress控制器的配置清单文件及Ingress对应规则。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 tee ingress-demo.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-demo namespace: demo annotations: kubernetes.io/ingress.class: "nginx" nginx.ingress.kubernetes.io/use-regex: "true" nginx.ingress.kubernetes.io/proxy-connect-timeout: "600" nginx.ingress.kubernetes.io/proxy-send-timeout: "600" nginx.ingress.kubernetes.io/proxy-read-timeout: "600" nginx.ingress.kubernetes.io/proxy-body-size: "10m" nginx.ingress.kubernetes.io/rewrite-target: / spec: defaultBackend: service: name: front-svc port: number: 80 tls: - hosts: - test.weiyigeek.top secretName: testweiyigeektop - hosts: - demo.weiyigeek.top secretName: demoweiyigeektop rules: - host: test.weiyigeek.top http: paths: - pathType: Prefix path: /tomcat backend: service: name: backend-svc port: number: 8080 - pathType: Prefix path: /nginx backend: service: name: front-svc port: number: 80 - host: demo.weiyigeek.top http: paths: - pathType: Prefix path: / backend: service: name: front-svc port: number: 80 EOF
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 $ kubectl create -f ingress-demo.yaml kubectl get svc -n ingress-nginx $ kubectl get ing -n demo ingress-demo $ kubectl describe ing -n demo ingress-demo
Step 6.验证Ingress规则配置的应用分别访问 https://demo.weiyigeek.top 与 https://test.weiyigeek.top 站点。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 $ curl 'https://10.10.107.221' -H 'host: demo.weiyigeek.top' -i -k $ curl 'https://10.10.107.221/tomcat' -H 'host: test.weiyigeek.top' -i -k
weiyigeek.top-Ingress 应用
