[TOC]

0x00 前言简述

描述: 在Kubernetes中可以非常方便的进行应用的扩容缩,也能非常方便的进行业务的迭代,此处由于开发测试的需求则在Kubernetes搭建了单实例和Redis集群主从同步的环境;


0x01 Redis 单实例实践

(1) 配置准备
配置文件: Redis.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# /nfsdisk/datastore/redis/single/Redis.conf
# 开启的端口
port 6379
# 密码设置
requirepass weiyigeek.top
# 开启Redis的AOF持久化 && appendonly日志文件
appendonly yes
appendfsync everysec
appendfilename appendonly.aof
# AOF持久化文件存在的位置及文件名称
dir /data
dbfilename dump.rdb
# 备份进程出错时,主进程停止接受写入操作,默认yes
stop-writes-on-bgsave-error yes
# 是否压缩,会占用部分cpu资源,默认yes
rdbcompression yes
# RDB自动触发策略是否启用,默认为yes
rdb-save-incremental-fsync yes
# 写入频率
save 300 10


Redis single 资源清单文件:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
# cat Redis-single.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis-single
namespace: database
spec:
serviceName: redis-single
replicas: 1
selector:
matchLabels:
app: redis-single
template:
metadata:
labels:
app: redis-single
spec:
containers:
- name: redis
image: redis:5.0.10-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 6379
name: client
command: [ "redis-server", "/conf/redis.conf"]
volumeMounts:
- name: conf
mountPath: /conf
- name: data
mountPath: /data
- name: timezone
mountPath: /etc/localtime # 时区设置
volumes:
- name: conf
hostPath: # 采用hostPath卷
type: DirectoryOrCreate # 卷类型此处选择如果DirectoryOrCreate如何子节点上没有该目录便会进行创建
path: /nfsdisk/datastore/redis/single # 各主机节点上已存在的目录
- name: timezone # 时区定义
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "managed-nfs-storage"
resources:
requests:
storage: 5Gi
---
apiVersion: v1
kind: Service
metadata:
name: redis-single
namespace: database
spec:
type: ClusterIP
ports:
- port: 6379
targetPort: 6379
name: client
selector:
app: redis-single


操作流程:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
# (1) 名称空间的创建 (PS:非常注意在删除名称空间时需要查看是否有存在Pod)
kubectl create namespace database

# (2) 按照资源清单构建Pod
kubectl create -f Redis-single.yaml

# (3) 查看部署结果(Pod以及SVC)
$ kubectl get pod -n database -l app=redis-single -o wide
# NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
# redis-single-0 1/1 Running 0 2d17h 172.16.182.221 weiyigeek-226 <none> <none>

$ kubectl get svc -n database
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# redis-single NodePort 10.109.0.159 <none> 6379:31001/TCP 2d17h


# (4) 查看Pod运行描述容器及其挂载信息
~$ kubectl describe pod -n database redis-single
# Status: Running
# IP: 172.16.182.221
# IPs:
# IP: 172.16.182.221
# Controlled By: StatefulSet/redis-single
# Containers:
# redis:
# Container ID: docker://c94f83f88e3a2cba36f6139b69746387981d0d6d0a3c44b72c0a21a91f977ad8
# Image: redis:5.0.10-alpine
# Image ID: docker-pullable://[email protected]:cab1de051c243a956749397af796d50bf235c7183e88bc4667259f66cad4f748
# Port: 6379/TCP
# Host Port: 0/TCP
# Command:
# redis-server
# /conf/redis.conf
# State: Running
# Started: Fri, 15 Jan 2021 09:48:39 +0000
# Ready: True
# Restart Count: 0
# Environment: <none>
# Mounts:
# /conf from conf (rw)
# /data from data (rw)
# /etc/localtime from timezone (rw)
# /var/run/secrets/kubernetes.io/serviceaccount from default-token-mhbbf (ro)
# Volumes:
# data:
# Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
# ClaimName: data-redis-single-0
# ReadOnly: false
# conf:
# Type: HostPath (bare host directory volume)
# Path: /nfsdisk-31/datastore/redis/single
# HostPathType: DirectoryOrCreate
# timezone:
# Type: HostPath (bare host directory volume)
# Path: /usr/share/zoneinfo/Asia/Shanghai
# HostPathType:
# default-token-mhbbf:
# Type: Secret (a volume populated by a Secret)
# SecretName: default-token-mhbbf
# Optional: false


# (5) Redis 连接测试
~$ kubectl exec -it -n database redis-single-0 sh
# /data # redis-c
# redis-check-aof redis-check-rdb redis-cli
# /data # redis-cli
# 127.0.0.1:6379> auth weiyigeek.top
# OK
# 127.0.0.1:6379> ping
# PONG
# 127.0.0.1:6379> info
# # Server
# redis_version:5.0.10
# redis_git_sha1:00000000
# redis_git_dirty:0
# redis_build_id:e19997e81345d7a5
# redis_mode:standalone
# os:Linux 5.4.0-42-generic x86_64
# arch_bits:64
# multiplexing_api:epoll
# atomicvar_api:atomic-builtin
# gcc_version:9.3.0
# process_id:1
# run_id:efc925af95781045bed44686f30a6e78190e0e66
# tcp_port:6379
# uptime_in_seconds:235758
# uptime_in_days:2
# hz:10
# configured_hz:10
# lru_clock:327141
# executable:/data/redis-server
# config_file:/conf/redis.conf

0x02 Redis 集群主从实践

(1) 自定义集群构建

配置文件说明:
redis 配置文件的使用方式

  • (1) 方式1.创建 Configmap kubectl create configmap redis-conf --from-file=redis.conf
  • (2) 方式2.利用 Hostpath Volumes 卷挂载该配置文件
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    cat > redis.conf <<'EOF'
    # 监听端口
    port 6379
    # 启用外部连接关闭安全模式
    protected-mode no
    masterauth weiyigeek.top
    requirepass weiyigeek.top
    # 开启Redis的AOF持久化 && 日志文件
    appendonly yes
    appendfilename appendonly.aof
    # AOF持久化文件存在的位置以及其文件名称
    dir /data
    dbfilename dump.rdb
    slave-read-only yes
    # 每秒钟同步一次折中的方案
    appendfsync everysec
    no-appendfsync-on-rewrite no
    auto-aof-rewrite-percentage 100
    auto-aof-rewrite-min-size 64mb
    # 集群模式打开
    cluster-enabled yes
    cluster-config-file /data/nodes.conf
    cluster-node-timeout 5000
    # 当负责一个插槽的主库下线且没有相应的从库进行故障恢复时集群仍然可用
    cluster-require-full-coverage no
    # 只有当一个主节点至少拥有其他给定数量个处于正常工作中的从节点的时候,才会分配从节点给集群中孤立的主节点
    cluster-migration-barrier 1
    EOF

补充说明:

  • Redis 配置文件: /conf/redis.conf
  • 集群配置更新文件: /conf/update-node.sh
  • 数据存储目录:/data
  • 集群节点配置文件: /data/nodes.conf


Redis-cluster 资源清单:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
cat > Redis-cluster-5.0.10.yaml <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-cluster
namespace: database
data:
# 外部命令参数传递执行, 精妙之处值得学习
update-node.sh: |
#!/bin/sh
REDIS_NODES="/data/nodes.conf"
sed -i -e "/myself/ s/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/${POD_IP}/" ${REDIS_NODES}
exec "[email protected]"
redis.conf: |+
# 监听端口
port 6379
# 启用外部连接关闭安全模式
protected-mode no
masterauth weiyigeek.top
requirepass weiyigeek.top
# 开启Redis的AOF持久化 && 日志文件
appendonly yes
appendfilename appendonly.aof
# AOF持久化文件存在的位置以及其文件名称
dir /data
dbfilename dump.rdb
slave-read-only yes
# 每秒钟同步一次折中的方案
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
# 集群模式打开
cluster-enabled yes
cluster-config-file /data/nodes.conf
cluster-node-timeout 5000
# 当负责一个插槽的主库下线且没有相应的从库进行故障恢复时集群仍然可用
cluster-require-full-coverage no
# 只有当一个主节点至少拥有其他给定数量个处于正常工作中的从节点的时候,才会分配从节点给集群中孤立的主节点
cluster-migration-barrier 1
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis-cluster
namespace: database
spec:
serviceName: redis-cluster
replicas: 6
selector:
matchLabels:
app: redis-cluster
template:
metadata:
labels:
app: redis-cluster
spec:
containers:
- name: redis
image: redis:5.0.10-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 6379
name: client
- containerPort: 16379
name: gossip
command: ["/conf/update-node.sh", "redis-server", "/conf/redis.conf"]
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
volumeMounts:
- name: conf
mountPath: /conf
readOnly: false
- name: data
mountPath: /data
readOnly: false
- name: timezone
mountPath: /etc/localtime # 在Pod中时区设置(挂载主机的时区)
volumes:
- name: conf
configMap:
name: redis-cluster
defaultMode: 0755
- name: timezone
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "managed-nfs-storage"
resources:
requests:
storage: 5Gi
---
# headless Service
apiVersion: v1
kind: Service
metadata:
name: redis-cluster
namespace: database
spec:
clusterIP: "None"
ports:
- port: 6379
targetPort: 6379
name: client
- port: 16379
targetPort: 16379
name: gossip
selector:
app: redis-cluster
EOF


Redis 资源清单构建:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# (1) 名称空间创建
$ kubectl create namespace database

# (2) 按照资源清单StatefulSet控制器创建Pod
$ kubectl create -f Redis-cluster-5.0.10.yaml
configmap/redis-cluster created
statefulset.apps/redis-cluster created
service/redis-cluster created

# (3) 查看应用的 statefulsets,pod,svc 等信息
$ kubectl get statefulsets,pod,svc -n database -o wide --show-labels | grep -v "redis-single"
# NAME READY AGE CONTAINERS IMAGES LABELS
# statefulset.apps/redis-cluster 6/6 27m redis redis:5.0.10-alpine <none>

# NAME READY STATUS RESTARTS AGE IP NODE LABELS
# pod/redis-cluster-0 1/1 Running 0 27m 172.16.183.81 weiyigeek-225 app=redis-cluster,controller-revision-hash=redis-cluster-6bf876ccf9,statefulset.kubernetes.io/pod-name=redis-cluster-0
# pod/redis-cluster-1 1/1 Running 0 26m 172.16.182.226 weiyigeek-226 app=redis-cluster,controller-revision-hash=redis-cluster-6bf876ccf9,statefulset.kubernetes.io/pod-name=redis-cluster-1
# pod/redis-cluster-2 1/1 Running 0 26m 172.16.24.204 weiyigeek-223 app=redis-cluster,controller-revision-hash=redis-cluster-6bf876ccf9,statefulset.kubernetes.io/pod-name=redis-cluster-2
# pod/redis-cluster-3 1/1 Running 0 26m 172.16.100.67 weiyigeek-224 app=redis-cluster,controller-revision-hash=redis-cluster-6bf876ccf9,statefulset.kubernetes.io/pod-name=redis-cluster-3
# pod/redis-cluster-4 1/1 Running 0 26m 172.16.243.67 weiyigeek-109 app=redis-cluster,controller-revision-hash=redis-cluster-6bf876ccf9,statefulset.kubernetes.io/pod-name=redis-cluster-4
# pod/redis-cluster-5 1/1 Running 0 26m 172.16.135.195 weiyigeek-108 app=redis-cluster,controller-revision-hash=redis-cluster-6bf876ccf9,statefulset.kubernetes.io/pod-name=redis-cluster-5

# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR LABELS
# service/redis-cluster ClusterIP None <none> 6379/TCP,16379/TCP 25m app=redis-cluster <none>


Redis 集群配置:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
# (1) 获取各个pod的IP地址
kubectl get pod -n database -l app=redis-cluster -o jsonpath='{ range.items [*]}{.status.podIP}:6379 '| sed "s# :6379 ##g"
# 172.16.182.219:6379 172.16.183.72:6379 172.16.24.202:6379 172.16.100.65:6379 172.16.135.193:6379 172.16.243.65:6379

# (2) 进入redis-cluster-0 pod中的shell
kubectl exec -n database -it redis-cluster-0 sh

# (3) 为此让我们运行以下命令并输入'yes'接受配置。我们将看到前三个节点将被选择为主节点,后三个节点将被选择为从节点。
/data $ redis-cli --cluster create --cluster-replicas 1 172.16.182.219:6379 172.16.183.72:6379 172.16.24.202:6379 172.16.100.65:6379 172.16.135.193:6379 172.16.243.65:6379
# >>> Performing hash slots allocation on 6 nodes...
# Master[0] -> Slots 0 - 5460
# Master[1] -> Slots 5461 - 10922
# Master[2] -> Slots 10923 - 16383
# Adding replica 172.16.100.65:6379 to 172.16.182.219:6379
# Adding replica 172.16.135.193:6379 to 172.16.183.72:6379
# Adding replica 172.16.243.65:6379 to 172.16.24.202:6379
# M: be20868bb9de5688d50c54d2654ffe1f11c28794 172.16.182.219:6379
# slots:[0-5460] (5461 slots) master
# M: ba71016f951653985ff1ad42528f4c09bcf51ddb 172.16.183.72:6379
# slots:[5461-10922] (5462 slots) master
# M: fb2f078d21d1efbcf93dc43cbead6aff9ea9eb76 172.16.24.202:6379
# slots:[10923-16383] (5461 slots) master
# S: 17244c7a73416380f9ed254ee9d8b1b56836e0a0 172.16.100.65:6379
# replicates be20868bb9de5688d50c54d2654ffe1f11c28794
# S: 677b964d754148db24e953bdd4e08de3a89c4eed 172.16.135.193:6379
# replicates ba71016f951653985ff1ad42528f4c09bcf51ddb
# S: 200f775a8e0f83b9e39cd1bf597d65aba53d5f26 172.16.243.65:6379
# replicates fb2f078d21d1efbcf93dc43cbead6aff9ea9eb76
# Can I set the above configuration? (type 'yes' to accept): yes
# >>> Nodes configuration updated
# >>> Assign a different config epoch to each node
# >>> Sending CLUSTER MEET messages to join the cluster
# Waiting for the cluster to join
# ....
# >>> Performing Cluster Check (using node 172.16.182.219:6379)
# M: be20868bb9de5688d50c54d2654ffe1f11c28794 172.16.182.219:6379
# slots:[0-5460] (5461 slots) master
# 1 additional replica(s)
# M: fb2f078d21d1efbcf93dc43cbead6aff9ea9eb76 172.16.24.202:6379
# slots:[10923-16383] (5461 slots) master
# 1 additional replica(s)
# S: 677b964d754148db24e953bdd4e08de3a89c4eed 172.16.135.193:6379
# slots: (0 slots) slave
# replicates ba71016f951653985ff1ad42528f4c09bcf51ddb
# M: ba71016f951653985ff1ad42528f4c09bcf51ddb 172.16.183.72:6379
# slots:[5461-10922] (5462 slots) master
# 1 additional replica(s)
# S: 17244c7a73416380f9ed254ee9d8b1b56836e0a0 172.16.100.65:6379
# slots: (0 slots) slave
# replicates be20868bb9de5688d50c54d2654ffe1f11c28794
# S: 200f775a8e0f83b9e39cd1bf597d65aba53d5f26 172.16.243.65:6379
# slots: (0 slots) slave
# replicates fb2f078d21d1efbcf93dc43cbead6aff9ea9eb76
# [OK] All nodes agree about slots configuration.
# >>> Check for open slots...
# >>> Check slots coverage...
# [OK] All 16384 slots covered.

Tips: 也可以利用下面两条命令一步到位的方式:

1
2
export REDIS_POD_IP=$(kubectl get pod -n database -l app=redis-cluster -o jsonpath='{ range.items [*]}{.status.podIP}:6379 '| sed "s# :6379 ##g")
kubectl exec -it -n database redis-cluster-0 -- sh -c "/usr/local/bin/redis-cli -a weiyigeek.top --cluster create --cluster-replicas 1 ${REDIS_POD_IP}"

ps : 在集群中我们可以利用SVC服务发现的机制采用 redis-cluster-1.redis-cluster.database.svc.cluster.local 域名的方式访问Pod;


Redis 集群访问验证:

  • 1.先以单台为例查看集群状态

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    ~/k8s/redis-master-slave$ kubectl exec -it -n database redis-cluster-0 -- sh -c "redis-cli -a weiyigeek.top ping"
    # PONG

    ~/k8s/redis-master-slave$ kubectl exec -it -n database redis-cluster-0 -- sh -c "redis-cli -a weiyigeek.top cluster info"
    # cluster_state:ok
    # cluster_slots_assigned:16384
    # cluster_slots_ok:16384
    # cluster_slots_pfail:0
    # cluster_slots_fail:0
    # cluster_known_nodes:6
    # cluster_size:3
    # cluster_current_epoch:6
    # cluster_my_epoch:1
    # cluster_stats_messages_ping_sent:1735
    # cluster_stats_messages_pong_sent:1797
    # cluster_stats_messages_sent:3532
    # cluster_stats_messages_ping_received:1792
    # cluster_stats_messages_pong_received:1735
    # cluster_stats_messages_meet_received:5
    # cluster_stats_messages_received:3532

    ~/k8s/redis-master-slave$ kubectl exec -it -n database redis-cluster-0 -- sh -c "redis-cli -a weiyigeek.top role"
    # 1) "master" # 当前主机角色
    # 2) (integer) 1288
    # 3) 1) 1) "172.16.243.67" # Slave IP
    # 2) "6379"
    # 3) "1288"

    ~/k8s/redis-master-slave$ kubectl exec -it -n database redis-cluster-0 -- sh -c "redis-cli -a weiyigeek.top info replication"
    # Replication
    # role:master
    # connected_slaves:1
    # slave0:ip=172.16.243.67,port=6379,state=online,offset=1372,lag=0
    # master_replid:222f4d18c5d0913cb367d8bff49edc717563a96d
    # master_replid2:0000000000000000000000000000000000000000
    # master_repl_offset:1372
    # second_repl_offset:-1
    # repl_backlog_active:1
    # repl_backlog_size:1048576
    # repl_backlog_first_byte_offset:1
    # repl_backlog_histlen:1372


    ~/k8s/redis-master-slave$ kubectl exec -it -n database redis-cluster-0 -- sh -c "redis-cli -a weiyigeek.top cluster nodes"
    # ee0aa0c0742e6a9362a96f182d54483ec065fdba 172.16.182.226:[email protected] master - 0 1611220585017 2 connected 5461-10922
    # 3324007374edeaf4bed02423148970ee8171a7cb 172.16.135.195:[email protected] slave ee0aa0c0742e6a9362a96f182d54483ec065fdba 0 1611220583514 6 connected
    # f5a740fcbfdcd398da6e380f496fb5cb92236748 172.16.183.81:[email protected] myself,master - 0 1611220583000 1 connected 0-5460
    # 7697cf77b6da8b58663d23e9ad8150f33a6ee9cd 172.16.24.204:[email protected] master - 0 1611220583515 3 connected 10923-16383
    # 99509917de27b2f9ba02fb1b09d137014a0ae5f9 172.16.100.67:[email protected] slave 7697cf77b6da8b58663d23e9ad8150f33a6ee9cd 0 1611220585016 4 connected
    # b5aa6a5f06f1ad2b9debccc6842bcc5f0d9ff732 172.16.243.67:[email protected] slave f5a740fcbfdcd398da6e380f496fb5cb92236748 0 1611220584015 5 connected
  • 2.查看各集群状态

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    for pod_name in $(kubectl get pod -n database -l app=redis-cluster -o jsonpath='{ range.items [*]}{.spec.hostname} ');do
    kubectl exec -it -n database ${pod_name} -- sh -c "redis-cli -a weiyigeek.top cluster nodes" | grep "myself";
    kubectl exec -it -n database ${pod_name} -- sh -c "redis-cli -a weiyigeek.top info replication" | egrep "role|slave"
    done

    # 前三个为主-Master
    # f5a740fcbfdcd398da6e380f496fb5cb92236748 172.16.183.81:[email protected] myself,master - 0 1611221210000 1 connected 0-5460 # 注意槽的分配端
    role:master
    connected_slaves:1
    slave0:ip=172.16.243.67,port=6379,state=online,offset=2436,lag=1
    # ee0aa0c0742e6a9362a96f182d54483ec065fdba 172.16.182.226:[email protected] myself,master - 0 1611221211000 2 connected 5461-10922
    role:master
    connected_slaves:1
    slave0:ip=172.16.135.195,port=6379,state=online,offset=2436,lag=0
    # 7697cf77b6da8b58663d23e9ad8150f33a6ee9cd 172.16.24.204:[email protected] myself,master - 0 1611221209000 3 connected 10923-16383
    role:master
    connected_slaves:1
    slave0:ip=172.16.100.67,port=6379,state=online,offset=2436,lag=0

    # 后三个为从-Slave
    # 99509917de27b2f9ba02fb1b09d137014a0ae5f9 172.16.100.67:[email protected] myself,slave 7697cf77b6da8b58663d23e9ad8150f33a6ee9cd 0 1611221211000 4 connected
    role:slave
    slave_repl_offset:2436
    slave_priority:100
    slave_read_only:1
    connected_slaves:0
    # b5aa6a5f06f1ad2b9debccc6842bcc5f0d9ff732 172.16.243.67:[email protected] myself,slave f5a740fcbfdcd398da6e380f496fb5cb92236748 0 1611221211000 5 connected
    role:slave
    slave_repl_offset:2436
    slave_priority:100
    slave_read_only:1
    connected_slaves:0
    # 3324007374edeaf4bed02423148970ee8171a7cb 172.16.135.195:[email protected] myself,slave ee0aa0c0742e6a9362a96f182d54483ec065fdba 0 1611221209000 6 connected
    role:slave
    slave_repl_offset:2436
    slave_priority:100
    slave_read_only:1
    connected_slaves:0
  • 3.Redis 数据写入和查询

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    ~/k8s/redis-master-slave$ kubectl exec -it -n database redis-cluster-0 -- sh -c "redis-cli -h redis-cluster-0 -c -a weiyigeek.top"
    redis-cluster-0:6379> set name weiyigeek
    -> Redirected to slot [5798] located at 172.16.182.226:6379
    OK
    172.16.182.226:6379> set demo Redis-Cluster
    -> Redirected to slot [903] located at 172.16.183.81:6379
    OK
    172.16.183.81:6379> set time 2021年1月21日
    OK
    172.16.183.81:6379> time
    1) "1611221505"
    2) "958367"
    172.16.183.81:6379> set time 2021年1月21日
    OK
    172.16.183.81:6379> set web www.weiyigeek.top
    -> Redirected to slot [9635] located at 172.16.182.226:6379
    OK
  • 4.利用客户端工具连接查看

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
cat > redis-StatefulSet.yaml <<'EOF'
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis-app
namespace: database
spec:
serviceName: "redis-service"
replicas: 3
selector:
matchLabels:
app: redis
appCluster: redis-cluster
template:
metadata:
labels:
app: redis
appCluster: redis-cluster
spec:
terminationGracePeriodSeconds: 20
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- redis
topologyKey: kubernetes.io/hostname
containers:
- name: redis
image: "redis"
imagePullPolicy: IfNotPresent
command:
- "redis-server" #redis启动命令
args:
- "/etc/redis/redis.conf" #redis-server后面跟的参数,换行代表空格
- "--protected-mode" #允许外网访问
- "no"
# command: redis-server /etc/redis/redis.conf --protected-mode no
resources: #资源
requests: #请求的资源
cpu: "300m" #m代表千分之,相当于0.1 个cpu资源
memory: "800Mi" #内存100m大小
ports:
- name: redis
containerPort: 6379
protocol: "TCP"
- name: cluster
containerPort: 16379
protocol: "TCP"
volumeMounts:
- name: "redis-conf" #挂载configmap生成的文件
mountPath: "/etc/redis" #挂载到哪个路径下
- name: "redis-data" #挂载持久卷的路径
mountPath: "/var/lib/redis"
volumes:
- name: "redis-conf" #引用configMap卷
configMap:
name: "redis-conf"
items:
- key: "redis.conf" #创建configMap指定的名称
path: "redis.conf" #里面的那个文件--from-file参数后面的文件
volumeClaimTemplates: #进行pvc持久卷声明,
- metadata:
name: redis-data
spec:
accessModes: ["ReadWriteMany"]
storageClassName: "managed-nfs-storage"
resources:
requests:
storage: 10G
EOF
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
cat > redis-headless-service.yml <<'EOF'
apiVersion: v1
kind: Service
metadata:
name: redis-service
namespace: database
labels:
app: redis
spec:
ports:
- name: redis-port
port: 6379
nodePort: 31001
clusterIP: None
type: NodePort
selector:
app: redis
appCluster: redis-cluster
EOF

kubectl create -f redis-headless-service.yml


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
cat > redis-cluster-5.0.10.yaml<<'EOF'
apiVersion: v1
kind: Secret
metadata:
annotations:
# if your operator run as cluster-scoped, add this annotations
redis.kun/scope: cluster-scoped
name: redis-secret
type: Opaque
data:
password: Q3F6ay5jb20uY24K
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: redis-cluster
annotations: # 空间标注
volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
accessModes: ["ReadWriteMany"]
resources:
requests:
storage: 1Gi
---
apiVersion: redis.kun/v1alpha1
kind: DistributedRedisCluster
metadata:
annotations:
# if your operator run as cluster-scoped, add this annotations
redis.kun/scope: cluster-scoped
name: example-distributedrediscluster
spec:
image: redis:5.0.10-alpine
masterSize: 3
clusterReplicas: 1
config:
activerehashing: "yes"
appendfsync: everysec
appendonly: "yes"
hash-max-ziplist-entries: "512"
hash-max-ziplist-value: "64"
hll-sparse-max-bytes: "3000"
list-compress-depth: "0"
maxmemory-policy: noeviction
maxmemory-samples: "5"
no-appendfsync-on-rewrite: "no"
notify-keyspace-events: ""
set-max-intset-entries: "512"
slowlog-log-slower-than: "10000"
slowlog-max-len: "128"
stop-writes-on-bgsave-error: "yes"
tcp-keepalive: "0"
timeout: "0"
zset-max-ziplist-entries: "128"
zset-max-ziplist-value: "64"
passwordSecret:
name: redis-secret
resources:
limits:
cpu: 500m
memory: 500Mi
requests:
cpu: 200m
memory: 100Mi
serviceName: redis-svc
storage:
type: persistent-claim
size: 1Gi
class: redis-cluster
deleteClaim: true
EOF
1
2
3
$ kubectl create -f redis-cluster-5.0.10.yaml -n redis-cluster
secret/redis-secret created
distributedrediscluster.redis.kun/example-distributedrediscluster created


(2) Redis-cluster-operator 集群构建

https://www.cnblogs.com/klvchen/p/10862607.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
tee redis-cli.yml<<'EOF'
apiVersion: v1
kind: Service
metadata:
name: redis-cli
labels:
app: redis-cli
spec:
type: ClusterIP
ports:
- name: redis-cli
port: 8080
selector:
app: redis-cli
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis-cli
namespace: monitor
labels:
app: redis-cli
spec:
replicas: 1
selector:
matchLabels:
app: redis-cli
template:
metadata:
labels:
app: redis-cli
spec:
containers:
- name: redis-cli
image: redis:6.2.4-alpine
command:
- "top"
- "-d"
- "360"
resources:
limits:
cpu: 1000m
memory: 1024Mi
EOF


0x03 Redis 集群连接

(1) 应用连接写入测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
cat > redis-cluster-connect.yaml <<'END'
apiVersion: v1
kind: Pod
metadata:
name: redis-test
namespace: default
labels:
app: redis-test
spec:
containers:
- name: redis-test
image: java:8
imagePullPolicy: IfNotPresent
command:
- "java"
- "-jar"
- "/app/redis.jar"
volumeMounts:
- name: app
mountPath: /app
- name: timezone
mountPath: /etc/localtime # 在Pod中时区设置(挂载主机的时区)
volumes:
- name: app
hostPath:
path: /nfsdisk-31/test
- name: timezone
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
END

redis如何实现主从复制?以及数据同步机制?
https://www.cnblogs.com/zhuifeng-mayi/p/9301574.html

官网: https://redis.io/topics/benchmarks/