[TOC]

0x00 前言简述

描述: 本章主要学习与记录了在进一步学习Docker容器中的一些基础名称解析与Docker与一些辅助软件配合使用来增加工作效率以及简化运维流程;

辅助工具比如: Docker Machine, Swarm(架构很小的时候使用不过现在推荐使用K8S) docker 集群, Docker-Compose;


0x01 名词解析

描述: 通过前面的学习,我们大概已经对docker容器有了个简单的认识,随着近些年云计算的流行,容器从出现至今广泛使用特别是在Kubernetes容器编排发布后由于其能快速为分布式架构的应用部署以及收缩,Docker 已经站在容器工具的顶端;

随着K8s的大热docker、oci、runc、containerd等等名词也逐渐传播开来。这么多的名词,也容易让人混淆。
本章对相关名词和其之间的联系进行一下梳理和总结,方便大家更好地理解。

container - 容器

描述: 我们所知Docker设计理念是一次Build到处运行,其Logo也是非常经典图标一条鲸鱼拖着若干个集装箱的经典形象已经深入人心;

Q: 现在问题来了在docker中container的翻译是译为容器还是集装箱?

答: 由于Container是在Docker出现以前产生,而在此之前Linux Container 就已经被翻译为Linux容器并被大家接受;从含义来看一开始选定把”容器”作为container的翻译,也应该是准确的。而随着docker出现,container的概念深入人心,而其与原来的linux container中的container含义应该说是一致的。

Q: 那何为容器?

答: 容器本质上是受到资源限制,彼此间相互隔离的若干个linux进程的集合,这是有别于基于模拟的虚拟机的。对于容器和虚拟机的区别的理解可以参考我前面所写的Docker基础文章;
一般来说容器技术主要指代用于资源限制的cgroup(Linux control group),用于隔离的namespace,以及基础的linux kernel等。


OCI - 镜像容器运行时标准

描述: Docker 公司与 CoreOS 和 Google 共同创建了OCI (Open Container Initial - 开放初始化容器),并由linux基金会进行管理致力于images spec 与 container runtime的标准的制定和runc的开发等工作。

Q: 何为container runtime?

答: 主要负责的是容器的生命周期的管理; oci的runtime spec标准中对于容器的状态描述,以及对于容器的创建、删除、查看等操作进行了定义。
其容器在运行时分为两类: Low-level Runtime 、 High-Level Runtime;

Docker容器镜像仓库存储原理(前世今身)与技巧文章中我们知道runc是Low-Level Runtime当然也当前应用最为广泛的;

下面针对于runc我们做一个简单描述, 它是对于OCI标准的一个参考实现,是一个可以用于创建和运行容器的CLI(command-line interface)工具;
简单的说 runc直接与容器所依赖的 cgroup/linux kernel等进行交互,负责为容器配置cgroup/namespace等启动容器所需的环境,创建启动容器的相关进程。


Docker-Engine - 容器引擎

描述: 容器引擎或者说容器平台,不仅包含对于容器的生命周期的管理,还包括了对于容器生态的管理,比如对于镜像等。现在的docker、rkt以及阿里推出的pouch均可属于此范畴。

从Docker开源发布至今笔者认为可以分为两个阶段来理解。在笔者接触docker之初,docker版本为1.2,当时的docker的主要作用是容器的生命周期管理和镜像管理,当时的docker在功能上更趋近于现在的container runtime。而后来,随着docker的发展,docker就不再局限于容器的管理,还囊括了存储(volume)、网络(net)等的管理,因此后来的docker更多的是一个容器及容器生态的管理平台。

在容器编排领域的两种架构技术Swarm集群与Kubernetes集群,最终以Google家的Kubernetes胜出并且在分布式架构中广泛应用;


containerd - 货箱

描述:为了兼容oci标准docker也做了架构调整。将容器运行时相关的程序从docker daemon剥离出来形成了containerd
Containerd向docker提供运行容器的API二者通过grpc进行交互, containerd最后会通过runc来实际运行容器。

Conatinerd 与 Docker 引擎以及 runC 关系图:

WeiyiGeek.container ship 集装箱运货船货柜船

WeiyiGeek.container ship 集装箱运货船货柜船


0x02 常用工具

Docker-Machine

描述: 它是一种可以让您在虚拟主机上安装 Docker 的工具,并可以使用 docker-machine 命令来管理主机, 其便于在Mac或者Windows上(需要和Docker Desktop for Windows-下载联合使用)进行安装, 还能进行远程的机器上安装与管理Docker宿主机,与之前说的vagrant非常类似;

官方手册:http://docs.docker.com/machine/
项目地址:https://github.com/docker/machine/releases/

简述作用:

  • (1) 旧的桌面系统版本中运行在Windows或者MAC的Docker环境中;
  • (2) 简化Docker和远程管理多个Docker Host机器,比如快速的给 100 台服务器安装上 docker。;
WeiyiGeek.图示

WeiyiGeek.图示

安装依赖:

  • 1.安装 Docker Machine 之前你需要先安装 Docker。
  • 2.您需要在您的机器上安装Virtualbox(MAC)或者Hyper-V(Windows)

安装流程:

  • Step1.在GitHub的docker/machine发布页面上找到最新版本的二进制文件进行拉取。
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    # Linux
    base=https://github.com/docker/machine/releases/download/v0.16.2 &&
    curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine &&
    sudo mv /tmp/docker-machine /usr/local/bin/docker-machine &&
    chmod +x /usr/local/bin/docker-machine

    #Windows (需借用Git Bash)
    if [[ ! -d "$HOME/bin" ]]; then mkdir -p "$HOME/bin"; fi && \
    curl -L https://github.com/docker/machine/releases/download/v0.16.2/docker-machine-Windows-x86_64.exe > "$HOME/bin/docker-machine.exe" && \
    chmod +x "$HOME/bin/docker-machine.exe"
  • Step 2.Docker-Machine自动生成的配置文件~/.docker/machine/machines/

    1
    2
    3
    4
    5
    # 查看docker信息

    # 查看docker-machine安装
    ./docker-machine-Windows-x86_64.exe version
    docker-machine-Windows-x86_64.exe version 0.16.2, build bd45ab13
  • Step 3.Docker-Machine卸载Remove the executable: rm $(which docker-machine)


docker-machine 命令
描述:使用 docker-machine 命令,您可以启动,检查,停止和重新启动托管主机,也可以升级 Docker 客户端和守护程序,以及配置 Docker 客户端与您的主机进行通信。

基础语法和参数:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
Usage: docker-machine-Windows-x86_64.exe [OPTIONS] COMMAND [arg...]
Options:
--debug, -D Enable debug mode
--storage-path, -s "C:\Users\WeiyiGeek\.docker\machine" Configures storage path [$MACHINE_STORAGE_PATH]
--tls-ca-cert CA to verify remotes against [$MACHINE_TLS_CA_CERT]
--tls-ca-key Private key to generate certificates [$MACHINE_TLS_CA_KEY]
--tls-client-cert Client cert to use for TLS [$MACHINE_TLS_CLIENT_CERT]
--tls-client-key Private key used in client TLS auth [$MACHINE_TLS_CLIENT_KEY]
--github-api-token Token to use for requests to the Github API [$MACHINE_GITHUB_API_TOKEN]
--native-ssh Use the native (Go-based) SSH implementation. [$MACHINE_NATIVE_SSH]
--bugsnag-api-token BugSnag API token for crash reporting [$MACHINE_BUGSNAG_API_TOKEN]
--help, -h show help
--version, -v print the version

Commands:
* config:查看当前激活状态 Docker 主机的连接信息。
* creat:创建 Docker 主机

* env:显示连接到某个主机需要的环境变量
* inspect: 以 json 格式输出指定Docker的详细信息
* ip: 获取指定 Docker 主机的地址
* kill: 直接杀死指定的 Docker 主机
* ls: 列出所有的管理主机
* provision: 重新配置指定主机
* regenerate-certs: 为某个主机重新生成 TLS 信息
* restart: 重启指定的主机
* rm: 删除某台 Docker 主机,对应的虚拟机也会被删除
* ssh: 通过 SSH 连接到主机上,执行命令
* scp: 在 Docker 主机之间以及 Docker 主机和本地主机之间通过 scp 远程复制数据
* mount: 使用 SSHFS 从计算机装载或卸载目录
* start: 启动一个指定的 Docker 主机,如果对象是个虚拟机,该虚拟机将被启动
* status: 获取指定 Docker 主机的状态(包括:Running、Paused、Saved、Stopped、Stopping、Starting、Error)等
* stop: 停止一个指定的 Docker 主机
* upgrade: 将一个指定主机的 Docker 版本更新为最新
* url: 获取指定 Docker 主机的监听 URL
* version: 显示 Docker Machine 的版本或者主机 Docker 版本
* help: 显示帮助信息

基础示例:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
$DOCKER_HOST=tcp://10.10.107.245:2375
$MACHINE_DRIVER=swarm # virtualbox | swarm | hyper-v

# 1.列出可用的机器,目前只有这里默认的 default 虚拟机。
docker-machine ls

# 2.在安装有Virutualbox的机器上创建机器
# --driver:指定用来创建机器的驱动类型这里是virtualbox。
docker-machine create --driver virtualbox test
docker-machine create -d hyperv --hyperv-virtual-switch "Primary Virtual Switch" worker2

# 3.查看机器的 ip
docker-machine ip test

# 4.停止机器
docker-machine stop test

# 5.启动机器
docker-machine start test

# 6.进入机器之中
docker-machine ssh test

# 7.查看当前激活状态的 Docker 主机
$ docker-machine active
$ docker-machine active
# staging

# 8.命令配置Shell
eval $(docker-machine env test)

WeiyiGeek.

WeiyiGeek.


实际案例:
在Win10开始菜单的“Windows管理工具”下看到Hyper-V管理器,打开后需要进行进一步的设置;

  • Step 1.确保Hyper-V是启用,可以正常打开Hyper-V管理器并可以看见虚拟机情况;

    WeiyiGeek.Hyper-V管理器

    WeiyiGeek.Hyper-V管理器

  • Step 2.然后进行建立一个新的外部网络交换机(可选),注意确定后会重新刷新网卡会导致短暂的断网(修改后建议重新机器以确保在虚拟交换机管理器中选择了正确的网络)

WeiyiGeek.VirtualSwitch

WeiyiGeek.VirtualSwitch

  • Step 3.使用Docker机器和微软Hyper-V驱动程序创建节点,注意需要采用管理员来运行cmd或者Powershell终端(推荐);

    1
    2
    3
    # 注意:区分大小写
    ./docker-machine.exe create -d hyperv --hyperv-virtual-switch Docker0 worker-01
    ./docker-machine.exe create -d hyperv --hyperv-virtual-switch Docker0 worker-02
    WeiyiGeek.虚拟

    WeiyiGeek.虚拟

  • Step 4.查看创建的虚拟机信息

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    # 创建的机器查看
    PS F:\ChoromeDownload> .\docker-machine.exe ls
    NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
    worker-01 - hyperv Running tcp://10.20.172.122:2376 v19.03.12
    worker-02 - hyperv Running tcp://10.20.172.123:2376 v19.03.12

    # 将你的Docker客户端连接到运行在这个虚拟机上的Docker引擎
    .\docker-machine.exe env worker-01
    $Env:DOCKER_TLS_VERIFY = "1"
    $Env:DOCKER_HOST = "tcp://10.20.172.122:2376"
    $Env:DOCKER_CERT_PATH = "C:\Users\WeiyiGeek\.docker\machine\machines\worker-01"
    $Env:DOCKER_MACHINE_NAME = "worker-01"
    $Env:COMPOSE_CONVERT_WINDOWS_PATHS = "true"
    # Run this command to configure your shell:
    & "F:\ChoromeDownload\docker-machine.exe" env worker-01 | Invoke-Expression

    # 当前活动机器切换
    PS F:\ChoromeDownload> .\docker-machine.exe active
    worker-01
    PS F:\ChoromeDownload> & "F:\ChoromeDownload\docker-machine.exe" env worker-02 | Invoke-Expression
    PS F:\ChoromeDownload> .\docker-machine.exe active
    worker-02

    # 配置进行查看
    .\docker-machine.exe config worker-01
    --tlsverify
    --tlscacert="C:\\Users\\WeiyiGeek\\.docker\\machine\\machines\\worker-01\\ca.pem"
    --tlscert="C:\\Users\\WeiyiGeek\\.docker\\machine\\machines\\worker-01\\cert.pem"
    --tlskey="C:\\Users\\WeiyiGeek\\.docker\\machine\\machines\\worker-01\\key.pem"
    -H=tcp://10.20.172.122:2376

    # 进入创建的机器的ssh之中;
    PS F:\ChoromeDownload> .\docker-machine.exe ssh worker-01
    ( '>')
    /) TC (\ Core is distributed with ABSOLUTELY NO WARRANTY.
    (/-_--_-\) www.tinycorelinux.net

    PS F:\ChoromeDownload> .\docker-machine.exe ssh worker-01 hostname
    worker-01

注意事项:

  • (1) 与Docker-compose一样如果您想使用docker-machine您必须在您的运行它的环境中安装docker;
  • (2) 不同操作系统建议不同的虚拟化程序
    1
    2
    Docker Desktop for Mac - virtualbox driver
    Docker Desktop for Windows - hyperv driver
  • (3) 构建镜像时候由于拉取Github的releases速度较慢会导致下载失败提示
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    # 错误信息
    (worker-01) Latest release for github.com/boot2docker/boot2docker is v19.03.12
    (worker-01) Downloading C:\Users\WeiyiGeek\.docker\machine\cache\boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v19.03.12/boot2docker.iso...
    worker-01) 0%Error removing file: Error removing temporary download file: remove C:\Users\WeiyiGeek\.docker\machine\cache\boot2docker.iso.tmp850714895: The process cannot access the file because it is being used by another process.
    (worker-01)
    Error with pre-create check: "read tcp 10.20.172.103:57715->52.216.89.11:443: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond."

    # 解决方法:
    ISO: https://github.com/boot2docker/boot2docker/releases/download/v19.03.12/boot2docker.iso
    1.迅雷下载
    2.Github下载加速站点:https://d.serctl.com/
    # 将下载的镜像放入到以下目录之中
    C:\Users\WeiyiGeek\.docker\machine\cache\boot2docker.iso

Docker Swarm 集群

描述: 由于Docker只能在单主机上进行运行,且跨HOST部署,运行与管理能力有限,此时就需要一个容器编排或者管理工具,自Docker 1.12版本开始,swarm模式已经被集成到Docker Engine之中;

此时以上面的Docker-machine和Hyper-V环境中创建的两个docker容器为例,假设我的一台node-2Linux服务器是管理者他被用来执行相关管理命令以及对加入的swarm集群节点进行授权,而worker-01/worker-02加入到swarm之中成为节点进行业务负载;


例如,在Windows中的利用Hyper-v + Docker进行实践docker swarm集群。

流程步骤:

  • Step1.创建swarm控制端在worker-01机器上或者加入集群的tokern

    1
    2
    3
    docker@worker-01:~$ docker swarm init --advertise-addr 10.20.172.122
    # docker swarm join --token SWMTKN-1-0w8bvf8gzw5hct845izbv38qvl45385r19p4gwo0v8fuwie68g-603ibqyz4od3mmnw1gmbahd8v 10.10.107.245:2377
    # To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
  • Step2.将worker-01/worker-02加入到swarm集群之中

    1
    2
    3
    4
    5
    6
    7
    8
    9
    # worker-01 节点
    PS F:\ChoromeDownload>.\docker-machine.exe ssh worker-01
    docker@worker-01:~$ docker swarm join --token SWMTKN-1-0w8bvf8gzw5hct845izbv38qvl45385r19p4gwo0v8fuwie68g-603ibqyz4od3mmnw1gmbahd8v 10.10.107.245:2377
    This node joined a swarm as a worker.

    # worker-02 节点
    PS F:\ChoromeDownload> .\docker-machine.exe ssh worker-02
    [root@node2 ~]$ docker swarm join --token SWMTKN-1-0w8bvf8gzw5hct845izbv38qvl45385r19p4gwo0v8fuwie68g-603ibqyz4od3mmnw1gmbahd8v 10.10.107.245:2377
    This node joined a swarm as a worker.
  • Step 3.在Swarm控制节点上查看加入的节点:

    1
    2
    3
    4
    5
    $ docker node ls
    # ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
    # tqulwewuzrgayabjomy3xw55u * node2 Ready Active Leader(领导者) 19.03.12
    # 7vp6qrabwp7ee2dkq0d1bf11l worker-01 Ready Active 19.03.12
    # kxhd5fdxvarn6vpjweadf8yfb worker-02 Ready Active 19.03.12
  • Step 4.Swarm 集群中部署应用准备

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    # DockerFile
    cat > dockerfile <<'END'
    FROM frolvlad/alpine-python3
    WORKDIR /opt/
    ADD ./hostname.txt /opt/
    RUN sh -c "ip addr" > ip.txt
    EXPOSE 8080
    ENV NAME Python-Web
    CMD ["python","-m","http.server","8080","-d","/opt/"]
    END

    # Docker-compose.yml
    # Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
    cat > docker-compose.yml<<'END'
    version: "3"
    services:
    web:
    image: python-web:latest
    deploy:
    replicas: 5
    resources:
    limits:
    cpus: "0.1"
    memory: "50M"
    restart_policy:
    condition: on-failure
    entrypoint: ["sh","-c","/opt/start.sh"]
    volumes:
    - "/tmp/:/opt/"
    ports:
    - "80:8080"
    networks:
    - webnet
    networks:
    webnet:
    END

    # [root@node2 test]# cat start.sh
    # ip addr > /opt/ip.txt && python -m http.server 8080

    # 镜像构建
    docker build -t python-web .

    # 镜像查看
    $ docker images
    REPOSITORY TAG IMAGE ID CREATED SIZE
    python-web latest 196e26d0d89f 9 seconds ago 57.8MB

    # 镜像导出与启动web下载端口
    docker save python-web -o python-web.tar
    python -m SimpleHTTPServer 80 #python 2语法
    # Serving HTTP on 0.0.0.0 port 80 ...
  • Step 5.各个节点拉取镜像进行导入到本地

    1
    2
    3
    4
    5
    6
    7
    8
    9
    # worker-01/worker-02 中执行拉取镜像
    wget http://10.10.107.245/python-web.tar && docker load -i python-web.tar
    # Connecting to 10.10.107.245 (10.10.107.245:80)
    # python-web.tar 100% |********************************| 58.1M 0:00:00 ETA
    # 50644c29ef5a: Loading layer 5.845MB/5.845MB
    # 47279ef47504: Loading layer 55.11MB/55.11MB
    # a126991f92ee: Loading layer 2.56kB/2.56kB
    # 89a1c87b5354: Loading layer 2.56kB/2.56kB
    # Loaded image: python-web:latest
  • Step 6.通过swarm集群进行部署应用(多个副本)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    # Swarm Master 控制节点上执行
    docker stack deploy -c docker-compose.yml swarm-python-web
    # Creating network swarm-python-web_webnet # 项目名称+网卡名 = 网络名称
    # Creating service swarm-python-web_web

    # swarm 集群信息与swarm-python-web服务应用信息
    docker stack ls
    # NAME SERVICES ORCHESTRATOR
    # swarm-python-web 1 Swarm
    docker stack services swarm-python-web
    # ID NAME MODE REPLICAS IMAGE PORTS
    # bzeeucg6n6s1 swarm-python-web_web replicated 5/5 python-web:latest *:80->8080/tcp

    # 应用启动后的正常状态
    docker stack ps swarm-python-web
    # ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
    # xv3doqj787ae swarm-python-web_web.1 python-web:latest worker-02 Running Running 5 minutes ago
    # njkppsnrf4nc swarm-python-web_web.2 python-web:latest worker-01 Running Running 5 minutes ago
    # sulpgnpg00v2 swarm-python-web_web.3 python-web:latest node2 Running Running 4 minutes ago
    # ny91qomtch7p swarm-python-web_web.4 python-web:latest worker-02 Running Running 5 minutes ago
    # fb1a0cp50wf4 swarm-python-web_web.5 python-web:latest worker-01 Running Running 5 minutes ago
  • Step 7.访问集群Web查看效果,即每个节点的IP:80
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    # master
    $ docker inspect $(docker ps -aq) | grep "IPAddress"
    "SecondaryIPAddresses": null,
    "IPAddress": "",
    "IPAddress": "10.0.0.28",
    "IPAddress": "10.0.2.11",

    # worker-01
    docker@worker-01:~$ docker inspect $(docker ps -aq) | grep "IPAddress"
    "SecondaryIPAddresses": null,
    "IPAddress": "",
    "IPAddress": "10.0.0.25",
    "IPAddress": "10.0.2.5",
    "SecondaryIPAddresses": null,
    "IPAddress": "",
    "IPAddress": "10.0.0.27",
    "IPAddress": "10.0.2.7",

    # worker-02
    docker@worker-02:~$ docker inspect $(docker ps -aq) | grep "IPAddress"
    "SecondaryIPAddresses": null,
    "IPAddress": "",
    "IPAddress": "10.0.0.24",
    "IPAddress": "10.0.2.4",
    "SecondaryIPAddresses": null,
    "IPAddress": "",
    "IPAddress": "10.0.0.26",
    "IPAddress": "10.0.2.6",
    WeiyiGeek.集群WEb

    WeiyiGeek.集群WEb

  • Step 8.如果此时停止或删除一个容器,swarm会在工作节点中新启动一个容器,使之副本数量一直为五;

    1
    2
    3
    4
    5
    6
    7
    8
    docker stack ps swarm-python-web
    # ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
    # xv3doqj787ae swarm-python-web_web.1 python-web:latest worker-02 Running Running 24 minutes ago
    # nkwtm8w9yfxp swarm-python-web_web.2 python-web:latest node2 Running Running 3 minutes ago
    # njkppsnrf4nc \_ swarm-python-web_web.2 python-web:latest worker-01 Shutdown Failed 3 minutes ago "task: non-zero exit (137)"
    # sulpgnpg00v2 swarm-python-web_web.3 python-web:latest node2 Running Running 24 minutes ago
    # ny91qomtch7p swarm-python-web_web.4 python-web:latest worker-02 Running Running 24 minutes ago
    # fb1a0cp50wf4 swarm-python-web_web.5 python-web:latest worker-01 Running Running 24 minutes ago
  • Step 9.删除在集群中部署的应用以及退出集群:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    $docker stack rm $(docker stack ls --format "{{.Name}}")
    Removing service swarm-python-web_web
    Removing network swarm-python-web_webnet

    # 排空节点上的集群容器
    docker@master-01:~$docker node update --availability drain j5auh1zg6qzvmzkf80wvpl9hr

    # 节点退出集群
    docker@worker-02:~$ docker swarm leave
    Node left the swarm.

    # 查看节点
    docker@master-01:~$ docker node ls
    ID HOSTNAME STATUS AVAILABILITY(可用性) MANAGER STATUS ENGINE VERSION
    j5auh1zg6qzvmzkf80wvpl9hr worker-02 Down 状态 Drain 已停止 19.03.12
    tjwo77l1i08gzextmfwn69i9z * worker-01 Ready Active Leader 19.03.12

    # 控制节点退出swarm集群
    docker@master-01:~$ docker swarm leave --force
    Node left the swarm.

0x03 在Docker中运行Docker(dind)

(1) dind 介绍

描述: 什么是dind(Docker in Docker)?
答: 就是在Docker容器内启动docker daemon从而对外提供服务, 每个运行容器都是在一个进程它托管在Docker Daemon。

dind 实现方式:

  • 1.利用宿主机的docker.sock文件
  • 2.利用容器中的运行docker:dind容器,在启动一个docker容器(b),容器b指定host为a容器内的Docker Daemon。


(2) dind 原理

基础代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
func (cli *Client) ImageBuild(ctx context.Context, buildContext io.Reader, options types.ImageBuildOptions) (types.ImageBuildResponse, error) {
query, err := cli.imageBuildOptionsToQuery(options)
if err != nil {
return types.ImageBuildResponse{}, err
}

headers := http.Header(make(map[string][]string))
buf, err := json.Marshal(options.AuthConfigs)
if err != nil {
return types.ImageBuildResponse{}, err
}
# 两句比较重要
headers.Add("X-Registry-Config", base64.URLEncoding.EncodeToString(buf))
headers.Set("Content-Type", "application/x-tar")

serverResp, err := cli.postRaw(ctx, "/build", query, buildContext, headers)
if err != nil {
return types.ImageBuildResponse{}, err
}

osType := getDockerOS(serverResp.header.Get("Server"))

return types.ImageBuildResponse{
Body: serverResp.body,
OSType: osType,
}, nil
}


(3) dind 实践

Tips : 注意不同的Docker版本其启动和访问有些许差异。
2.1 低版本启动及访问

1
2
3
4
5
6
7
# 启动1.12.6-dind 并在其它容器中访问
docker run --privileged -d --name mydocker docker:1.12.6-dind
docker run --rm --link mydocker:docker docker:1.12.6 version

# 在宿主机访问
docker -H 172.17.0.2:2375 version
docker -H 0.0.0.0:${hostport} version


2.2 例18.09以上的dind镜像
描述: 19.03+默认tls模式,通过DOCKER_TLS_CERTDIR=''可以关闭;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# A 容器:启动 stable 19.03.1 版本
docker run --privileged --name some-docker -d \
#--network some-network --network-alias docker \
-e DOCKER_TLS_CERTDIR=/certs \
-v some-docker-certs-ca:/certs/ca \
-v some-docker-certs-client:/certs/client \
docker:stable-dind

# B 容器: 容器内访问A容器
docker run --rm \
#--network some-network \
-e DOCKER_TLS_CERTDIR=/certs \
-v some-docker-certs-client:/certs/client:ro \
--link some-docker:docker \
docker:stable version


(4) dind DockerFile 学习

Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
FROM alpine:3.10
RUN apk add --no-cache ca-certificates

# set up nsswitch.conf for Go's "netgo" implementation (which Docker explicitly uses)
# - https://github.com/docker/docker-ce/blob/v17.09.0-ce/components/engine/hack/make.sh#L149
# - https://github.com/golang/go/blob/go1.9.1/src/net/conf.go#L194-L275
# - docker run --rm debian:stretch grep '^hosts:' /etc/nsswitch.conf

RUN [ ! -e /etc/nsswitch.conf ] && echo 'hosts: files dns' > /etc/nsswitch.conf
ENV DOCKER_CHANNEL stable
ENV DOCKER_VERSION 19.03.1

# todo ENV DOCKER_SHA256
# https://github.com/docker/docker-ce/blob/5b073ee2cf564edee5adca05eee574142f7627bb/components/packaging/static/hash_files !!
# (no SHA file artifacts on download.docker.com yet as of 2017-06-07 though)

RUN set -eux; \
\
# this "case" statement is generated via "update.sh"
apkArch="$(apk --print-arch)"; \
case "$apkArch" in \
# amd64
x86_64) dockerArch='x86_64' ;; \
# arm32v6
armhf) dockerArch='armel' ;; \
# arm32v7
armv7) dockerArch='armhf' ;; \
# arm64v8
aarch64) dockerArch='aarch64' ;; \
*) echo >&2 "error: unsupported architecture ($apkArch)"; exit 1 ;;\
esac; \
\
if ! wget -O docker.tgz "https://download.docker.com/linux/static/${DOCKER_CHANNEL}/${dockerArch}/docker-${DOCKER_VERSION}.tgz"; then \
echo >&2 "error: failed to download 'docker-${DOCKER_VERSION}' from '${DOCKER_CHANNEL}' for '${dockerArch}'"; \
exit 1; \
fi; \
\
tar --extract \
--file docker.tgz \
--strip-components 1 \
--directory /usr/local/bin/ \
; \
rm docker.tgz; \
\
dockerd --version; \
docker --version

COPY modprobe.sh /usr/local/bin/modprobe
COPY docker-entrypoint.sh /usr/local/bin/

# https://github.com/docker-library/docker/pull/166
# dockerd-entrypoint.sh uses DOCKER_TLS_CERTDIR for auto-generating TLS certificates
# docker-entrypoint.sh uses DOCKER_TLS_CERTDIR for auto-setting DOCKER_TLS_VERIFY and DOCKER_CERT_PATH
# (For this to work, at least the "client" subdirectory of this path needs to be shared between the client and server containers via a volume, "docker cp", or other means of data sharing.)

ENV DOCKER_TLS_CERTDIR=/certs
# also, ensure the directory pre-exists and has wide enough permissions for "dockerd-entrypoint.sh" to create subdirectories, even when run in "rootless" mode
RUN mkdir /certs /certs/client && chmod 1777 /certs /certs/client
# (doing both /certs and /certs/client so that if Docker does a "copy-up" into a volume defined on /certs/client, it will "do the right thing" by default in a way that still works for rootless users)

ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["sh"]


docker-entrypoint.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
#!/bin/sh
set -eu

# first arg is `-f` or `--some-option`
if [ "${1#-}" != "$1" ]; then
set -- docker "$@"
fi

# if our command is a valid Docker subcommand, let's invoke it through Docker instead
# (this allows for "docker run docker ps", etc)
if docker help "$1" > /dev/null 2>&1; then
set -- docker "$@"
fi

_should_tls() {
[ -n "${DOCKER_TLS_CERTDIR:-}" ] \
&& [ -s "$DOCKER_TLS_CERTDIR/client/ca.pem" ] \
&& [ -s "$DOCKER_TLS_CERTDIR/client/cert.pem" ] \
&& [ -s "$DOCKER_TLS_CERTDIR/client/key.pem" ]
}

# if we have no DOCKER_HOST but we do have the default Unix socket (standard or rootless), use it explicitly
if [ -z "${DOCKER_HOST:-}" ] && [ -S /var/run/docker.sock ]; then
export DOCKER_HOST=unix:///var/run/docker.sock
elif [ -z "${DOCKER_HOST:-}" ] && XDG_RUNTIME_DIR="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}" && [ -S "$XDG_RUNTIME_DIR/docker.sock" ]; then
export DOCKER_HOST="unix://$XDG_RUNTIME_DIR/docker.sock"
fi

# if DOCKER_HOST isn't set (no custom setting, no default socket), let's set it to a sane remote value
if [ -z "${DOCKER_HOST:-}" ]; then
if _should_tls || [ -n "${DOCKER_TLS_VERIFY:-}" ]; then
export DOCKER_HOST='tcp://docker:2376'
else
export DOCKER_HOST='tcp://docker:2375'
fi
fi

if [ "${DOCKER_HOST#tcp:}" != "$DOCKER_HOST" ] \
&& [ -z "${DOCKER_TLS_VERIFY:-}" ] \
&& [ -z "${DOCKER_CERT_PATH:-}" ] \
&& _should_tls \
; then
export DOCKER_TLS_VERIFY=1
export DOCKER_CERT_PATH="$DOCKER_TLS_CERTDIR/client"
fi

if [ "$1" = 'dockerd' ]; then
cat >&2 <<-'EOW'
📎 Hey there! It looks like you're trying to run a Docker daemon.
You probably should use the "dind" image variant instead, something like:
docker run --privileged --name some-docker ... docker:dind ...
See https://hub.docker.com/_/docker/ for more documentation and usage examples.
EOW
sleep 3
fi

exec "$@"