[TOC]

Docker Runtime kubernetes 云原生 CNCF containerd.io containerd 容器运行时 WeiyiGeek 容器客户端工具

0x00 Containerd 容器运行时配置指南

描述: containerd 是一个在任何系统上运行的简单守护程序。它提供了一个带有旋钮的最小配置，用于配置守护程序以及在必要时使用哪些插件。

在当前最新的 Containerd v1.6.2版本中其/etc/containerd/config.toml配置文件项存在变化, 下面的文档中提供了常用配置以及CRI插件配置的说明。

官方参考地址: https://github.com/containerd/containerd/blob/main/docs/cri/config.md

如何配置 Containerd 的 systemd 守护进程服务?

$ cat /lib/systemd/system/containerd.service

[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target

[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/bin/containerd
Type=notify
# 允许 containerd 及其运行时管理它所创建的容器的 cgroups
Delegate=yes
# 确保在 containerd 被关闭时仅终止 containerd 守护程序，而不终止任何子进程，如填充程序和容器
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999

[Install]
WantedBy=multi-user.target

如何查看 Containerd 相关插件及其存目录?

描述：在 containerd 配置文件中，您将找到持久性和运行时存储位置的设置，以及各种 API 的 grpc、调试和指标地址。
containerd 在主机系统上还有两个不同的存储位置, 一个用于持久性数据，另一个用于运行时状态。

• root将用于存储容器的任何类型的持久性数据。快照、内容、容器和映像的元数据以及任何插件数据都将保留在此位置。根目录也为容器加载的插件提供了命名空间。每个插件都有自己的目录来存储数据。containerd本身实际上没有任何需要存储的持久性数据，其功能来自加载的插件。

  /var/lib/containerd/
  ├── io.containerd.content.v1.content
  │   ├── blobs
  │   └── ingest
  ├── io.containerd.metadata.v1.bolt
  │   └── meta.db
  ├── io.containerd.runtime.v2.task
  │   ├── default
  │   └── example
  ├── io.containerd.snapshotter.v1.btrfs
  └── io.containerd.snapshotter.v1.overlayfs
      ├── metadata.db
      └── snapshots

• state将用于存储任何类型的临时数据, 套接字、pids、运行时状态、挂载点和其他在重新启动之间不得保留的插件数据存储在此位置。

/run/containerd
├── containerd.sock
├── debug.sock
├── io.containerd.runtime.v2.task
│   └── default

│       └── redis
│           ├── config.json
│           ├── init.pid
│           ├── log.json
│           └── rootfs
│               ├── bin
│               ├── data
│               ├── dev
│               ├── etc
│               ├── home
│               ├── lib
│               ├── media
│               ├── mnt
│               ├── proc
│               ├── root
│               ├── run
│               ├── sbin
│               ├── srv
│               ├── sys
│               ├── tmp
│               ├── usr
│               └── var
└── runc
    └── default
        └── redis
            └── state.json

如何生成 Containerd 默认的 config.toml 配置文件?

$ mkdir -vp /etc/containerd/ && containerd config default > /etc/containerd/config.toml

如何配置 Containerd 让其支持懒拉取 (eStargz)?

描述: 从 v1.4 版本开始 containerd 便支持懒拉取, Stargz Snapshotter 是使 containerd 能够处理 eStargz的插件，在 Kubernetes 上我们可以通过如下配置/etc/containerd/config.toml启用了此功能。

Stargz Snapshotter插件（containerd-stargz-grpc二进制文件）也需要作为单独的进程运行, 其启动示例文件为:

# etc/systemd/system/stargz-snapshotter.service
[Unit]
Description=stargz snapshotter (Rootless)
PartOf=containerd.service

[Service]
Environment=PATH=/usr/local/bin:/sbin:/usr/sbin:/usr/local/elasticsearch-7.15.0/jdk/bin:/usr/local/elasticsearch-7.15.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
Environment=IPFS_PATH=/home/weiyigeek/.local/share/ipfs
ExecStart="/usr/local/bin/containerd-rootless-setuptool.sh" nsenter -- containerd-stargz-grpc -address "/run/user/1000/containerd-stargz-grpc/containerd-stargz-grpc.sock" -root "/home/weiyigeek/.local/share/containerd-stargz-grpc" -config "/home/weiyigeek/.config/containerd-stargz-grpc/config.toml"
ExecReload=/bin/kill -s HUP $MAINPID
RestartSec=2
Restart=always
Type=simple
KillMode=mixed

[Install]
WantedBy=default.target

温馨提示: 官方systemd示例地址(https://github.com/containerd/stargz-snapshotter/blob/v0.6.0/script/config/etc/systemd/system/stargz-snapshotter.service)

修改 /etc/containerd/config.toml 配置文件示例:

version = 2
# Plug stargz snapshotter into containerd Containerd recognizes stargz snapshotter through specified socket address.
[proxy_plugins]
  [proxy_plugins.stargz]
    type = "snapshot"
    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"

# Use stargz snapshotter through CRI
[plugins."io.containerd.grpc.v1.cri".containerd]
  snapshotter = "stargz"
  disable_snapshot_annotations = false

如何配置 Containerd 在 harbor 私有仓库拉取镜像?

描述: 在k8s的1.20版本发布之后，对外宣称在1.23.x不再使用dokershim作为默认的底层容器运行时，而是通过Container Runtime Interface(CRI)使用containerd来作为容器运行时, 因此原来在docker中配置的个人仓库环境不再起作用，导致k8s配置pods时拉取镜像失败, 本节将进行演示如何在 containerd 配置从私有仓库拉取镜像。

Step 1.kubernetes 使用 containerd 拉取harbor仓库中镜像配置说明, 项目地址介绍: https://github.com/containerd/cri/blob/master/docs/registry.md

Step 2.containerd 的配置文件 (相当于docker 的 daemon.json)

$ vim /etc/containerd/config.toml
[plugins."io.containerd.grpc.v1.cri".registry]
   [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
        endpoint = ["https://xlx9erfu.mirror.aliyuncs.com"]                         # 使用阿里镜像源到此为配置文件默认生成，之后为需要添加的内容
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors."harbor.weiyigeek.top"] # 内部私有仓库配置
        endpoint = ["https://harbor.weiyigeek.top"]                             
   [plugins."io.containerd.grpc.v1.cri".registry.configs]
     [plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.weiyigeek.to"] 
       [plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.weiyigeek.top".auth]   # harbor 认证的账号密码 配置
         username = "admin"                                                      
         password = "Harbor12345"
         auth = ""
         identitytoken = ""
       [plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.weiyigeek.top".tls]    # harbor 证书认证配置
         insecure_skip_verify = false                     # 是否跳过证书认证
         ca_file   = "/etc/containerd/harbor/ca.crt"      # CA 证书
         cert_file = "/etc/containerd/harbor/harbor.crt"  # harbor 证书
         key_file  = "/etc/containerd/harbor/harbor.key"  # harbor 私钥

Step 3.重载 systemd 的 daemon守护进程并重启containerd.service服务, 然后k8s集群便可正常从 harbor.weiyigeek.top 拉取镜像了。

systemctl daemon-reload && systemctl restart containerd.service

Step 4.虽然上面的方式可以使k8s直接拉取镜像,但是在利用 ctl命令 进行手动拉取镜像此时会报如下错误（巨坑-经过无数次失败测试，原本以为是CA证书签发的harbor证书问题），即使你在config.toml中配置insecure_skip_verify为true也是不行的。

# 错误信息
$ ctr -n k8s.io i pull harbor.weiyigeek.top/devops/jenkins-jnlp:3.13.8-alpine
INFO[0000] trying next host  error="failed to do request: Head \"https://harbor.weiyigeek.top/v2/devops/jenkins-jnlp/manifests/3.13.8-alpine\": x509: certificate signed by unknown authority" host=harbor.weiyigeek.top
ctr: failed to resolve reference "harbor.weiyigeek.top/devops/jenkins-jnlp:3.13.8-alpine": failed to do request: Head "https://harbor.weiyigeek.top/v2/devops/jenkins-jnlp/manifests/3.13.8-alpine": x509: certificate signed by unknown authority

# 解决办法1.指定 -k 参数跳过证书校验。
$ ctr -n k8s.io i pull -k harbor.weiyigeek.top/devops/jenkins-jnlp:3.13.8-alpine

# 解决办法2.指定CA证书、Harbor相关证书文件路径。
$ ctr -n k8s.io i pull --tlscacert ca.crt --tlscert harbor.crt --tlskey harbor.key  harbor.weiyigeek.top/devops/jenkins-jnlp:3.13.6-alpine

如何进行 Containerd 基本配置?

• 自定义配置 containerd 持久化目录和 containerd 运行时状态信息进行。

  cat /etc/containerd/config.toml
  version = 2
  # containerd 持久化目录
  root = "/var/lib/containerd"
  # containerd 运行时状态信息
  state = "/run/containerd"
  oom_score = 0

• 由于国内环境原因我们需要将 sandbox_image 镜像源设置为阿里云google_containers镜像源。

$ vim /etc/containerd/config.toml
  [plugins."io.containerd.grpc.v1.cri"]
  sandbox_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6"
# 或者
$ sed -i "s#k8s.gcr.io/pause#registry.cn-hangzhou.aliyuncs.com/google_containers/pause#g"  /etc/containerd/config.toml

• 为了快速docker.io镜像拉取速度,同样我们需要为其设置国内镜像源。

$ vim /etc/containerd/config.toml
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
      endpoint = ["https://xlx9erfu.mirror.aliyuncs.com"]

# 或者执行
sed -i '/registry.mirrors]/a\ \ \ \ \ \ \ \ [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]' /etc/containerd/config.toml
sed -i '/registry.mirrors."docker.io"]/a\ \ \ \ \ \ \ \ \ \ endpoint = ["https://xlx9erfu.mirror.aliyuncs.com"]' /etc/containerd/config.toml

• 使用虽然 containerd 和 Kubernetes 默认使用旧版驱动程序来管理 cgroups，但建议在基于 systemd 的主机上使用该驱动程序，以符合 cgroup 的“单编写器”规则。

$ vim /etc/containerd/config.toml
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
  SystemdCgroup = true
# 或者
$ sed -i 's#SystemdCgroup = false#SystemdCgroup = true#g' /etc/containerd/config.toml

于此同时, 我们需要在K8S的KubeletConfiguration资源清单中, 还必须将 cgroup 驱动程序配置为使用 “systemd”。

kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: "systemd"

• 设置CRI插件默认快照器设置为 overlayfs（类似于 Docker 的存储驱动程序）, 一般是无需更改默认即为overlayfs.

$ vim /etc/containerd/config.toml
  [plugins."io.containerd.grpc.v1.cri".containerd]
    snapshotter = "overlayfs"
# 或者
sed -ri 's#snapshotter = "\w*"#snapshotter = "overlayfs"#' /etc/containerd/config.toml

• 设置运行时类, 下面的示例将自定义运行时注册到 containerd 中：

  # 默认配置
  [plugins."io.containerd.grpc.v1.cri".containerd]
    default_runtime_name = "runc"
   [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
        base_runtime_spec = ""
        container_annotations = []
        pod_annotations = []
        privileged_without_host_devices = false
        runtime_engine = ""
        runtime_root = ""
        runtime_type = "io.containerd.runc.v2"
        
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
          BinaryName = ""
          CriuImagePath = ""
          CriuPath = ""
          CriuWorkPath = ""
          IoGid = 0
          IoUid = 0
          NoNewKeyring = false
          NoPivotRoot = false
          Root = ""
          ShimCgroup = ""
          SystemdCgroup = true
  
  # 自定义别名为crun的运行时
  $ vim /etc/containerd/config.toml
  version = 2
  [plugins."io.containerd.grpc.v1.cri".containerd]
    default_runtime_name = "crun"   # 注意别名
    [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
      # crun: https://github.com/containers/crun
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.crun]
        runtime_type = "io.containerd.runc.v2"
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.crun.options]
          BinaryName = "/usr/local/bin/crun"
      # gVisor: https://gvisor.dev/
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.gvisor]
        runtime_type = "io.containerd.runsc.v1"
      # Kata Containers: https://katacontainers.io/
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata]
        runtime_type = "io.containerd.kata.v2"

  此外，还必须将以下RuntimeClass资源安装到具有该角色的群集中 cluster-admin

  apiVersion: node.k8s.io/v1
  kind: RuntimeClass
  metadata:
    name: crun
  handler: crun
  ---
  apiVersion: node.k8s.io/v1
  kind: RuntimeClass
  metadata:
    name: gvisor
  handler: gvisor
  ---
  apiVersion: node.k8s.io/v1
  kind: RuntimeClass
  metadata:
    name: kata
  handler: kata

要将运行时类应用于 pod，请设置.spec.runtimeClassName

apiVersion: v1
kind: Pod
spec:
  runtimeClassName: crun

• 在 config.toml 中为容器设置私有仓库主机配置的简单示例
  在配置路径处创建一个目录树, 其中包含一个表示要配置的主机命名空间的目录, 然后在中添加一个文件以配置主机命名空间。

# 例如, 公共的 docker.io 镜像仓库为例
$ tree /etc/containerd/certs.d
/etc/containerd/certs.d
└── docker.io
    └── hosts.toml
$ cat /etc/containerd/certs.d/docker.io/hosts.toml
server = "https://docker.io"
[host."https://registry-1.docker.io"]
  capabilities = ["pull", "resolve"]

# 例如, 私有的 hub.weiyigeek.top 镜像仓库配置
$ mkdir -vp /etc/containerd/certs.d/hub.weiyigeek.top/
$ touch /etc/containerd/certs.d/hub.weiyigeek.top/hosts.toml
$ vim  /etc/containerd/certs.d/hub.weiyigeek.top/hosts.toml
server = "https://hub.weiyigeek.top"
[host."https://hub.weiyigeek.top"]
  ca = "/path/to/ca.crt"

• Bolt 元数据插件允许在命名空间之间配置内容共享策略。
  默认模式“shared”将使 Blob 在被拉入任何命名空间后在所有命名空间中都可用, 如果使用后端中已存在的“Expected”摘要打开编写器，则 Blob 将被拉入命名空间, 备用模式“隔离”要求客户端在将 Blob 添加到命名空间之前，通过向引入提供所有内容来证明它们有权访问内容。

默认值为“共享”。虽然这在很大程度上是最需要的策略，但可以通过以下配置更改为“隔离”模式：

$ vim /etc/containerd/config.toml
version = 2
[plugins."io.containerd.metadata.v1.bolt"]
  content_sharing_policy = "isolated"

补充说明: 在“隔离”模式下，还可以通过将标签添加到特定命名空间来仅共享该命名空间的内容。这将使其 Blob 在所有其他命名空间中都可用，即使内容共享策略设置为“隔离”也是如此。如果标签值设置为 除 以外的任何内容，则不会共享命名空间内容。

---

0x01 使用 nerdctl 工具配合 Containerd 替代 Docker

描述: Containerd 是一个工业级标准的容器运行时，它强调简单性、健壮性和可移植性。Containerd 可以在宿主机中管理完整的容器生命周期：容器镜像的传输和存储、容器的执行和管理、存储和网络等

学过Docker的童鞋都知道, 我们利用docker-cli与docker-compose工具能帮助快速的进行镜像和容器的相关操作。自从 Containerd 发布 1.5 以后，Kubernetes 的CRI接口使用 Containerd 来替代 dockershim 时, 我们便可以使用 nerdctl 工具(替代 docker cli) 配合 Containerd 的情况下基本已经可以替换掉 Docker 和 Docker Compose;

介绍 nerdctl 工具

什么是nerdctl?
nerdctl 是 与 Docker 兼容的CLI for Containerd，其支持Compose、Rootless、eStargz、OCIcrypt和IPFS，与docker命令行语法类似，所以其上手使用非常简单😊。

nerdctl 官方发布包包含两个安装版本:

Minimal: 仅包含 nerdctl 二进制文件以及 rootless 模式下的辅助安装脚本
Full: 看名字就能知道是个全量包，其包含了 Containerd、CNI、runc、BuildKit 等完整组件

项目地址： GitHub - containerd/nerdctl: Docker-compatible CLI for containerd, with support for Compose ( https://github.com/containerd/nerdctl )

安装 nerdctl 工具

最新nerdctl版本releases下载地址: https://github.com/containerd/nerdctl/releases

Step 1.从Github Release页面中下载适用于你系统的 Full 包，当前最新版本【2022年4月25日 11:22:59】为v.0.19.0。

# 断点与后台下载
# 全量包
wget -c -b https://github.com/containerd/nerdctl/releases/download/v0.19.0/nerdctl-full-0.19.0-linux-amd64.tar.gz

# 仅包含 nerdctl 二进制
wget -c -b https://github.com/containerd/nerdctl/releases/download/v0.19.0/nerdctl-0.19.0-linux-amd64.tar.gz

Step 2.下载完成后解压然后进行相应的安装操作。

# 1.安装仅包含 nerdctl 二进制文件包 （Minimal - 不推荐）
tar Cxzvvf /usr/local/bin nerdctl-0.19.0-linux-amd64.tar.gz
  # -rwxr-xr-x root/root  27578368 2022-04-22 12:03 nerdctl
  # -rwxr-xr-x root/root     21562 2022-04-22 12:02 containerd-rootless-setuptool.sh
  # -rwxr-xr-x root/root      7032 2022-04-22 12:02 containerd-rootless.sh
cd /usr/local/bin && ./containerd-rootless-setuptool.sh install # 注意不能以root用户运行


# 2.安装 nerdctl 全量包 (Full - 推荐)
tar Cxzvvf /usr/local nerdctl-full-0.19.0-linux-amd64.tar.gz
  # drwxr-xr-x 0/0               0 2022-04-22 12:16 bin/
  # -rwxr-xr-x 0/0        25371420 2015-10-21 08:00 bin/buildctl
  # -rwxr-xr-x 0/0        39651613 2015-10-21 08:00 bin/buildkitd
  # drwxr-xr-x 0/0               0 2022-04-22 12:16 share/doc/nerdctl-full/
  # -rw-r--r-- 0/0            1135 2022-04-22 12:16 share/doc/nerdctl-full/README.md
  # -rw-r--r-- 0/0            5425 2022-04-22 12:16 share/doc/nerdctl-full/SHA256SUMS

# 3.初始化安装以全量包为例,如出现如下命令表示安装成功。
/usr/local/bin$ ./containerd-rootless-setuptool.sh install
  # + systemctl --user enable containerd.service
  # Created symlink /home/weiyigeek/.config/systemd/user/default.target.wants/containerd.service → /home/weiyigeek/.config/systemd/user/containerd.service.
  # [INFO] Installed "containerd.service" successfully.
  # [INFO] To control "containerd.service", run: `systemctl --user (start|stop|restart) containerd.service`
  # [INFO] To run "containerd.service" on system startup automatically, run: `sudo loginctl enable-linger weiyigeek`
  # [INFO] ------------------------------------------------------------------------------------------
  # [INFO] Use `nerdctl` to connect to the rootless containerd.
  # [INFO] You do NOT need to specify $CONTAINERD_ADDRESS explicitly.

Step 3.启动 containerd 和 buildkitd 服务。

# 启动 containerd &&  验证服务状态
systemctl enable --now containerd && systemctl restart containerd && systemctl status containerd

# 启用 BuildKit 为了构建 Dockerfile，您需要使用以下命令启用 BuildKit。 
/usr/local/bin$ CONTAINERD_NAMESPACE=default containerd-rootless-setuptool.sh install-buildkit-containerd
  # [INFO] Creating "/home/weiyigeek/.config/systemd/user/default-buildkit.service"
  # [INFO] Starting systemd unit "default-buildkit.service"
  # + systemctl --user start default-buildkit.service
  # + sleep 3
  # + systemctl --user --no-pager --full status default-buildkit.service
  # ● default-buildkit.service - BuildKit (Rootless)
  #     Loaded: loaded (/home/weiyigeek/.config/systemd/user/default-buildkit.service; disabled; vendor preset: enabled)
  #     Active: active (running) since Mon 2022-04-25 13:05:25 CST; 3s ago
  #   Main PID: 329750 (buildkitd)
  #     CGroup: /user.slice/user-1000.slice/user@1000.service/default-buildkit.service
  #             └─329750 buildkitd --addr=unix:///run/user/1000/buildkit-default/buildkitd.sock --root=/home/weiyigeek/.local/share/buildkit-default --containerd-worker-namespace=default

  # Apr 25 13:05:25 node-2 systemd[327840]: Started BuildKit (Rootless).
  # Apr 25 13:05:26 node-2 containerd-rootless-setuptool.sh[329750]: time="2022-04-25T13:05:26+08:00" level=info msg="found worker \"ffl4pd8j6x7fh6t9o85a70l11\", labels=map[org.mobyproject.buildkit.worker.containerd.namespace:default org.mobyproject.buildkit.worker.containerd.uuid:4f029882-9edc-4c20-ab49-0363abe0d40e org.mobyproject.buildkit.worker.executor:containerd org.mobyproject.buildkit.worker.hostname:node-2 org.mobyproject.buildkit.worker.network:host org.mobyproject.buildkit.worker.snapshotter:overlayfs], platforms=[linux/amd64 linux/amd64/v2 linux/amd64/v3 linux/386]"
  # Apr 25 13:05:26 node-2 containerd-rootless-setuptool.sh[329750]: time="2022-04-25T13:05:26+08:00" level=info msg="found 1 workers, default=\"ffl4pd8j6x7fh6t9o85a70l11\""
  # Apr 25 13:05:26 node-2 containerd-rootless-setuptool.sh[329750]: time="2022-04-25T13:05:26+08:00" level=warning msg="currently, only the default worker can be used."
  # Apr 25 13:05:26 node-2 containerd-rootless-setuptool.sh[329750]: time="2022-04-25T13:05:26+08:00" level=info msg="running server on /run/user/1000/buildkit-default/buildkitd.sock"
  # + systemctl --user enable default-buildkit.service
  # Created symlink /home/weiyigeek/.config/systemd/user/default.target.wants/default-buildkit.service → /home/weiyigeek/.config/systemd/user/default-buildkit.service.
  # [INFO] Installed "default-buildkit.service" successfully.
  # [INFO] To control "default-buildkit.service", run: `systemctl --user (start|stop|restart) default-buildkit.service`

Step 4.查看nerdctl工具执行结果及其版本，至此安装完毕

~$ nerdctl --version
nerdctl version 0.19.0

使用 nerdctl 工具

描述: 在某一个用户执行时nerdctl命令时，我们可以其家目录中创建一个如下路径文件/home/weiyigeek/.config/nerdctl/nerdctl.toml,该文件可配置包含nerdctl相关配置项目。

简述尝试

• Step 1.镜像拉取与查看

  $ nerdctl pull hello-world:latest
    # docker.io/library/hello-world:latest:                                             resolved       |++++++++++++++++++++++++++++++++++++++|
    # index-sha256:10d7d58d5ebd2a652f4d93fdd86da8f265f5318c6a73cc5b6a9798ff6d2b2e67:    done           |++++++++++++++++++++++++++++++++++++++|
    # manifest-sha256:f54a58bc1aac5ea1a25d796ae155dc228b3f0e11d046ae276b39c4bf2f13d8c4: done           |++++++++++++++++++++++++++++++++++++++|
    # config-sha256:feb5d9fea6a5e9606aa995e879d862b825965ba48de054caab5ef356dc6b3412:   done           |++++++++++++++++++++++++++++++++++++++|
    # layer-sha256:2db29710123e3e53a794f2694094b9b4338aa9ee5c40b930cb8063a1be392c54:    done           |++++++++++++++++++++++++++++++++++++++|
    # elapsed: 34.3s                                                                    total:  6.9 Ki (205.0 B/s)
  
  $ nerdctl image ls
    # REPOSITORY     TAG       IMAGE ID        CREATED           PLATFORM       SIZE        BLOB SIZE
    # hello-world    latest    10d7d58d5ebd    31 seconds ago    linux/amd64    20.0 KiB    6.9 KiB

• Step 2.运行拉取的 hello-world 镜像

  # 创建并后台运行一个名为 hello-containerd 的容器
  $ nerdctl run -d --privileged --name hello-containerd hello-world:latest
    # 17cc212e69b2387b0aa1e4c4e679e41072eeeb14f909fff2acdb24d3c1033c0d
  
  # 查看 hello-containerd 容器日志
  $ nerdctl logs hello-containerd
    # Hello from Docker!
    # This message shows that your installation appears to be working correctly.
    # .......
    # Share images, automate workflows, and more with a free Docker ID: https://hub.docker.com/
    # For more examples and ideas, visit:https://docs.docker.com/get-started/

• Step 3.查看创建的容器相关信息以及删除容器。

  # 容器查看
  $ nerdctl ps -a
    # CONTAINER ID    IMAGE                                   COMMAND     CREATED          STATUS                      PORTS    NAMES
    # 17cc212e69b2    docker.io/library/hello-world:latest    "/hello"    2 minutes ago    Exited (0) 2 minutes ago             hello-containerd
  
  # 容器详细信息
  $ nerdctl inspect 17cc212e69b2
  [
    {
      "Id": "17cc212e69b2387b0aa1e4c4e679e41072eeeb14f909fff2acdb24d3c1033c0d",
      "Created": "2022-04-25T05:13:16.690822727Z",
      "Path": "/hello",
      "Args": null,
      "State": { "Status": "exited", "Running": false, "Paused": false,"Pid": 330226, "ExitCode": 0,"FinishedAt": "2022-04-25T05:13:17.251972898Z"},
      "Image": "docker.io/library/hello-world:latest",
      "ResolvConfPath": "/home/weiyigeek/.local/share/nerdctl/1935db59/containers/default/17cc212e69b2387b0aa1e4c4e679e41072eeeb14f909fff2acdb24d3c1033c0d/resolv.conf",
      "HostnamePath": "/home/weiyigeek/.local/share/nerdctl/1935db59/containers/default/17cc212e69b2387b0aa1e4c4e679e41072eeeb14f909fff2acdb24d3c1033c0d/hostname",
      "LogPath": "/home/weiyigeek/.local/share/nerdctl/1935db59/containers/default/17cc212e69b2387b0aa1e4c4e679e41072eeeb14f909fff2acdb24d3c1033c0d/17cc212e69b2387b0aa1e4c4e679e41072eeeb14f909fff2acdb24d3c1033c0d-json.log",
      "Name": "hello-containerd",
      "Driver": "overlayfs",
      "Platform": "linux",
      "AppArmorProfile": "",
      "Mounts": null,
      "NetworkSettings": null
    }
  ]
  
  # 移除 hello-containerd 容器
  $ nerdctl rm hello-containerd
  hello-containerd

• Step 4.删除指定 hello-world 镜像

$ nerdctl rmi hello-world
Untagged: docker.io/library/hello-world:latest@sha256:10d7d58d5ebd2a652f4d93fdd86da8f265f5318c6a73cc5b6a9798ff6d2b2e67
Deleted: sha256:e07ee1baac5fae6a26f30cabfe54a36d3402f96afda318fe0a96cec4ca393359

• Step 5.我们也可以拉取 nginx:alpine 镜像并创建、后台运行nginx容器、进入容器内部。

$ sudo nerdctl run -d --name nginx -p 80:80 nginx:alpine
  # docker.io/library/nginx:alpine:                                                   resolved       |++++++++++++++++++++++++++++++++++++++|
  # index-sha256:5a0df7fb7c8c03e4158ae9974bfbd6a15da2bdfdeded4fb694367ec812325d31:    done           |++++++++++++++++++++++++++++++++++++++|
  # manifest-sha256:efc09388b15fb423c402f0b8b28ca70c7fd20fe31f8d7531ae1896bbb4944999: done           |++++++++++++++++++++++++++++++++++++++|
  # config-sha256:51696c87e77e4ff7a53af9be837f35d4eacdb47b4ca83ba5fd5e4b5101d98502:   done           |++++++++++++++++++++++++++++++++++++++|
  # layer-sha256:4071be97c256d6f5ab0e05ebdebcfec3d0779a5e199ad0d71a5fccba4b3e2ce4:    done           |++++++++++++++++++++++++++++++++++++++|
  # layer-sha256:df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139:    done           |++++++++++++++++++++++++++++++++++++++|
  # layer-sha256:5867cba5fcbd3ae827c5801e76d20e7dc91cbb626ac5c871ec6c4d04eb818b16:    done           |++++++++++++++++++++++++++++++++++++++|
  # layer-sha256:4b639e65cb3ba47e77db93f93c6625a62ba1b9eec99160b254db380115ae009d:    done           |++++++++++++++++++++++++++++++++++++++|
  # layer-sha256:061ed9e2b9762825b9869a899a696ce8b56e7e0ec1e1892b980969bf7bcda56a:    done           |++++++++++++++++++++++++++++++++++++++|
  # layer-sha256:bc19f3e8eeb1bb75268787f8689edec9a42deda5cdecdf2f95b3c6df8eb57a48:    done           |++++++++++++++++++++++++++++++++++++++|
  # elapsed: 33.7s                                                                    total:  9.7 Mi (294.8 KiB/s)
  # 98eb2f1d4639b173dc21c30e40be5f3e2c410e4ca325b6bd6bafcaab46ab6c11

# 查看创建的nginx容器
$ sudo nerdctl ps
  # CONTAINER ID    IMAGE                             COMMAND                   CREATED              STATUS    PORTS                 NAMES
  # 98eb2f1d4639    docker.io/library/nginx:alpine    "/docker-entrypoint.…"    About an hour ago    Up        0.0.0.0:80->80/tcp    nginx

# 进入nginx容器内部
$ nerdctl exec -it nginx -- sh
/ # whoami
root
/ # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if4: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
    link/ether ee:e6:a6:9a:07:48 brd ff:ff:ff:ff:ff:ff
    inet 10.4.0.2/24 brd 10.4.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::ece6:a6ff:fe9a:748/64 scope link
       valid_lft forever preferred_lft forever

/usr/share/nginx/html # hostname > index.html
/usr/share/nginx/html # ip addr >> index.html

# 访问创建的nginx容器 或者 浏览器访问 10.10.107.227 宿主机
$ curl -I localhost

• Step 6.nginx 容器详细信息查看与容器停止、删除。

  $ nerdctl inspect --format "{{ .Name }} {{ .Id }}" nginx
    # nginx 98eb2f1d4639b173dc21c30e40be5f3e2c410e4ca325b6bd6bafcaab46ab6c11
  
  $ nerdctl stop nginx
    # nginx
  $ nerdctl ps  -a
    # CONTAINER ID    IMAGE                             COMMAND                   CREATED        STATUS                      PORTS                 NAMES
    # 98eb2f1d4639    docker.io/library/nginx:alpine    "/docker-entrypoint.…"    2 hours ago    Exited (0) 4 seconds ago    0.0.0.0:80->80/tcp    nginx
  
  $ nerdctl rm nginx
  nginx

weiyigeek.top-nerdctl-containerd

温馨提示: 不知读者是否发现, 其子命令及其使用方法与docker客户端工具基本是类似的，所以前面说到其学习成本较低, 熟悉docker的朋友可以快速上手

工具 nerdctl 命令

描述 : nerdctl 是 containerd 的命令行界面的工具。

命令参数:

# Usage:
  nerdctl [flags]
  nerdctl [command]

# Management commands:
  apparmor    Manage AppArmor profiles
  builder     Manage builds
  container   Manage containers
  image       Manage images
  ipfs        Distributing images on IPFS
  namespace   Manage containerd namespaces
  network     Manage networks
  system      Manage containerd
  volume      Manage volumes
Commands:
  build       Build an image from a Dockerfile. Needs buildkitd to be running.
  commit      Create a new image from a container changes
  completion  Generate the autocompletion script for the specified shell
  compose     Compose
  cp          Copy files/folders between a running container and the local filesystem.
  create      Create a new container. Optionally specify "ipfs://" or "ipns://" scheme to pull image from IPFS.
  events      Get real time events from the server
  exec        Run a command in a running container
  help        Help about any command
  history     Show the history of an image
  images      List images
  info        Display system-wide information
  inspect     Return low-level information on objects.
  kill        Kill one or more running containers
  load        Load an image from a tar archive or STDIN
  login       Log in to a Docker registry
  logout      Log out from a Docker registry
  logs        Fetch the logs of a container. Currently, only containers created with `nerdctl run -d` are supported.
  pause       Pause all processes within one or more containers
  port        List port mappings or a specific mapping for the container
  ps          List containers
  pull        Pull an image from a registry. Optionally specify "ipfs://" or "ipns://" scheme to pull image from IPFS.
  push        Push an image or a repository to a registry. Optionally specify "ipfs://" or "ipns://" scheme to push image to IPFS.
  restart     Restart one or more running containers
  rm          Remove one or more containers
  rmi         Remove one or more images
  run         Run a command in a new container. Optionally specify "ipfs://" or "ipns://" scheme to pull image from IPFS.
  save        Save one or more images to a tar archive (streamed to STDOUT by default)
  start       Start one or more running containers
  stats       Display a live stream of container(s) resource usage statistics.
  stop        Stop one or more running containers
  tag         Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
  top         Display the running processes of a container
  unpause     Unpause all processes within one or more containers
  update      Update one or more running containers
  version     Show the nerdctl version information
  wait        Block until one or more containers stop, then print their exit codes.

# Flags:
  -H, --H string                 Alias of --address (default "/run/containerd/containerd.sock")
  -a, --a string                 Alias of --address (default "/run/containerd/containerd.sock")
      --address string           containerd address, optionally with "unix://" prefix [$CONTAINERD_ADDRESS] (default "/run/containerd/containerd.sock")
      --cgroup-manager string    Cgroup manager to use ("cgroupfs"|"systemd") (default "none")
      --cni-netconfpath string   cni config directory [$NETCONFPATH] (default "/home/weiyigeek/.config/cni/net.d")
      --cni-path string          cni plugins binary directory [$CNI_PATH] (default "/usr/local/libexec/cni")
      --data-root string         Root directory of persistent nerdctl state (managed by nerdctl, not by containerd) (default "/home/weiyigeek/.local/share/nerdctl")
      --debug                    debug mode
      --debug-full               debug mode (with full output)
  -h, --help                     help for nerdctl
      --host string              Alias of --address (default "/run/containerd/containerd.sock")
      --hosts-dir strings        A directory that contains <HOST:PORT>/hosts.toml (containerd style) or <HOST:PORT>/{ca.cert, cert.pem, key.pem} (docker style) (default [/home/weiyigeek/.config/containerd/certs.d,/home/weiyigeek/.config/docker/certs.d])
      --insecure-registry        skips verifying HTTPS certs, and allows falling back to plain HTTP
  -n, --n string                 Alias of --namespace (default "default") , 支持名称空间。
      --namespace string         containerd namespace, such as "moby" for Docker, "k8s.io" for Kubernetes [$CONTAINERD_NAMESPACE] (default "default")
      --snapshotter string       containerd snapshotter [$CONTAINERD_SNAPSHOTTER] (default "overlayfs")
      --storage-driver string    Alias of --snapshotter (default "overlayfs")
  -v, --version                  version for nerdctl

nerdctl 使用示例

• 仓库认证

$ nerdctl login -u weiyigeek index.docker.io
  # Enter Password:
  # WARNING: Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See
  # https://docs.docker.com/engine/reference/commandline/login/#credentials-store
  # Login Succeeded

$ nerdctl logout
  # Removing login credentials for https://index.docker.io/v1/

• 镜像操作

# 拉取
nerdctl pull docker.io/library/nginx:alpine
nerdctl -n k8s.io pull docker.io/library/nginx:alpine

# 查看
nerdctl -n default image ls
nerdctl -n default inspect nginx:alpine

# 删除
nerdctl -n default rmi nginx:alpine

• 容器操作

  # 创建运行容器
  sudo nerdctl -n default run -d --name nginx -p 80:80 nginx:alpine
  sudo nerdctl -n k8s.io run -d --privileged --restart=always --name rancher -p 80:80 -p 443:443 docker.io/cnrancher/rancher:v2.4.17-ent
  
  # 查看容器
  sudo nerdctl -n k8s.io ps -a
  sudo nerdctl -n k8s.io inpsect rancher
  
  # 进入容器内部
  sudo nerdctl -n k8s.io exec rancher -- sh
  
  # 停止容器
  sudo nerdctl -n k8s.io stop rancher
  
  # 删除容器
  sudo nerdctl -n k8s.io rm rancher

---

有趣的实验功能

描述: 我在验证环境中使用时发现特别吸引人的最新功能一个是图像快速拉取（eStargz）, 另外一个是高速 rootless 模式。

1.使用懒加载加速图像拉取（eStargz）

描述: 在过去的一年中，可以在容器相关的工具（包括Kubernetes，contained，nerdctl，CRI-O，Podman，BuildKit，Kaniko等）上使用eStargz镜像的懒拉取功能, 在2021年懒拉取将成为越来越普遍的镜像分发技术，并且正在把eStargz增加到OCI容器镜像规范当中。

从 1.4 版开始 Containered 支持懒拉取, Stargz Snapshotter 是使 containerd 能够处理eStargz的插件。 这是一种用于懒拉取的镜像分发技术。这使容器运行时无需等待整个镜像内容即可启动容器。取而代之的是，按需获取必要的内容块（例如单个文件）。这样可以将容器启动延迟从数十秒缩短到最佳的几秒。

weiyigeek.top-eStargz加速拉取

配置参考地址: https://github.com/containerd/nerdctl/blob/master/docs/rootless.md#stargz-snapshotter

我们将继续执行如下步骤, 以启用图像拉取的加速（延迟拉取）, 即加速镜像拉取。

实践操作

1.安装与启用 Stargz 快照程序。

/usr/local/bin$ containerd-rootless-setuptool.sh install-stargz
[INFO] Creating "/home/weiyigeek/.config/systemd/user/stargz-snapshotter.service"
[INFO] Starting systemd unit "stargz-snapshotter.service"
+ systemctl --user start stargz-snapshotter.service
+ sleep 3
+ systemctl --user --no-pager --full status stargz-snapshotter.service
...........
[INFO] Installed "stargz-snapshotter.service" successfully.
[INFO] To control "stargz-snapshotter.service", run: `systemctl --user (start|stop|restart) stargz-snapshotter.service`
[INFO] Add the following lines to "/home/weiyigeek/.config/containerd/config.toml" manually, and then run `systemctl --user restart containerd.service`:
### BEGIN ###
[proxy_plugins]
  [proxy_plugins."stargz"]
    type = "snapshot"
    address = "/run/user/1000/containerd-stargz-grpc/containerd-stargz-grpc.sock"
###  END  ###
[INFO] Set `export CONTAINERD_SNAPSHOTTER="stargz"` to use the stargz snapshotter.

2.准备一个包含以下 containerd 内容的配置, 为了安全我们可以指定一个用户, 例如使用 nerdctl 的用户的 UID 为 1000 的配置放置位置为/run/user/1000。

# 确认方法当前用户UID
$ echo $UID
1000

# 在 containerd 配置文件中启用该插件
tee ~/.config/containerd/config.toml  <<'EOF'
# Plug stargz snapshotter into containerd
# Containerd recognizes stargz snapshotter through specified socket address.
[proxy_plugins]
  [proxy_plugins."stargz"]
    type = "snapshot"
    # NOTE: replace "1000" with your actual UID
    address = "/run/user/1000/containerd-stargz-grpc/containerd-stargz-grpc.sock"

# Use stargz snapshotter through CRI

EOF

3.重新启动 restargz-snapshotter 与 containerd 以使配置更改生效。

systemctl daemon-reload 
systemctl --user start restargz-snapshotter.service && systemctl --user restart containerd.service
systemctl status stargz-snapshotter.service && systemctl status containerd.service

● stargz-snapshotter.service - stargz snapshotter (Rootless)
     Loaded: loaded (/home/weiyigeek/.config/systemd/user/stargz-snapshotter.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-04-25 16:48:56 CST; 7s ago
   Main PID: 360664 (containerd-star)
     CGroup: /user.slice/user-1000.slice/user@1000.service/stargz-snapshotter.service
             └─360664 containerd-stargz-grpc -address /run/user/1000/containerd-stargz-grpc/containerd-stargz-grpc.sock -root /home/weiyigeek/.local/share/containerd-stargz-grpc -config /home/weiyigeek/.config/container>
Apr 25 16:48:56 node-2 systemd[358295]: Started stargz snapshotter (Rootless).

4.您可以选择执行以下命令,并指定使用--snapshotter=stargz进行快速图像拉取。

export CONTAINERD_SNAPSHOTTER="stargz"
nerdctl run --snapshotter=stargz ghcr.io/stargz-containers/python:3.10-esgz python -c "print('hello')"

2.配置高速的 rootless 模式

描述: Rootless模式允许你在没有特权的情况下运行容器，具有安全优势，但它的缺点是容器外的通信速度比特权 Rootful 模式慢。

但是我们可以通过使用 bypass4netns, 使得 Rootless 的容器外通信速度将比 Rootful 更快, 操作如下所示:

1.安装与启用 bypass4netns

/usr/local/bin$ containerd-rootless-setuptool.sh install-bypass4netnsd

2.您可以选择执行如下命令，使用快速无根功能启动容器和以下--label nerdctl/bypass4netns=true参数快速图像拉取。

$ nerdctl run --label nerdctl/bypass4netns=true ghcr.io/stargz-containers/python:3.10-esgz python -c "print('hello')"

---

实践.使用 containerd & nerdctl 工具快速部署 redis 数据库服务容器

Step 1.准备 redis 数据库配置文件极其相关目录

mkdir -vp /app/redis/data
tee /app/redis/redis.conf <<'EOF'
# 绑定任意接口、服务端口、后台运行。
bind 0.0.0.0
port 6379
# 容器里必须设置为no
daemonize no
supervised auto

# redis服务pid进程文件名
pidfile "/var/run/redis.pid"

# 关闭保护模式，并配置使用密码访问
protected-mode no
requirepass "123456"
# echo -e "weiyigeek"|sha256sum 
# requirepass 097575a79efcd7ea7b1efa2bcda78a4fc7cbd0820736b2f2708e72c3d21f8b61

# 数据文件保存路径，rdb/AOF文件也保存在这里
dir "/data"

# 日志文件记录文件(notice / verbose)
# logfile "/logs/redis.log"
# loglevel verbose  

# 最大客户端连接数
maxclients 10000

# 客户端连接空闲多久后断开连接，单位秒，0表示禁用
timeout 60
tcp-keepalive 60 

# Redis 数据持久化(rdb/aof)配置
# RDB 文件名
dbfilename "dump.rdb"
# 数据自动保存脚本条件例如300s中有10key发生变化
save 300 10
save 60 10000
# 对RDB文件进行压缩，建议以（磁盘）空间换（CPU）时间。
rdbcompression yes
# 版本5的RDB有一个CRC64算法的校验和放在了文件的最后。这将使文件格式更加可靠。
rdbchecksum yes
# RDB自动触发策略是否启用，默认为yes
rdb-save-incremental-fsync yes

# AOF开启
appendonly yes
# AOF文件名
appendfilename "appendonly.aof"
# 可选值 always， everysec，no，建议设置为everysec
appendfsync everysec

# Redis风险命令重命名
# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
rename-command FLUSHDB b840fc02d524045429941cc15f59e41cb7be6c53
rename-command FLUSHALL b840fc02d524045429941cc15f59e41cb7be6c54
rename-command EVAL b840fc02d524045429941cc15f59e41cb7be6c55
rename-command DEBUG b840fc02d524045429941cc15f59e41cb7be6c56
# rename-command SHUTDOWN SHUTDOWN
EOF

Step 2.执行如下命令进行快速创建容器并运行redis服务。

$ nerdctl run -d -p 6379:6379 \
 -v /app/redis/redis.conf:/etc/redis/redis.conf \
 -v /app/redis/data:/data \
 --name redis-server redis:6.2.6-alpine3.15 redis-server /etc/redis/redis.conf
5e854a58087ae1bba5a661b2941474560cbecc37f54c7f4e7a28afbaed6aebf0

Step 3.查看创建的容器并验证redis服务是否正常。

$ nerdctl ps
CONTAINER ID    IMAGE                                       COMMAND                   CREATED           STATUS    PORTS                     NAMES
5e854a58087a    docker.io/library/redis:6.2.6-alpine3.15    "docker-entrypoint.s…"    42 seconds ago    Up        0.0.0.0:6379->6379/tcp    redis-server

$ nerdctl logs redis-server
1:C 25 Apr 2022 13:22:59.597 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 25 Apr 2022 13:22:59.597 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 25 Apr 2022 13:22:59.597 # Configuration loaded


# 如下返回，表示 redis 服务状态是正常的
$ nerdctl exec -it redis-server redis-cli -a 123456 ping
PONG
$ nerdctl exec -it redis-server redis-cli -a 123456 info
# Server
redis_version:6.2.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:63421500bb103677
redis_mode:standalone
os:Linux 5.4.0-96-generic x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin

Step 4.这里我们直接用telnet工具连接创建的redis容器中的redis数据库服务。

$ telnet 10.10.107.227 6379
Trying 10.10.107.227...
Connected to 10.10.107.227.
Escape character is '^]'.
auth 123456  # 认证
+OK
ping   # 服务验证
+PONG 
set name weiyigeek # 设置 字符串 类型的key
+OK
get name  # 获取 key 的值
$9
weiyigeek

---

0x0n 入坑出坑

• 错误1.安装nerdctl（Minimal）时执行containerd-rootless-setuptool.sh脚本安装时报[ERROR] Refusing to install rootless containerd as the root user错误
  解决办法: 请切换到普通用户执行。

• 错误2.安装nerdctl（Minimal）时执行containerd-rootless-setuptool.sh脚本安装时报containerd-rootless-setuptool.sh: 110: rootlesskit: Permission denied错误

  /usr/local/bin$ containerd-rootless-setuptool.sh install
  [INFO] Checking RootlessKit functionality
  /usr/local/bin/containerd-rootless-setuptool.sh: 110: rootlesskit: Permission denied

  问题原因: 由于未安装 rootlesskit 相关依赖工具
  解决办法: 执行如下命令进行安装操作 apt install -y rootlesskit rootlessctl

• 错误3.安装nerdctl (full) 时执行containerd-rootless-setuptool.sh脚本安装时报exec: "newuidmap": executable file not found in $PATH错误

  ./containerd-rootless-setuptool.sh install
  [INFO] Checking RootlessKit functionality
  [rootlesskit:parent] error: failed to setup UID/GID map: newuidmap 328299 [0 1000 1 1 100000 65536] failed: : exec: "newuidmap": executable file not found in $PATH

  问题原因: newuidmap 在没有安装的环境下会出现上述错误，所以执行apt命令安装uidmap软件包。
  解决版本: sudo apt install uidmap