Docker 容器與 Virtual Machine 不同，它本質上是一個運作環境被隔離起來的系統子程序，在 Host 上共用了底層的資源，例如 Linux 核心。
當上述兩者相遇，就有不直覺的事情產生了：當我們打包一些程式，裡面包含了 SystemD Service File，若我們希望可以在 Docker Container 裡面透過 systemd 的指令 systemctl 啟動程序的話，需要一些先決條件跟額外處置才行。
[UPDATE] 這篇示範的東西，其實應該不太算 Docker 的正規用法 ～ 如果有「在 Container 裡面透過 SystemD 操作軟體服務」的需求，可以參考看看用 LXC 打造的測試環境系列筆記看看～～
[UPDATE2] 比起 LXC 更方便的衍生界面 LXD 用起來更方便～～～
這裡以打包一個具備 ssh 服務與 vi 編輯軟體的 Docker Image 為範例：這個 Image 的用亦是拿來作為測試環境用的，當成一個輕量的「假 VM」來使用。因此，這違反了所謂容器微服務化的精神相牴觸～～
以下先示範失敗的狀況：
當我們想也不想（還是要花一點腦筋～），直接基於一個 CentOS 7 的 Docker Image，用 YUM 指令裝 sshd 服務，然後要啟動的話，會導致下面最末出現的狀況（容器啟動失敗）
[docker@docker_host ~]$ mkdir -p ~/sshd_container
[docker@docker_host sshd_container]$ cd ~/sshd_container
[docker@docker_host sshd_container]$ cat << "EOF" > ./dockerfile
FROM centos:7
RUN yum install -y \
--setopt=tsflags=nodocs \
--disableplugin=fastestmirror \
openssh-clients \
openssh-server \
sed \
vim-minimal \
&& yum clean all \
&& rm -rf /var/cache/yum \
&& sed -e 's/#PermitRootLogin yes/PermitRootLogin yes/g' \
-i /etc/ssh/sshd_config \
&& echo "root:password" | chpasswd
#CMD ["service", "sshd", "start"]
CMD ["systemctl", "start", "sshd.service"]
EOF
[docker@docker_host sshd_container]$ docker build -t centos7-sshd-systemd ./
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM centos:7
7: Pulling from library/centos
7dc0dca2b151: Pull complete
Digest: sha256:b67d21dfe609ddacf404589e04631d90a342921e81c40aeaf3391f6717fa5322
Status: Downloaded newer image for centos:7
---> 49f7960eb7e4
Step 2/3 : RUN yum install -y --setopt=tsflags=nodocs --disableplugin=fastestmirror openssh-clients openssh-server sed vim-minimal && yum clean all && rm -rf /var/cache/yum && sed -e 's/#PermitRootLogin yes/PermitRootLogin yes/g' -i /etc/ssh/sshd_config && echo "root:password" | chpasswd
---> Running in 97471d100549
Loaded plugins: ovl
Package sed-4.2.2-5.el7.x86_64 already installed and latest version
Package 2:vim-minimal-7.4.160-4.el7.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package openssh-clients.x86_64 0:7.4p1-16.el7 will be installed
--> Processing Dependency: openssh = 7.4p1-16.el7 for package: openssh-clients-7.4p1-16.el7.x86_64
--> Processing Dependency: fipscheck-lib(x86-64) >= 1.3.0 for package: openssh-clients-7.4p1-16.el7.x86_64
--> Processing Dependency: libfipscheck.so.1()(64bit) for package: openssh-clients-7.4p1-16.el7.x86_64
--> Processing Dependency: libedit.so.0()(64bit) for package: openssh-clients-7.4p1-16.el7.x86_64
---> Package openssh-server.x86_64 0:7.4p1-16.el7 will be installed
--> Processing Dependency: libwrap.so.0()(64bit) for package: openssh-server-7.4p1-16.el7.x86_64
--> Running transaction check
---> Package fipscheck-lib.x86_64 0:1.4.1-6.el7 will be installed
--> Processing Dependency: /usr/bin/fipscheck for package: fipscheck-lib-1.4.1-6.el7.x86_64
---> Package libedit.x86_64 0:3.0-12.20121213cvs.el7 will be installed
---> Package openssh.x86_64 0:7.4p1-16.el7 will be installed
---> Package tcp_wrappers-libs.x86_64 0:7.6-77.el7 will be installed
--> Running transaction check
---> Package fipscheck.x86_64 0:1.4.1-6.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository
Size
================================================================================
Installing:
openssh-clients x86_64 7.4p1-16.el7 base 655 k
openssh-server x86_64 7.4p1-16.el7 base 458 k
Installing for dependencies:
fipscheck x86_64 1.4.1-6.el7 base 21 k
fipscheck-lib x86_64 1.4.1-6.el7 base 11 k
libedit x86_64 3.0-12.20121213cvs.el7 base 92 k
openssh x86_64 7.4p1-16.el7 base 510 k
tcp_wrappers-libs x86_64 7.6-77.el7 base 66 k
Transaction Summary
================================================================================
Install 2 Packages (+5 Dependent packages)
Total download size: 1.8 M
Installed size: 5.8 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/base/packages/fipscheck-lib-1.4.1-6.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Public key for fipscheck-lib-1.4.1-6.el7.x86_64.rpm is not installed
--------------------------------------------------------------------------------
Total 2.2 MB/s | 1.8 MB 00:00
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) "
Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
Package : centos-release-7-5.1804.el7.centos.2.x86_64 (@Updates)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : fipscheck-1.4.1-6.el7.x86_64 1/7
Installing : fipscheck-lib-1.4.1-6.el7.x86_64 2/7
Installing : openssh-7.4p1-16.el7.x86_64 3/7
Installing : libedit-3.0-12.20121213cvs.el7.x86_64 4/7
Installing : tcp_wrappers-libs-7.6-77.el7.x86_64 5/7
Installing : openssh-server-7.4p1-16.el7.x86_64 6/7
Installing : openssh-clients-7.4p1-16.el7.x86_64 7/7
Verifying : fipscheck-lib-1.4.1-6.el7.x86_64 1/7
Verifying : openssh-7.4p1-16.el7.x86_64 2/7
Verifying : tcp_wrappers-libs-7.6-77.el7.x86_64 3/7
Verifying : fipscheck-1.4.1-6.el7.x86_64 4/7
Verifying : libedit-3.0-12.20121213cvs.el7.x86_64 5/7
Verifying : openssh-clients-7.4p1-16.el7.x86_64 6/7
Verifying : openssh-server-7.4p1-16.el7.x86_64 7/7
Installed:
openssh-clients.x86_64 0:7.4p1-16.el7 openssh-server.x86_64 0:7.4p1-16.el7
Dependency Installed:
fipscheck.x86_64 0:1.4.1-6.el7 fipscheck-lib.x86_64 0:1.4.1-6.el7
libedit.x86_64 0:3.0-12.20121213cvs.el7 openssh.x86_64 0:7.4p1-16.el7
tcp_wrappers-libs.x86_64 0:7.6-77.el7
Complete!
Loaded plugins: fastestmirror, ovl
Cleaning repos: base extras updates
Cleaning up everything
Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
---> 7359492fd7e4
Removing intermediate container 97471d100549
Step 3/3 : CMD systemctl start sshd.service
---> Running in fae3af4bfbf6
---> 63227558a887
Removing intermediate container fae3af4bfbf6
Successfully built 63227558a887
Successfully tagged centos7-sshd-systemd:latest
[docker@docker_host sshd_container]$ docker run -d --name mimic_vm centos7-sshd-systemd
335032ef2314b25c6074708d0fde615f24971d9446eb0f47a3447458861a457d
[docker@docker_host sshd_container]$
[brandon@ai-node-01 sshd_container]$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
[brandon@ai-node-01 sshd_container]$ docker container ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
335032ef2314 centos7-sshd-systemd "systemctl start s..." About a minute ago Exited (1) About a minute ago mimic_vm
[docker@docker_host sshd_container]$
原因在於
1. 由於 SystemD init 在 OS 內緊咬著 PID 1 不放。因此在容器裡面啟動 SystemD init，並執行 systemctl，也會去跟 Host 的 SystemD init 搶 PID 1，因而會跟外面「真正的 SystemD init」衝突。
2. SystemD 需要的 DBus 跟 cgroup 相關環境，在容器的內部從缺。
如果「非得要」在容器裡面使用 SystemD 的指令的話，以下為解決方式
[docker@docker_host sshd_container]$ docker image rm -f 63227558a887 Untagged: centos7-sshd-systemd-2:latest Deleted: sha256:63227558a8876426ee47e642e06333cf58a960d323d42e0637d771e5e8f966fd Deleted: sha256:7359492fd7e43e0a77f6c4a394590d0ff4f0f8d9b6be0c773e99b5f28f286046 [docker@docker_host sshd_container]$ cat << "EOF" > ./dockerfile FROM centos/systemd:latest # 將打算啟動的服務用 systemctl enable 打開，就會在 container「開機」時啟動 RUN yum install -y \ --setopt=tsflags=nodocs \ --disableplugin=fastestmirror \ openssh-clients \ openssh-server \ sed \ vim-minimal \ && yum clean all \ && rm -rf /var/cache/yum \ && sed -e 's/#PermitRootLogin yes/PermitRootLogin yes/g' \ -i /etc/ssh/sshd_config \ && echo "root:password" | chpasswd \ && systemctl enable sshd.service CMD ["/usr/sbin/init"] EOF [docker@docker_host sshd_container]$ docker build -t centos7-sshd-systemd ./ Sending build context to Docker daemon 2.048kB Step 1/3 : FROM centos/systemd:latest ---> 5477808193ce Step 2/3 : RUN yum install -y --setopt=tsflags=nodocs --disableplugin=fastestmirror openssh-clients openssh-server sed vim-minimal && yum clean all && rm -rf /var/cache/yum && sed -e 's/#PermitRootLogin yes/PermitRootLogin yes/g' -i /etc/ssh/sshd_config && echo "root:password" | chpasswd && systemctl enable sshd.service ---> Running in 5ffe827ff116 Loaded plugins: ovl Package sed-4.2.2-5.el7.x86_64 already installed and latest version Resolving Dependencies --> Running transaction check ---> Package openssh-clients.x86_64 0:7.4p1-16.el7 will be installed --> Processing Dependency: openssh = 7.4p1-16.el7 for package: openssh-clients-7.4p1-16.el7.x86_64 --> Processing Dependency: fipscheck-lib(x86-64) >= 1.3.0 for package: openssh-clients-7.4p1-16.el7.x86_64 --> Processing Dependency: libfipscheck.so.1()(64bit) for package: openssh-clients-7.4p1-16.el7.x86_64 --> Processing Dependency: libedit.so.0()(64bit) for package: openssh-clients-7.4p1-16.el7.x86_64 ---> Package openssh-server.x86_64 0:7.4p1-16.el7 will be installed --> Processing Dependency: libwrap.so.0()(64bit) for package: openssh-server-7.4p1-16.el7.x86_64 ---> Package vim-minimal.x86_64 2:7.4.160-2.el7 will be updated ---> Package vim-minimal.x86_64 2:7.4.160-4.el7 will be an update --> Running transaction check ---> Package fipscheck-lib.x86_64 0:1.4.1-6.el7 will be installed --> Processing Dependency: /usr/bin/fipscheck for package: fipscheck-lib-1.4.1-6.el7.x86_64 ---> Package libedit.x86_64 0:3.0-12.20121213cvs.el7 will be installed ---> Package openssh.x86_64 0:7.4p1-16.el7 will be installed ---> Package tcp_wrappers-libs.x86_64 0:7.6-77.el7 will be installed --> Running transaction check ---> Package fipscheck.x86_64 0:1.4.1-6.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: openssh-clients x86_64 7.4p1-16.el7 base 655 k openssh-server x86_64 7.4p1-16.el7 base 458 k Updating: vim-minimal x86_64 2:7.4.160-4.el7 base 437 k Installing for dependencies: fipscheck x86_64 1.4.1-6.el7 base 21 k fipscheck-lib x86_64 1.4.1-6.el7 base 11 k libedit x86_64 3.0-12.20121213cvs.el7 base 92 k openssh x86_64 7.4p1-16.el7 base 510 k tcp_wrappers-libs x86_64 7.6-77.el7 base 66 k Transaction Summary ================================================================================ Install 2 Packages (+5 Dependent packages) Upgrade 1 Package Total download size: 2.2 M Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. Public key for fipscheck-1.4.1-6.el7.x86_64.rpm is not installed warning: /var/cache/yum/x86_64/7/base/packages/fipscheck-1.4.1-6.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY -------------------------------------------------------------------------------- Total 1.5 MB/s | 2.2 MB 00:01 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 Importing GPG key 0xF4A80EB5: Userid : "CentOS-7 Key (CentOS 7 Official Signing Key)" Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5 Package : centos-release-7-4.1708.el7.centos.x86_64 (@CentOS) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : fipscheck-1.4.1-6.el7.x86_64 1/9 Installing : fipscheck-lib-1.4.1-6.el7.x86_64 2/9 Installing : openssh-7.4p1-16.el7.x86_64 3/9 Installing : libedit-3.0-12.20121213cvs.el7.x86_64 4/9 Installing : tcp_wrappers-libs-7.6-77.el7.x86_64 5/9 Installing : openssh-server-7.4p1-16.el7.x86_64 6/9 Installing : openssh-clients-7.4p1-16.el7.x86_64 7/9 Updating : 2:vim-minimal-7.4.160-4.el7.x86_64 8/9 Cleanup : 2:vim-minimal-7.4.160-2.el7.x86_64 9/9 Verifying : fipscheck-lib-1.4.1-6.el7.x86_64 1/9 Verifying : openssh-7.4p1-16.el7.x86_64 2/9 Verifying : tcp_wrappers-libs-7.6-77.el7.x86_64 3/9 Verifying : fipscheck-1.4.1-6.el7.x86_64 4/9 Verifying : libedit-3.0-12.20121213cvs.el7.x86_64 5/9 Verifying : openssh-clients-7.4p1-16.el7.x86_64 6/9 Verifying : 2:vim-minimal-7.4.160-4.el7.x86_64 7/9 Verifying : openssh-server-7.4p1-16.el7.x86_64 8/9 Verifying : 2:vim-minimal-7.4.160-2.el7.x86_64 9/9 Installed: openssh-clients.x86_64 0:7.4p1-16.el7 openssh-server.x86_64 0:7.4p1-16.el7 Dependency Installed: fipscheck.x86_64 0:1.4.1-6.el7 fipscheck-lib.x86_64 0:1.4.1-6.el7 libedit.x86_64 0:3.0-12.20121213cvs.el7 openssh.x86_64 0:7.4p1-16.el7 tcp_wrappers-libs.x86_64 0:7.6-77.el7 Updated: vim-minimal.x86_64 2:7.4.160-4.el7 Complete! Loaded plugins: fastestmirror, ovl Cleaning repos: base extras updates Cleaning up everything Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos ---> 2f9bd1a066ad Removing intermediate container 5ffe827ff116 Step 3/3 : CMD /usr/sbin/init ---> Running in 2fa4d4494b89 ---> e1dab9368a87 Removing intermediate container 2fa4d4494b89 Successfully built e1dab9368a87 Successfully tagged centos7-sshd-systemd:latest [docker@docker_host sshd_container]$ docker run -d --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:ro --name mimic_vm centos7-sshd-systemd [docker@docker_host sshd_container]$ docker inspect mimic_vm | grep IPAddress "SecondaryIPAddresses": null, "IPAddress": "172.17.0.2", "IPAddress": "172.17.0.2", [docker@docker_host sshd_container]$ [docker@docker_host sshd_container]$ ssh root@172.17.0.2 The authenticity of host '172.17.0.2 (172.17.0.2)' can't be established. ECDSA key fingerprint is SHA256:c8c3RYWJ5lDWhNFvAQRdqHGPmBMpsg63sXiFVpA4A60. ECDSA key fingerprint is MD5:f2:e2:00:63:0b:7d:81:ef:87:a0:5c:ff:b7:be:1e:1f. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '172.17.0.2' (ECDSA) to the list of known hosts. root@172.17.0.2's password: System is booting up. See pam_nologin(8) [root@2c7bb254bacd ~]# [root@2c7bb254bacd ~]# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 42768 3056 ? Ss 03:03 0:00 /usr/sbin/init root 17 0.1 0.0 37932 5516 ? Ss 03:03 0:00 /usr/lib/systemd/systemd-journald root 40 0.0 0.0 105996 4100 ? Ss 03:03 0:00 /usr/sbin/sshd -D dbus 45 0.0 0.0 24284 1604 ? Ss 03:05 0:00 /bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation root 46 0.0 0.0 24204 1648 ? Ss 03:05 0:00 /usr/lib/systemd/systemd-logind root 60 0.1 0.0 144236 5940 ? Ss 03:06 0:00 sshd: root@pts/0 root 62 0.0 0.0 15208 2008 pts/0 Ss 03:06 0:00 -bash root 77 0.0 0.0 50880 1812 pts/0 R+ 03:07 0:00 ps aux [root@2c7bb254bacd ~]# [root@2c7bb254bacd ~]# exit logout Connection to 172.17.0.2 closed. [brandon@ai-node-01 sshd_container]$ docker container exec -it mimic_vm bash [root@2c7bb254bacd /]# [root@2c7bb254bacd /]# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 42768 3056 ? Ss 03:03 0:00 /usr/sbin/init root 17 0.0 0.0 37932 5520 ? Ss 03:03 0:00 /usr/lib/systemd/systemd-journald root 40 0.0 0.0 105996 4100 ? Ss 03:03 0:00 /usr/sbin/sshd -D dbus 45 0.0 0.0 24284 1604 ? Ss 03:05 0:00 /bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation root 46 0.0 0.0 24204 1648 ? Ss 03:05 0:00 /usr/lib/systemd/systemd-logind root 78 0.0 0.0 11776 1888 pts/0 Ss 03:08 0:00 bash root 91 0.0 0.0 47448 1672 pts/0 R+ 03:08 0:00 ps aux [root@2c7bb254bacd /]#
上面的作業，是使用官方修剪過的 base image，已經把會跟 Host OS 搶核心程序的服務拔掉。然後從 Host 以唯讀的方式把 /sys/fs/cgroup 目錄掛載進來，提供 systemctl 所需的資料。接著，想要在 docker 裡面使用 systemd service file 的東西，只要設定開機啟動就好了。
這個解決方式有一個大前提：現在運行 Docker Runtime 的作業系統，本身就要基於 SystemD init 才行，不然 . . . 在 /sys/fs/cgroup 裡面就沒有需要的東西了（但是，這些 Linux 也有這些目錄喔）。例如，Host OS 叫做 Slackware、Alpine Linux、Boot2docker（基於 Tiny Core Linux）就沒轍了～
因此，這裡講的東西，是沒辦法在 Docker Toolbox 環境下面使用（能掛載的目錄是 Host OS 的，而不是背後的 boot2docker VM） . . . 更不用說在 Docker for Windows 環境裡面跑了～ 不過倒是可以考慮透過 Docker machine 處理一個 CentOS 7 / Debian Host VM。
最後，這裡打包的「假 VM」，當作測試環境還不錯，可以不用困擾開多台 VM 來測試而讓電腦鈍鈍了，想要打掉重練也更簡易一點～～而且經過測試，只要用 docker network 畫出一個網段，如此一來，連 Virtual IP 也可以使用 OS 指令，在「docker 內網」啟動喔。
但是，千萬不要把這樣的東西放到正式環境裡面，因為這裡有寫死內部的 root 密碼！
參考資料
centos/systemd - Docker Hub 的說明
Running systemd within a Docker Container - RHD Blog
Running systemd in a non-privileged container - RHD Blog
Is there any concrete and acceptable solution for running systemd inside the docker container? - DevOps Stack Exchange
Running systemd inside a docker container (arch linux) - Server Fault
玩具烏托邦: Docker 新手第二課： 前景/背景/收工休息 圖解狀態轉換（Note: 在 2018 的今日是可以用在 Production 的～只是使用思維不是當成 VM 看待）
[2019/5 更新]
gdraheim/docker-systemctl-replacement: docker systemctl replacement - allows to deploy to systemd-controlled containers without starting an actual systemd daemon (e.g. centos7, ubuntu16) 沒用過，不知道這個替代品好不好～
How to run systemd in a container - Red Hat Developer Blog：鼓吹 SystemD 的公司所開發的競爭方案
沒有留言:
張貼留言