結果。。。竟然發現！三兩下就設置完畢了，而且前面好幾篇的設置，竟然在 LXD 都處理妥當了～
也就是說，使用 LXD 作叢集測試環境，是更加更加方便的～這篇筆記，簡要紀錄一下設置環境
LXD 的操作設計，有別於 LXC 原始的操作方式，比較像 Docker 風格的操作方式。也支援一些進階的功能（例如，RESTful API 互動界面、LXD 管理節點作叢集等），但。。這都不是這邊關心的。。這邊，只有關心如何省省自己的電腦記憶體，方便軟體叢集演練而已。
首先是安裝，方式超簡單（如果顯示安裝不成功，執行一下 sudo apt-get update 再執行即可）
elementary@elementary:~$ sudo apt-get install -y lxd Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: ebtables liblxc-common liblxc1 libuv1 lxcfs lxd-client uidmap xdelta3 Suggested packages: criu lxd-tools The following NEW packages will be installed: ebtables liblxc-common liblxc1 libuv1 lxcfs lxd lxd-client uidmap xdelta3 0 upgraded, 9 newly installed, 0 to remove and 531 not upgraded. Need to get 9,097 kB/9,246 kB of archives. After this operation, 35.2 MB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 liblxc1 amd64 3.0.3-0ubuntu1~18.04.1 [264 kB] Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 liblxc-common amd64 3.0.3-0ubuntu1~18.04.1 [438 kB] Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 libuv1 amd64 1.18.0-3 [64.4 kB] Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 lxcfs amd64 3.0.3-0ubuntu1~18.04.1 [38.9 kB] Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 lxd-client amd64 3.0.3-0ubuntu1~18.04.1 [3,027 kB] Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 uidmap amd64 1:4.5-1ubuntu2 [65.7 kB] Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 lxd amd64 3.0.3-0ubuntu1~18.04.1 [5,199 kB] Fetched 9,097 kB in 16s (579 kB/s) Selecting previously unselected package ebtables. (Reading database ... 146647 files and directories currently installed.) Preparing to unpack .../0-ebtables_2.0.10.4-3.5ubuntu2.18.04.3_amd64.deb ... Unpacking ebtables (2.0.10.4-3.5ubuntu2.18.04.3) ... Selecting previously unselected package liblxc1. Preparing to unpack .../1-liblxc1_3.0.3-0ubuntu1~18.04.1_amd64.deb ... Unpacking liblxc1 (3.0.3-0ubuntu1~18.04.1) ... Selecting previously unselected package liblxc-common. Preparing to unpack .../2-liblxc-common_3.0.3-0ubuntu1~18.04.1_amd64.deb ... Unpacking liblxc-common (3.0.3-0ubuntu1~18.04.1) ... Selecting previously unselected package libuv1:amd64. Preparing to unpack .../3-libuv1_1.18.0-3_amd64.deb ... Unpacking libuv1:amd64 (1.18.0-3) ... Selecting previously unselected package lxcfs. Preparing to unpack .../4-lxcfs_3.0.3-0ubuntu1~18.04.1_amd64.deb ... Unpacking lxcfs (3.0.3-0ubuntu1~18.04.1) ... Selecting previously unselected package lxd-client. Preparing to unpack .../5-lxd-client_3.0.3-0ubuntu1~18.04.1_amd64.deb ... Unpacking lxd-client (3.0.3-0ubuntu1~18.04.1) ... Selecting previously unselected package uidmap. Preparing to unpack .../6-uidmap_1%3a4.5-1ubuntu2_amd64.deb ... Unpacking uidmap (1:4.5-1ubuntu2) ... Selecting previously unselected package xdelta3. Preparing to unpack .../7-xdelta3_3.0.11-dfsg-1ubuntu1_amd64.deb ... Unpacking xdelta3 (3.0.11-dfsg-1ubuntu1) ... Selecting previously unselected package lxd. Preparing to unpack .../8-lxd_3.0.3-0ubuntu1~18.04.1_amd64.deb ... Adding system user `lxd' (UID 121) ... Adding new user `lxd' (UID 121) with group `nogroup' ... Creating home directory `/var/lib/lxd/' ... Adding group `lxd' (GID 128) ... Done. Unpacking lxd (3.0.3-0ubuntu1~18.04.1) ... Processing triggers for ureadahead (0.100.0-20) ... Setting up libuv1:amd64 (1.18.0-3) ... Setting up uidmap (1:4.5-1ubuntu2) ... Setting up lxd-client (3.0.3-0ubuntu1~18.04.1) ... Processing triggers for libc-bin (2.27-3ubuntu1) ... Processing triggers for systemd (237-3ubuntu10.3) ... Setting up ebtables (2.0.10.4-3.5ubuntu2.18.04.3) ... Created symlink /etc/systemd/system/multi-user.target.wants/ebtables.service → /lib/systemd/system/ebtables.service. update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults Setting up lxcfs (3.0.3-0ubuntu1~18.04.1) ... Created symlink /etc/systemd/system/multi-user.target.wants/lxcfs.service → /lib/systemd/system/lxcfs.service. Processing triggers for man-db (2.8.3-2) ... Setting up xdelta3 (3.0.11-dfsg-1ubuntu1) ... Setting up liblxc1 (3.0.3-0ubuntu1~18.04.1) ... Setting up liblxc-common (3.0.3-0ubuntu1~18.04.1) ... apparmor.service is not active, cannot reload. invoke-rc.d: initscript apparmor, action "reload" failed. Setting up lxd (3.0.3-0ubuntu1~18.04.1) ... Created symlink /etc/systemd/system/multi-user.target.wants/lxd-containers.service → /lib/systemd/system/lxd-containers.service. Created symlink /etc/systemd/system/sockets.target.wants/lxd.socket → /lib/systemd/system/lxd.socket. Setting up lxd dnsmasq configuration. To go through the initial LXD configuration, run: lxd init Processing triggers for ureadahead (0.100.0-20) ... Processing triggers for systemd (237-3ubuntu10.3) ... Processing triggers for libc-bin (2.27-3ubuntu1) ... elementary@elementary:~$
安裝完畢之後，原則上 LXD 就已經啟動了（重開機也會啟動）。
此外，由於 Ubuntu 系列的 OS 都以 sudoer 來操作，剛剛執行安裝的 sudoer 帳號，基本上已經在 lxd 群組裡面，只要重登入，就可以直接操作 LXD。
重新登入之後，就可以直接操作 LXC。。。這邊寫的，的確是 LXC 沒錯，不是 LXD～這是因為 LXD 分成 LXD Server 跟 LXD Client 兩塊，操作時的指令幾乎都是 LXD Client。
接著的下一步，要初始化 LXD 服務。初始化是一個互動式問答，問題大概有
- 是否要設置 LXD 叢集。不過這叢集並非 Docker Swarm/K8S 那一種容器高可用，而是 LXD 服務本身
- 採用的檔案系統，這邊採用最陽春的目錄模式，比較能支援各種怪怪需求，例如直接複製東西進去容器裡面～
- 網路的設置，可以選內網或對 Host 的外部發布，這邊選取主機內部網路，就是 NAT 模式
elementary@elementary:~$ sudo lxd init
Would you like to use LXD clustering? (yes/no) [default=no]: no
Do you want to configure a new storage pool? (yes/no) [default=yes]: yes
Name of the new storage pool [default=default]: ↲
Name of the storage backend to use (btrfs, dir, lvm) [default=btrfs]: dir
Would you like to connect to a MAAS server? (yes/no) [default=no]: no
Would you like to create a new local network bridge? (yes/no) [default=yes]: yes
What should the new bridge be called? [default=lxdbr0]: ↲
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: ↲
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: ↲
Would you like LXD to be available over the network? (yes/no) [default=no]: no
Would you like stale cached images to be updated automatically? (yes/no) [default=yes] ↲
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: yes
config: {}
networks:
- config:
ipv4.address: auto
ipv6.address: auto
description: ""
managed: false
name: lxdbr0
type: ""
storage_pools:
- config: {}
description: ""
name: default
driver: dir
profiles:
- config: {}
description: ""
devices:
eth0:
name: eth0
nictype: bridged
parent: lxdbr0
type: nic
root:
path: /
pool: default
type: disk
name: default
cluster: null
elementary@elementary:~$
初始化完畢，就可以使用了。
為了方便以後的使用，可以把目前使用的帳號加入 lxd 群組，在重新登入後，就可以不需要 sudo 指令，操作 LXD：
elementary@elementary:~$ sudo usermod --append --groups lxd $USER
LXD 執行的風格，跟 docker 相似，以 sub-command 進行，比較不像原生 LXC 的指令設計方式。
以下為列出 LXD Image，主要跟先前 LXC download Template 一樣，到官網的 Image Server 去下載。
LXD 的設計，有點在模仿 Docker，也使用所謂的 Image 來初始化新的 Container Instance。不過因為 LXD 是所謂的 System Container，所以不會有特製化的 AP Image。
elementary@elementary:~$ lxc image list images:centos
+---------------------------+--------------+--------+-----------------------------------+---------+----------+------------------------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCH | SIZE | UPLOAD DATE |
+---------------------------+--------------+--------+-----------------------------------+---------+----------+------------------------------+
| centos/6 (3 more) | 8e4717a57a9b | yes | Centos 6 amd64 (20190608_07:08) | x86_64 | 109.74MB | Jun 8, 2019 at 12:00am (UTC) |
+---------------------------+--------------+--------+-----------------------------------+---------+----------+------------------------------+
| centos/6/i386 (1 more) | e793cef1bde7 | yes | Centos 6 i386 (20190608_07:08) | i686 | 109.98MB | Jun 8, 2019 at 12:00am (UTC) |
+---------------------------+--------------+--------+-----------------------------------+---------+----------+------------------------------+
| centos/7 (3 more) | 1cff685e5fd8 | yes | Centos 7 amd64 (20190608_07:08) | x86_64 | 124.91MB | Jun 8, 2019 at 12:00am (UTC) |
+---------------------------+--------------+--------+-----------------------------------+---------+----------+------------------------------+
| centos/7/arm64 (1 more) | e37e392c098e | yes | Centos 7 arm64 (20190608_07:11) | aarch64 | 124.90MB | Jun 8, 2019 at 12:00am (UTC) |
+---------------------------+--------------+--------+-----------------------------------+---------+----------+------------------------------+
| centos/7/armhf (1 more) | 01140b7639a2 | yes | Centos 7 armhf (20190608_07:08) | armv7l | 122.74MB | Jun 8, 2019 at 12:00am (UTC) |
+---------------------------+--------------+--------+-----------------------------------+---------+----------+------------------------------+
| centos/7/i386 (1 more) | e054c6587969 | yes | Centos 7 i386 (20190608_07:08) | i686 | 125.34MB | Jun 8, 2019 at 12:00am (UTC) |
+---------------------------+--------------+--------+-----------------------------------+---------+----------+------------------------------+
| centos/7/ppc64el (1 more) | b829bad4a16a | yes | Centos 7 ppc64el (20190608_07:08) | ppc64le | 126.73MB | Jun 8, 2019 at 12:00am (UTC) |
+---------------------------+--------------+--------+-----------------------------------+---------+----------+------------------------------+
elementary@elementary:~$
這邊，要初始化一個 Container。如果本地沒有 image 的話，就會從 image Server 下載，然後初始化。
elementary@elementary:~$ lxc launch images:centos/7/amd64 epas1p Creating epas1p Starting epas1p elementary@elementary:~$
LXD 也支援裡面裝 Docker，這需要在建立 Container 時，指定一個參數（也可以在建立後補上設定）。但由於一些細節問題，啟動後仍然半殘。。。這邊就留著這段紀錄
要注意要使用 Nested Container 功能，選用的 File System 要注意一下，這邊選目錄，在這方面不太有問題。
elementary@elementary:~$ lxc launch images:centos/7/amd64 dockerce1 -c security.nesting=true Creating dockerce1 Starting dockerce1 elementary@elementary:~$
列出 Container 清單
elementary@elementary:~$ lxc list +-----------+---------+----------------------+-----------------------------------------------+------------+-----------+ | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | +-----------+---------+----------------------+-----------------------------------------------+------------+-----------+ | dockerce1 | RUNNING | 10.111.132.3 (eth0) | fd42:4114:febd:dec2:216:3eff:feca:30f2 (eth0) | PERSISTENT | 0 | +-----------+---------+----------------------+-----------------------------------------------+------------+-----------+ | epas1p | RUNNING | 10.111.132.21 (eth0) | fd42:4114:febd:dec2:216:3eff:fe67:f680 (eth0) | PERSISTENT | 0 | +-----------+---------+----------------------+-----------------------------------------------+------------+-----------+ elementary@elementary:~$
接著就是登入 Container 裡面。LXD 登入容器的方式跟 Docker 的 docker exec 指令比較像，直接 attach 到容器裡面執行指令，跟 LXC 直接模擬 TTY console 不同。這邊有用到一個 -- 的指令來區分外部的 LXD 參數跟 Container 裡面的指令，這一點反而跟原生 LXC 很像。
以下就啟動一個 root 帳號的 bash shell，進入 Container 裡面，並確認一些網路的測通
elementary@elementary:~$ lxc shell epas1p [root@epas1p ~]# ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=54 time=25.3 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=54 time=24.9 ms ^C --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 24.993/25.194/25.395/0.201 ms [root@epas1p ~]#
要複製東西進去的話，除了直接複製之外，也提供了指令
elementary@elementary:~$ lxc file push ~/下載/edb_11_redhat_rhel-7-x86_64.tar.gz epas1p/root/ Pushing /home/elementary/下載/edb_11_redhat_rhel-7-x86_64.tar.gz to /root/edb_11_redhat_rhel-7-x86_64.tar.gz: 100% (35.01MB/s) elementary@elementary:~$
基本操作就紀錄到這邊。
雖然，測試過程中，也有試著確認 Docker 與 Docker Swarm 的功能。不過測試後發現，目前 Docker 18.09 的 Docker Swarm 跟 LXD 有網路界面的整合問題。主要是 Docker Swarm 要產生 Service （docker service create）時，會試圖自行載入 Kernel Driver：在 Container 裡面，當然就會找不到 Kernel 檔案。。。因此這邊就不特地紀錄這部份的測試。
待以後有機會，兩個軟體比較友好的話，再來試試看。
最後小感想：
1) 雖然 LXD 比直接操作 LXC 更加更加的方便～但也多靠 LXC，才順便趁這個機會了解了防火牆、IP 轉發等等的操作，也是不錯的體驗。
2) LXD 跟 Docker/rkt/podman 等 OCI 標準的 runc-based Container 比較起來，儘管是基於上一代的軟體 LXC，捲土重來的新平台。但依照使用模式，應該算是不同使用情境的軟體。只是在商業上，還是有主流/不夠主流的差別～
3) 雖然 LXC/LXD 不是現在流行的 AP-based Container，而是類似獨立 VM 的 System-based Container，但對我現在的軟體演練需求夠用就好了～
4) LXC/LXD 現在還是活躍的開源專案～一直有再改善、維護，也有相當比例的人在使用的（例如，搭配 Proxmox VE）
參考資料：
LXD 教學
Introduction to Ubuntu's LXD Containers - Make Tech Easier
Learn and use LXD system containers — especially for development and testing | by Cristian Posoiu | Medium
Learn and use LXD system containers — especially for development and testing | by Cristian Posoiu | Medium
(在 Fedora 安裝 LXD：畢竟 LXD 背後是 Ubuntu 當靠山，Red Hat 系列要簡單安裝就得遵守別人規則)
在套件集更舊的 RHEL/CentOS 7 Linux 裝 LXD：LXD on Centos 7 - LXD - Linux Containers Forum
設定 CentOS container 的 HTTPd 服務
把 LXC 搬到 LXD?
以下是 LXD 裡面跑 Docker 的議題相關文章（截至 2019/8）。目前結論是，只能等等看有沒有機會改善 docker 跟 docker swarm 了
沒有留言:
張貼留言