另外，Greenplum 6.0 之後，正式根基於 PostgreSQL 9.4 系列，與之前版本都只有 PostgreSQL 8.x 相比，差異很多了～因此值得去了解一下。
近期釋出的是 6.2.1 版，由 Pivotal 公司發行的企業版 Pivotal Greenplum 與開源的 Greenplum 功能差異可以參考 Release Note：主要是一些進階功能（如，GPCC、GPTEXT）不提供。
這邊用 LXC 啟動一個 GPDB 的 1 Master + 2 Segment 叢集。用來簡單安裝，並觀察一下下 Streaming Replication 的狀態。
以下的紀錄為簡單安裝：正式環境的設置，還需要作一些 OS 層面的設定，以承受較高壓力的運作需求。秉持著 PostgreSQL 只需要少少資源就能喘的想法，相信 Greenplum 也可以在很低銷的程度上喘著～所以這邊不做複雜的設定紀錄～
首先先長出環境
user@lab:~$ lxc launch images:centos/7/amd64 gpdb-mdw user@lab:~$ lxc launch images:centos/7/amd64 gpdb-sdw1 user@lab:~$ lxc launch images:centos/7/amd64 gpdb-sdw2
user@lab:~$ lxc list +-----------+---------+-----------------------+-----------------------------------------------+------------+-----------+ | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | +-----------+---------+-----------------------+-----------------------------------------------+------------+-----------+ | gpdb-mdw | RUNNING | 10.236.247.98 (eth0) | fd42:b925:969f:cf73:216:3eff:fee1:b1b4 (eth0) | PERSISTENT | 0 | +-----------+---------+-----------------------+-----------------------------------------------+------------+-----------+ | gpdb-sdw1 | RUNNING | 10.236.247.234 (eth0) | fd42:b925:969f:cf73:216:3eff:feca:9143 (eth0) | PERSISTENT | 0 | +-----------+---------+-----------------------+-----------------------------------------------+------------+-----------+ | gpdb-sdw2 | RUNNING | 10.236.247.15 (eth0) | fd42:b925:969f:cf73:216:3eff:fe6a:5a5d (eth0) | PERSISTENT | 0 | +-----------+---------+-----------------------+-----------------------------------------------+------------+-----------+ user@lab:~$
安裝 OpenSSH-Server，然後補正 LXC 裡面的 ssh 整合 PAM 的小問題
user@lab:~$ lxc exec gpdb-mdw -- yum install -y openssh-server user@lab:~$ lxc exec gpdb-sdw1 -- yum install -y openssh-server user@lab:~$ lxc exec gpdb-sdw2 -- yum install -y openssh-server user@lab:~$ lxc exec gpdb-mdw -- sed -e "s/^UsePAM yes/UsePAM no/g" -i /etc/ssh/sshd_config user@lab:~$ lxc exec gpdb-sdw1 -- sed -e "s/^UsePAM yes/UsePAM no/g" -i /etc/ssh/sshd_config user@lab:~$ lxc exec gpdb-sdw2 -- sed -e "s/^UsePAM yes/UsePAM no/g" -i /etc/ssh/sshd_config user@lab:~$ lxc exec gpdb-mdw -- chkconfig sshd on user@lab:~$ lxc exec gpdb-sdw1 -- chkconfig sshd on user@lab:~$ lxc exec gpdb-sdw2 -- chkconfig sshd on user@lab:~$ lxc exec gpdb-mdw -- service sshd start user@lab:~$ lxc exec gpdb-sdw1 -- service sshd start user@lab:~$ lxc exec gpdb-sdw2 -- service sshd start
下載 GPDB 執行檔的 RPM，推送到全部的 Container 裡面，並進行安裝
user@lab:~$ wget https://github.com/greenplum-db/gpdb/releases/download/6.2.1/greenplum-db-6.2.1-rhel7-x86_64.rpm user@lab:~$ user@lab:~$ lxc file push ./greenplum-db-6.2.1-rhel7-x86_64.rpm gpdb-mdw/root/ user@lab:~$ lxc file push ./greenplum-db-6.2.1-rhel7-x86_64.rpm gpdb-sdw1/root/ user@lab:~$ lxc file push ./greenplum-db-6.2.1-rhel7-x86_64.rpm gpdb-sdw2/root/ user@lab:~$ user@lab:~$ lxc exec gpdb-mdw -- yum install -y ~/greenplum-db-6.2.1-rhel7-x86_64.rpm user@lab:~$ lxc exec gpdb-sdw1 -- yum install -y ~/greenplum-db-6.2.1-rhel7-x86_64.rpm user@lab:~$ lxc exec gpdb-sdw2 -- yum install -y ~/greenplum-db-6.2.1-rhel7-x86_64.rpm user@lab:~$
增加 /etc/hosts 的內容
user@lab:~$ lxc exec gpdb-mdw -- bash <<'EOF' cat >> /etc/hosts 10.236.247.98 gpdb-mdw 10.236.247.234 gpdb-sdw1 10.236.247.15 gpdb-sdw2 EOF user@lab:~$ user@lab:~$ lxc exec gpdb-sdw1 -- bash <<'EOF' cat >> /etc/hosts 10.236.247.98 gpdb-mdw 10.236.247.234 gpdb-sdw1 10.236.247.15 gpdb-sdw2 EOF user@lab:~$ user@lab:~$ lxc exec gpdb-sdw2 -- bash <<'EOF' cat >> /etc/hosts 10.236.247.98 gpdb-mdw 10.236.247.234 gpdb-sdw1 10.236.247.15 gpdb-sdw2 EOF user@lab:~$
建立 gpadmin 帳號：這是 Greenplum 的慣用帳號名稱。。。
user@lab:~$ lxc exec gpdb-mdw -- groupadd gpadmin user@lab:~$ lxc exec gpdb-sdw1 -- groupadd gpadmin user@lab:~$ lxc exec gpdb-sdw2 -- groupadd gpadmin user@lab:~$ lxc exec gpdb-mdw -- useradd gpadmin -r -m -g gpadmin user@lab:~$ lxc exec gpdb-sdw1 -- useradd gpadmin -r -m -g gpadmin user@lab:~$ lxc exec gpdb-sdw2 -- useradd gpadmin -r -m -g gpadmin user@lab:~$ lxc exec gpdb-mdw -- passwd gpadmin user@lab:~$ lxc exec gpdb-sdw1 -- passwd gpadmin user@lab:~$ lxc exec gpdb-sdw2 -- passwd gpadmin
在 gpadmin 帳號增加 ssh 金鑰，以及環境變數（有一兩個變數是預先加進來的～）
user@lab:~$ lxc exec gpdb-mdw -- su - gpadmin -c 'ssh-keygen' user@lab:~$ lxc exec gpdb-sdw1 -- su - gpadmin -c 'ssh-keygen' user@lab:~$ lxc exec gpdb-sdw2 -- su - gpadmin -c 'ssh-keygen' user@lab:~$ lxc exec gpdb-mdw -- su - gpadmin -c 'echo "source /usr/local/greenplum-db/greenplum_path.sh" >> ~/.bash_profile' user@lab:~$ lxc exec gpdb-sdw1 -- su - gpadmin -c 'echo "source /usr/local/greenplum-db/greenplum_path.sh" >> ~/.bash_profile' user@lab:~$ lxc exec gpdb-sdw2 -- su - gpadmin -c 'echo "source /usr/local/greenplum-db/greenplum_path.sh" >> ~/.bash_profile' user@lab:~$ lxc exec gpdb-mdw -- su - gpadmin -c 'echo "source /usr/local/greenplum-db/greenplum_path.sh" >> ~/.bashrc' user@lab:~$ lxc exec gpdb-sdw1 -- su - gpadmin -c 'echo "source /usr/local/greenplum-db/greenplum_path.sh" >> ~/.bashrc' user@lab:~$ lxc exec gpdb-sdw2 -- su - gpadmin -c 'echo "source /usr/local/greenplum-db/greenplum_path.sh" >> ~/.bashrc' user@lab:~$ lxc exec gpdb-mdw -- bash <<'EOF' cat >> /home/gpadmin/.bashrc export MASTER_DATA_DIRECTORY=/data/master/gpseg-1/ export PGPORT=5432 export PGUSER=gpadmin export PGDATABASE=postgres EOF user@lab:~$ lxc exec gpdb-mdw -- bash <<'EOF' cat >> /home/gpadmin/.bash_profile export MASTER_DATA_DIRECTORY=/data/master/gpseg-1/ export PGPORT=5432 export PGUSER=gpadmin export PGDATABASE=postgres EOF user@lab:~$
在 gpdb-mdw 增加 GPDB 佈署時，需要的 Host Info 檔案
user@lab:~$ lxc exec gpdb-mdw -- bash <<'EOF' cat >> /home/gpadmin/hostfile_exkeys gpdb-mdw gpdb-sdw1 gpdb-sdw2 EOF user@lab:~$ lxc exec gpdb-mdw -- chown gpadmin. /home/gpadmin/hostfile_exkeys
先從 gpdb-mdw 的 gpadmin 帳號，使用 ssh-copy-id 單向打通到其他節點
user@lab:~$ lxc exec gpdb-mdw -- ssh-copy-id gpdb-smdw user@lab:~$ lxc exec gpdb-mdw -- ssh-copy-id gpdb-sdw1 user@lab:~$ lxc exec gpdb-mdw -- ssh-copy-id gpdb-sdw2
透過 Greenplum 提供的輔助指令 gpssh-exkeys 使 SSH 互相打通（網絡拓樸上，形成完全圖）
user@lab:~$ lxc exec gpdb-mdw -- su - gpadmin -c 'gpssh-exkeys -f ~/hostfile_exkeys' [STEP 1 of 5] create local ID and authorize on local host ... /home/gpadmin/.ssh/id_rsa file exists ... key generation skipped [STEP 2 of 5] keyscan all hosts and update known_hosts file [STEP 3 of 5] retrieving credentials from remote hosts ... send to gpdb-smdw ... send to gpdb-sdw1 ... send to gpdb-sdw2 [STEP 4 of 5] determine common authentication file content [STEP 5 of 5] copy authentication files to all remote hosts ... finished key exchange with gpdb-smdw ... finished key exchange with gpdb-sdw1 ... finished key exchange with gpdb-sdw2 [INFO] completed successfully user@lab:~$
驗證 gpadmin 帳號的 gpssh 互通
user@lab:~$ lxc exec gpdb-mdw -- su - gpadmin -c 'gpssh -f ~/hostfile_exkeys -e ls -l' [gpdb-sdw2] ls -l [gpdb-sdw2] total 0 [gpdb-smdw] ls -l [gpdb-smdw] total 0 [gpdb-sdw1] ls -l [gpdb-sdw1] total 0 [ gpdb-mdw] ls -l [ gpdb-mdw] total 4 [ gpdb-mdw] -rw-rw-r-- 1 gpadmin gpadmin 39 Nov 1 13:53 hostfile_exkeys user@lab:~$
準備資料庫目錄：這步驟基本上是用 gpssh 處理的，但我不習慣用。。
user@lab:~$ lxc exec gpdb-mdw -- mkdir -p /data/master user@lab:~$ lxc exec gpdb-mdw -- chown -R gpadmin. /data/master user@lab:~$ lxc exec gpdb-sdw1 -- mkdir -p /data/primary user@lab:~$ lxc exec gpdb-sdw1 -- mkdir -p /data/mirror user@lab:~$ lxc exec gpdb-sdw1 -- chown -R gpadmin. /data user@lab:~$ lxc exec gpdb-sdw2 -- mkdir -p /data/primary user@lab:~$ lxc exec gpdb-sdw2 -- mkdir -p /data/mirror user@lab:~$ lxc exec gpdb-sdw2 -- chown -R gpadmin. /data
準備 GPDB 資料庫的初始化：這邊開始就進去 gpdb-mdw 環境裡面操作就好了～
初始化 GPDB 要準備兩個 Config，分別是 hostfile_gpinitsystem 以及 gpinitsystem_config。其中 gpinitsystem_config 可以在 $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config 找到範本。
這邊把 $PGDATA 目錄都放在 /data 底下，因此複製 gpinitsystem_config 後需要確認調整一下
在 Config 中預設在一個 Segment 電腦中放三個 Primary Segment Instance，且沒有先啟用 Mirror Segment Instance。這邊做了調整，改成一個 Segment OS 只放一個 Primary Instance 以及一個 Mirror Instance，這樣對測試環境的壓力也比較小。因此會長的像是：
┏ mdw : 1 Master Instance
┣ sdw1 : 1 Primary Instance, 1 Mirror Instance
┗ sdw2 : 1 Primary Instance, 1 Mirror Instance
編輯設定檔
這邊有一點要稍微注意一下：在啟用 X-Window 桌面的作業系統上，Port 6000 會被 X-Window 佔用，會造成初始化失敗，需要注意避免（調整 PORT_BASE=6000）。當然，這邊用 LXC 不會遇到這種問題～
user@lab:~$ lxc shell gpdb-mdw [root@gpdb-mdw ~]$ su - gpadmin [gpadmin@gpdb-mdw ~]$ echo $GPHOME /usr/local/greenplum-db/. [gpadmin@gpdb-mdw ~]$ cat << EOF >> ~/hostfile_gpinitsystem gpdb-sdw1 gpdb-sdw2 EOF [gpadmin@gpdb-mdw ~]$ [gpadmin@gpdb-mdw ~]$ cp $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config ~ [gpadmin@gpdb-mdw ~]$ vi ~/gpinitsystem_config [gpadmin@gpdb-mdw ~]$ diff -Naur $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config ~/gpinitsystem_config --- /usr/local/greenplum-db/./docs/cli_help/gpconfigs/gpinitsystem_config 2019-10-11 19:00:01.000000000 +0000 +++ /home/gpadmin/gpinitsystem_config 2019-11-01 14:42:58.428632785 +0000 @@ -22,10 +22,10 @@ #### physical host (if multiple addresses for a host are listed in #### the hostfile, the number of segments will be spread evenly across #### the specified interface addresses). -declare -a DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary) +declare -a DATA_DIRECTORY=(/data/primary) #### OS-configured hostname or IP address of the master host. -MASTER_HOSTNAME=mdw +MASTER_HOSTNAME=gpdb-mdw #### File system location where the master data directory #### will be created. @@ -49,13 +49,13 @@ #### Base number by which mirror segment port numbers #### are calculated. -#MIRROR_PORT_BASE=7000 +MIRROR_PORT_BASE=7000 #### File system location(s) where mirror segment data directories #### will be created. The number of mirror locations must equal the #### number of primary locations as specified in the #### DATA_DIRECTORY parameter. -#declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror) +declare -a MIRROR_DATA_DIRECTORY=(/data/mirror) ################################################ [gpadmin@gpdb-mdw ~]$ [gpadmin@gpdb-mdw ~]$ ls gpinitsystem_config hostfile_exkeys hostfile_gpinitsystem [gpadmin@gpdb-mdw ~]$
執行初始化。Standby Master 節點需要用 -s 參數指定，這邊略過～
另外，在普通環境中，可能有啟用防火牆服務（firewallD），建議將叢集內溝通的網路界面加到 trusted Zone 上，或是將防火牆停用。這邊的 LXC/LXD 環境裡面沒有防火牆，不用擔心～
[gpadmin@gpdb-mdw ~]$ gpinitsystem -c ~/gpinitsystem_config -h ~/hostfile_gpinitsystem
20191101:14:45:24:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Checking configuration parameters, please wait...
20191101:14:45:25:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Reading Greenplum configuration file /home/gpadmin/gpinitsystem_config
20191101:14:45:25:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Locale has not been set in /home/gpadmin/gpinitsystem_config, will set to default value
20191101:14:45:25:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Locale set to en_US.utf8
20191101:14:45:25:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-No DATABASE_NAME set, will exit following template1 updates
20191101:14:45:25:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-MASTER_MAX_CONNECT not set, will set to default value 250
20191101:14:45:25:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Checking configuration parameters, Completed
20191101:14:45:25:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Commencing multi-home checks, please wait...
..
20191101:14:45:26:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Configuring build for standard array
20191101:14:45:26:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Commencing multi-home checks, Completed
20191101:14:45:26:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Building primary segment instance array, please wait...
..
20191101:14:45:27:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Building group mirror array type , please wait...
..
20191101:14:45:28:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Checking Master host
20191101:14:45:28:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Checking new segment hosts, please wait...
....
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Checking new segment hosts, Completed
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Greenplum Database Creation Parameters
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:---------------------------------------
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master Configuration
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:---------------------------------------
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master instance name = Greenplum Data Platform
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master hostname = gpdb-mdw
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master port = 5432
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master instance dir = /data/master/gpseg-1
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master LOCALE = en_US.utf8
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Greenplum segment prefix = gpseg
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master Database =
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master connections = 250
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master buffers = 128000kB
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Segment connections = 750
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Segment buffers = 128000kB
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Checkpoint segments = 8
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Encoding = UNICODE
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Postgres param file = Off
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Initdb to be used = /usr/local/greenplum-db/./bin/initdb
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-GP_LIBRARY_PATH is = /usr/local/greenplum-db/./lib
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-HEAP_CHECKSUM is = on
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-HBA_HOSTNAMES is = 0
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Ulimit check = Passed
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Array host connect type = Single hostname per node
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master IP address [1] = 10.207.69.186
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master IP address [2] = ::1
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master IP address [3] = fd42:ebd6:90e3:9a71:216:3eff:fe16:d7bd
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Master IP address [4] = fe80::216:3eff:fe16:d7bd
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Standby Master = gpdb-smdw
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Number of primary segments = 1
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Standby IP address = 10.207.69.183
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Standby IP address = ::1
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Standby IP address = fd42:ebd6:90e3:9a71:216:3eff:fec2:7186
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Standby IP address = fe80::216:3eff:fec2:7186
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Total Database segments = 2
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Trusted shell = ssh
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Number segment hosts = 2
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Mirror port base = 7000
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Number of mirror segments = 1
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Mirroring config = ON
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Mirroring type = Group
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:----------------------------------------
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Greenplum Primary Segment Configuration
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:----------------------------------------
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-gpdb-sdw1 /data/primary/gpseg0 6000 2 0
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-gpdb-sdw2 /data/primary/gpseg1 6000 3 1
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:---------------------------------------
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Greenplum Mirror Segment Configuration
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:---------------------------------------
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-gpdb-sdw2 /data/mirror/gpseg0 7000 4 0
20191101:14:45:34:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-gpdb-sdw1 /data/mirror/gpseg1 7000 5 1
Continue with Greenplum creation Yy|Nn (default=N):
> Y
20191101:14:48:21:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Building the Master instance database, please wait...
20191101:14:48:32:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Starting the Master in admin mode
20191101:14:48:36:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Commencing parallel build of primary segment instances
20191101:14:48:36:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Spawning parallel processes batch [1], please wait...
..
20191101:14:48:36:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Waiting for parallel processes batch [1], please wait...
.........................
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:------------------------------------------------
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Parallel process exit status
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:------------------------------------------------
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Total processes marked as completed = 2
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Total processes marked as killed = 0
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Total processes marked as failed = 0
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:------------------------------------------------
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Deleting distributed backout files
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Removing back out file
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-No errors generated from parallel processes
20191101:14:49:02:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Restarting the Greenplum instance in production mode
20191101:14:49:05:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Starting gpstop with args: -a -l /home/gpadmin/gpAdminLogs -m -d /data/master/gpseg-1
20191101:14:49:05:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Gathering information and validating the environment...
20191101:14:49:06:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20191101:14:49:06:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20191101:14:49:06:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.0.1 build commit:053a66ae19cd7301ec8c8910ed85ec2c20ad60cc'
20191101:14:49:06:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Commencing Master instance shutdown with mode='smart'
20191101:14:49:06:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Master segment instance directory=/data/master/gpseg-1
20191101:14:49:06:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Stopping master segment and waiting for user connections to finish ...
server shutting down
20191101:14:49:07:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process
20191101:14:49:07:003939 gpstop:gpdb-mdw:gpadmin-[INFO]:-Terminating processes for segment /data/master/gpseg-1
20191101:14:49:08:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Starting gpstart with args: -a -l /home/gpadmin/gpAdminLogs -d /data/master/gpseg-1
20191101:14:49:08:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Gathering information and validating the environment...
20191101:14:49:08:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.0.1 build commit:053a66ae19cd7301ec8c8910ed85ec2c20ad60cc'
20191101:14:49:08:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232'
20191101:14:49:08:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Starting Master instance in admin mode
20191101:14:49:09:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20191101:14:49:09:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20191101:14:49:09:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Setting new master era
20191101:14:49:09:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Master Started...
20191101:14:49:09:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Shutting down master
20191101:14:49:10:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait...
....
20191101:14:49:15:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Process results...
20191101:14:49:15:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20191101:14:49:15:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:- Successful segment starts = 2
20191101:14:49:15:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:- Failed segment starts = 0
20191101:14:49:15:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:- Skipped segment starts (segments are marked down in configuration) = 0
20191101:14:49:15:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20191101:14:49:15:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Successfully started 2 of 2 segment instances
20191101:14:49:15:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20191101:14:49:15:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Starting Master instance gpdb-mdw directory /data/master/gpseg-1
20191101:14:49:16:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Command pg_ctl reports Master gpdb-mdw instance active
20191101:14:49:16:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Connecting to dbname='template1' connect_timeout=15
20191101:14:49:17:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-No standby master configured. skipping...
20191101:14:49:17:003963 gpstart:gpdb-mdw:gpadmin-[INFO]:-Database successfully started
20191101:14:49:17:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Completed restart of Greenplum instance in production mode
20191101:14:49:17:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Commencing parallel build of mirror segment instances
20191101:14:49:17:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Spawning parallel processes batch [1], please wait...
..
20191101:14:49:17:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Waiting for parallel processes batch [1], please wait...
.......................
20191101:14:49:41:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:------------------------------------------------
20191101:14:49:42:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Parallel process exit status
20191101:14:49:42:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:------------------------------------------------
20191101:14:49:42:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Total processes marked as completed = 2
20191101:14:49:42:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Total processes marked as killed = 0
20191101:14:49:42:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Total processes marked as failed = 0
20191101:14:49:42:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:------------------------------------------------
20191101:14:49:45:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Starting initialization of standby master gpdb-smdw
20191101:14:49:50:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Validating environment and parameters for standby initialization...
20191101:14:49:56:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Checking for data directory /data/master/gpseg-1 on gpdb-smdw
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:------------------------------------------------------
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Greenplum standby master initialization parameters
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:------------------------------------------------------
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Greenplum master hostname = gpdb-mdw
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Greenplum master data directory = /data/master/gpseg-1
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Greenplum master port = 5432
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Greenplum standby master hostname = gpdb-smdw
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Greenplum standby master port = 5432
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Greenplum standby master data directory = /data/master/gpseg-1
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Greenplum update system catalog = On
20191101:14:49:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Syncing Greenplum Database extensions to standby
20191101:14:49:58:004671 gpinitstandby:gpdb-mdw:gpadmin-[WARNING]:-Syncing of Greenplum Database extensions has failed.
20191101:14:49:58:004671 gpinitstandby:gpdb-mdw:gpadmin-[WARNING]:-Please run gppkg --clean after successful standby initialization.
20191101:14:49:59:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Adding standby master to catalog...
20191101:14:49:59:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Database catalog updated successfully.
20191101:14:49:59:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Updating pg_hba.conf file...
20191101:14:50:11:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-pg_hba.conf files updated successfully.
20191101:14:50:30:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Starting standby master
20191101:14:50:30:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Checking if standby master is running on host: gpdb-smdw in directory: /data/master/gpseg-1
20191101:14:52:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[WARNING]:-Could not start standby master
20191101:14:52:57:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Cleaning up pg_hba.conf backup files...
20191101:14:53:15:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Backup files of pg_hba.conf cleaned up successfully.
20191101:14:53:15:004671 gpinitstandby:gpdb-mdw:gpadmin-[INFO]:-Successfully created standby master on gpdb-smdw
20191101:14:53:15:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Successfully completed standby master initialization
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Scanning utility log file for any warning messages
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[WARN]:-*******************************************************
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[WARN]:-Scan of log file indicates that some warnings or errors
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[WARN]:-were generated during the array creation
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Please review contents of log file
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-/home/gpadmin/gpAdminLogs/gpinitsystem_20191101.log
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-To determine level of criticality
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[WARN]:-*******************************************************
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Greenplum Database instance successfully created
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-------------------------------------------------------
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-To complete the environment configuration, please
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-update gpadmin .bashrc file with the following
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-1. Ensure that the greenplum_path.sh file is sourced
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-2. Add "export MASTER_DATA_DIRECTORY=/data/master/gpseg-1"
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:- to access the Greenplum scripts for this instance:
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:- or, use -d /data/master/gpseg-1 option for the Greenplum scripts
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:- Example gpstate -d /data/master/gpseg-1
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Script log file = /home/gpadmin/gpAdminLogs/gpinitsystem_20191101.log
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-To remove instance, run gpdeletesystem utility
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Standby Master gpdb-smdw has been configured
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-To activate the Standby Master Segment in the event of Master
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-failure review options for gpactivatestandby
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-------------------------------------------------------
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-The Master /data/master/gpseg-1/pg_hba.conf post gpinitsystem
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-has been configured to allow all hosts within this new
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-array to intercommunicate. Any hosts external to this
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-new array must be explicitly added to this file
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-Refer to the Greenplum Admin support guide which is
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-located in the /usr/local/greenplum-db/./docs directory
20191101:14:53:16:001355 gpinitsystem:gpdb-mdw:gpadmin-[INFO]:-------------------------------------------------------
[gpadmin@gpdb-mdw ~]$
這樣就長出來了～
查看一下狀態無誤
[gpadmin@gpdb-mdw ~]$ gpstate 20191231:15:07:48:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:-Starting gpstate with args: 20191231:15:07:48:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4' 20191231:15:07:48:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 16 2019 03:07:22' 20191231:15:07:48:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:-Obtaining Segment details from master... 20191231:15:07:48:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:-Gathering data from segments... . 20191231:15:07:49:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:-Greenplum instance status summary 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:----------------------------------------------------- 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Master instance = Active 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Master standby = No master standby configured 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total segment instance count from metadata = 4 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:----------------------------------------------------- 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Primary Segment Status 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:----------------------------------------------------- 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total primary segments = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total primary segment valid (at master) = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total primary segment failures (at master) = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of postmaster.pid files found = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of /tmp lock files found = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number postmaster processes missing = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number postmaster processes found = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:----------------------------------------------------- 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Mirror Segment Status 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:----------------------------------------------------- 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total mirror segments = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total mirror segment valid (at master) = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total mirror segment failures (at master) = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of postmaster.pid files found = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number of /tmp lock files found = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number postmaster processes missing = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number postmaster processes found = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number mirror segments acting as primary segments = 0 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:- Total number mirror segments acting as mirror segments = 2 20191231:15:07:50:000557 gpstate:gpdb-mdw:gpadmin-[INFO]:----------------------------------------------------- [gpadmin@gpdb-mdw ~]$ [gpadmin@gpdb-mdw ~]$ psql -x -c "select * from gp_segment_configuration;" -[ RECORD 1 ]--+--------------------- dbid | 1 content | -1 role | p preferred_role | p mode | n status | u port | 5432 hostname | gpdb-mdw address | gpdb-mdw datadir | /data/master/gpseg-1 -[ RECORD 2 ]--+--------------------- dbid | 2 content | 0 role | p preferred_role | p mode | s status | u port | 6000 hostname | gpdb-sdw1 address | gpdb-sdw1 datadir | /data/primary/gpseg0 -[ RECORD 3 ]--+--------------------- dbid | 4 content | 0 role | m preferred_role | m mode | s status | u port | 7000 hostname | gpdb-sdw2 address | gpdb-sdw2 datadir | /data/mirror/gpseg0 -[ RECORD 4 ]--+--------------------- dbid | 3 content | 1 role | p preferred_role | p mode | s status | u port | 6000 hostname | gpdb-sdw2 address | gpdb-sdw2 datadir | /data/primary/gpseg1 -[ RECORD 5 ]--+--------------------- dbid | 5 content | 1 role | m preferred_role | m mode | s status | u port | 7000 hostname | gpdb-sdw1 address | gpdb-sdw1 datadir | /data/mirror/gpseg1 [gpadmin@gpdb-mdw ~]$
在 Master 主機上查看 Process：這邊用的 LXC 環境有一個好處，就是可見範圍的 Process 很少，看起來很乾淨，適合用來觀察服務的運作～
user@lab:~$ lxc shell gpdb-mdw [root@gpdb-mdw ~]# ps auxww USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 43200 5116 ? Ss 06:33 0:00 /sbin/init root 33 0.0 0.0 39096 4092 ? Ss 06:33 0:00 /usr/lib/systemd/systemd-journald root 49 0.0 0.0 41540 3152 ? Ss 06:33 0:00 /usr/lib/systemd/systemd-udevd root 51 0.0 0.0 26392 2912 ? Ss 06:33 0:00 /usr/lib/systemd/systemd-logind dbus 54 0.0 0.0 58136 4452 ? Ss 06:33 0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation root 95 0.0 0.0 22704 2720 ? Ss 06:33 0:00 /usr/sbin/crond -n root 96 0.0 0.0 6528 1740 console Ss+ 06:33 0:00 /sbin/agetty --noclear --keep-baud console 115200,38400,9600 linux root 245 0.0 0.0 102908 4628 ? Ss 06:33 0:00 /sbin/dhclient -1 -q -lf /var/lib/dhclient/dhclient--eth0.lease -pf /var/run/dhclient-eth0.pid -H gpdb-mdw eth0 root 306 0.0 0.0 218560 5004 ? Ssl 06:33 0:00 /usr/sbin/rsyslogd -n root 307 0.0 0.1 112932 7748 ? Ss 06:33 0:00 /usr/sbin/sshd -D gpadmin 407 0.0 3.6 474180 220704 ? Ss 06:36 0:01 /usr/local/greenplum-db-6.2.1/bin/postgres -D /data/master/gpseg-1 -p 5432 -E gpadmin 408 0.0 0.1 243616 9580 ? Ss 06:36 0:00 postgres: 5432, master logger process gpadmin 412 0.0 0.2 474312 12632 ? Ss 06:36 0:00 postgres: 5432, checkpointer process gpadmin 413 0.0 0.1 474180 9280 ? Ss 06:36 0:00 postgres: 5432, writer process gpadmin 414 0.0 0.2 474180 13212 ? Ss 06:36 0:00 postgres: 5432, wal writer process gpadmin 415 0.0 0.1 246008 8916 ? Ss 06:36 0:00 postgres: 5432, stats collector process gpadmin 417 0.0 0.3 541972 23340 ? Ssl 06:36 0:01 postgres: 5432, bgworker: ftsprobe process gpadmin 424 0.0 0.1 319988 6440 ? Ssl 06:36 0:04 /usr/local/greenplum-db-6.2.1/bin/gpmmon -D /data/master/gpseg-1/gpperfmon/conf/gpperfmon.conf -p 5432 gpadmin 425 0.0 0.1 474180 7832 ? Ss 06:36 0:00 postgres: 5432, bgworker: sweeper process gpadmin 426 0.0 0.1 474180 7832 ? Ss 06:36 0:03 postgres: 5432, bgworker: stats sender process gpadmin 487 0.0 0.0 25376 3168 ? Ss 06:36 0:02 /usr/local/greenplum-db/./bin/gpsmon -m 0 -t 150 -l /data/master/gpseg-1/gpperfmon/logs -v 0 8888 root 7427 0.0 0.0 185592 4196 ? Ss 09:31 0:00 su -l root 7428 0.0 0.0 11840 2876 ? S 09:31 0:00 -bash root 7453 0.0 0.0 51764 3576 ? R+ 09:32 0:00 ps auxww [root@gpdb-mdw ~]#
在 Segment 主機上
user@lab:~$ lxc shell gpdb-sdw1 [root@gpdb-sdw1 ~]# ps auxww USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 43200 4992 ? Ss 06:33 0:00 /sbin/init root 32 0.0 0.0 39096 4040 ? Ss 06:33 0:00 /usr/lib/systemd/systemd-journald root 47 0.0 0.0 41540 3176 ? Ss 06:33 0:00 /usr/lib/systemd/systemd-udevd root 52 0.0 0.0 26392 2888 ? Ss 06:33 0:00 /usr/lib/systemd/systemd-logind dbus 54 0.0 0.0 58136 4252 ? Ss 06:33 0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation root 95 0.0 0.0 22708 2748 ? Ss 06:33 0:00 /usr/sbin/crond -n root 96 0.0 0.0 6528 1880 console Ss+ 06:33 0:00 /sbin/agetty --noclear --keep-baud console 115200,38400,9600 linux root 259 0.0 0.0 102908 4680 ? Ss 06:33 0:00 /sbin/dhclient -1 -q -lf /var/lib/dhclient/dhclient--eth0.lease -pf /var/run/dhclient-eth0.pid -H gpdb-sdw1 eth0 root 320 0.0 0.1 112932 7644 ? Ss 06:33 0:00 /usr/sbin/sshd -D root 321 0.0 0.0 284096 5084 ? Ssl 06:33 0:00 /usr/sbin/rsyslogd -n gpadmin 382 0.0 4.0 513480 242496 ? Ss 06:35 0:00 /usr/local/greenplum-db-6.2.1/bin/postgres -D /data/mirror/gpseg1 -p 7000 gpadmin 383 0.0 4.0 513480 242468 ? Ss 06:35 0:01 /usr/local/greenplum-db-6.2.1/bin/postgres -D /data/primary/gpseg0 -p 6000 gpadmin 384 0.0 0.1 243608 8816 ? Ss 06:35 0:00 postgres: 7000, logger process gpadmin 385 0.0 0.2 513708 13752 ? Ss 06:35 0:00 postgres: 7000, startup process recovering 00000001000000000000003B gpadmin 388 0.0 0.1 243608 8892 ? Ss 06:35 0:00 postgres: 6000, logger process gpadmin 411 0.0 0.2 513616 12216 ? Ss 06:35 0:00 postgres: 7000, checkpointer process gpadmin 412 0.0 0.1 513480 9260 ? Ss 06:35 0:00 postgres: 7000, writer process gpadmin 414 0.0 0.2 515100 12852 ? Ss 06:35 0:03 postgres: 7000, wal receiver process streaming 0/EE1B7240 gpadmin 463 0.0 0.2 513632 15080 ? Ss 06:36 0:00 postgres: 6000, checkpointer process gpadmin 464 0.0 0.1 513480 9292 ? Ss 06:36 0:00 postgres: 6000, writer process gpadmin 465 0.0 0.2 513480 13388 ? Ss 06:36 0:00 postgres: 6000, wal writer process gpadmin 466 0.0 0.1 245872 8892 ? Ss 06:36 0:00 postgres: 6000, stats collector process gpadmin 467 0.0 0.1 513672 7748 ? Ss 06:36 0:00 postgres: 6000, bgworker: sweeper process gpadmin 468 0.0 0.1 513672 7748 ? Ss 06:36 0:03 postgres: 6000, bgworker: stats sender process gpadmin 470 0.0 0.2 516744 17652 ? Ss 06:36 0:00 postgres: 6000, wal sender process gpadmin 10.236.247.15(53116) streaming 0/E445E3C0 gpadmin 495 0.0 0.0 25376 3084 ? Ss 06:36 0:01 /usr/local/greenplum-db/./bin/gpsmon -m 0 -t 150 -l /data/primary/gpseg0/gpperfmon -v 0 8888 root 2454 0.0 0.0 185592 3996 ? Ss 09:33 0:00 su -l root 2455 0.0 0.0 11840 2880 ? S 09:33 0:00 -bash root 2468 0.0 0.0 51764 3548 ? R+ 09:33 0:00 ps auxww [root@gpdb-sdw1 ~]#
直接登入 Segment 主機裡面的 Primary DB Instance，查看同步狀態：
這邊需要說明一下，GPDB 一般登入，是直接登入 Master 的 5432 Port，進行的 SQL 大多都會被轉換成全叢集的操作。在執行計畫的結構上，分成 Master 上面的 Query Dispatcher 與個別 Segment 的 Query Executor。
而實際上的每個底下的資料庫服務，預設是不能直接進入的。要進去的話，必須使用本處用的 PGOPTIONS 變數，指定 gp_session_role 參數才行
user@lab:~$ lxc shell gpdb-mdw
[root@gpdb-mdw ~]# su - gpadmin
[gpadmin@gpdb-mdw ~]$ export PGOPTIONS='-c gp_session_role=utility'
[gpadmin@gpdb-mdw ~]$ psql -h gpdb-sdw1 -p 6000
psql (9.4.24)
Type "help" for help.
postgres=# \x
Expanded display is on.
postgres=# select * from pg_stat_replication ;
-[ RECORD 1 ]----+------------------------------
pid | 470
usesysid | 10
usename | gpadmin
application_name | gp_walreceiver
client_addr | 10.236.247.15
client_hostname |
client_port | 53116
backend_start | 2019-12-18 06:36:02.908296+00
backend_xmin |
state | streaming
sent_location | 0/E44510B8
write_location | 0/E44510B8
flush_location | 0/E44510B8
replay_location | 0/E44510B8
sync_priority | 1
sync_state | sync
postgres=# show hot_standby;
hot_standby
-------------
off
(1 row)
postgres=# \q
[gpadmin@gpdb-mdw ~]$ psql -h gpdb-sdw1 -p 6000 -F' ' -c 'IDENTIFY_SYSTEM' 'dbname=replication replication=true'
systemid | timeline | xlogpos | dbname
---------------------+----------+------------+--------
6758086678634251403 | 1 | 0/E731AE18 |
(1 row)
[gpadmin@gpdb-mdw ~]$
[gpadmin@gpdb-mdw ~]$ psql -h gpdb-sdw1 -p 7000
psql: FATAL: the database system is in recovery mode
DETAIL: last replayed record at 0/F1097858
- VERSION: PostgreSQL 9.4.24 (Greenplum Database 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 16 2019 03:07:22
[gpadmin@gpdb-mdw ~]$
這邊看到，Segment 節點的 Primary DB 真的有 Streaming Replication 在進行著～
不過 GPDB CE v6 的 Mirror 並沒有啟動 hot_standby 功能。
進一步嘗試看看 hot_standby 參數的設定值：注意要記得 unset PGOPTIONS 變數才行～
[gpadmin@gpdb-mdw ~]$ gpconfig -s hot_standby
20191231:15:36:16:001721 gpconfig:gpdb-mdw:gpadmin-[ERROR]:-Failed to retrieve GUC information: error 'server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
' in 'select * from gp_toolkit.gp_param_setting('hot_standby')'
'NoneType' object is not iterable
[gpadmin@gpdb-mdw ~]$
[gpadmin@gpdb-mdw ~]$
[gpadmin@gpdb-mdw ~]$ gpstate -e
20191231:15:36:42:001798 gpstate:gpdb-mdw:gpadmin-[INFO]:-Starting gpstate with args: -e
20191231:15:36:42:001798 gpstate:gpdb-mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4'
20191231:15:36:42:001798 gpstate:gpdb-mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 16 2019 03:07:22'
20191231:15:36:42:001798 gpstate:gpdb-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20191231:15:36:42:001798 gpstate:gpdb-mdw:gpadmin-[INFO]:-Gathering data from segments...
...
20191231:15:36:46:001798 gpstate:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20191231:15:36:46:001798 gpstate:gpdb-mdw:gpadmin-[INFO]:-Segment Mirroring Status Report
20191231:15:36:46:001798 gpstate:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20191231:15:36:46:001798 gpstate:gpdb-mdw:gpadmin-[INFO]:-All segments are running normally
[gpadmin@gpdb-mdw ~]$ unset PGOPTIONS
[gpadmin@gpdb-mdw ~]$ gpconfig -s hot_standby
Values on all segments are consistent
GUC : hot_standby
Master value: off
Segment value: off
[gpadmin@gpdb-mdw ~]$
[gpadmin@gpdb-mdw ~]$ gpconfig -s wal_level
Values on all segments are consistent
GUC : wal_level
Master value: archive
Segment value: archive
[gpadmin@gpdb-mdw ~]$
以上發現 hot_standby 停用，而且。。。wal_level 竟然是 archive 而非 hot_standby。。。
看一下 Mirror 的 recovery.conf，可以發現到設定上使用了 Replication Slot
[gpadmin@gpdb-mdw ~]$ gpssh -f ~/hostfile_exkeys -e "cat /data/mirror/gpseg*/recovery.conf" [gpdb-sdw1] cat /data/mirror/gpseg*/recovery.conf [gpdb-sdw1] standby_mode = 'on' [gpdb-sdw1] primary_conninfo = 'user=gpadmin host=gpdb-sdw2 port=6000 sslmode=prefer sslcompression=1 krbsrvname=postgres application_name=gp_walreceiver' [gpdb-sdw1] primary_slot_name = 'internal_wal_replication_slot' [ gpdb-mdw] cat /data/mirror/gpseg*/recovery.conf [ gpdb-mdw] cat: /data/mirror/gpseg*/recovery.conf: No such file or directory [gpdb-sdw2] cat /data/mirror/gpseg*/recovery.conf [gpdb-sdw2] standby_mode = 'on' [gpdb-sdw2] primary_conninfo = 'user=gpadmin host=gpdb-sdw1 port=6000 sslmode=prefer sslcompression=1 krbsrvname=postgres application_name=gp_walreceiver' [gpdb-sdw2] primary_slot_name = 'internal_wal_replication_slot' [gpadmin@gpdb-mdw ~]$
接著仿照一般 Postgres 的設定，嘗試調整 wal_level 與 hot_standby 看看（需重啟資料庫叢集）。
[gpadmin@gpdb-mdw ~]$ gpconfig -c wal_level -v hot_standby
20200102:02:58:14:008891 gpconfig:gpdb-mdw:gpadmin-[INFO]:-completed successfully with parameters '-c wal_level -v hot_standby'
[gpadmin@gpdb-mdw ~]$ gpconfig -c hot_standby -v on
20200102:03:05:28:009257 gpconfig:gpdb-mdw:gpadmin-[INFO]:-completed successfully with parameters '-c hot_standby -v on'
[gpadmin@gpdb-mdw ~]$
[gpadmin@gpdb-mdw ~]$ gpstop -r
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Starting gpstop with args: -r
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Gathering information and validating the environment...
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4'
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Master instance parameters
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Master Greenplum instance process active PID = 8736
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Database = template1
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Master port = 5432
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Master directory = /data/master/gpseg-1/
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Shutdown mode = smart
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Timeout = 120
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Shutdown Master standby host = Off
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Segment instances that will be shutdown:
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Host Datadir Port Status
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- gpdb-sdw1 /data/primary/gpseg0 6000 u
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- gpdb-sdw2 /data/mirror/gpseg0 7000 u
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- gpdb-sdw2 /data/primary/gpseg1 6000 u
20200102:03:08:19:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- gpdb-sdw1 /data/mirror/gpseg1 7000 u
Continue with Greenplum instance shutdown Yy|Nn (default=N):
> Y
20200102:03:08:43:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Commencing Master instance shutdown with mode='smart'
20200102:03:08:43:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Master segment instance directory=/data/master/gpseg-1/
20200102:03:08:43:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Stopping master segment and waiting for user connections to finish ...
server shutting down
20200102:03:08:44:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process
20200102:03:08:44:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Terminating processes for segment /data/master/gpseg-1/
20200102:03:08:44:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-No standby master host configured
20200102:03:08:44:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Targeting dbid [2, 4, 3, 5] for shutdown
20200102:03:08:44:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Commencing parallel primary segment instance shutdown, please wait...
20200102:03:08:44:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-0.00% of jobs completed
20200102:03:08:46:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-100.00% of jobs completed
20200102:03:08:46:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Commencing parallel mirror segment instance shutdown, please wait...
20200102:03:08:46:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-0.00% of jobs completed
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-100.00% of jobs completed
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Segments stopped successfully = 4
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:- Segments with errors during stop = 0
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Successfully shutdown 4 of 4 segment instances
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Database successfully shutdown with no errors reported
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Cleaning up leftover gpmmon process
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-No leftover gpmmon process found
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Cleaning up leftover gpsmon processes
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-No leftover gpsmon processes on some hosts. not attempting forceful termination on these hosts
20200102:03:08:48:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Cleaning up leftover shared memory
20200102:03:08:49:009400 gpstop:gpdb-mdw:gpadmin-[INFO]:-Restarting System...
20200102:03:09:56:009400 gpstop:gpdb-mdw:gpadmin-[CRITICAL]:-Error occurred: non-zero rc: 1
Command was: '$GPHOME/bin/gpstart -a -d /data/master/gpseg-1/'
rc=1, stdout='20200102:03:08:50:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Starting gpstart with args: -a -d /data/master/gpseg-1/
20200102:03:08:50:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Gathering information and validating the environment...
20200102:03:08:50:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4'
20200102:03:08:50:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232'
20200102:03:08:50:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Starting Master instance in admin mode
20200102:03:08:51:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20200102:03:08:51:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20200102:03:08:51:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Setting new master era
20200102:03:08:51:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Master Started...
20200102:03:08:51:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Shutting down master
20200102:03:08:52:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Commencing parallel primary and mirror segment instance startup, please wait...
...
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Process results...
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-DBID:5 FAILED host:'gpdb-sdw1' datadir:'/data/mirror/gpseg1' with reason:'PG_CTL failed.'
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-DBID:4 FAILED host:'gpdb-sdw2' datadir:'/data/mirror/gpseg0' with reason:'PG_CTL failed.'
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:- Successful segment starts = 2
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Failed segment starts = 2 <<<<<<<<
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:- Skipped segment starts (segments are marked down in configuration) = 0
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Successfully started 2 of 4 segment instances <<<<<<<<
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Segment instance startup failures reported
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Failed start 2 of 4 segment instances <<<<<<<<
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Review /home/gpadmin/gpAdminLogs/gpstart_20200102.log
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-For more details on segment startup failure(s)
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Run gpstate -s to review current segment instance status
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:08:55:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Starting Master instance gpdb-mdw directory /data/master/gpseg-1/
20200102:03:08:56:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Command pg_ctl reports Master gpdb-mdw instance active
20200102:03:08:56:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-Connecting to dbname='template1' connect_timeout=15
20200102:03:09:11:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Timeout expired connecting to template1, attempt 1/4
20200102:03:09:26:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Timeout expired connecting to template1, attempt 2/4
20200102:03:09:41:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Timeout expired connecting to template1, attempt 3/4
20200102:03:09:56:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Timeout expired connecting to template1, attempt 4/4
20200102:03:09:56:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Failed to connect to template1
20200102:03:09:56:009504 gpstart:gpdb-mdw:gpadmin-[INFO]:-No standby master configured. skipping...
20200102:03:09:56:009504 gpstart:gpdb-mdw:gpadmin-[WARNING]:-Number of segments which failed to start: 2
', stderr=''
[gpadmin@gpdb-mdw ~]$
[gpadmin@gpdb-mdw ~]$ gpstate -e
20200102:03:16:01:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-Starting gpstate with args: -e
20200102:03:16:01:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4'
20200102:03:16:01:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 16 2019 03:07:22'
20200102:03:16:01:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
20200102:03:16:01:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-Gathering data from segments...
.
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[WARNING]:-pg_stat_replication shows no standby connections
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[WARNING]:-pg_stat_replication shows no standby connections
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-Segment Mirroring Status Report
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-Unsynchronized Segment Pairs
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:- Current Primary Port Mirror Port
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:- gpdb-sdw1 6000 gpdb-sdw2 7000
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:- gpdb-sdw2 6000 gpdb-sdw1 7000
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-----------------------------------------------------
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:-Downed Segments (may include segments where status could not be retrieved)
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:- Segment Port Config status Status
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:- gpdb-sdw2 7000 Down Down in configuration
20200102:03:16:02:009823 gpstate:gpdb-mdw:gpadmin-[INFO]:- gpdb-sdw1 7000 Down Down in configuration
[gpadmin@gpdb-mdw ~]$
[gpadmin@gpdb-mdw ~]$ psql -c "select * from gp_segment_configuration;"
dbid | content | role | preferred_role | mode | status | port | hostname | address | datadir
------+---------+------+----------------+------+--------+------+-----------+-----------+----------------------
1 | -1 | p | p | n | u | 5432 | gpdb-mdw | gpdb-mdw | /data/master/gpseg-1
3 | 1 | p | p | n | u | 6000 | gpdb-sdw2 | gpdb-sdw2 | /data/primary/gpseg1
5 | 1 | m | m | n | d | 7000 | gpdb-sdw1 | gpdb-sdw1 | /data/mirror/gpseg1
2 | 0 | p | p | n | u | 6000 | gpdb-sdw1 | gpdb-sdw1 | /data/primary/gpseg0
4 | 0 | m | m | n | d | 7000 | gpdb-sdw2 | gpdb-sdw2 | /data/mirror/gpseg0
(5 rows)
[gpadmin@gpdb-mdw ~]$
結果竟然 Mirror Down。。。看起來。。。可能 hot_standby 功能沒有解鎖。。。只能先調整回來了
[gpadmin@gpdb-mdw ~]$ gpconfig -s wal_level Values on all segments are consistent GUC : wal_level Master value: hot_standby Segment value: hot_standby [gpadmin@gpdb-mdw ~]$ [gpadmin@gpdb-mdw ~]$ ssh gpdb-sdw1 [gpadmin@gpdb-sdw1 ~]$ cat /data/mirror/gpseg1/pg_log/gpdb-2020-01-02_030855.csv 2020-01-02 03:08:55.243623 UTC,,,p6589,th1353083008,,,,0,,,seg1,,,,,"LOG","00000","database system was shut down in recovery at 2020-01-02 03:08:47 UTC",,,,,,,0,,"xlog.c",6407, 2020-01-02 03:08:55.247113 UTC,"gpadmin","postgres",p6590,th1353083008,"::1","52098",2020-01-02 03:08:55 UTC,0,,,seg1,,,,,"FATAL","57P03","the database system is starting up",,,,,,,0,,"postmaster.c",2531, 2020-01-02 03:08:55.267460 UTC,,,p6589,th1353083008,,,,0,,,seg1,,,,,"LOG","00000","entering standby mode",,,,,,,0,,"xlog.c",6492, 2020-01-02 03:08:55.349269 UTC,"gpadmin","postgres",p6592,th1353083008,"::1","52132",2020-01-02 03:08:55 UTC,0,,,seg1,,,,,"FATAL","57P03","the database system is starting up",,,,,,,0,,"postmaster.c",2531, 2020-01-02 03:08:55.475836 UTC,"gpadmin","postgres",p6600,th1353083008,"::1","52142",2020-01-02 03:08:55 UTC,0,,,seg1,,,,,"FATAL","57P03","the database system is starting up",,,,,,,0,,"postmaster.c",2531, 2020-01-02 03:08:55.495899 UTC,,,p6589,th1353083008,,,,0,,,seg1,,,,,"FATAL","XX000","hot standby is not possible because wal_level was not set to ""hot_standby"" or higher on the master server (xlog.c:6340)",,"Either set wal_level to ""hot_standby"" on the master, or turn off hot_standby here.",,,,,0,,"xlog.c",6340,"Stack trace: 1 0x9f2ddc postgres errstart (elog.c:558) 2 0x54e836 postgres(xlog.c:6338) 3 0x558774 postgres StartupXLOG (xlog.c:6933) 4 0x8161e3 postgres StartupProcessMain (startup.c:248) 5 0x595832 postgres AuxiliaryProcessMain (bootstrap.c:447) 6 0x81225c postgres (postmaster.c:5824) 7 0x815ad2 postgres PostmasterMain (postmaster.c:1509) 8 0x4ce991 postgres main (main.c:205) 9 0x7f764c3cf505 libc.so.6 __libc_start_main + 0xf5 10 0x4cef6c postgres + 0x4cef6c " 2020-01-02 03:08:55.498284 UTC,,,p6582,th1353083008,,,,0,,,seg1,,,,,"LOG","00000","startup process (PID 6589) exited with exit code 1",,,,,,,0,,"postmaster.c",3940, 2020-01-02 03:08:55.498320 UTC,,,p6582,th1353083008,,,,0,,,seg1,,,,,"LOG","00000","aborting startup due to startup process failure",,,,,,,0,,"postmaster.c",3196, [gpadmin@gpdb-sdw1 ~]$
[gpadmin@gpdb-mdw ~]$ gpconfig -c wal_level -v archive 20200102:03:29:28:008891 gpconfig:gpdb-mdw:gpadmin-[INFO]:-completed successfully with parameters '-c wal_level -v archive' [gpadmin@gpdb-mdw ~]$ gpconfig -c hot_standby -v off 20200102:03:29:42:010489 gpconfig:gpdb-mdw:gpadmin-[INFO]:-completed successfully with parameters '-c hot_standby -v off' [gpadmin@gpdb-mdw ~]$ gprecoverseg 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Starting gprecoverseg with args: 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4' 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 16 2019 03:07:22' 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Obtaining Segment details from master... 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Heap checksum setting is consistent between master and the segments that are candidates for recoverseg 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Greenplum instance recovery parameters 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------------------- 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Recovery type = Standard 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------------------- 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Recovery 1 of 2 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------------------- 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Synchronization mode = Incremental 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Failed instance host = gpdb-sdw2 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Failed instance address = gpdb-sdw2 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Failed instance directory = /data/mirror/gpseg0 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Failed instance port = 7000 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Source instance host = gpdb-sdw1 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Source instance address = gpdb-sdw1 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Source instance directory = /data/primary/gpseg0 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Source instance port = 6000 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Target = in-place 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------------------- 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Recovery 2 of 2 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------------------- 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Synchronization mode = Incremental 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Failed instance host = gpdb-sdw1 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Failed instance address = gpdb-sdw1 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Failed instance directory = /data/mirror/gpseg1 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Failed instance port = 7000 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Source instance host = gpdb-sdw2 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Source instance address = gpdb-sdw2 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Source instance directory = /data/primary/gpseg1 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Source instance port = 6000 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:- Recovery Target = in-place 20200102:03:29:50:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:---------------------------------------------------------- Continue with segment recovery procedure Yy|Nn (default=N): > Y 20200102:03:30:21:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-2 segment(s) to recover 20200102:03:30:21:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Ensuring 2 failed segment(s) are stopped 20200102:03:30:22:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Ensuring that shared memory is cleaned up for stopped segments 20200102:03:30:24:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Updating configuration with new mirrors 20200102:03:30:24:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Updating mirrors 20200102:03:30:24:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Running pg_rewind on required mirrors 20200102:03:30:25:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Starting mirrors 20200102:03:30:25:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-era is 88b34f9c8bf1e7f6_200102030851 20200102:03:30:25:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Commencing parallel segment instance startup, please wait... ... 20200102:03:30:28:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Process results... 20200102:03:30:28:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Triggering FTS probe 20200102:03:30:30:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-****************************************************************** 20200102:03:30:30:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Updating segments for streaming is completed. 20200102:03:30:30:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-For segments updated successfully, streaming will continue in the background. 20200102:03:30:30:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-Use gpstate -s to check the streaming progress. 20200102:03:30:30:010544 gprecoverseg:gpdb-mdw:gpadmin-[INFO]:-****************************************************************** [gpadmin@gpdb-mdw ~]$ [gpadmin@gpdb-mdw ~]$ gpstate -e 20200102:03:31:11:010659 gpstate:gpdb-mdw:gpadmin-[INFO]:-Starting gpstate with args: -e 20200102:03:31:11:010659 gpstate:gpdb-mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4' 20200102:03:31:11:010659 gpstate:gpdb-mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.24 (Greenplum Database 6.2.1 build commit:d90ac1a1b983b913b3950430d4d9e47ee8827fd4) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 16 2019 03:07:22' 20200102:03:31:11:010659 gpstate:gpdb-mdw:gpadmin-[INFO]:-Obtaining Segment details from master... 20200102:03:31:11:010659 gpstate:gpdb-mdw:gpadmin-[INFO]:-Gathering data from segments... .. 20200102:03:31:14:010659 gpstate:gpdb-mdw:gpadmin-[INFO]:----------------------------------------------------- 20200102:03:31:14:010659 gpstate:gpdb-mdw:gpadmin-[INFO]:-Segment Mirroring Status Report 20200102:03:31:14:010659 gpstate:gpdb-mdw:gpadmin-[INFO]:----------------------------------------------------- 20200102:03:31:14:010659 gpstate:gpdb-mdw:gpadmin-[INFO]:-All segments are running normally [gpadmin@gpdb-mdw ~]$
小結：
- Greenplum 6.x 開始，骨幹資料庫更新到 PostgreSQL 9.4，導入 Streaming Replication 機制取代之前版本的 Block-Level 同步
- 雖然是使用 Streaming Replication，但截至 v6.2.1 為止，預設值 wal_level = archive，hot_standby = off；且無法啟動 Mirror 的 hot_standby 功能，就算將 wal_level 開的夠高也是一樣
- 在 Primary/Mirror 之間使用 physical replication slot 機制防止同步脫勾
- Greenplum 管理指令 gprecoverseg 會使用 pg_rewind 嘗試救回 Mirror。不過 Postgres 在 9.5 才將 pg_rewind 正式納入內部，在 PGSQL 9.3～9.4 版上，pg_rewind 是一個外部專案
參考資料
同事的筆記～
Greenplum Overview for Postgres Hackers - Greenplum Summit 2018
使用gprecoverseg修复Segment节点 - Liang Log
FTS (fault tolerance server) in Pivotal Greenplum (GPDB) - Pivotal Community Knowledge Article
About Greenplum Query Processing | Pivotal Greenplum Docs
沒有留言:
張貼留言