一、GP集群运行状态检查
- 查gp_segment_configuration，status='d'表示segment失败，status='u'表示segment正常
SELECT * FROM gp_segment_configuration WHERE status='d';
- gpstate查看各个primary、mirror segment状态信息 集群状态： 9个 primary segment失败 3个 mirror segment失败 此时，GP集群增删改查均正常，猜测是mirror segment起了作用。
[gpadmin@gpm ~]$ gpstate20160829:17:36:08:035340 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: 20160829:17:36:08:035340 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:17:36:08:035340 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:17:36:09:035340 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:17:36:09:035340 gpstate:gpm:gpadmin-[INFO]:-Gathering data from segments.... 20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:-Greenplum instance status summary20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Master instance = Active20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Master standby = gps120160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Standby master state = Standby host passive20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total segment instance count from metadata = 2420160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Primary Segment Status20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total primary segments = 1220160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total primary segment valid (at master) = 320160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total primary segment failures (at master) = 9 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total number of postmaster.pid files missing = 9 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid files found = 320160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total number of postmaster.pid PIDs missing = 9 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 320160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total number of /tmp lock files missing = 9 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total number of /tmp lock files found = 320160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total number postmaster processes missing = 9 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total number postmaster processes found = 320160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Mirror Segment Status20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total mirror segments = 1220160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total mirror segment valid (at master) = 920160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total mirror segment failures (at master) = 3 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total number of postmaster.pid files missing = 3 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid files found = 920160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total number of postmaster.pid PIDs missing = 3 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 920160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total number of /tmp lock files missing = 3 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total number of /tmp lock files found = 920160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total number postmaster processes missing = 3 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total number postmaster processes found = 920160829:17:36:10:035340 gpstate:gpm:gpadmin-[WARNING]:-Total number mirror segments acting as primary segments = 9 <<<<<<<<20160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:- Total number mirror segments acting as mirror segments = 320160829:17:36:10:035340 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------
- gpstate -m列出mirror 实例的状态和配置信息。 失败的primary segment被mirror segment顶起来了，可以看到，有9个mirror segment Acting as Primary。
[gpadmin@gpm ~]$ gpstate -m20160829:17:38:03:035819 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: -m20160829:17:38:03:035819 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:17:38:03:035819 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:17:38:03:035819 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:--Current GPDB mirror list and status20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:--Type = Group20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- Mirror Datadir Port Status Data Status 20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data5/gpdata/mirror/gpseg0 50000 Acting as Primary Change Tracking20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data6/gpdata/mirror/gpseg1 50001 Acting as Primary Change Tracking20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data7/gpdata/mirror/gpseg2 50002 Acting as Primary Change Tracking20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data8/gpdata/mirror/gpseg3 50003 Acting as Primary Change Tracking20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data5/gpdata/mirror/gpseg4 50000 Acting as Primary Change Tracking20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data6/gpdata/mirror/gpseg5 50001 Acting as Primary Change Tracking20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data7/gpdata/mirror/gpseg6 50002 Acting as Primary Change Tracking20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data8/gpdata/mirror/gpseg7 50003 Acting as Primary Change Tracking20160829:17:38:04:035819 gpstate:gpm:gpadmin-[WARNING]:-gps1 /hdfs/data5/gpdata/mirror/gpseg8 50000 Failed <<<<<<<<20160829:17:38:04:035819 gpstate:gpm:gpadmin-[WARNING]:-gps1 /hdfs/data6/gpdata/mirror/gpseg9 50001 Failed <<<<<<<<20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data7/gpdata/mirror/gpseg10 50002 Acting as Primary Change Tracking20160829:17:38:04:035819 gpstate:gpm:gpadmin-[WARNING]:-gps1 /hdfs/data8/gpdata/mirror/gpseg11 50003 Failed <<<<<<<<20160829:17:38:04:035819 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:17:38:04:035819 gpstate:gpm:gpadmin-[WARNING]:-9 segment(s) configured as mirror(s) are acting as primaries20160829:17:38:04:035819 gpstate:gpm:gpadmin-[WARNING]:-3 segment(s) configured as mirror(s) have failed20160829:17:38:04:035819 gpstate:gpm:gpadmin-[WARNING]:-9 mirror segment(s) acting as primaries are in change tracking
- gpstate -c 查看primary instance 和 mirror instance 的对应关系
[gpadmin@gpm ~]$ gpstate -c20160829:17:38:53:035970 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: -c20160829:17:38:53:035970 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:17:38:53:035970 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:17:38:53:035970 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:17:38:54:035970 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:17:38:54:035970 gpstate:gpm:gpadmin-[INFO]:--Current GPDB mirror list and status20160829:17:38:54:035970 gpstate:gpm:gpadmin-[INFO]:--Type = Group20160829:17:38:54:035970 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:17:38:54:035970 gpstate:gpm:gpadmin-[INFO]:- Status Data State Primary Datadir Port Mirror Datadir Port20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Mirror Active, Primary Failed Change Tracking gps1 /hdfs/data1/gpdata/primary/gpseg0 40000 gps2 /hdfs/data5/gpdata/mirror/gpseg0 50000 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Mirror Active, Primary Failed Change Tracking gps1 /hdfs/data2/gpdata/primary/gpseg1 40001 gps2 /hdfs/data6/gpdata/mirror/gpseg1 50001 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Mirror Active, Primary Failed Change Tracking gps1 /hdfs/data3/gpdata/primary/gpseg2 40002 gps2 /hdfs/data7/gpdata/mirror/gpseg2 50002 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Mirror Active, Primary Failed Change Tracking gps1 /hdfs/data4/gpdata/primary/gpseg3 40003 gps2 /hdfs/data8/gpdata/mirror/gpseg3 50003 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Mirror Active, Primary Failed Change Tracking gps2 /hdfs/data1/gpdata/primary/gpseg4 40000 gps3 /hdfs/data5/gpdata/mirror/gpseg4 50000 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Mirror Active, Primary Failed Change Tracking gps2 /hdfs/data2/gpdata/primary/gpseg5 40001 gps3 /hdfs/data6/gpdata/mirror/gpseg5 50001 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Mirror Active, Primary Failed Change Tracking gps2 /hdfs/data3/gpdata/primary/gpseg6 40002 gps3 /hdfs/data7/gpdata/mirror/gpseg6 50002 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Mirror Active, Primary Failed Change Tracking gps2 /hdfs/data4/gpdata/primary/gpseg7 40003 gps3 /hdfs/data8/gpdata/mirror/gpseg7 50003 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Primary Active, Mirror Failed Change Tracking gps3 /hdfs/data1/gpdata/primary/gpseg8 40000 gps1 /hdfs/data5/gpdata/mirror/gpseg8 50000 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Primary Active, Mirror Failed Change Tracking gps3 /hdfs/data2/gpdata/primary/gpseg9 40001 gps1 /hdfs/data6/gpdata/mirror/gpseg9 50001 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Mirror Active, Primary Failed Change Tracking gps3 /hdfs/data3/gpdata/primary/gpseg10 40002 gps1 /hdfs/data7/gpdata/mirror/gpseg10 50002 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-Primary Active, Mirror Failed Change Tracking gps3 /hdfs/data4/gpdata/primary/gpseg11 40003 gps1 /hdfs/data8/gpdata/mirror/gpseg11 50003 <<<<<<<<20160829:17:38:54:035970 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-9 segment(s) configured as mirror(s) are acting as primaries20160829:17:38:54:035970 gpstate:gpm:gpadmin-[WARNING]:-12 segment(s) are in change tracking
二、失败segment修复
经过上述操作，基本了解了GP集群各节点状态，接下来借助Greenplum自带工具进行失败segment修复。
- gprecoverseg 工具修复失败segment，出现提示是否继续恢复程序时，输出‘Y’。
# 该过程打印信息忘了保存，借用网上的$ gprecoverseg 20160118:14:08:37:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Starting gprecoverseg with args: 20160118:14:08:37:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.6.2 build 1'20160118:14:08:37:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.6.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Nov 12 2015 23:50:28'20160118:14:08:37:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Checking if segments are ready20160118:14:08:37:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Obtaining Segment details from master...20160118:14:08:37:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Obtaining Segment details from master...20160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Greenplum instance recovery parameters20160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:----------------------------------------------------------20160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Recovery type = Standard20160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:----------------------------------------------------------20160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Recovery 1 of 120160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:----------------------------------------------------------20160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Synchronization mode = Incremental20160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Failed instance host = gp-s320160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Failed instance address = gp-s320160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Failed instance directory = /home/gpadmin/primary/gpseg220160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Failed instance port = 4000020160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Failed instance replication port = 4100020160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Recovery Source instance host = gp-s120160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Recovery Source instance address = gp-s120160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Recovery Source instance directory = /home/gpadmin/mirror/gpseg220160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Recovery Source instance port = 5000020160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Recovery Source instance replication port = 5100020160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:- Recovery Target = in-place20160118:14:08:38:025645 gprecoverseg:gp-master:gpadmin-[INFO]:----------------------------------------------------------Continue with segment recovery procedure Yy|Nn (default=N):> y20160118:14:08:42:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-1 segment(s) to recover20160118:14:08:42:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Ensuring 1 failed segment(s) are stopped20160118:14:08:43:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-23723: /home/gpadmin/primary/gpseg2 20160118:14:08:51:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Ensuring that shared memory is cleaned up for stopped segmentsupdating flat files20160118:14:08:56:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Updating configuration with new mirrors20160118:14:08:56:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Updating mirrors. 20160118:14:08:57:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Starting mirrors20160118:14:08:57:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Commencing parallel primary and mirror segment instance startup, please wait..... 20160118:14:08:59:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Process results...20160118:14:08:59:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Updating configuration to mark mirrors up20160118:14:08:59:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Updating primaries20160118:14:08:59:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Commencing parallel primary conversion of 1 segments, please wait.... 20160118:14:09:00:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Process results...20160118:14:09:00:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Done updating primaries20160118:14:09:00:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-******************************************************************20160118:14:09:00:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Updating segments for resynchronization is completed.20160118:14:09:00:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-For segments updated successfully, resynchronization will continue in the background.20160118:14:09:00:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-20160118:14:09:00:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-Use gpstate -s to check the resynchronization progress.20160118:14:09:00:025645 gprecoverseg:gp-master:gpadmin-[INFO]:-******************************************************************
2 确认primary和mirror已同步，状态从不正常变为正常的过程:Change Tracking>>Resynchronizing>> Synchronized。
[gpadmin@gpm ~]$ gpstate -m20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: -m20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--Current GPDB mirror list and status20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--Type = Group20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- Mirror Datadir Port Status Data Status 20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data5/gpdata/mirror/gpseg0 50000 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data6/gpdata/mirror/gpseg1 50001 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data7/gpdata/mirror/gpseg2 50002 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data8/gpdata/mirror/gpseg3 50003 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data5/gpdata/mirror/gpseg4 50000 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data6/gpdata/mirror/gpseg5 50001 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data7/gpdata/mirror/gpseg6 50002 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data8/gpdata/mirror/gpseg7 50003 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data5/gpdata/mirror/gpseg8 50000 Passive Synchronized20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data6/gpdata/mirror/gpseg9 50001 Passive Synchronized20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data7/gpdata/mirror/gpseg10 50002 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data8/gpdata/mirror/gpseg11 50003 Passive Synchronized20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------[gpadmin@gpm ~]$ gpstate -m20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: -m20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--Current GPDB mirror list and status20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--Type = Group20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- Mirror Datadir Port Status Data Status 20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data5/gpdata/mirror/gpseg0 50000 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data6/gpdata/mirror/gpseg1 50001 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data7/gpdata/mirror/gpseg2 50002 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data8/gpdata/mirror/gpseg3 50003 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data5/gpdata/mirror/gpseg4 50000 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data6/gpdata/mirror/gpseg5 50001 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data7/gpdata/mirror/gpseg6 50002 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data8/gpdata/mirror/gpseg7 50003 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data5/gpdata/mirror/gpseg8 50000 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data6/gpdata/mirror/gpseg9 50001 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data7/gpdata/mirror/gpseg10 50002 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data8/gpdata/mirror/gpseg11 50003 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------
3 恢复所有segment到原有状态
当primary segment宕机，mirror成为了primary segment。执行gprecoverseg命令恢复失败segment后，当前primary segment是原来的mirror，原来失败的primary segment变成了现在的mirror。并没有恢复到系统初始化时候的角色状态，造成一个主机上有多个primary segment，会导致系统处于非平衡状态，消耗更多的系统资源。-
gpstate -e命令，能够检查出segment是否是原来的角色
-
运行gpstate -m，确认所有mirrors是否同步完成(Resynchronized)。 如果mirrors的状态是Resynchronizing，等待完成
-
运行gprecoverseg -r命令，将所有segments恢复到原有角色。当提示是否继续segment 负载均衡时，输入‘Y’。
[gpadmin@gpm ~]$ gprecoverseg -r20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Starting gprecoverseg with args: -r20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Checking if segments are ready20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Greenplum instance recovery parameters20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Recovery type = Rebalance20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 1 of 1820160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps220160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps220160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data5/gpdata/mirror/gpseg020160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 5000020160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 5100020160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Mirror20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Primary20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 2 of 1820160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data1/gpdata/primary/gpseg020160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 4000020160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 4100020160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Primary20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Mirror20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 3 of 1820160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps220160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps220160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data6/gpdata/mirror/gpseg120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 5000120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 5100120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Mirror20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Primary20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 4 of 1820160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data2/gpdata/primary/gpseg120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 4000120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 4100120160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Primary20160829:18:34:09:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 5 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data7/gpdata/mirror/gpseg220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 5000220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 5100220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 6 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data3/gpdata/primary/gpseg220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 4000220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 4100220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 7 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data8/gpdata/mirror/gpseg320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 5000320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 5100320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 8 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data4/gpdata/primary/gpseg320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 4000320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 4100320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 9 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data5/gpdata/mirror/gpseg420160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 5000020160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 5100020160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 10 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data1/gpdata/primary/gpseg420160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 4000020160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 4100020160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 11 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data6/gpdata/mirror/gpseg520160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 5000120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 5100120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 12 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data2/gpdata/primary/gpseg520160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 4000120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 4100120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 13 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data7/gpdata/mirror/gpseg620160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 5000220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 5100220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 14 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data3/gpdata/primary/gpseg620160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 4000220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 4100220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 15 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data8/gpdata/mirror/gpseg720160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 5000320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 5100320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 16 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data4/gpdata/primary/gpseg720160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 4000320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 4100320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 17 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps120160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data7/gpdata/mirror/gpseg1020160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 5000220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 5100220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Unbalanced segment 18 of 1820160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance host = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance address = gps320160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance directory = /hdfs/data3/gpdata/primary/gpseg1020160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance port = 4000220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Unbalanced instance replication port = 4100220160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Balanced role = Primary20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:- Current role = Mirror20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[INFO]:----------------------------------------------------------20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[WARNING]:-This operation will cancel queries that are currently executing.20160829:18:34:10:048215 gprecoverseg:gpm:gpadmin-[WARNING]:-Connections to the database however will not be interrupted.Continue with segment rebalance procedure Yy|Nn (default=N):> y20160829:18:34:11:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Getting unbalanced segments20160829:18:34:11:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Stopping unbalanced primary segments..... 20160829:18:34:13:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Triggering segment reconfiguration20160829:18:34:18:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Starting segment synchronization............. 20160829:18:34:31:048215 gprecoverseg:gpm:gpadmin-[INFO]:-******************************************************************20160829:18:34:31:048215 gprecoverseg:gpm:gpadmin-[INFO]:-The rebalance operation has completed successfully.20160829:18:34:31:048215 gprecoverseg:gpm:gpadmin-[INFO]:-There is a resynchronization running in the background to bring all20160829:18:34:31:048215 gprecoverseg:gpm:gpadmin-[INFO]:-segments in sync.20160829:18:34:31:048215 gprecoverseg:gpm:gpadmin-[INFO]:-20160829:18:34:31:048215 gprecoverseg:gpm:gpadmin-[INFO]:-Use gpstate -e to check the resynchronization progress.20160829:18:34:31:048215 gprecoverseg:gpm:gpadmin-[INFO]:-******************************************************************
- 查看同步状态
[gpadmin@gpm ~]$ gpstate -e20160829:18:34:47:048559 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: -e20160829:18:34:48:048559 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:18:34:48:048559 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:18:34:48:048559 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:18:34:48:048559 gpstate:gpm:gpadmin-[INFO]:-Gathering data from segments.... 20160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:-Segment Mirroring Status Report20160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:-Segment Pairs in Resynchronization20160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- Current Primary Port Resync mode Est. resync progress Total resync objects Objects to resync Data synced Est. total to sync Est. resync end time Change tracking size Mirror Port20160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- gps1 40000 Incremental 100% 0 0 384 kB Sync complete; awaiting config change 128 bytes gps2 5000020160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- gps1 40001 Incremental 100% 0 0 384 kB Sync complete; awaiting config change 128 bytes gps2 5000120160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- gps1 40002 Incremental 100% 0 0 384 kB Sync complete; awaiting config change 128 bytes gps2 5000220160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- gps1 40003 Incremental 100% 0 0 384 kB Sync complete; awaiting config change 128 bytes gps2 5000320160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- gps2 40000 Incremental 100% 0 0 384 kB Sync complete; awaiting config change 128 bytes gps3 5000020160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- gps2 40001 Incremental 100% 0 0 384 kB Sync complete; awaiting config change 128 bytes gps3 5000120160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- gps2 40002 Incremental 100% 0 0 384 kB Sync complete; awaiting config change 128 bytes gps3 5000220160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- gps2 40003 Incremental 100% 0 0 384 kB Sync complete; awaiting config change 128 bytes gps3 5000320160829:18:34:49:048559 gpstate:gpm:gpadmin-[INFO]:- gps3 40002 Incremental 100% 0 0 384 kB Sync complete; awaiting config change 128 bytes gps1 50002
[gpadmin@gpm ~]$ gpstate -m20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: -m20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--Current GPDB mirror list and status20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--Type = Group20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- Mirror Datadir Port Status Data Status 20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data5/gpdata/mirror/gpseg0 50000 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data6/gpdata/mirror/gpseg1 50001 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data7/gpdata/mirror/gpseg2 50002 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data8/gpdata/mirror/gpseg3 50003 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data5/gpdata/mirror/gpseg4 50000 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data6/gpdata/mirror/gpseg5 50001 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data7/gpdata/mirror/gpseg6 50002 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data8/gpdata/mirror/gpseg7 50003 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data5/gpdata/mirror/gpseg8 50000 Passive Synchronized20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data6/gpdata/mirror/gpseg9 50001 Passive Synchronized20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data7/gpdata/mirror/gpseg10 50002 Passive Resynchronizing20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data8/gpdata/mirror/gpseg11 50003 Passive Synchronized20160829:18:35:02:048639 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------[gpadmin@gpm ~]$ gpstate -m20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: -m20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--Current GPDB mirror list and status20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--Type = Group20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- Mirror Datadir Port Status Data Status 20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data5/gpdata/mirror/gpseg0 50000 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data6/gpdata/mirror/gpseg1 50001 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data7/gpdata/mirror/gpseg2 50002 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps2 /hdfs/data8/gpdata/mirror/gpseg3 50003 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data5/gpdata/mirror/gpseg4 50000 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data6/gpdata/mirror/gpseg5 50001 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data7/gpdata/mirror/gpseg6 50002 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps3 /hdfs/data8/gpdata/mirror/gpseg7 50003 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data5/gpdata/mirror/gpseg8 50000 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data6/gpdata/mirror/gpseg9 50001 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data7/gpdata/mirror/gpseg10 50002 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:- gps1 /hdfs/data8/gpdata/mirror/gpseg11 50003 Passive Synchronized20160829:18:36:01:048898 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------
- 检查恢复结果 可以看到primary 和mirror均没有失败的segment了，恢复成功。
[gpadmin@gpm ~]$ gpstate20160829:18:36:10:049017 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: 20160829:18:36:10:049017 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:18:36:10:049017 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:18:36:10:049017 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:18:36:11:049017 gpstate:gpm:gpadmin-[INFO]:-Gathering data from segments.... 20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:-Greenplum instance status summary20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Master instance = Active20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Master standby = gps120160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Standby master state = Standby host passive20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total segment instance count from metadata = 2420160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Primary Segment Status20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total primary segments = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total primary segment valid (at master) = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total primary segment failures (at master) = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid files found = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of /tmp lock files missing = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of /tmp lock files found = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number postmaster processes missing = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number postmaster processes found = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Mirror Segment Status20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total mirror segments = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total mirror segment valid (at master) = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total mirror segment failures (at master) = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid files found = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of /tmp lock files missing = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number of /tmp lock files found = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number postmaster processes missing = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number postmaster processes found = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number mirror segments acting as primary segments = 020160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:- Total number mirror segments acting as mirror segments = 1220160829:18:36:12:049017 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------
[gpadmin@gpm ~]$ gpstate -e20160829:19:14:45:008450 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: -e20160829:19:14:45:008450 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:19:14:45:008450 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:19:14:45:008450 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:19:14:46:008450 gpstate:gpm:gpadmin-[INFO]:-Gathering data from segments.... 20160829:19:14:47:008450 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:19:14:47:008450 gpstate:gpm:gpadmin-[INFO]:-Segment Mirroring Status Report20160829:19:14:47:008450 gpstate:gpm:gpadmin-[INFO]:-----------------------------------------------------20160829:19:14:47:008450 gpstate:gpm:gpadmin-[INFO]:-All segments are running normally
[gpadmin@gpm ~]$ gpstate -c20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:-Starting gpstate with args: -c20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.8.2 build 1'20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.8.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on May 9 2016 22:28:40'20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:-Obtaining Segment details from master...20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:--Current GPDB mirror list and status20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:--Type = Group20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Status Data State Primary Datadir Port Mirror Datadir Port20160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps1 /hdfs/data1/gpdata/primary/gpseg0 40000 gps2 /hdfs/data5/gpdata/mirror/gpseg0 5000020160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps1 /hdfs/data2/gpdata/primary/gpseg1 40001 gps2 /hdfs/data6/gpdata/mirror/gpseg1 5000120160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps1 /hdfs/data3/gpdata/primary/gpseg2 40002 gps2 /hdfs/data7/gpdata/mirror/gpseg2 5000220160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps1 /hdfs/data4/gpdata/primary/gpseg3 40003 gps2 /hdfs/data8/gpdata/mirror/gpseg3 5000320160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps2 /hdfs/data1/gpdata/primary/gpseg4 40000 gps3 /hdfs/data5/gpdata/mirror/gpseg4 5000020160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps2 /hdfs/data2/gpdata/primary/gpseg5 40001 gps3 /hdfs/data6/gpdata/mirror/gpseg5 5000120160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps2 /hdfs/data3/gpdata/primary/gpseg6 40002 gps3 /hdfs/data7/gpdata/mirror/gpseg6 5000220160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps2 /hdfs/data4/gpdata/primary/gpseg7 40003 gps3 /hdfs/data8/gpdata/mirror/gpseg7 5000320160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps3 /hdfs/data1/gpdata/primary/gpseg8 40000 gps1 /hdfs/data5/gpdata/mirror/gpseg8 5000020160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps3 /hdfs/data2/gpdata/primary/gpseg9 40001 gps1 /hdfs/data6/gpdata/mirror/gpseg9 5000120160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps3 /hdfs/data3/gpdata/primary/gpseg10 40002 gps1 /hdfs/data7/gpdata/mirror/gpseg10 5000220160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:- Primary Active, Mirror Available Synchronized gps3 /hdfs/data4/gpdata/primary/gpseg11 40003 gps1 /hdfs/data8/gpdata/mirror/gpseg11 5000320160829:19:15:28:008817 gpstate:gpm:gpadmin-[INFO]:--------------------------------------------------------------
三、遗留问题
- 按以上操作成功恢复了GP集群失败segment，但对恢复工具机制还需进一步了解
- 以上只是部分segment失败、集群仍能正常运行场景的恢复，对其他场景(如GP集群不能正常工作)的恢复，尚未遇到。
参考资料：