Start and stop a cluster

Pavel Semyonov

Contents

Start a cluster
- Start master instance only
Stop a cluster
Restart a cluster
Reload cluster configuration
Check cluster state

This topic explains how to start, stop, and restart a Greengage DB (based on Greenplum) cluster.

A Greengage DB cluster typically comprises multiple processes distributed across different servers. These processes run PostgreSQL databases underlying the Greengage DB instances on their respective servers. To start or stop all cluster processes consistently with a single command, the following utilities are available:

gpstart — start the cluster.
gpstop — stop or restart the cluster, reload configuration.

The utilities are located in the bin subdirectory of the Greengage DB installation directory ($GPHOME).

Running gpstart on the master host starts an initialized Greengage DB cluster:

$ gpstart

NOTE

Running gpstart is not required for the first cluster startup after its initialization using gpinitsystem. The cluster is started automatically.

This command checks the environment and displays a summary of the cluster configuration:

[INFO]:---------------------------
[INFO]:-Master instance parameters
[INFO]:---------------------------
[INFO]:-Database                 = template1
[INFO]:-Master Port              = 5432
[INFO]:-Master directory         = /data1/master/gpseg-1
[INFO]:-Timeout                  = 600 seconds
[INFO]:-Master standby start     = On
[INFO]:---------------------------------------
[INFO]:-Segment instances that will be started
[INFO]:---------------------------------------
[INFO]:-   Host   Datadir                 Port    Role
[INFO]:-   sdw1   /data1/primary/gpseg0   10000   Primary
[INFO]:-   sdw2   /data1/mirror/gpseg0    10500   Mirror
[INFO]:-   sdw1   /data1/primary/gpseg1   10001   Primary
[INFO]:-   sdw2   /data1/mirror/gpseg1    10501   Mirror
[INFO]:-   sdw2   /data1/primary/gpseg2   10000   Primary
[INFO]:-   sdw1   /data1/mirror/gpseg2    10500   Mirror
[INFO]:-   sdw2   /data1/primary/gpseg3   10001   Primary
[INFO]:-   sdw1   /data1/mirror/gpseg3    10501   Mirror

Type y and press Enter to confirm the configuration and continue the cluster startup. To skip confirmation, use the -a option:

$ gpstart -a

The command initiates and orchestrates a startup of all cluster instances. All of them — master, standby master, and both primary and mirror segments — are started on the corresponding hosts in parallel. If the startup is successful, the output includes the following message:

[INFO]:-Database successfully started

By default, gpstart uses up to 64 processes for parallel instance startup. Use the -B option to change this number:

$ gpstart -B 16

The master-only startup mode allows starting only the master instance without starting the segment instances. This mode is primarily used for system maintenance tasks that do not affect stored data.

To start only the master instance, run gpstart with the -m option on the master host:

$ gpstart -m

Enter y and press Enter to confirm startup.

CAUTION

Manual changes to the system catalog can bring the cluster to an inconsistent state. Use the master-only mode carefully.

To perform cluster maintenance in the master-only mode:

Start the cluster in the master-only mode:
```
$ gpstart -m
```

Connect to the master database in utility mode:

$ PGOPTIONS='-c gp_session_role=utility' psql postgres

Complete the maintenance tasks.
Stop the master instance:
```
$ gpstop -m
```
(Optional) Start the entire cluster:
```
$ gpstart
```

TIP

For maintenance tasks that require a fully running cluster or access to stored data, there is also a restricted startup mode. In this mode, all instances are started, but only superusers are allowed to connect:

$ gpstart -R

CAUTION

Don’t use the kill command to stop Greengage DB instances. This can lead to data corruption and cluster inconsistency.

To stop a Greengage DB cluster, run gpstop on the master host:

$ gpstop

The command displays the information about the instances it is going to shut down:

[INFO]:---------------------------------------------
[INFO]:-Master instance parameters
[INFO]:---------------------------------------------
[INFO]:-   Master Greengage instance process active PID   = 14255
[INFO]:-   Database                                       = template1
[INFO]:-   Master port                                    = 5432
[INFO]:-   Master directory                               = /data1/master/gpseg-1
[INFO]:-   Shutdown mode                                  = smart
[INFO]:-   Timeout                                        = 120
[INFO]:-   Shutdown Master standby host                   = On
[INFO]:---------------------------------------------
[INFO]:-Segment instances that will be shutdown:
[INFO]:---------------------------------------------
[INFO]:-   Host   Datadir                 Port    Status
[INFO]:-   sdw1   /data1/primary/gpseg0   10000   u
[INFO]:-   sdw2   /data1/mirror/gpseg0    10500   u
[INFO]:-   sdw1   /data1/primary/gpseg1   10001   u
[INFO]:-   sdw2   /data1/mirror/gpseg1    10501   u
[INFO]:-   sdw2   /data1/primary/gpseg2   10000   u
[INFO]:-   sdw1   /data1/mirror/gpseg2    10500   u
[INFO]:-   sdw2   /data1/primary/gpseg3   10001   u
[INFO]:-   sdw1   /data1/mirror/gpseg3    10501   u

Type y and press Enter to confirm the cluster shutdown. To skip confirmation, use the -a option:

$ gpstop -a

If the shutdown is successful, the output includes the following message:

[INFO]:-Database successfully shutdown with no errors reported

By default, gpstop uses up to 64 processes to shut down instances in parallel. Use the -B option to change this number:

$ gpstop -B 16

Greengage DB provides three shutdown modes for handling active transactions during cluster shutdown: smart, fast, and immediate. Each mode determines how active transactions and client connections are managed when stopping the cluster.

Shutdown modes
Mode	Command	Description
Smart	gpstop gpstop -M smart	Default mode. Waits for active transactions to complete. If uncommitted transactions remain after two minutes, prompts the user to choose whether to continue in the same mode or switch to another
Fast	gpstop -M fast	Rolls back active transactions, terminates client connections, and gracefully shuts down the cluster
Immediate	gpstop -M immediate	Terminates cluster processes without waiting or cleaning up temporary artifacts. Includes the following steps: Send `SIGQUIT` to child Postgres processes and wait for them to terminate. Terminate the remaining ones with `SIGKILL`. Exit the Greengage DB master server process.

CAUTION

Using the immediate shutdown mode can lead to database corruption and cluster inconsistency.

A typical Greengage DB cluster shutdown goes as follows:

Attempt shutdown in the default (smart) mode:
```
$ gpstop
```
If active transactions prevent shutdown, clean them up using the pg_cancel_backend() and pg_terminate_backend() system functions. Learn more in the Stop client queries and processes section.
When it is safe to terminate client connections, continue shutdown in the fast mode:
```
$ gpstop -M fast
```

Each client connection initiates a dedicated backend PostgreSQL process. When stopping a cluster, you can manage active client transactions and connections using the following functions:

pg_cancel_backend() cancels the currently executing query for the specified backend process ID (PID), leaving the connection open.
pg_terminate_backend() forcefully terminates the specified backend process, closing the associated client connection.

Both functions require superuser privileges and accept the backend process ID (PID) as an argument. Additionally, you can provide an optional message to send to the client:

SELECT pg_cancel_backend(1234);
SELECT pg_terminate_backend(2345, 'Shutting down the server for maintenance.');

PIDs of active client backends are stored in the pg_stat_activity system view:

SELECT pid, usename, state, query FROM pg_stat_activity;

The query result can look as follows:

 pid  | usename |        state        |                          query
------+---------+---------------------+----------------------------------------------------------
 7662 | gpadmin | active              | SELECT pid, usename, state, query FROM pg_stat_activity;
 7831 | alice   | idle in transaction | select count (*) from pg_settings;
(2 rows)

With information from this view, you can cancel or terminate multiple backends at once:

Cancel all running queries:

SELECT pg_cancel_backend(pid) FROM pg_stat_activity
WHERE state = 'active' AND pid <> pg_backend_pid();

NOTE

The pid <> pg_backend_pid() condition prevents canceling this query from itself.

Terminate all client connections to a specified database:

SELECT pg_terminate_backend (pid) FROM pg_stat_activity
WHERE pg_stat_activity.datname = '<database_name>';

where <database_name> is the name of the database.

To perform maintenance of a single host, you can stop all segments that run on it:

$ gpstop --host <hostname>

where <hostname> is a segment host name.

Specify the --host argument according to the hostname column of the gp_segment_configuration table.

As a result of this command:

For each primary segment that runs on this host, the corresponding mirror segment on another host is promoted to primary.
Primary and mirror segments on this host are shut down.

IMPORTANT

Host segments cannot be stopped with gpstop --host if:

Segment mirroring is not enabled.
A mirror of a primary segment to shut down is running on the same host.
A primary segment to shut down is not synchronized with its mirror.
Master or standby master is running on this host.

To restart a Greengage DB cluster, run gpstop with the -r option on the master host:

$ gpstop -r

This shuts down all instances and then starts them again.

TIP

For conciseness, you can join multiple options into a single string. For example, use gpstop -arM fast instead of gpstop -a -r -M fast to restart the cluster using fast shutdown mode without confirmation.

To reload the cluster configuration without interruption, run gpstop with the -u option:

$ gpstop -u

Such a reload is required, for example, after making changes to the pg_hba.conf or postgresql.conf (only runtime parameters) configuration files. The changes are propagated to all active sessions without reconnection.

NOTE

To reload parameters that cannot be changed in runtime, a restart with gpstop -r is required.

To display the cluster state summary, use the gpstate utility:

$ gpstate

It outputs general information about the cluster, such as a total number of segments, processes, failures, and so on.

Example of gpstate output

[INFO]:-Starting gpstate with args:
[INFO]:-local Greengage Version: 'postgres (Greengage Database) 6.28.0+dev.1.g8552cb80e5 build dev'
[INFO]:-master Greengage Version: 'PostgreSQL 9.4.26 (Greengage Database 6.28.0+dev.1.g8552cb80e5 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit compiled on Mar  7 2025 05:20:53'
[INFO]:-Obtaining Segment details from master...
[INFO]:-Gathering data from segments...
.
[INFO]:-Greengage instance status summary
[INFO]:-----------------------------------------------------
[INFO]:-   Master instance                                           = Active
[INFO]:-   Master standby                                            = smdw
[INFO]:-   Standby master state                                      = Standby host passive
[INFO]:-   Total segment instance count from metadata                = 16
[INFO]:-----------------------------------------------------
[INFO]:-   Primary Segment Status
[INFO]:-----------------------------------------------------
[INFO]:-   Total primary segments                                    = 8
[INFO]:-   Total primary segment valid (at master)                   = 8
[INFO]:-   Total primary segment failures (at master)                = 0
[INFO]:-   Total number of postmaster.pid files missing              = 0
[INFO]:-   Total number of postmaster.pid files found                = 8
[INFO]:-   Total number of postmaster.pid PIDs missing               = 0
[INFO]:-   Total number of postmaster.pid PIDs found                 = 8
[INFO]:-   Total number of /tmp lock files missing                   = 0
[INFO]:-   Total number of /tmp lock files found                     = 8
[INFO]:-   Total number postmaster processes missing                 = 0
[INFO]:-   Total number postmaster processes found                   = 8
[INFO]:-----------------------------------------------------
[INFO]:-   Mirror Segment Status
[INFO]:-----------------------------------------------------
[INFO]:-   Total mirror segments                                     = 8
[INFO]:-   Total mirror segment valid (at master)                    = 8
[INFO]:-   Total mirror segment failures (at master)                 = 0
[INFO]:-   Total number of postmaster.pid files missing              = 0
[INFO]:-   Total number of postmaster.pid files found                = 8
[INFO]:-   Total number of postmaster.pid PIDs missing               = 0
[INFO]:-   Total number of postmaster.pid PIDs found                 = 8
[INFO]:-   Total number of /tmp lock files missing                   = 0
[INFO]:-   Total number of /tmp lock files found                     = 8
[INFO]:-   Total number postmaster processes missing                 = 0
[INFO]:-   Total number postmaster processes found                   = 8
[INFO]:-   Total number mirror segments acting as primary segments   = 0
[INFO]:-   Total number mirror segments acting as mirror segments    = 8
[INFO]:-----------------------------------------------------

To view detailed information about each segment, add the -s option:

$ gpstate -s

Example of gpstate -s output

Cluster and master instance information:

[INFO]:-Starting gpstate with args: -s
[INFO]:-local Greengage Version: 'postgres (Greengage Database) 6.28.0+dev.1.g8552cb80e5 build dev'
[INFO]:-master Greengage Version: 'PostgreSQL 9.4.26 (Greengage Database 6.28.0+dev.1.g8552cb80e5 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit compiled on Mar  7 2025 05:20:53'
[INFO]:-Obtaining Segment details from master...
[INFO]:-Gathering data from segments...

[INFO]:-----------------------------------------------------
[INFO]:--Master Configuration & Status
[INFO]:-----------------------------------------------------
[INFO]:-   Master host                    = mdw
[INFO]:-   Master postgres process ID     = 2118
[INFO]:-   Master data directory          = /data1/master/gpseg-1
[INFO]:-   Master port                    = 5432
[INFO]:-   Master current role            = dispatch
[INFO]:-   Greengage initsystem version   = 6.28.0+dev.1.g8552cb80e5 build dev
[INFO]:-   Greengage current version      = PostgreSQL 9.4.26 (Greengage Database 6.28.0+dev.1.g8552cb80e5 build dev) on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit compiled on Mar  7 2025 05:20:53
[INFO]:-   Postgres version               = 9.4.26
[INFO]:-   Master standby                 = smdw
[INFO]:-   Standby master state           = Standby host passive

Information about each segment in the following format:

[INFO]:-   Segment Info
[INFO]:-      Hostname                              = sdw1
[INFO]:-      Address                               = sdw1
[INFO]:-      Datadir                               = /data1/primary/gpseg0
[INFO]:-      Port                                  = 10000
[INFO]:-   Mirroring Info
[INFO]:-      Current role                          = Primary
[INFO]:-      Preferred role                        = Primary
[INFO]:-      Mirror status                         = Synchronized
[INFO]:-   Replication Info
[INFO]:-      Current write location                = 0/CFAB210
[INFO]:-      Bytes remaining to send to mirror     = 0
[INFO]:-   Status
[INFO]:-      PID                                   = 2757
[INFO]:-      Configuration reports status as       = Up
[INFO]:-      Database status                       = Up

To quickly find potential mirror status issues, you can display segments with such issues using the -e option:

$ gpstate -e

Example of gpstate -e output with issues

[INFO]:-----------------------------------------------------
[INFO]:-Segment Mirroring Status Report
[INFO]:-----------------------------------------------------
[INFO]:-Segments with Primary and Mirror Roles Switched
[INFO]:-   Current Primary   Port    Mirror   Port
[INFO]:-   sdw3              10500   sdw4     10000
[INFO]:-   sdw3              10501   sdw4     10001
[INFO]:-----------------------------------------------------
[INFO]:-Unsynchronized Segment Pairs
[INFO]:-   Current Primary   Port    WAL sync remaining bytes   Mirror   Port
[INFO]:-   sdw3              10000   Unknown                    sdw4     10500
[INFO]:-   sdw3              10001   Unknown                    sdw4     10501
[INFO]:-   sdw3              10500   Unknown                    sdw4     10000
[INFO]:-   sdw3              10501   Unknown                    sdw4     10001
[INFO]:-----------------------------------------------------
[INFO]:-Downed Segments (may include segments where status could not be retrieved)
[INFO]:-   Segment   Port    Config status   Status
[INFO]:-   sdw4      10500   Down            Down in configuration
[INFO]:-   sdw4      10501   Down            Down in configuration
[INFO]:-   sdw4      10000   Down            Down in configuration
[INFO]:-   sdw4      10001   Down            Down in configuration

Logging

Expand

Start and stop a cluster

Start a cluster

Start master instance only

Stop a cluster

Cluster shutdown modes

Stop client queries and processes

Stop segments on a host

Restart a cluster

Reload cluster configuration

Check cluster state