gplogfilter

Contents

Synopsis
Description
Options
Log format
Examples
See also

Searches through Greengage DB log files for specified entries.

gplogfilter [ <timestamp_options> ]
            [ <pattern_options> ]
            [ <output_options> ]
            [ <input_options> ]
            [ <input_file> ]

gplogfilter -? | -h | --help

gplogfilter --version

Description

The gplogfilter utility can be used to search through a Greengage DB log file for entries matching the specified criteria. If an input file is not supplied, then gplogfilter will use the MASTER_DATA_DIRECTORY environment variable to locate the Greengage DB master log file in the standard logging location. To read from standard input, use a dash (-) as the input file name. Input files may be compressed using gzip. In an input file, a log entry is identified by its timestamp in YYYY-MM-DD [hh:mm[:ss]] format.

You can also use gplogfilter to search through all segment log files at once by running it through the gpssh utility. For example, to display the last three lines of each segment log file:

$ gpssh -f seg_host_file
=> source /usr/local/gpdb/greengage_path.sh
=> gplogfilter -n 3 /data1/*/log/gpdb*.csv

By default, the output of gplogfilter is sent to standard output. Use the -o option to send the output to a file or a directory. If you supply an output file name ending in .gz, the output file will be compressed by default using maximum compression. If the output destination is a directory, the output file is given the same name as the input file.

-b <datetime> | --begin=<datetime>

Specify a starting date and time to begin searching for log entries in the format of YYYY-MM-DD [hh:mm[:ss]].

If a time is specified, the date and time must be enclosed in either single or double quotes. This example encloses the date and time in single quotes:

$ gplogfilter -b '2025-05-23 14:33'

-e <datetime> | --end=<datetime>

Specify an ending date and time to stop searching for log entries in the format of YYYY-MM-DD [hh:mm[:ss]].

If a time is specified, the date and time must be enclosed in either single or double quotes. This example encloses the date and time in single quotes:

$ gplogfilter -e '2025-05-23 14:33'

-d <time> | --duration=<time>

Specify a time duration to search for log entries in the format of [hh][:mm[:ss]]. If used without either the -b or -e option, uses the current time as a basis.

-c i [gnore] | r [espect] | --case=i [gnore] | r [espect]: Matching of alphabetic characters is case-sensitive by default unless preceded by the --case=ignore option.
-C '<string>' | --columns='<string>': Select specific columns from the log file. Specify the desired columns as a comma-delimited string of column numbers beginning with 1, where the second column from left is 2, the third is 3, and so on.
-f '<string>' | --find='<string>': Find the log entries containing the specified string.
-F '<string>' | --nofind='<string>': Reject the log entries containing the specified string.
-m <regex> | --match=<regex>: Find log entries that match the specified Python regular expression. See Regular expression operations for Python regular expression syntax.
-M <regex> | --nomatch=<regex>: Reject log entries that match the specified Python regular expression. See Regular expression operations for Python regular expression syntax.
-t | --trouble: Find only the log entries that have ERROR:, FATAL:, or PANIC: in the first line.

-n <integer> | --tail=<integer>: Limit the output to the last <integer> of qualifying log entries found.
-s <offset> [<limit>] | --slice=<offset> [<limit>]: From the list of qualifying log entries, return the limit number of entries starting at the offset entry number, where an offset of zero denotes the first entry in the result set and an offset of any number greater than zero counts back from the end of the result set.
-o <output_file> | --out=<output_file>: Write the output to the specified file or directory location instead of STDOUT.
-z 0-9 | --zip=0-9: Compress the output file to the specified compression level using gzip, where 0 is no compression and 9 is maximum compression. If you supply an output file name ending in .gz, the output file will be compressed by default using maximum compression.
-a | --append: If the output file already exists, append to the file instead of overwriting it.

<input_file>: The name of the input log file(s) to search through. If an input file is not supplied, gplogfilter will use the MASTER_DATA_DIRECTORY environment variable to locate the Greengage DB master log file. To read from standard input, use a dash (-) as the input file name.
-u | --unzip: Uncompress the input file using gunzip. If the input file name ends in .gz, it will be uncompressed by default.

--help: Display help.
--version: Display the version of this utility.

Field number Field name Data type Description

Field number	Field name	Data type	Description
1	event_time	timestamp with time zone	The time when the log entry was written to the log. Example: `2025-02-28 07:22:42.388447 UTC`
2	user_name	varchar(100)	The database user name. Examples: `gpadmin`, `alice`
3	database_name	varchar(100)	The database name. Examples: `postgres`, `crm`
4	process_id	varchar(10)	The system process ID (prefixed with `p`). Example: `p2062`
5	thread_id	varchar(50)	The thread ID (prefixed with `th`). Example: `th-1924802432`
6	remote_host	varchar(100)	On the master, represents the client host address. On segments, returns the master host address. Examples: `192.168.10.55`, `[local]`
7	remote_port	varchar(10)	The remote port number. Example: `49206`
8	session_start_time	timestamp with time zone	The time when the session connection was initiated
9	transaction_id	int	The top-level transaction ID on the master. This ID is the parent of any subtransactions. On segments, it identifies a local transaction that participates in the distributed transaction
10	gp_session_id	text	The session identifier (prefixed with `con`)
11	gp_command_count	text	The number of commands the master has received from the client (prefixed with `cmd`). Example: `cmd9`
12	gp_segment	text	The segment content identifier. Prefixed with `seg` for primaries or `mir` for mirrors. The master always has a content ID of `-1`. Examples: `seg-1`, `seg1`
13	slice_id	text	The slice ID, which represents the portion of the query plan being run
14	distr_tranx_id	text	The distributed transaction ID
15	local_tranx_id	text	The local transaction ID
16	sub_tranx_id	text	The subtransaction ID
17	event_severity	varchar(10)	The event severity. Examples: `LOG`, `ERROR`, `FATAL`
18	sql_state_code	varchar(10)	The SQL state code associated with the log message. Example: `00000`
19	event_message	text	The text of the log or error message. Examples: `statement: CREATE DATABASE crm;`, `connection authorized: user=alice database=crm`
20	event_detail	text	The detailed message text associated with an error or warning message
21	event_hint	text	A hint message text associated with an error or warning message
22	internal_query	text	The internally generated query text
23	internal_query_pos	int	The cursor position within the internally generated query text
24	event_context	text	The context in which this message is generated
25	debug_query_string	text	A user-supplied query string with full details for debugging
26	error_cursor_pos	int	The cursor position within the query string
27	func_name	text	The function in which this message is generated
28	file_name	text	The internal code file where the message originated. Example: `postgres.c`
29	file_line	int	The line of the code file where the message originated. Example: `1688`
30	stack_trace	text	The stack trace text associated with this message

event_time

timestamp with time zone

The time when the log entry was written to the log.

Example: 2025-02-28 07:22:42.388447 UTC

user_name

varchar(100)

The database user name.

Examples: gpadmin, alice

database_name

varchar(100)

The database name.

Examples: postgres, crm

process_id

varchar(10)

The system process ID (prefixed with p).

Example: p2062

thread_id

varchar(50)

The thread ID (prefixed with th).

Example: th-1924802432

remote_host

varchar(100)

On the master, represents the client host address. On segments, returns the master host address.

Examples: 192.168.10.55, [local]

remote_port

varchar(10)

The remote port number.

Example: 49206

session_start_time

timestamp with time zone

The time when the session connection was initiated

transaction_id

int

The top-level transaction ID on the master. This ID is the parent of any subtransactions. On segments, it identifies a local transaction that participates in the distributed transaction

gp_session_id

text

The session identifier (prefixed with con)

gp_command_count

text

The number of commands the master has received from the client (prefixed with cmd).

Example: cmd9

gp_segment

text

The segment content identifier. Prefixed with seg for primaries or mir for mirrors. The master always has a content ID of -1.

Examples: seg-1, seg1

slice_id

text

The slice ID, which represents the portion of the query plan being run

distr_tranx_id

text

The distributed transaction ID

local_tranx_id

text

The local transaction ID

sub_tranx_id

text

The subtransaction ID

event_severity

varchar(10)

The event severity.

Examples: LOG, ERROR, FATAL

sql_state_code

varchar(10)

The SQL state code associated with the log message.

Example: 00000

event_message

text

The text of the log or error message.

Examples: statement: CREATE DATABASE crm;, connection authorized: user=alice database=crm

event_detail

text

The detailed message text associated with an error or warning message

event_hint

text

A hint message text associated with an error or warning message

internal_query

text

The internally generated query text

internal_query_pos

int

The cursor position within the internally generated query text

event_context

text

The context in which this message is generated

debug_query_string

text

A user-supplied query string with full details for debugging

error_cursor_pos

int

The cursor position within the query string

func_name

text

The function in which this message is generated

file_name

text

The internal code file where the message originated.

Example: postgres.c

file_line

int

The line of the code file where the message originated.

Example: 1688

stack_trace

text

The stack trace text associated with this message

Display the last three error messages in the master log file:

$ gplogfilter -t -n 3

Display all log messages in the master log file timestamped in the last 10 minutes:

$ gplogfilter -d :10

Display log messages in the master log file containing the string |con6 cmd11|:

$ gplogfilter -f '|con6 cmd11|'

Using gpssh, run gplogfilter on the segment hosts and search for log messages in the segment log files containing the ERROR, FATAL, or PANIC messages and save output to a file:

$ gpssh -f hostfile_segment_hosts -e " \
      source /usr/local/gpdb/greengage_path.sh && \
      gplogfilter /data1/*/*/pg_log/gpdb*.csv \
      --trouble \
  " > seglog.out

gpssh
gpscp

gpload

gpmemreport

gplogfilter

Synopsis

Description

Options

Timestamp options

Pattern matching options

Output options

Input options

Informational options

Log format

Examples

See also