Open search page
Hello, I’m DocuDroid!
Open Ask AI page
Submitting feedback
Thank you for rating our AI Search!
We would be grateful if you could share your thoughts so we can improve our AI Search for you and other readers.
GitHub

plcontainer

The plcontainer utility installs Docker images and manages the PL/Container configuration. The utility consists of two sets of commands:

  • image commands manage Docker images on the Greengage DB system hosts.

  • runtime commands manage the PL/Container configuration file on the Greengage DB instances.

You can add Docker image information to the PL/Container configuration file including the image name, location, and shared folder information. You can also edit the configuration file.

To configure PL/Container to use a Docker image, you install the Docker image on all the Greengage DB hosts and then add configuration information to the PL/Container configuration.

PL/Container configuration values, such as image names, runtime IDs, and parameter values and names are case-sensitive.

Syntax

plcontainer [ <command> ] [-h | --help [<command>]] [ --verbose ]

where command is one of the following:

image-add { -f | --file } <image_file> [ -ulc | --use_local_copy ]
image-add { -u | --URL } <image_url>

image-delete { -i | --image } <image_name>

image-list

runtime-add { -r | --runtime } <runtime_id>
            { -i | --image } <image_name>
            { -l | --language } { python | python3 | r }
            [ { -v | --volume } <shared_volume> [ { -v | --volume } <shared_volume> ... ] ]
            [ { -s | --setting } <param>=<value> [ { -s | --setting } <param>=<value> ... ] ]

runtime-replace { -r | --runtime } <runtime_id>
                { -i | --image } <image_name>
                { -l | --language } { python | python3 | r }
                [ { -v | --volume } <shared_volume> [ { -v | --volume } <shared_volume> ... ] ]
                [ { -s | --setting } <param>=<value> [ { -s | --setting } <param>=<value> ... ] ]

runtime-show { -r | --runtime } <runtime_id>

runtime-delete { -r | --runtime } <runtime_id>

runtime-edit [ { -e | --editor } <editor> ]

runtime-backup { -f | --file } <config_file>

runtime-restore { -f | --file } <config_file>

runtime-verify

Commands and options

image-add <location>

Install a Docker image on the Greengage DB hosts. Specify either the location of the Docker image file on the host or the URL to the Docker image. These are the supported location options:

  • {-f | --file} <image_file> — specify the file system location of the Docker image tar archive file on the local host. This example specifies an image file in the gpadmin user’s home directory: /home/gpadmin/test_image.tar.gz.

  • {-u | --URL} <image_url> — specify the URL of the Docker repository and image. This example URL points to a local Docker repository: 192.168.0.1:5000/images/plc_python_shared:latest.

By default, the image-add command copies the image to each Greengage DB segment and standby master host, and installs the image. When you specify an image_file and provide the [-ulc | --use_local_copy] option, plcontainer installs the image only on the host on which you run the command.

After installing the Docker image, use the runtime-add command to configure PL/Container to use the Docker image.

image-delete {-i | --image} <image_name>

Remove an installed Docker image from all Greengage DB hosts.

image-list

List the Docker images installed on the host. The command lists only the images on the local host, not remote hosts. The command lists all installed Docker images, including images installed with Docker commands.

runtime-add <runtime_id>

Add configuration information to the PL/Container configuration file on all Greengage DB hosts. If the specified runtime_id exists, the utility returns an error and the configuration information is not added.

These are the supported options:

  • {-i | --image} <image_name> — (required) specify the full Docker image name, including the tag, that is installed on the Greengage DB hosts.

The utility returns a warning if the specified Docker image is not installed.

The plcontainer image-list command displays installed image information including the name and tag (the Repository and Tag columns).

{-l | --language} python | python3 | r

(Required) Specify the PL/Container language type. Supported values are python (PL/Python using Python 2), python3 (PL/Python using Python 3), and r (PL/R). When adding configuration information for a new runtime, the utility adds a startup command to the configuration based on the language you specify.

Startup command for the Python 2 language:

/clientdir/pyclient.sh

Startup command for the Python 3 language:

/clientdir/py3client.sh

Startup command for the R language:

/clientdir/rclient.sh
{-r | --runtime} <runtime_id>

(Required) Add the runtime ID. When adding a runtime element in the PL/Container configuration file, this is the value of the id element in the PL/Container configuration file. Maximum length is 63 Bytes.

You specify the name in the Greengage DB UDF on the # container line.

{-s | --setting} <param>=<value>

(Optional) Specify a setting to add to the runtime configuration information. You can specify this option multiple times. The setting applies to the runtime configuration specified by runtime_id. These are valid parameters:

  • cpu_share — set the CPU limit for each container in the runtime configuration. The default value is 1024. The value is a relative weighting of CPU usage compared to other containers.

  • memory_mb — set the memory limit for each container in the runtime configuration. The default value is 1024. The value is an integer that specifies the amount of memory in MB.

  • resource_group_id — assign the specified resource group to the runtime configuration. The resource group limits the total CPU and memory resource usage for all containers that share this runtime configuration. You must specify the groupid of the resource group.

  • roles — specify the Greengage DB roles that are allowed to run a container for the runtime configuration. You can specify a single role name or a comma-separated list of role names. The default is no restriction.

  • use_container_logging — activate or deactivate Docker logging for the container. The value yes activates logging. The value no deactivates logging (the default).

    The Greengage DB server configuration parameter log_min_messages controls the log level. The default log level is warning.

  • enable_network — activate or deactivate network access for the UDF container. The value yes enables UDFs to access the network. The value no deactivates network access (the default).

{-v | --volume} <shared_volume>

(Optional) Specify a Docker volume to bind mount. You can specify this option multiple times to define multiple volumes.

The format for a shared volume: <host-dir>:<container-dir>:[rw|ro]. The information is stored as attributes in the shared_directory element of the runtime element in the PL/Container configuration file:

  • host-dir — absolute path to a directory on the host system. The Greengage DB administrator user (gpadmin) must have appropriate access to the directory.

  • container-dir — absolute path to a directory in the Docker container.

  • [rw|ro] — read-write or read-only access to the host directory from the container.

When adding configuration information for a new runtime, the utility adds this read-only shared volume information:

/usr/local/gpdb/bin/plcontainer_clients:/clientdir:ro

If needed, you can specify other shared directories. The utility returns an error if the specified container-dir is the same as the one that is added by the utility, or if you specify multiple shared volumes with the same container-dir.

CAUTION

Allowing read-write access to a host directory requires special considerations:

  • When specifying read-write access to a host directory, ensure that the specified host directory has the correct permissions.

  • When running PL/Container user-defined functions, multiple concurrent Docker containers that are running on a host could change data in the host directory.

Ensure that the functions support multiple concurrent access to the data in the host directory.

runtime-backup {-f | --file} <config_file>

Copy the PL/Container configuration file to the specified file on the local host.

runtime-delete {-r | --runtime} <runtime_id>

Remove runtime configuration information in the PL/Container configuration file on all Greengage DB instances. The utility returns a message if the specified runtime_id does not exist in the file.

runtime-edit [{-e | --editor} <editor>]

Edit the XML file plcontainer_configuration.xml with the specified editor. The default editor is vi.

Saving the file updates the configuration file on all Greengage DB hosts. If errors exist in the updated file, the utility returns an error and does not update the file.

runtime-replace <runtime_id>

Replace runtime configuration information in the PL/Container configuration file on all Greengage DB instances. If the runtime_id does not exist, the information is added to the configuration file. The utility adds a startup command and shared directory to the configuration.

runtime-restore {-f | --file} <config_file>

Replace information in the PL/Container configuration file plcontainer_configuration.xml on all Greengage DB instances with the information from the specified file on the local host.

runtime-show [{-r | --runtime} <runtime_id>]

Display formatted PL/Container runtime configuration information. If a runtime_id is not specified, the configuration for all runtime IDs is displayed.

runtime-verify

Check the PL/Container configuration information on the Greengage DB instances with the configuration information on the master. If the utility finds inconsistencies, you are prompted to replace the remote copy with the local copy. The utility also performs XML validation.

-h | --help [<command>]

Display help text. If specified without a command, displays help for all plcontainer commands. If specified with a command, displays help for the command.

--verbose

Enable verbose logging for the command.

Examples

These are examples of common commands to manage PL/Container:

  • Install a Docker image on all Greengage DB hosts. This example loads a Docker image from a file. The utility displays progress information on the command line as the utility installs the Docker image on all the hosts:

    $ plcontainer image-add -f plcontainer-python3-image-2.4.0-gp6.tar.gz

    After installing the Docker image, you add or update a runtime entry in the PL/Container configuration file to give PL/Container access to the Docker image to start Docker containers.

  • Install the Docker image only on the local Greengage DB host:

    $ plcontainer image-add -f plcontainer-python3-image-2.4.0-gp6.tar.gz --use_local_copy

  • Add a container entry to the PL/Container configuration file. This example adds configuration information for a Python 3 runtime:

    $ plcontainer runtime-add -r plc_python_shared -i python39.alpine:latest -l python3

    The utility displays progress information on the command line as it adds the runtime configuration to the configuration file and distributes the updated configuration to all instances.

  • Show specific runtime with given runtime ID in configuration file:

    $ plcontainer runtime-show -r plc_python_shared

  • Edit the configuration in an interactive editor of your choice. This example edits the configuration file with the vim editor:

    $ plcontainer runtime-edit -e vim

    When you save the file, the utility displays progress information on the command line as it distributes the file to the Greengage DB hosts.

  • Save the current PL/Container configuration to a file. This example saves the file to the local file /home/gpadmin/saved_plc_config.xml:

    $ plcontainer runtime-backup -f /home/gpadmin/saved_plc_config.xml

  • Overwrite PL/Container configuration file with an XML file. This example replaces the information in the configuration file with the information from the file in the /home/gpadmin directory.

    $ plcontainer runtime-restore -f /home/gpadmin/new_plcontainer_configuration.xml

    The utility displays progress information on the command line as it distributes the updated file to the Greengage DB instances.

PL/Container configuration file

The Greengage DB utility plcontainer manages the PL/Container configuration files. The utility ensures that the configuration files are consistent across the master and segment instances.

CAUTION

Modifying the configuration files on the segment instances without using the utility might create different, incompatible configurations on different Greengage DB segments that could cause unexpected behavior.

PL/Container maintains a configuration file plcontainer_configuration.xml in the data directory of all Greengage DB segments.

This query lists the Greengage DB system data directories:

SELECT hostname, datadir
FROM gp_segment_configuration;

A sample PL/Container configuration file is in $GPHOME/share/postgresql/plcontainer.

In an XML file, names, such as element and attribute names, and values are case-sensitive.

In this XML file, the root element configuration contains one or more runtime elements. You specify the id of the runtime element in the # container: line of a PL/Container function definition.

This is an example file. Note that all XML elements, names, and attributes are case sensitive.

<?xml version="1.0" ?>
<configuration>
    <runtime>
        <id>plc_python_shared</id>
        <image>python39.alpine:latest</image>
        <command>/clientdir/py3client.sh</command>
        <shared_directory access="ro" container="/clientdir" host="/usr/local/gpdb/bin/plcontainer_clients"/>
    </runtime>
    <runtime>
        <id>plc_r_shared</id>
        <image>r.alpine:latest</image>
        <command>/clientdir/rclient.sh</command>
        <shared_directory access="ro" container="/clientdir" host="/usr/local/gpdb/bin/plcontainer_clients"/>
        <setting use_container_logging="yes"/>
        <setting enable_network="no"/>
        <setting roles="gpadmin,user1"/>
    </runtime>
</configuration>

These are the XML elements and attributes in a PL/Container configuration file.

configuration

Root element for the XML file.

runtime

One element for each specific container available in the system. These are child elements of the configuration element.

id

(Required) The value is used to reference a Docker container from a PL/Container user-defined function. The id value must be unique in the configuration. The id must start with a character or digit (a-z, A-Z, or 0-9) and can contain characters, digits, or the characters _ (underscore), . (period), or - (dash). Maximum length is 63 Bytes.

The id specifies which Docker image to use when PL/Container creates a Docker container to run a user-defined function.

image

(Required) The value is the full Docker image name, including image tag. Specify it the same way you specify it for starting this container in Docker. Configuration allows you to have many container objects referencing the same image name. In this way, in Docker they would be represented by identical containers.

For example, you might have two runtime elements, with different id elements both referencing the Docker image python39.alpine:latest.

command

(Required) The value is the command to be run inside the container to start the client process inside the container. When creating a runtime element, the plcontainer utility adds a command element based on the language (the -l option).

command element for the Python 2 language:

<command>/clientdir/pyclient.sh</command>

command element for the Python 3 language:

<command>/clientdir/py3client.sh</command>

command element for the R language:

<command>/clientdir/rclient.sh</command>

You should modify the value only if you build a custom container and want to implement some additional initialization logic before the container starts.

NOTE

This element cannot be set with the plcontainer utility. You can update the configuration file with the plcontainer runtime-edit command.

shared_directory

(Optional) This element specifies a Docker shared volume for a container with access information. Multiple shared_directory elements are allowed. Each shared_directory element specifies a single shared volume. XML attributes for the shared_directory element:

  • host — a directory location on the host system.

  • container — a directory location inside the container.

  • access — access level to the host directory, which can be either ro (read-only) or rw (read-write).

When creating a runtime element, the plcontainer utility adds a shared_directory element:

<shared_directory access="ro" container="/clientdir" host="/usr/local/gpdb/bin/plcontainer_clients"/>

For each runtime element, the container attribute of the shared_directory elements must be unique. For example, a runtime element cannot have two shared_directory elements with attribute container="/clientdir".

CAUTION

Allowing read-write access to a host directory requires special consideration. When specifying read-write access to a host directory, ensure that the specified host directory has the correct permissions. When running PL/Container user-defined functions, multiple concurrent Docker containers that are running on a host could change data in the host directory. Ensure that the functions support multiple concurrent access to the data in the host directory.

setting

(Optional) This element specifies Docker container configuration information. Each setting element contains one attribute. The element attribute specifies logging, memory, or networking information. For example, this element enables logging:

<setting use_container_logging="yes"/>

These are the valid attributes:

cpu_share

(Optional) Specify the CPU usage for each PL/Container container in the runtime. The value of the attribute is a positive integer. The default value is 1024. The value is a relative weighting of CPU usage compared to other containers.

For example, a container with a cpu_share of 2048 is allocated double the CPU slice time compared with a container with the default value of 1024.

memory_mb="<size>"

(Optional) Specify the amount of memory, in MB, that each container is allowed to use. Each container starts with this amount of RAM and twice the amount of swap space. The container memory consumption is limited by the host system cgroups configuration, which means in case of memory overcommit, the container is terminated by the system.

resource_group_id="<rg_groupid>"

(Optional) Specify the groupid of the resource group to assign to the PL/Container runtime. The resource group limits the total CPU and memory resource usage for all running containers that share this runtime configuration. You must specify the groupid of the resource group. If you do not assign a resource group to a PL/Container runtime configuration, its container instances are limited only by system resources.

roles="<list_of_roles>"

(Optional) The value is a Greengage DB role name or a comma-separated list of roles. PL/Container runs a container that uses the PL/Container runtime configuration only for the listed roles. If the attribute is not specified, any Greengage DB role can run an instance of this container runtime configuration. For example, you create a UDF that specifies the plcontainer language and identifies a # container: runtime configuration that has the roles attribute set. When a role (user) runs the UDF, PL/Container checks the list of roles and runs the container only if the role is on the list.

use_container_logging="{yes | no}"

(Optional) Activate or deactivate Docker logging for the container. The attribute value yes activates logging. The attribute value no deactivates logging (the default).

enable_network="{yes | no}"

(Optional) Activate or deactivate network access for the UDF container. The attribute value yes enables UDFs to access the network. The attribute value no deactivates network access (the default).

Update the PL/Container configuration

You can add a runtime element to the PL/Container configuration file with the plcontainer runtime-add command. The command options specify information such as the runtime ID, Docker image, and language. You can use the plcontainer runtime-replace command to update an existing runtime element. The utility updates the configuration file on the master and all segment instances.

The PL/Container configuration file can contain multiple runtime elements that reference the same Docker image specified by the XML element image. In the example configuration file, the runtime elements have id values of plc_python_128 and plc_python_256, both referencing the Docker image python39.alpine:latest. The first runtime element is defined with a 128 MB RAM limit and the second one with a 256 MB RAM limit.

<configuration>
  <runtime>
    <id>plc_python_128</id>
    <image>python39.alpine:latest</image>
    <command>/clientdir/py3client.sh</command>
    <shared_directory access="ro" container="/clientdir" host="/usr/local/gpdb/bin/plcontainer_clients"/>
    <setting memory_mb="128"/>
  </runtime>
  <runtime>
    <id>plc_python_256</id>
    <image>python39.alpine:latest</image>
    <command>/clientdir/py3client.sh</command>
    <shared_directory access="ro" container="/clientdir" host="/usr/local/gpdb/bin/plcontainer_clients"/>
    <setting memory_mb="256"/>
    <setting resource_group_id="24993"/>
  </runtime>
</configuration>

Configuration changes that are made with the utility are applied to the XML files on all Greengage DB segments. However, PL/Container configurations of currently running sessions use the configuration that existed during session start up. To update the PL/Container configuration in a running session, run this command in the session:

SELECT * FROM plcontainer_refresh_config;

The command runs a PL/Container function that updates the session configuration on the master and segment instances.

Previous
pgbouncer
Next
psql