Support user-supplied sidecars for SGCluster pods
Some applications might want to access Postgres, or its filesystem, or part of other components, directly (i.e. not through the Postgres wire protocol). For this cases, highly user dependent, we might want to add the possibility to "inject" a user sidecar into the SGCluster pods.
This could potentially imply many options like:
- Container image.
- Annotations.
- Environment variables, config maps?
- Inject some additional env vars to the container, like the postgres superuser and password.
- Potentially access the Postgres filesystem (PGDATA) and/or the logs, etc.
- Resource limits for the user-added sidecar, to control its potential impact.
Proposal
apiVersion: stackgres.io/v1
kind: SGCluster
spec:
postgresServices:
primary:
customPorts: # The list of extra ports that will be exposed by the primary service
- <ServicePort> # See https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#serviceport-v1-core
replicas:
customPorts: # The list of extra ports that will be exposed by the replicas service
- <ServicePort> # See https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#serviceport-v1-core
pods:
customVolumes:
- <Volume> # See https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#volume-v1-core
customInitContainers:
- <Container> # See https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#container-v1-core
customContainers:
- <Container> # See https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#container-v1-core
Sections .spec.postgresServices.primary.customPorts
and .spec.postgresServices.replicas.customPorts
will allow to expose any custom service defined by a custom sidecars. We may think about renaming section .spec.postgresServices
to .spec.services
in a future version of SGCluster
CRD since we will expose ports from custom containers and user may not want to expose Postgres ports at all but just those of their custom services.
The .spec.pods.customContainers[]
and .spec.pods.customInitContainers[]
are the same as the Container
object present in the Pod resource under section .spec.containers[]
and .spec.initContainers[]
.
The .name
field in Container
object will forbid using any container name that is used by StackGres.
~~We will forbid following fields:
-
.securityContext
: The users of the container will use the main security context and will be replaced by mounting/etc/passwd
,/etc/shadow
,/etc/groups
and/etc/gshadow
. This will allow to use postgres UNIX socket and the data volume with the same user as postgres. Also it will allow to use the same container in OpenShift with arbitrary user ids. -
.stdin
: This will be left unset -
.stdinOnce
: This will be left unset -
.terminationMessagePath
: This will be left unset -
.terminationMessagePolicy
: This will be left unset -
.tty
: This will be left unset -
.volumeDevices
: This will be left unset since StatefulSet can not work with it. -
.resources
: This will be handled bySGInstanceProfile
(see also #144 (closed)) and first implementation should be the same as for other containers and init containers other thanpatroni
. We will open an issue to implement resources for custom containers.~~
Instead of forbidding we should document the fields that should not be touch and why. This is a very advanced feature and user that does not know how to define correctly a container and how StackGres internal works will be warned that using such feature may break the cluster in unexpected ways.
We may also forbid reduce definition of following fields in the UI to make a first implementation easier:
.lifecycle
.startupProbe
.readinessProbe
.livenessProbe
User will be able to use any volume that is defined in the StatefulSet including the data PV by setting the section .volumeMounts[]
in Container
object. To use the custom volumes defined in the section .spec.pods.customVolumes
the names user in the .volumeMounts[]
section of Container
object will have to be prepended by the custom-
prefix string (this is to avoid collisions). This will be documented properly in the SGCluster
CRD to specify the names of such container and what they do.
The .name
and .ports[].name
fields of Container
object as the .name
and .targetPort
fields of ServicePort
object will also be prepended with the custom-
string prefix to avoid collisions.
Acceptance Criteria
-
Implement the feature -
Create tests -
Documentation