[Xavier-Do]

Runbot currently work by sharing a postgresql cluster across multiple build, convenient to share, maintain and re-use database but this is not perfect for isolation. Mainly, the cpu limits of docker is not reflected to the PostgreSQL server, meaning that high intensive cpu/memory operation on postgres can exhaust resources to other builds, making the test time vary a lot depending on the load for some builds. This also cause issues where the server load is high making the auto-vaccum behavior unpredictable.

With this change the docker limits will be applied to the PostgreSQL server and we will be able to tweak the config per build if needed (memory, vacuum, ...).

The main change is the introduction of an entrypoint in the docker image. This entrypoint does not depend on the runbot infrastructure and should be generic, to allow to use the image locally. When started, the docker image entrypoint will create/restore a cluster if needed and start postgress. On stop, it will export the cluster if requested.
The entrypoint logic can be skipped by mounting /var/run/postgresql in the docker, forcing the docker file to use another existing socket. This will help in the transition between the two options. a parameter use_host_postgres can be given in the params to prevent mounting /var/run/postgresql in the docker image, as well as a param on the dockerfile, allowing to axis to migrate to the full dockerized postgress: one using the trigger, the other using the docker image config. In the future all of that will be most likely removed to only support docker in postgress.

A little "hacky" but this files are only mounted if PG_VERSION is defined, to avoid to add an entry point in images that don't have/need a pg_server.

restore

While on it a few changes are made in the restore step, in order to make it easier to debug and configure. As a side effect some changes of behavior are expected:

• the docker file that will be used to restore will be the same as the source build
• restore_rename_db_suffix is now only applied when downloading a dump from an url
• generally, once a reference build is found the behavior to select a suffix to download and a target name should be more coherent, only the dump_url part is different.
  - in general this change will need in depth testing and most likely need to adapt some existing restore

Upgrade cluster

A typical upgrade flow will be like this

• restore a 19 database
• run pre tests in 19
• run upgrade to 20
• run post test in 20

Since the Docker 19 used in the first two can be in a different ubuntu version, the postgresl version is not always the same between the pre steps and the upgrade step. The cluster will be in a different place if it is the case, PGDATA=/data/build/postgres/${PG_VERSION} so if the PG_VERSION changes, we need to restore a database exported from the pre. In this case we can only import the sql file

The current version is far from ideal, we need to explicitly have a zip generation in the previous step, and this will generate a zip file, but detect that we may need the raw dump and leave the files in place. This will work for now but could be replace by a more precise database export, only if the next step will really need it, without always generating the zip of the database and the cluster. In the context of the pre step, it makes sense. We could also want to immediately restore the database in the right docker version if the next won't use the same docker-file. Some cleanup should be done here, most likely by introducing the pre test in the runbot code to make everything more formal.

[bepro-odoo]

the start/stop scripts are not valid in dash-dialect. dash is the interpreter mode selected when "/bin/sh"-ing in ubuntu resolute containers, verified with runbot image odoo:DockerMaster.future (aka DockerMasterPostgres).

check yourself running shellcheck;

shellcheck --norc --shell dash /tmp/start_postgres.sh

Easy fix is to add the shebang for bash at the top of scripts

#!/bin/bash

[Xavier-Do]

The scripts are not executable and explicitly called with bash in the entrypoint. But I don't mind adding the shebang.