The official recommendation for creating multiple databases is as follows:

If you would like to do additional initialization in an image derived from this one, add one or more *.sql, *.sql.gz, or *.sh scripts under /docker-entrypoint-initdb.d (creating the directory if necessary). After the entrypoint calls initdb to create the default postgres user and database, it will run any *.sql files and source any *.sh scripts found in that directory to do further initialization before starting the service.

This directory contains a script to create multiple databases using that mechanism.

Clone the repository, mount its directory as a volume into /docker-entrypoint-initdb.d and declare database names separated by commas in POSTGRES_MULTIPLE_DATABASES environment variable as follows (github actions syntax):

postgres:
    image: postgres:12
    volumes:
        - docker-postgresql-multiple-databases:/docker-entrypoint-initdb.d
    env:
        POSTGRES_MULTIPLE_DATABASES: db1,db2
        POSTGRES_USER: postgres
        POSTGRES_PASSWORD: postgres

If you need to use non-standard database names (hyphens, uppercase letters etc), quote them in POSTGRES_MULTIPLE_DATABASES:

    environment:
        - POSTGRES_MULTIPLE_DATABASES: "test-db-1","test-db-2"