How to load custom config (e.g. gwc-gs.xml) into pgconfig at GeoServer Cloud startup?

Hello,

In my pgconfig database, I can see entries like this in the resourcestore table:

image

I would like to know how I can preload my own configuration (such as a custom gwc-gs.xml file) into the database when starting GeoServer Cloud for the first time. From my tests, I noticed that the database gets populated automatically when the web-ui service starts. So it seems the initialization happens at that point.

See my compose file:

# Define shared environment variables for all services
x-variables:
  environment: &common_env
    SPRING_PROFILES_ACTIVE: "pgconfig,acl"
    GEOWEBCACHE_CACHE_DIR: /data/geowebcache
    #    GEOSERVER_DATA_DIRECTORY_ENABLE: "true"
    #    SPRING_PROFILES_ACTIVE: "fileconfig"
    #GEOSERVER_DATA_DIRECTORY_ENABLE: true
    ACL_URL: http://acl:8080/acl/api
    ACL_USERNAME: admin
    ACL_PASSWORD: s3cr3t

x-gs-dependencies: &gs-dependencies
  rabbitmq:
    condition: service_healthy
  discovery:
    condition: service_healthy
  pgconfigdb:
    condition: service_started
  config:
    condition: service_healthy
  acl:
    condition: service_healthy

volumes:
  geowebcache_data: # Stores GeoWebCache tile data
  pgconfigdb_data:  # Stores PostgreSQL data for pgconfig backend

services:
  pgconfigdb:
    # use an image with linux/amd64 and linux/arm64 support. Doesn't need to be postgis
    image: imresamu/postgis:17-3.5
    shm_size: 2g
    environment:
      POSTGRES_DB: pgconfig
      POSTGRES_USER: pgconfig
      POSTGRES_PASSWORD: pgconfig
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U pgconfig"]
      interval: 5s
      timeout: 5s
      retries: 5
    volumes:
      - pgconfigdb_data:/var/lib/postgresql/data
    ports:
      - "5433:5432"  # expose le port PostgreSQL à l'hôte
    deploy:
      resources:
        limits:
          cpus: '4.0'
          memory: 4G

  acl:
    image: geoservercloud/geoserver-acl:2.3.1
    user: "1000:1000"
    depends_on:
      pgconfigdb:
        condition: service_healthy
      rabbitmq:
        condition: service_healthy
    environment:
      PG_HOST: pgconfigdb
      PG_PORT: 5432
      PG_DB: pgconfig
      PG_SCHEMA: acl
      PG_USERNAME: pgconfig
      PG_PASSWORD: pgconfig
      'ACL_USERS_ADMIN_PASSWORD': '{noop}s3cr3t'
      'ACL_USERS_GEOSERVER_PASSWORD': '{noop}s3cr3t'
      GEOSERVER_BUS_ENABLED: true
      RABBITMQ_HOST: rabbitmq
      RABBITMQ_PORT: 5672
      RABBITMQ_USER: guest
      RABBITMQ_PASSWORD: guest
    deploy:
      resources:
        limits:
          cpus: '4.0'
          memory: 2G

  rabbitmq:
    image: rabbitmq:4-management-alpine
    user: "1000:1000"
    restart: unless-stopped
    tmpfs:
      - /var/lib/rabbitmq:size=512m,mode=1777  # Store RabbitMQ data in memory
    healthcheck:
      test: rabbitmq-diagnostics is_running
      start_period: 10s
      interval: 15s
      timeout: 30s
      retries: 3
    deploy:
      resources:
        limits:
          cpus: '4.0'
          memory: 2G

  # Spring Cloud Config service, provides centralized configuration to all
  # microservices. Being a Discovery First Bootstrap configuration, it'll
  # register itself with the Eureka discovery service and can be scaled
  config:
    image: geoservercloud/geoserver-cloud-config:2.27.0.0
    user: 1000:1000 # set the userid:groupid the container runs as
    environment:
      # default to `native` loading the config embedded in /etc/geoserver
      # use `git` to fetch the config from a git repository, and CONFIG_GIT_URI to change
      # the default repository https://github.com/geoserver/geoserver-cloud-config.git
      SPRING_PROFILES_ACTIVE: native
      # If using the `git` profile, get the config from this tag
      SPRING_CLOUD_CONFIG_SERVER_GIT_DEFAULT_LABEL: v2.27.0.0
    # Uncomment to bind to a local filesystem directory if using the 'native' profile
    #volumes:
    #  - ./config:/etc/geoserver
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 256M

  discovery:
    image: geoservercloud/geoserver-cloud-discovery:2.27.0.0
    user: "1000:1000"
    depends_on:
      - config
    ports:
      - 8761:8761
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 256M

  # Application facade, provides a single entry point routing to all
  # microservices (e.g. http://localhost:9090/geoserver/cloud/wms, http://localhost:9090/geoserver/cloud/wfs, etc)
  gateway:
    image: geoservercloud/geoserver-cloud-gateway:2.27.0.0
    user: 1000:1000 # set the userid:groupid the container runs as
    depends_on:
      discovery:
        condition: service_healthy
    environment:
      # eat our own dogfood and set a base path
      GEOSERVER_BASE_PATH: /geoserver/cloud
    ports:
      - 9090:8080
    deploy:
      resources:
        limits:
          cpus: '4.0'
          memory: 1G

  geoserver_template:
    image: geoservercloud/geoserver-cloud-webui:2.27.0.0
    user: "1000:1000"
    environment:
      <<: *common_env
    volumes:
      - geowebcache_data:/data/geowebcache
    deploy:
      mode: replicated
      replicas: 1
      resources:
        limits:
          cpus: '2.0'
          memory: 1G

Hello @khercouet and welcome!

Generally I manage these by using the REST API of GeoServer (including GeoServer Cloud) to push the required files in the configuration (/resources endpoint).

If you don’t write your own script to do so, you can have a look to the different Python clients built around the REST API or you can also use the terraform provider we maintain to manage the configuration.

Regards
Alexandre

1 Like

Thank you for your quick response and the different solutions you provided.

Is it your repository right ? : terraform

Does that mean you have to restart the container after adding a resource to see the changes in the UI?

Yes it is this repo.

Depending on the resource you could have to restart some services to have it taken into account (but only the first time you upload it since after it is persisted into db).

What kind of customization have you in mind?

I’m currently thinking about the configuration of services like wms.xml and gwc-gs.xml.It would also be useful to have a way to automatically add tiles to the layers cached by GeoWebCache. I noted for that the api gwc/rest/layers (not sure how to use this)

The requirements may evolve over time.

For these files, a reload can be indeed required. Or you can start the web UI after your setup phase.

I am working on being able to configure the WMS service with the provider (since there is an API for it).

Regarding the seeding of tiles, you can use the GWC REST API but since it is an asynchronous task which can take some time, I suggest not to use the same pattern as changing the configuration.

Thank you for your advices, have nice weekend