dimanche 15 décembre 2019

Testing an application with multiple Docker containers

I have a pretty simple application interacting with different kinds of databases (supported by SQLAlchemy). I need to test if the application is working correctly with every required database engine, checking different versions of those engines and different Python versions.

My current solution is using Docker and Docker Compose. I defined a number of docker-compose.yml files and in my CI tool, I just start the tests for every single configuration file. More or less, it looks like that:

├── python-35
│   ├── postgresql-10
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   ├── postgresql-11
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   ├── postgresql-12
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   ├── postgresql-9
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   └── sqlite-3
│       ├── Dockerfile
│       └── docker-compose.yml
├── python-36
│   ├── postgresql-10
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   ├── postgresql-11
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   ├── postgresql-12
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   ├── postgresql-9
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   └── sqlite-3
│       ├── Dockerfile
│       └── docker-compose.yml
├── python-37
│   ├── postgresql-10
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   ├── postgresql-11
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   ├── postgresql-12
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   ├── postgresql-9
│   │   ├── Dockerfile
│   │   └── docker-compose.yml
│   └── sqlite-3
│       ├── Dockerfile
│       └── docker-compose.yml
└── python-38
    ├── postgresql-10
    │   ├── Dockerfile
    │   └── docker-compose.yml
    ├── postgresql-11
    │   ├── Dockerfile
    │   └── docker-compose.yml
    ├── postgresql-12
    │   ├── Dockerfile
    │   └── docker-compose.yml
    ├── postgresql-9
    │   ├── Dockerfile
    │   └── docker-compose.yml
    └── sqlite-3
        ├── Dockerfile
        └── docker-compose.yml

To run a single test suite, I use the following commands:

docker-compose --file=var/docker/python-35/postgresql-9/docker-compose.yml build --no-cache
docker-compose --file=var/docker/python-35/postgresql-9/docker-compose.yml run -w /app app_python_35_postgresql_9 /venv/bin/pytest
docker-compose --file=var/docker/python-35/postgresql-9/docker-compose.yml down

I have a few problems with that. The first one is that I need to define it like that in the first place. I find if a little hard to maintain. It is possible to start all the test suite in a loop, but it's still a pretty ugly thing to keep in the repository. The next problem is the performance. A single test suite takes 30 seconds to run. When I use Docker to build an image every single time, it takes 4 minutes. When I multiply it by 20 (the current number of configurations) it gives 80 minutes - this is how long it takes to run the tests after each push. I'm not sure if this is something that can be fixed, but it's worth to mention anyway.

There is one final problem though - it's not always working. When I run the full suite on my local mashine, it takes a while to complete, but it's working. When I do the same in my CI (TeamCity) - well, I have not managed to run the full test suite (20 configurations) without getting an error.

Here is sample docker-compose.yml:

version: '2'
services:
  db_python_35_postgresql_9:
    container_name: db_python_35_postgresql_9
    image: postgres:9.6.16-alpine
    restart: always
    ports:
      - 15432:15432
    environment:
      - POSTGRES_DB=app
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - PGPORT=15432
  app_python_35_postgresql_9:
    build:
      context: ../../../../
      dockerfile: ./var/docker/python-35/postgresql-9/Dockerfile
      args:
        MIGRATOR_CONNECTION_STRING: postgresql://postgres:postgres@db_python_35_postgresql_9:15432/app
    depends_on:
      - db_python_35_postgresql_9
    links:
      - db_python_35_postgresql_9

And here is sample error that I'm getting:

2 matches found based on name: network postgresql-9_default is ambiguous
Removing network postgresql-9_default
network postgresql-9_default is ambiguous (2 matches found based on name)
Process exited with code 1

It does not always occur at the same time. I remove the containers after each run with docker-compose down but it still tends to crash like that. I tried to rename the containers so now each one has a unique name (like db_python_35_postgresql_9 or app_python_35_postgresql_9) but it's still crashing.

Now, is there any way to do achieve what I need, but simpler? I'm not very advanced with Docker. Maybe it allows to do the same and avoiding the problems I have? I tried to read a bit about Kubernetes, but I'm not sure if that's the correct way to deal with tests like that. I could really use some help :-)

Aucun commentaire:

Enregistrer un commentaire