Recently, while prototyping a new side product on my own server, I looked for a faster way than manually spinning up services every time or dealing with complex deployment scripts. Docker Compose has always been a practical solution for me, especially for single-host environments, for quickly deploying and managing applications. In this post, I’ll share five fundamental approaches I frequently use to run your self-hosted applications more efficiently and reliably with Docker Compose.
Docker Compose is a tool for defining and running multi-container Docker applications. It allows you to configure all your application’s services, networks, and storage volumes from a single place using a YAML file. This way, you can spin up, stop, and manage applications with complex dependencies with a single command. This approach provides great convenience, especially for rapid prototyping, development environments, and small-scale production deployments.
1. Reduce Initial Setup Cost with Minimal Compose Files
When developing or deploying a self-hosted application, minimizing initial setup complexity is crucial. Often, all we need to get a service up and running is an image definition and perhaps a few port and volume settings. Trying to create a massive docker-compose.yaml file by thinking of every detail at first is both a waste of time and unnecessary fatigue. I usually start with the most basic requirements and iteratively develop the file as the application runs and needs become clearer.
For a typical web application consisting of a database, an API service, and a frontend, my initial docker-compose.yaml file usually looks like this:
version: '3.8'
services:
database:
image: postgres:15-alpine
restart: unless-stopped
environment:
POSTGRES_DB: myapp_db
POSTGRES_USER: myuser
POSTGRES_PASSWORD: mypassword
volumes:
- db_data:/var/lib/postgresql/data
api:
build: ./api
restart: unless-stopped
ports:
- "8000:8000"
environment:
DATABASE_URL: postgres://myuser:mypassword@database:5432/myapp_db
depends_on:
- database
frontend:
build: ./frontend
restart: unless-stopped
ports:
- "3000:3000"
depends_on:
- api
volumes:
db_data:In this example, I’ve defined database, api, and frontend services. Each service has a basic image or build command, a restart policy, necessary environment variables, and ports. Specifying inter-service dependencies with depends_on ensures that services start in the correct order. The volumes definition ensures the persistence of database data. This simple approach allows me to quickly test the core functionalities of the application and avoid getting bogged down with unnecessary details in the initial iterations.
This minimal structure is a method I frequently use when testing a new module for a production ERP or developing the backend for my own Android application. Especially when I need to quickly validate a concept, avoiding the overhead of complex YAML files helps me maintain project momentum. Instead of trying to perfect everything at first, starting with a working skeleton allows me to see the next steps more clearly.
2. Smartly Managing Environment Variables: .env and Compose Variables
When self-hosting my applications, I never want sensitive information (API keys, database passwords) to remain explicitly written within the docker-compose.yaml file. This poses both a security risk and makes managing configuration across different environments (development, test, production) difficult. This is where .env files and Docker Compose’s variable management capabilities come into play. For me, this is one of the cornerstones of a flexible and secure deployment.
An .env file is a simple text file located in the same directory as your docker-compose.yaml file, containing key-value pairs:
# .env file
POSTGRES_PASSWORD=mysupersecretpassword
API_KEY=anothersecretkey
FRONTEND_PORT=3000In the docker-compose.yaml file, I can reference these variables using $VARIABLE_NAME or ${VARIABLE_NAME}:
version: '3.8'
services:
database:
image: postgres:15-alpine
environment:
POSTGRES_DB: myapp_db
POSTGRES_USER: myuser
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} # Pulled from .env file
volumes:
- db_data:/var/lib/postgresql/data
api:
build: ./api
ports:
- "8000:8000"
environment:
DATABASE_URL: postgres://myuser:${POSTGRES_PASSWORD}@database:5432/myapp_db
API_SECRET: ${API_KEY}
depends_on:
- database
frontend:
build: ./frontend
ports:
- "${FRONTEND_PORT}:${FRONTEND_PORT}" # Pulled from .env file
depends_on:
- api
volumes:
db_data:This way, I can separate sensitive information from the YAML file and exclude the .env file from version control systems like git (I usually add it to .gitignore). Furthermore, I can reuse the same docker-compose.yaml file across different environments by using different .env files. For example, I can create dev.env for the development environment and prod.env for production, and then spin up specific environments with commands like docker compose --env-file prod.env up -d.
When working on a production ERP, managing database connection details or external service keys between test and production environments was always a challenge. .env files made these transitions seamless, allowing us to safely use the same Compose file for different environments. This way, we not only limited the concept of “working code” to application code but also included deployment configuration within this scope.
3. Correctly Using Volumes for Persistent Data
Containers are inherently ephemeral; that is, all data within them is lost when they are stopped or deleted. When I need persistent data in my self-hosted applications, such as databases, user uploads, or configuration files, using Docker Volumes is essential. Incorrect volume configuration can lead to serious problems, such as losing all data when a server restarts. Therefore, correctly configuring volumes is critical for application continuity and data security.
I use two main types of volumes in Docker Compose:
-
Named Volumes: These are volumes managed by Docker, usually visible with the
docker volume lscommand. They are the best way to store data independently of the container and are often preferred for critical data like databases.version: '3.8' services: database: image: postgres:15-alpine volumes: - db_data:/var/lib/postgresql/data # Named volume volumes: db_data: # Volume definition -
Bind Mounts: These directly mount a specific directory from the host machine into the container. This is useful for instantly reflecting code changes during development or providing configuration files from the host to the container. However, for databases in production, Named Volumes are a safer and more manageable option.
version: '3.8' services: api: build: ./api volumes: - ./api:/app # Binds host directory to container
When I use named volumes like db_data, my database data remains safe even if I stop the services with docker compose down. This provides great peace of mind, especially when I need to restart a service due to an update or an error.
I remember accidentally binding database data to a temporary directory on the host using a bind mount in a client project. All data was lost when the server restarted. This incident painfully taught me the difference between Named Volumes and Bind Mounts and when each should be used. Since then, I always make sure to use Named Volumes for critical data and even design my backup strategy with these volumes in mind.
4. Optimizing Network Configuration: Custom Networks and Port Mapping
When running multiple services with Docker Compose, inter-service communication needs to be both secure and efficient. By default, Docker Compose creates a single default network for your application and connects all services to it. However, when more isolation, security, or a complex network topology is required, defining custom networks and optimizing port mapping strategies becomes very useful.
For example, a frontend service should only be accessible by an Nginx reverse proxy, while a backend service should only be accessible from the frontend or another internal service. In this scenario, I can restrict inter-service access by defining different networks:
version: '3.8'
services:
nginx:
image: nginx:alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
networks:
- external_network
- internal_network
frontend:
build: ./frontend
networks:
- internal_network
api:
build: ./api
networks:
- internal_network
database:
image: postgres:15-alpine
networks:
- internal_network
environment:
POSTGRES_DB: myapp_db
POSTGRES_USER: myuser
POSTGRES_PASSWORD: mypassword
volumes:
- db_data:/var/lib/postgresql/data
networks:
external_network:
driver: bridge
internal_network:
driver: bridge
volumes:
db_data:In this configuration, the nginx service is on both the external_network (internet-facing) and the internal_network (inter-application service communication). The frontend, api, and database services, however, only run on the internal_network. This prevents direct external access to the database service. Additionally, services can communicate with each other by their names (e.g., the api service can access the database service via database:5432), which simplifies configuration.
When developing an internal platform for a bank, different services needed to operate in different security zones. By using Custom networks, we achieved strict isolation between these zones. By allowing only specific services to communicate with each other, we limited the spread of a potential security breach. This is a reflection of network segmentation and Zero-Trust architecture principles, not just in Docker Compose environments, but in general. Nginx being only externally exposed, while other services reside securely within their internal networks, is a fundamental step in such architectures.
5. Increasing Reliability with Health Checks and Restart Policies
A self-hosted application doesn’t just need to be “running”; it also needs to be “healthy” and capable of automatically recovering from unexpected situations. Docker Compose allows us to increase this reliability with its healthcheck mechanism and restart policies. Understanding whether a service is truly operational and ensuring it automatically restarts when it crashes significantly reduces my operational burden.
A healthcheck definition allows Docker to periodically check the status of the application inside a container. For example, for a web service, we can check if it responds by making a request to a specific HTTP endpoint:
version: '3.8'
services:
api:
build: ./api
ports:
- "8000:8000"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 20s
restart: unless-stopped
depends_on:
- databaseIn this example, I’ve defined a healthcheck for the api service. Docker will make a curl request to http://localhost:8000/health every 30 seconds. If it doesn’t receive a response within 10 seconds or if an HTTP error code is returned (with curl’s -f flag), it will be counted as a failure. After three failed attempts, the container will be marked as unhealthy. start_period allows sufficient time for the application to start; even if the health check fails during this period, a restart won’t be triggered.
restart policies determine when a container will automatically restart:
- no: No automatic restart.
- on-failure: Restarts if the container exits with an error code (non-zero).
- always: The container is always restarted (until manually stopped).
- unless-stopped: The container is always restarted unless manually stopped (my most frequently used).
I was using a Redis service in the backend of my own side product, and sometimes the Redis container would crash due to out-of-memory (OOM) issues. Thanks to the restart: unless-stopped policy, Redis would automatically restart, ensuring my application ran without interruption. However, adding a healthcheck to verify that Redis was truly operational ensured that the application could actually serve requests, beyond just the container being alive. This dual approach is an indispensable combination for increasing the resilience of my self-hosted systems.
graph TD;
A["User Access"] --> B["Nginx Reverse Proxy"];
B --> C{"External_Network"};
C --> N["Nginx"];
N --> D{"Internal_Network"};
D --> E["Frontend Service"];
E --> F["API Service"];
F --> G["Database Service (PostgreSQL)"];
style A fill:#D4E6F1,stroke:#3498DB,stroke-width:2px;
style B fill:#D4E6F1,stroke:#3498DB,stroke-width:2px;
style C fill:#D4E6F1,stroke:#3498DB,stroke-width:2px;
style N fill:#D4E6F1,stroke:#3498DB,stroke-width:2px;
style D fill:#D4E6F1,stroke:#3498DB,stroke-width:2px;
style E fill:#D4E6F1,stroke:#3498DB,stroke-width:2px;
style F fill:#D4E6F1,stroke:#3498DB,stroke-width:2px;
style G fill:#D4E6F1,stroke:#3498DB,stroke-width:2px;
Conclusion
Docker Compose is an invaluable tool for someone with field experience like me, for quickly and reliably spinning up self-hosted applications. Starting quickly with minimalist docker-compose.yaml files, smartly managing environment variables with .env, using Named Volumes for critical data, optimizing network configuration with Custom Networks, and finally increasing application resilience with Health Checks and Restart Policies are my frequently used core strategies.
These approaches not only run an application but also make it more manageable, secure, and flexible. Each step has been distilled from real-world problems and solutions I’ve encountered. I hope this guide helps you get the most out of Docker Compose in your own self-hosted projects and saves you from operational headaches. In my next post, perhaps I’ll share my experiences on CI/CD integration with Docker Compose.