Building in Public: Deploy a PHP application with Kamal

The Challenge

After Michael Kimsal released his proof of concept project sherldoc, I was wondering if I could deploy this using Kamal, a deployment tool typically used for Rails web applications. Supposedly Kamal is application and framework agnostic.

Getting this to work is like looking for Bigfoot or that Giant Squid you see in those late night shows on the History Channel. It should be possible!

First, let us look at sherldoc’s description:

Web service endpoint to scan a document for

• existence of keyword or phrase
• absence of keyword or phrase

The project provides a Docker Compose configuration for deployment. Several assumptions are built into the current configuration, which I will outline later. The challenge will be to convert these assumptions into Kamal configuration directives and then get a running sherldoc instance.

Why Kamal?

Kamal promises:

Kamal offers zero-downtime deploys, rolling restarts, asset bridging, remote builds, accessory service management, and everything else you need to deploy and manage your web app in production with Docker. Originally built for Rails apps, Kamal will work with any type of web app that can be containerized.

and

Kamal basically is Capistrano for Containers, without the need to carefully prepare servers in advance. 

Capistrano was the old reliable deployment tool for many years in the Rails world. The idea that we can use the same ease of deployment with containers on a naked VM is attractive.

The whole point is to make it easy to put your application on a low-cost VM, either in some hosted environment or perhaps an on-premise machine. Consider this to be a potent weapon in the crusade against BigVM^TM.

I am not a PHP expert, so getting this to work will push me outside of my comfort zone. I may fail. I may not find the Giant Squid or Bigfoot. In the worst case, I’ll learn something new.

Analyzing the Docker Compose configuration

I start by examining the original docker-compose.yml file for clues:

services:
     nginx:
          image: nginx:stable-bullseye
...
     app:
          image: sherldoc/web:1.0
...
     redis:
          image: redis:6
...
     tika:
          image: apache/tika:2.9.2.1-full
...
     supervisor:
...
     psql:
          image:postgres:16.1-bullseye
...

We start with the entry “app”, which is the PHP web application. This will be the source of our Kamal application image. Kamal creates this image and then stores it in a registry we specify. Once successfully connected to our provisioned virtual machine, Kamal installs docker and then download the image from the registry.

We can see that the original configuration required a docker supervisor process “supervisor”. Since we will only require a single image, the PHP application, the supervisor is not needed. Therefore, we omit this in our Kamal configuration. [Edit: This is not correct. The supervisor image is actually the sherldoc application “jobs” worker process. We will need to replicate this and as we will see, Kamal anticipates this need.]

The “nginx” entry hints that the PHP web application depends on the Nginx application proxy. We can peek inside the associated Nginx dockerfile (docker/app/nginx/conf.d/app.conf) and see that the application directives. Kamal provides an application proxy (“kamal-proxy”, https://kamal-deploy.org/docs/configuration/proxy/) which, in theory, provides the same capabilities.

The “redis”, “tika”, and “postgres” entries indicate additional services that the web application relies on. Each of these services has an associated container image.

Kamal provides configuration options for “accessory” services as well (https://kamal-deploy.org/docs/configuration/accessories/). As long as we can use the same images and apply similar configuration options to match the original values in the docker-compose.yml file it should work.

Preparing for Kamal

I forked the project to avoid bombarding the original project with PR requests. Perhaps my work will be merged in later.

Next I installed Kamal and initialized the local workspace with kamal init.

Next, I edited the default Kamal configuration (config/deploy.yml) with the following, removing all the comments for easier reading:

service: sherldoc

# Name of the container image.
image: sherldoc

# Deploy to these servers.
servers:
  web:
    - 159.203.76.193
registry:
  server: registry.digitalocean.com/team-james-demo
  username: my-user

  password:
    - KAMAL_REGISTRY_PASSWORD

builder:
  arch: amd64
  dockerfile: docker/app/php/php.dockerfile

Let’s review the contents:

service: sherldoc

# Name of the container image.
image: sherldoc

The name is just the name of the original project. No magic here.

servers:
  web:
    - 159.203.76.193

The IP address is the same address as the provisioned in my VM provider of choice at Digital Ocean. This is the cheapest configuration I could find. It might be too small or under provisioned, but we can fix that later.

registry:
  server: registry.digitalocean.com/team-james-demo
  username: my-user

  password:
    - KAMAL_REGISTRY_PASSWORD

Kamal will generate a docker image then push that image into your registry. Because I am using Digital Ocean I can use the Digital Ocean registry service. I could have also used Docker Hub, AWS Elastic Container Store, or any other container registry.

The KAMAL_REGISTRY_PASSWORD is an environment variable set to the credentials (an authentication token) provided by Digital Ocean. For security reasons, I don’t want to commit the actual value to the configuration file. I’ll leave this to be constituted at runtime.

First deployment attempt

All these things in place, we kick off the build with “kamal setup”.

INFO [fe0776d2] Running /usr/bin/env mkdir -p .kamal on 159.203.76.193
INFO [fe0776d2] Finished in 1.702 seconds with exit status 0 (successful).
Acquiring the deploy lock...
Ensure Docker is installed...
INFO [edde3944] Running docker -v on 159.203.76.193
INFO [edde3944] Finished in 0.186 seconds with exit status 0 (successful).
Log into image registry...
INFO [8eb7c038] Running docker login registry.digitalocean.com/team-james-demo -u [REDACTED] -p [REDACTED] as jdjeffers@localhost
... (lots of logs cut out here) 
INFO [9ce887de] Running docker container ls --all --filter name=^sherldoc-web-7ceb4de587a2119c9b007f40973a40cd7eb88b8e$ --quiet | xargs docker inspect --format '{{json .State.Health}}' on 159.203.76.193
INFO [9ce887de] Finished in 0.249 seconds with exit status 0 (successful).
 ERROR null
INFO [54c5ab35] Running docker container ls --all --filter name=^sherldoc-web-7ceb4de587a2119c9b007f40973a40cd7eb88b8e$ --quiet | xargs docker stop on 159.203.76.193
INFO [54c5ab35] Finished in 0.404 seconds with exit status 0 (successful).
  Finished all in 571.2 seconds
Releasing the deploy lock...
  Finished all in 573.5 seconds
ERROR (SSHKit::Command::Failed): Exception while executing on host 159.203.76.193: docker exit status: 1
docker stdout: Nothing written
docker stderr: Error: target failed to become healthy

This result is expected for several reasons. The original application:

1. doesn’t provide a default 200OK to the kamal heartbeat request at “/up”,
2. expects a redis instance,
3. expects an Nginx application proxy,
4. expects a tika server process,
5. expects a PostgreSQL database.

Without these other services, the sherldoc PHP application is probably not going work! We’ll fix these issues next.

Want to follow this quest? Read part 2!