Waiting: The exact art that you must master

Recently, while working in a workshop entitled your cloud request test on Kubernetes with GKE and GitHub procedures, I faced the same problem twice: service B. service needs, but service A begins faster than service B, and system failure. In this post, I would like to describe the context of these issues and how I solved them with the same tool.

Waiting in kubernetes

It may seem strange to wait in Kubernetes. Kubernetes is one of its biggest benefits. Let’s look at two pods: Python app and postgresql database.

The app begins very quickly and tries impatiently to create a database connection. Meanwhile, preparing the database itself with the data provided; Communication failure. The pods end in Failed state.

After a while, kubernetes requests the mandate of the application. Because he failed, it ends it and begins a new pod. At this stage, two things can happen: the database pod is not yet ready, returned to one box, or it is ready, and the application finally connects.

To accelerate the process, kubernetes make startup investigations:

startupProbe:
  httpGet:
    path: /health
    port: 8080
  failureThreshold: 30
  periodSeconds: 10

With the probe above, kubernetes awaits for ten second seconds before ordering POD condition. If it fails, it is waiting for another ten seconds. Rinse and repeat 30 times before it fails.

Maybe you noticed http /health The end point above. Kubernetes offers exclusive preparations to form the investigation: httpGet or exec. Previous suitable for web applications, while the latter is for other applications. It means that we need to know what kind of containers that POD contains and how to check its condition, provided that it is. I am not an expert in postgresql, so I looked for the case verification. The Bitnami Helm chart is similar to the following:

startupProbe:
  exec:
    command:
      - /bin/sh
      - -c
      - -e
      - exec pg_isready -U $PG_USER -h $PG_HOST -p $PG_PORT

Note that the foregoing is simplification, because it delightly ignores the name of the database and the SSL certificate.

The startup probe speeds up matters compared to the default situation if you create it properly. You can set a long first delay, then shorter increases. However, the more the container diversity, the more difficult it is, as you should be an expert in each of the basic containers.

It will be useful to search for alternatives.

Wait 4x

The alternatives are the tools that focus on waiting. A long time ago, I found the scenario waiting for this. The idea is clear:

./wait-for It is a text designed to synchronize services such as Docker containers. that it sh and alpine harmonic.

Here’s how to wait for the HTTP app programming interface:

sh -c './wait-for http://my.api/health -- echo "The api is up! Let's use it"'

The task has been accomplished, but at that time, you had to copy the text program and manually check the updates. I have reviewed, and the project now provides an ordinary container.

WAIT4X plays the same role, but it is available as an issue with a version and provides more services to wait: http, DNS, databases, and messages. This is my current choice.

Whatever the tool you use, you can use it inside the Init container:

Pod can have multiple containers that operate applications inside them, but it can also contain one or more Init containers, which are operated before the start of the application containers.

Init containers are regular containers, except:

• Init containers always run to finish.
• Each Init container should be successfully completed before the next start.

Imagine the following Pod This depends on posgresql Deployment:

apiVersion: v1
kind: Pod
metadata:
  labels:
    type: app
    app: recommandations
spec:
  containers:
    - name: recommandations
      image: recommandations:latest
      envFrom:
        - configMapRef:
            name: postgres-config

The app is Bithon and starts very quickly. He tries to contact the postgresql database. Unfortunately, the database did not finish the preparation, so the connection failed, and the kubernetes restart POD.

We can fix it using initContainer And waiting container:

apiVersion: v1
kind: Pod
metadata:
  labels:
    type: app
    app: recommandations
spec:
  initContainers:
    - name: wait-for-postgres
      image: atkrad/wait4x:3.1
      command:
        - wait4x
        - postgresql
        - postgres://$(DATABASE_URL)?sslmode=disable
      envFrom:
        - configMapRef:
            name: postgres-config
  containers:
    - name: recommandations
      image: recommandations:latest
      envFrom:
        - configMapRef:
            name: postgres-config

In preparation above, initContainer Do not stop until the contact database is accepted. When you do it, it ends, and recommandations The container can start. Kubernetes does not need to be terminated Pod As in the previous preparation! Less number of records and perhaps less alerts are required.

When waiting becomes mandatory

The above is a slight improvement, but you can dispense with it. In other cases, waiting becomes mandatory. I recently tried it when preparing for the aforementioned workshop. The scenario is the following:

• The pipeline applies a statement on the side of Kubernetes
• In the next step, it runs the test
• When the test begins before reading the application, it fails.

We must wait for the back interface ready before the test. Let’s use wait4x To be waiting Pod To accept applications before the tests are launched:

      - name: Wait until the application has started
        uses: addnab/docker-run-action@v3                                       #1
        with:
          image: atkrad/wait4x:latest
          run: wait4x http ${{ env.BASE_URL }}/health --expect-status-code 200  #2

1. GitHub is allowed to operate a container. I could download The Go Binary instead.
2. Wait for even /health The end point return a 200 Response icon.

conclusion

Kubernetes starting investigations are a great way to avoid unnecessary restarts when services that depend on each other begin. The alternative is an external waiting tool formed in initContainer. wait4x It is a tool that can be used in other contexts. It is now part of my tool base.

To go further:

It was originally published in Java Al -Muhaous on April 20, 2025