Creating Docker in Docker (DinD) Image with Proto for GRPC

When working on a multi-Gradle project, the implementation of a CI/CD pipeline can present unique challenges, especially when integrating services that utilize gRPC. This article discusses the context of the problem, the specific issues encountered, and the solution developed to address these challenges.

The Context

In the course of developing a CI/CD pipeline for a multi-Gradle project, the team faced difficulties with microservices that relied on gRPC. While the microservices built successfully on local environment across various operating systems, including Ubuntu, MacOS, and Windows, errors related to Protocol Buffers (Proto) arose during the CI/CD pipeline execution. The CI/CD environment utilized a GitLab runner configured with a Docker executor, which complicated the build process.

The Problem

The primary issue stemmed from the absence of the Protocol Buffers compiler (protoc) in the default Docker-in-Docker (DinD) image based on Alpine Linux. This lightweight operating system is commonly used in CI/CD environments due to its minimal footprint, but it lacks certain development tools, including the protobuf compiler necessary for building gRPC services. Consequently, the microservices that depended on gRPC were unable to compile successfully in the CI/CD pipeline.

The Solution

To resolve the issue, a custom DinD Docker image was created that included the protoc compiler. This involved the following steps:

To modify docker image use below Dockerfile content. (You can use any alpine image as a base.)

FROM iaminci/dind-jdk21-gradle-8:0.0.1
RUN apk add --no-cache protoc
RUN apk add --no-cache go
RUN go install github.com/golang/protobuf/protoc-gen-go@latest
RUN apk add --no-cache gcompat

To run proto in CI/CD image we have used below images.

For Alpine (Also have openjdk 21 and gradle 8.1.1)

iaminci/dind-jdk21-gradle-8:0.0.3-test

For Ubuntu (Also have openjdk 21 and gradle 8.1.1)

iaminci/ubuntu-dind:jdk21-gradle8.1

The Conclusion

By creating a custom DinD Docker image that includes the Protocol Buffers compiler, the team was able to resolve the build issues encountered in the CI/CD pipeline for their multi-Gradle project. This solution not only facilitated the successful compilation of gRPC services but also enhanced the overall reliability of the CI/CD process across different environments.