There is no autostart for pods and containers. Of course you can generate systemd service units for all components but managing them is not easy without an automation tool. There are separate service files for each pod and container.
For updates you might have to stop, remove, recreate everything from scratch, unless you script it. There is no ‘up‘, ‘down‘ or ‘build‘ features like you have with docker-compose.
There is no single point of configuration which I could use to describe all the pods and containers. I could write and maintain Kubernetes YAML files, but that’s even harder than using the CLI syntax I am already familiar with.
I needed a tool which makes managing Podman pods easier. podman-compose looked promising but it did not really work for me and I also did not like its CLI interface. So I decided to write my own tool.
Design goals of pods-compose
I did not want to put a lot of effort into this. I only wanted the following additional abilities.
Be able to automatically start and stop all pods and containers upon reboot.
Tear down existing pods at once.
Create pods and containers from a description at once.
Build all the images I define with a single command.
I did not want to rely on docker-compose’s YAML format. Intentionally there is no support for using an existing compose configuration. Although I was already familiar with that format, I wanted a complete migration not just a partial one.
Managing Podman pods with pods-compose
As a kickstart, let’s get a glimpse into the similarities between pods-compose and docker-compose.
Tear down pod(s)
Build all container images
Status of pods and containers
Generate Kubernetes Pod YAML(s)
Autostart pods and containers
Podman can generate systemd units for pods and containers. However there will be many of them, making it hard to overview and maintain it. Because pods-compose takes care of starting and stopping pods with a single command line option, I could create a single systemd service file instead of many.
The install script will deploy that systemd service file for pods-compose. Enabling it makes your pods and containers to start automatically upon reboot. And of course gracefully stop before the system halts.
Creating containers at least first time is a manual procedure. People usually start with ‘docker run’ commands then once the result looks okay then will create a docker-compose YAML.
This will not change with pods-compose. You still have to create your pods and containers with ‘podman run’. However you do not have to create any YAML files. The tool will create them for you. Luckily podman CLI syntax is almost the same as docker’s, so it is easy to make progress fast.
The other part is defining which image should be built by pods-compose. Because this information cannot be set in Kubernetes YAML files, you can use pods-compose‘s INI formatted configuration file to define the TAG and the CONTEXT of images. As a result, pods-compose will build all the images for you.
Let me know if you are still missing some features you would love to see implemented in pods-compose. Also please share if you liked it.
How to deploy pods with Podman when you only need a single-host system and not a complex Kubernetes. Convert your docker-compose services to pods with Podman.
For a single host setup or even for a now officially dead Docker Swarm setup using docker-compose is pretty convenient. But I wanted to get rid of Docker completely and migrate my docker-compose services to pods with Podman.
The reasons why I convert docker-compose services to pods
I have been using Docker’s container technology for about 4-5 years. Both in production and in different labs. Call me an old fashioned but I always managed to set up systems either with pure Docker containers or with docker-compose. However there are things I cannot easily forget.
Here are the top reasons why I decided to convert my docker-compose services to pods with Podman and get rid of Docker completely.
Recurring errors like failing to create many bridged network at once on a clean system, claiming ‘ERROR: Pool overlaps with other one on this address space‘.
Too many fiddling with iptables rules on a system using firewalld. This may not be a problem where a host OS’ only role is to run containers. But there are legit cases where containers may run on a host serving other purposes as well.
Daemon changes causing data losses. I learned the hard way why putting a production SQL database (state-full) into a container is a NO GO.
Inconsistency between recommendations and real life experience. Like “Don’t run more than one process in a single container” – Have you seen GitLab’s official Docker image?
I know some of these reasons may not apply to recent versions of Docker. And I am also aware that some issues are container technology related, so they may apply to Podman containers as well.
The basis of migration
Any migration requires planning and testing. So I started off with my home lab which hosts different systems. In my lab docker-compose took care of composing all services with a single YAML file. The following simplified figure shows a high level overview of the network architecture. Although the picture may indicate, the reverse proxy is not the gateway for the containers.
Although this system worked pretty well, I have some issues with it.
All networks use a bridge network driver to provide network isolation of service groups. Therefore you have to create many networks, which in turn improves complexity.
The network of Reverse Proxy has to be literally connected to all other bridges to have access to the web servers. However, this way the proxy container could access all exposed ports of all containers on any networks the proxy container is attached to. It provides a bigger attack surface.
Docker makes these networking possible with lots of iptables rules (so as Podman) which are hard to overview and pollute the iptables rules you may already have.
Planning the conversion of docker-compose services to pods
There is a very fundamental difference between Docker and Podman. Podman supports the concepts of pods for instance. This is intentionally very similar to Kubernetes’ pods. Containers in a pod shares the same namespace, like network. So all containers in the same pod looks like sharing the same localhost network. And each pod has its own localhost.
With Docker (Figure 1) there are 5 networks for 9 containers. With Podman by using pods there is only 1 network for 5 pods (Figure 2).
Pods provide another layer of isolation I really like. This way containers of any pods could only access ports published by other pods and not the containers themselves.
Challenges with Podman
Migrating to a new technology is not without compromises or challenges. Podman is around for a while and is rapidly evolving. Here are the challenges I had to handle.
Assign IP addresses to pods and not to containers
You can join a container to any networks. But a pod can be only joined to the default network. According to my understanding this will be changed later. This is the reason why I stick to the default network in my setup.
“Most of the attributes that make up the Pod are actually assigned to the “infra” container. Port bindings, cgroup-parent values, and kernel namespaces are all assigned to the “infra” container. “
By default pods will connect to network labeled cni_default_network in libpod.conf. If you join the pod’s containers to other networks, the pod will still have its IP assigned from the default network. However containers will have IPs assigned from the specified networks. As far as I know this symptom looks like a bug.
While I am waiting for support of DNS on pod level, I worked around this limitation. I publish the exposed ports of pods to their gateway’s IP address 10.88.0.1 and not the IP address of their infra container itself. As long as the gateway’s IP address static this will work.
Replacing functionalities of docker-compose
The YAML format of docker-compose uses an abstraction above ‘docker run‘ command. However I realized that all the hard work docker-compose did to me was to create networks and assign container’s to them. And of course deploying services.
Networking: Rootfull container networking (CNI)
Luckily describing how a network should look like is not the role of Podman but CNI and its plugins. You can see the layout of the default network in Figure 3.
The published ports are not visible from the outside network unless you set up routes externally. Or you can simply set the IP address of the host for serving published ports (--publish 192.168.122.253:80:80). Effectively it will be another DNAT rule. For my simple case it is enough.
P.s. Do not forget to enable IP forwarding with sysctl to persist across reboots.
P.s. 2: You may need to change the default firewall backend from iptables to firewalld in CNI configuration. So you will have a cleaner overview of your chains and rules.
I really like that you do not have to learn another language to build an image. Use the same Dockerfile format you are already familiar with.
Building an image is not the task of Podman but another tool called buildah. Although you can even use podman build, it will actually use Buildah in the background. Assuming you have your Dockerfile in the current working directory, it will look like this. It can even publish the image to a Docker repository.
I did a lot of testing, so I managed to convert all docker-compose services to pods with Podman and with some shell scripting. I still have to figure out how to auto start of pods. There are shareable systemd devices for containers, but I want to test it for pods. See you next time.
When I created a central file sever, I mentioned that some of the problems with the solution are yet to be resolved.
YaST created an import rule in file /etc/fstab, which is the de-facto place for storing such information. Its content and the mounts are usually static in server environments. On most client (in term of using an export of an NFS server) the network connectivity rarely or never changes in traditional environments. However in case of mobile devices like on laptops, the network state could vary a lot. It can be offline, or on WiFi, or on wired connection, maybe using VPNs. We need much more flexibility than a mostly static file.
Users would like to mount exports on their own. The system should be as transparent as possible to the end users.
Lucky for us, mounting NFS exports by using autofs service help us and gives the following advantages too. Continue reading →
In 2019 almost everyone has a digital life, so as I. Having digital photos or videos taken with our smartphones is an every day action. Year by year the number of smart phones and computers rises in households. Files started to be found everywhere. In your computers, in the cloud, everywhere Why do not we store them in one place and access them from everywhere?
I wanted to create a file server providing a central location for all our digital data. Creating an NFS file server looked promising. Continue reading →