Switching from Docker to Podman

April 17th, 2024

For some time I had Podman on my list to look at. Mainly because it focuses on rootless container setups. With docker and the docker daemon I always felt that this is a bit too much. Overall reasons for me to switch:

  • Rootless container without a daemon running in the background
  • The pod setup in podman is reducing complexity. You start a webserver and database in one pod and they can talk to each other via localhost, no networking needed
  • Auto updating of containers and restarting services
  • Systemd integration (although I did not know the last two in advance)

Reading a lot of other howtos, it sounded like a piece of cake. It is easy, however the switch in thinking from having a daemon running in root to a rootless user namespace setup is hard. Solutions you used before might now work and new settings are needed to get things running.

Also, all of this is within a Debian home server setup, so nothing big/professional.

The hardest part is podmans release and development style. A lot of features get partly developed, released and advertised as the “new thing” to do, but they are not ready and you need to switch back to older features. With running 4.9.x on Debian, I had this with features like

  • Quadlets would not produce pod systemd files (needs v5 or use old systemd generator)
  • Auto-update would not work with containers in pods (needs v5)
  • Pasta networking, is far from stable (ipv6 connections take time to come up, start/stop of containers in pods kill networking at some point) and it makes DNS setups in pods hard to understand
  • runc and crun have different feature sets

All of this can be worked around, however if you just read documentation, blog posts, etc. you start configuring and then figure out it is not feature complete and need to redo the whole config. So be warned and always check to what version the documentation refers to.

So how is the move done?

First of all: You need to stop all docker, whilst working on podman. I would even recommend to uninstall docker or set alias docker=podman right away. It is easy to get confused between the two and - believe me - the docker command is sort of muscle memory for containers. I worked on podman, whilst having docker installed and it does not pose a problem, other than me staring on docker ps and being baffled that my podman container has not started. ;-)

Podman is very compatible to the docker commands, so you won’t feel a difference. You mostly can start your docker configurations right away. However I would recommend to work on it in the following way. Next steps are executed as root / sudo.

Fundamental things on the host to get podman running

First you need to install Podman. Currently it is very hard to get latest versions in the non-RedHat world. My server is on Debian Bookworm with quite old podman versions. However There is a sort of up-to-date version (time of writing 4.9.x) in testing, hence you can get it with a bit of Pinning.

First create a new source file /apt/sources.list.d/testing.list for access to testing:

deb http://deb.debian.org/debian testing main contrib

This allows now to use testing packages in your system.

Next we need to pin single packages, so that not all packages get upgraded to testing, but only needed ones. Pinning basically sorts priorities. Highest number wins and standard is 500. For that create /etc/apt/preferences.d/testing:

Package: *
Pin: release a=testing
Pin-Priority: -1

Package: podman
Pin: release a=testing
Pin-Priority: 1001

Package: podman-compose
Pin: release a=testing
Pin-Priority: 1001

Package: podman-docker
Pin: release a=testing
Pin-Priority: 1001

Package: passt
Pin: release a=testing
Pin-Priority: 1001

Package: libgpgme11t64
Pin: release a=testing
Pin-Priority: 1001

This will reduce the priority for all packages of testing to -1, making sure, that your system stays on bookworm. Otherwise the testing packages would all override the bookworm packages due to same priority but higher version number. We then have exemptions for the podman packages.

  • podman and podman-compose are the fundamental packages needed
  • podman-docker is needed if you want to use podman as a full drop-in for docker and e.g. use something like the docker socket in Traefik
  • passt is the package to use the pasta networking (which really looks promising)
  • libgpgme11t64 is a dependency needed

After this a apt-get update will reload the apt cache and with apt-cache policy podman you can check which version will be installed.

IMPORTANT: Check with apt-get upgrade -s first on what effect it will have. If on a fully upgraded system more packages get an upgrade than the pinned ones above you made something wrong. Do not upgrade!

Now that we have podman installed, there is some configuration to do. I used a new user for running my rootless containers. You can simply create a regular user with a home folder.

Important read: Here you will have the first difference to rootful Docker: Your containers will run as that user, hence will only have access to what the user has access to. If you need devices in the container, it is not enough to add them in the config, but the user must have access to the device. The same holds true for file access.

To make it more complicated, there are different modes for the rootless setup. Two fundamental ideas are important to understand:

  • Running with root inside the container: Even though the processes inside the container run as ID 0 = root, they will be mapped to your user on the outside. It is playing root without being one. This is - I feel - the easiest to understand and to administer. Just configure your host user to have access on what you need in the container (direct ownership or groups)
  • Running as a different user in the container: Now it becomes tricky, as the internal user gets an offset as the external ones, defined in the subuid configuration.

Both types are explained here.

But lets first make sure the system is setup, before we get to container specific configurations.

As a regular user, we have some troubles handling system processes. Two settings I found essential

  1. Starting processes on privileged ports (<1024), e.g. a http/https webserver.
  2. Pinging from containers to check connectivity.

To allow this, create /etc/sysctl.d/podman-settings.conf with

net.ipv4.ip_unprivileged_port_start=0
net.ipv4.ping_group_range=0 2000000

It is not very specific of a configuration, but I do not know any other way. Reload it with sysctl --load /etc/sysctl.d/podman-settings.conf.

After that we need to make sure, that rootless containers can continue to run after the user logs out. For that a simple loginctl enable-linger <your unprivileged username> will suffice.

Next we need to check if subuid and subgid is set. For that run cat /etc/subuid and see if there is a line for your user. Normally it is created when adding the user. If it is missing run: usermod --add-subuids 100000-165535 --add-subgids 100000-165535 <your unprivileged username>.

This is the offset I mentioned earlier. If you run a user inside the container, e.g. with ID 999, it will be on the host 100998. Similar for gids. This delivers even more user separation, but also is complicated to track over all containers, especially if you need to interact with container files on the host or over different containers.

Finally we need to add container registries to podman. You can always use the registry in the name of the image you are pulling, but I felt it is a good idea to have the standard Docker Hub included, to simplify search. To explain: If you want to get traefik:latest, you could pull docker.io/library/traefik:latest. But with a registry configured you can just use traefik:latest again.

For that create /etc/containers/registries.conf and add

unqualified-search-registries = ["docker.io"]

You can add more here, but this should be enough for now.

All this was fundamental configuration made by root. Now you can start building containers. Most things you had with docker will work right away, or will need minimal changes to work with the rootless user. It is possible to partially use sudo -u <your unprivileged username> podman..., however this has limits. I would advise to log in as that user and continue from there.

Some additional helpful things

I did the following little helpful things in /etc/containers/containers.conf, as this applies to all containers and reduces definition work. To get the full options for the conf file, you find a well commented version (Debian) in /usr/share/containers/containers.conf.

[containers]
# For bookworm with journald logging
log_driver = "journald"
log_tag = "podman/{{.Name}}"
tz = "Europe/Berlin"

[network]
# Make default network fit to general setup
default_network = "podman"
default_subnet = "192.168.14.1/24"
default_subnet_pools = [
  {"base" = "192.168.15.1/24", "size" = 24},
  {"base" = "192.168.16.1/24", "size" = 24},
  {"base" = "192.168.17.1/24", "size" = 24},
  {"base" = "192.168.18.1/24", "size" = 24},
  {"base" = "192.168.19.1/24", "size" = 24},
]

Christian

Author