Unauthorised pull when using compose file with podman

[ngudbhav]

Issue Description

Using the compose file to run podman compose build errors out the image pull from docker.io with unauthorised access.

Steps to reproduce the issue

Steps to reproduce the issue

1. Simple podman pull of an image from docker.io

╰─❯ podman pull docker://golang:1.21-alpine                                                                                                                             ─╯
Trying to pull docker.io/library/golang:1.21-alpine...
Getting image source signatures
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:690e87867337b8441990047e169b892933e9006bdbcbed52ab7a356945477a4d
Copying blob sha256:2a6022646f09ee78a83ef4abd0f5af04071b6563cf16a18e00fb2dcfe63ca0a3
Copying blob sha256:171883aaf475f5dea5723bb43248d9cf3f3c3a7cf5927947a8bed4836bbccb62
Copying blob sha256:e495e1face5cc12777f452389e1da15202c37ec00ba024f12f841b5c90a47057
Copying config sha256:2bbe4e7e4d4e0f6f1b6c7192f01b9c7099e921b9fe8eae0c5c939a1d257f7e81
Writing manifest to image destination
2bbe4e7e4d4e0f6f1b6c7192f01b9c7099e921b9fe8eae0c5c939a1d257f7e81

2. Podman compose pull with same image

podman compose pull docker://golang:1.21-alpine
>>>> Executing external compose provider "/usr/local/bin/docker-compose". Please see podman-compose(1) for how to disable this message. <<<<

no such service: docker://golang:1.21-alpine
Error: executing /usr/local/bin/docker-compose pull docker://golang:1.21-alpine: exit status 1

3. Using the compose file to pull image from podman

podman compose build builder
>>>> Executing external compose provider "/usr/local/bin/docker-compose". Please see podman-compose(1) for how to disable this message. <<<<

[+] Building 0/1
 ⠏ Service builder  Building                                                                                                                                          0.9s 
Sending build context to Docker daemon  738.8kB
STEP 1/9: FROM golang:1.21-alpine AS builder
[+] Building 0/1il>) to docker.io (enforced by caller)
 ⠼ Service builder  Building                                                                                                                                          2.4s 
creating build container: initializing source docker://golang:1.21-alpine: unable to retrieve auth token: invalid username/password: unauthorized: incorrect username or password

Error: executing /usr/local/bin/docker-compose build builder: exit status 1

Describe the results you received

creating build container: initializing source docker://golang:1.21-alpine: unable to retrieve auth token: invalid username/password: unauthorized: incorrect username or password

This error message is similar to the output from podman compose pull. So, this could be an issue with podman compose.

Describe the results you expected

The build process should have completed successfully.

podman info output

Client:
  APIVersion: 5.4.1
  BuildOrigin: brew
  Built: 1741713733
  BuiltTime: Tue Mar 11 22:52:13 2025
  GitCommit: ""
  GoVersion: go1.24.1
  Os: darwin
  OsArch: darwin/arm64
  Version: 5.4.1
host:
  arch: arm64
  buildahVersion: 1.39.0
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-3.fc41.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 97.68
    systemPercent: 0.63
    userPercent: 1.68
  cpus: 10
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "41"
  eventLogger: journald
  freeLocks: 2029
  hostname: localhost.localdomain
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.12.9-200.fc41.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 5674049536
  memTotal: 15559577600
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.13.1-1.fc41.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.13.1
    package: netavark-1.13.1-1.fc41.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.13.1
  ociRuntime:
    name: crun
    package: crun-1.19.1-1.fc41.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.19.1
      commit: 3e32a70c93f5aa5fea69b50256cca7fd4aa23c80
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20241211.g09478d5-1.fc41.aarch64
    version: |
      pasta 0^20241211.g09478d5-1.fc41.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: unix:///run/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.3.1-1.fc41.aarch64
    version: |-
      slirp4netns version 1.3.1
      commit: e5e368c4f5db6ae75c2fce786e31eef9da6bf236
      libslirp: 4.8.0
      SLIRP_CONFIG_VERSION_MAX: 5
      libseccomp: 2.5.5
  swapFree: 0
  swapTotal: 0
  uptime: 0h 25m 46.00s
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 12
    paused: 0
    running: 9
    stopped: 3
  graphDriverName: overlay
  graphOptions:
    overlay.additionalImageStores:
    - /usr/lib/containers/storage
    overlay.imagestore: /usr/lib/containers/storage
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 498331758592
  graphRootUsed: 38083604480
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 26
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 5.4.0
  BuildOrigin: Fedora Project
  Built: 1739232000
  BuiltTime: Tue Feb 11 05:30:00 2025
  GitCommit: ""
  GoVersion: go1.23.5
  Os: linux
  OsArch: linux/arm64
  Version: 5.4.0

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

Yes

Additional environment details

NA

Additional information

NA

[baude]

can you replace the pics in your report with text blobs? it makes it hard to read?

[ngudbhav]

@baude, Apologies and Thanks for taking a look at the issue. I have updated the issue description. Please let me know if you need further details.

[ngudbhav]

I performed another test. This time with Docker-GO SDK (testcontainers-go) and I am facing the same issue.

Failed to pull image: Error response from daemon: {"message":"unable to retrieve auth token: invalid username/password: unauthorized: incorrect username or password"}, will retry

I tried to pull the image wiremock/wiremock:3.9.1

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[gy-mate]

I have the same issue.

Commands to reproduce:

git checkout https://github.com/kami911/osm_poi_matchmaker/
cd osm_poi_matchmaker
git checkout py3_11
git checkout f520554fd2df400068beb692dcdd710830567d94
./restart.sh

Relevant output with the error:

Sending build context to Docker daemon  3.433MB
STEP 1/13: FROM python:3.11-slim-bullseye
Resolving %!q(<nil>) to docker.io (enforced by caller)
Trying to pull docker.io/library/python:3.11-slim-bullseye...
creating build container: internal error: unable to copy from source docker://python:3.11-slim-bullseye: initializing source docker://python:3.11-slim-bullseye: unable to retrieve auth token: invalid username/password: unauthorized: incorrect username or password

Sending build context to Docker daemon  3.433MB
STEP 1/11: FROM alpine:3.11.3
Resolving %!q(<nil>) to docker.io (enforced by caller)
Trying to pull docker.io/library/alpine:3.11.3...
creating build container: internal error: unable to copy from source docker://alpine:3.11.3: initializing source docker://alpine:3.11.3: unable to retrieve auth token: invalid username/password: unauthorized: incorrect username or password

podman login docker.io output:

Authenticating with existing credentials for docker.io
Existing credentials are valid. Already logged in to docker.io

podman info output:

Client:
  APIVersion: 5.5.0
  BuildOrigin: pkginstaller
  Built: 1747237432
  BuiltTime: Wed May 14 17:43:52 2025
  GitCommit: 0dbcb51477ee7ab8d3b47d30facf71fc38bb0c98
  GoVersion: go1.24.2
  Os: darwin
  OsArch: darwin/arm64
  Version: 5.5.0
host:
  arch: arm64
  buildahVersion: 1.40.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-3.fc41.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 99.76
    systemPercent: 0.11
    userPercent: 0.13
  cpus: 8
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "41"
  eventLogger: journald
  freeLocks: 2048
  hostname: localhost.localdomain
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.12.13-200.fc41.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 7072681984
  memTotal: 7715565568
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.15.0-1.fc41.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.15.0
    package: netavark-1.15.2-1.fc41.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.15.2
  ociRuntime:
    name: crun
    package: crun-1.20-2.fc41.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.20
      commit: 9c9a76ac11994701dd666c4f0b869ceffb599a66
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20250121.g4f2c8e7-2.fc41.aarch64
    version: |
      pasta 0^20250121.g4f2c8e7-2.fc41.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: unix:///run/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.3.1-1.fc41.aarch64
    version: |-
      slirp4netns version 1.3.1
      commit: e5e368c4f5db6ae75c2fce786e31eef9da6bf236
      libslirp: 4.8.0
      SLIRP_CONFIG_VERSION_MAX: 5
      libseccomp: 2.5.5
  swapFree: 0
  swapTotal: 0
  uptime: 11h 31m 29.00s (Approximately 0.46 days)
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.additionalImageStores:
    - /usr/lib/containers/storage
    overlay.imagestore: /usr/lib/containers/storage
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 30180323328
  graphRootUsed: 2894282752
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 5.5.1
  BuildOrigin: 'Copr: packit/containers-podman-26294'
  Built: 1749081600
  BuiltTime: Thu Jun  5 02:00:00 2025
  GitCommit: 850db76dd78a0641eddb9ee19ee6f60d2c59bcfa
  GoVersion: go1.23.9
  Os: linux
  OsArch: linux/arm64
  Version: 5.5.1

[MrPostie]

The issue also seems to be present on my Windows 11 system with a private repository. When doing a podman compose up I get ✘ simpleprintserver Error {"message":"unauthorized: authentication required"}. The image is defined in the compose.yml file as image: <image-url>

When I copy/paste the <image-url> and run a podman pull <image url> it pulls the image without issues

[gy-mate]

@baude Could you please remove the stale-issue label? Thanks in advance! :)

[rzym-on]

Same issue here. Found out it's because podman compose uses docker auth file to pull from registry.

When you're logging in to (whatever) container registry:

podman login ghcr.io

Authenticating with existing credentials for ghcr.io
Existing credentials are valid. Already logged in to ghcr.io

The system stores those credentials in auth.json file. Usually it's in one of those locations:

• ${XDG_RUNTIME_DIR}/containers/auth.json (general)
• /run/user/$UID/containers/auth.json (rootfull)
• ~/.config/containers/auth.json (rootless)

But podman compose (if it's configured to use docker-compose under the hood) is trying to use one of those files:

• /root/.docker/config.json (rootfull mode)
• ~/.docker/config.json (rootless mode)

So what's worked for me for now:

1. Login with podman login to registry;
2. Copy file auth.json to place where docker would look for auth file;
3. podman compose pull works fine;