after quite a few frustrating attempts to setup a built environment, fixing some issues, jumping over some hoops for the build, I managed to get to the 'starting point' of a build

 

./compile.sh
[🌱] uuidgen not found [ uuidgen not installed yet ]
[🌱] Using prebuilt Armbian image as base for 'ubuntu-jammy' [ DOCKER_ARMBIAN_BASE_IMAGE: ghcr.io/armbian/docker-armbian-build:armbian-ubuntu-jammy-latest ]
[🌿] Docker info [ Docker 19.03.11 Kernel:4.12.14-lp151.28.91-default RAM:31.28GiB CPUs:8 OS:'openSUSE Leap 15.1' hostname 'snoopy1.internal' under 'Linux' - buildx:yes - loop-hacks:yes static-loops:no ]
[🌱] Creating [ .dockerignore ]
[🌱] Docker launcher [ enabling all extensions looking for Docker dependencies ]
[🌱] Extension manager [ processed 27 Extension Methods calls and 82 Extension Method implementations ]
[🌱] Adding rootfs encryption related packages [ cryptsetup cryptsetup-initramfs ]
[🌱] Preparing rkdevflash host-side dependencies [ rkdevflash ]
[🌱] Creating [ Dockerfile; FROM ghcr.io/armbian/docker-armbian-build:armbian-ubuntu-jammy-latest ]
[🌱] Armbian docker image [ already exists: ghcr.io/armbian/docker-armbian-build:armbian-ubuntu-jammy-latest ]
[🌱] Building [ Dockerfile via 'buildx build --progress=plain --load' ]
[🔨]   #0 building with "default" instance using docker driver
[🔨]   
[🔨]   #1 [internal] load build definition from Dockerfile
[🔨]   #1 transferring dockerfile: 2.12kB done
...
[🔨]   #6 [2/6] RUN echo "--> CACHE MISS IN DOCKERFILE: apt packages." &&  DEBIAN_FRONTEND=noninteractive apt-get -y update &&  DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends bash git psmisc uuid-runtime bc binfmt-support bison libc6-dev make dpkg-dev gcc ca-certificates ccache cpio debootstrap device-tree-compiler dialog dirmngr dosfstools dwarves flex gawk gnupg gpg imagemagick jq kmod libbison-dev libelf-dev libfdt-dev libfile-fcntllock-perl libmpc-dev libfl-dev liblz4-tool libncurses-dev libssl-dev libusb-1.0-0-dev linux-base locales lsof ncurses-base ncurses-term ntpdate patchutils pkg-config pv qemu-user-static rsync swig u-boot-tools udev uuid-dev zlib1g-dev file tree expect colorized-logs unzip zip pigz xz-utils pbzip2 lzop zstd parted gdisk fdisk aria2 curl wget axel parallel python3-dev python3-distutils python3-setuptools python3-pip python2 python2-dev gcc-x86-64-linux-gnu gcc-aarch64-linux-gnu gcc-arm-linux-gnueabihf gcc-arm-linux-gnueabi gcc-riscv64-linux-gnu debian-archive-keyring libc6-amd64-cross g++-aarch64-linux-gnu g++ btrfs-progs cryptsetup openssh-client f2fs-tools nilfs-tools xfsprogs zerofree qemu-utils qemu-utils libudev-dev libusb-1.0-0-dev dh-autoreconf build-essential gcc-arm-linux-gnueabi gcc-or1k-elf qemu-utils
[🔨]   #6 0.958 --> CACHE MISS IN DOCKERFILE: apt packages.
[🔨]   #6 49.04 Ign:1 http://archive.ubuntu.com/ubuntu jammy InRelease
[🔨]   #6 49.04 Ign:2 http://security.ubuntu.com/ubuntu jammy-security InRelease
[🔨]   #6 97.09 Ign:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
...
[🔨]   #6 200.2 Err:2 http://security.ubuntu.com/ubuntu jammy-security InRelease
[🔨]   #6 200.2   Temporary failure resolving 'security.ubuntu.com'
...
[🔨]   #6 481.5 Err:1 http://archive.ubuntu.com/ubuntu jammy InRelease
[🔨]   #6 481.5   Temporary failure resolving 'archive.ubuntu.com'
[🔨]   #6 529.5 Err:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
[🔨]   #6 529.5   Temporary failure resolving 'archive.ubuntu.com'

 

Is anyone else encountering a similar issue? The builds seemed to be failing repeatedly at DNS resolution to the repositories. Didn't manage to get past this point as the builds failed repeatedly here many minutes to like 1/2 hour and abort, many times with no successes so far.

 

They also seem to be the same few accesses and possibly packages.

[🔨]   #6 481.4 Err:1 http://archive.ubuntu.com/ubuntu jammy InRelease
[🔨]   #6 481.4   Temporary failure resolving 'archive.ubuntu.com'
[🔨]   #6 529.4 Err:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
[🔨]   #6 529.4   Temporary failure resolving 'archive.ubuntu.com'
[🔨]   #6 577.5 Err:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
[🔨]   #6 577.5   Temporary failure resolving 'archive.ubuntu.com'
[🔨]   #6 577.5 Reading package lists...
[🔨]   #6 578.1 W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/jammy/InRelease  Temporary failure resolving 'archive.ubuntu.com'
[🔨]   #6 578.1 W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/jammy-updates/InRelease  Temporary failure resolving 'archive.ubuntu.com'
[🔨]   #6 578.1 W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/jammy-backports/InRelease  Temporary failure resolving 'archive.ubuntu.com'
[🔨]   #6 578.1 W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/jammy-security/InRelease  Temporary failure resolving 'security.ubuntu.com'
[🔨]   #6 578.1 W: Some index files failed to download. They have been ignored, or old ones used instead.

 

 

Edited November 20, 20232 yr by ag123

For the curious the build is run from within Ubuntu Jammy Jellyfish (22.04) release, running in a systemd-nspwan container, the docker engine is 'outside' the container in the host os.

There are lots of issues attempting to run docker engine from within a container.

 

That helps avoid dependency problems for the build as the build environment Ubuntu Jammy 22.04.x amd64 is stated.

https://docs.armbian.com/Developer-Guide_Build-Preparation/

 

The trick is the 'simple' docker.sock method

https://devopscube.com/run-docker-in-docker/

 

i.e. docker client (e.g. compile.sh) runs from within the Ubuntu Jammy Jellyfish container, but setup /var/run/docker.sock so that it can be accessed from within the container by docker client.

And outside the container - the host os can run the docker engine (e.g. dockerd, containerd), but sharing out /var/run/docker.sock so that it is the same unix socket accessed by docker client inside the container.

 

Jumped over many hoops, problems to get to this, but the 'network', 'dns resolver' issues remains a bummer.

 

i'm thinking this 'docker in docker' trick may even make it possible to build from within a docker container. e.g. from one docker container use a Ubuntu Jammy Jellyfish (22.04) and run compile.sh .

But that pror to that you need to 'remap' the docker.sock so that it can be accessed from the container. e.g.

https://devopscube.com/run-docker-in-docker/

 

Edited November 20, 20232 yr by ag123

made an attempt to extract /etc/apt/sources.list from the pulled image ghcr.io/armbian/docker-armbian-build,  armbian-ubuntu-jammy-latest.

 

replaced that in the Ubuntu Jammy build container /etc/apt/sources.list

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://archive.ubuntu.com/ubuntu/ jammy main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ jammy main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu/ jammy-updates main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ jammy universe
# deb-src http://archive.ubuntu.com/ubuntu/ jammy universe
deb http://archive.ubuntu.com/ubuntu/ jammy-updates universe
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://archive.ubuntu.com/ubuntu/ jammy multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy multiverse
deb http://archive.ubuntu.com/ubuntu/ jammy-updates multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ jammy-backports main restricted universe multiverse

deb http://security.ubuntu.com/ubuntu/ jammy-security main restricted
# deb-src http://security.ubuntu.com/ubuntu/ jammy-security main restricted
deb http://security.ubuntu.com/ubuntu/ jammy-security universe
# deb-src http://security.ubuntu.com/ubuntu/ jammy-security universe
deb http://security.ubuntu.com/ubuntu/ jammy-security multiverse
# deb-src http://security.ubuntu.com/ubuntu/ jammy-security multiverse

 

then

sudo apt-get update

 

but that did not resolve the issue, the errors seemed to be always the same hosts and seemingly same specific packages.

Edited November 20, 20232 yr by ag123

tried switching the base image to debian-bookworm, unfortunately a same issue appeared after some time during built

[🔨]   #6 [2/6] RUN echo "--> CACHE MISS IN DOCKERFILE: apt packages." &&  DEBIAN_FRONTEND=noninteractive apt-get -y update &&  DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends bash git psmisc uuid-runtime bc binfmt-support bison libc6-dev make dpkg-dev gcc ca-certificates ccache cpio debootstrap device-tree-compiler dialog dirmngr dosfstools dwarves flex gawk gnupg gpg imagemagick jq kmod libbison-dev libelf-dev libfdt-dev libfile-fcntllock-perl libmpc-dev libfl-dev liblz4-tool libncurses-dev libssl-dev libusb-1.0-0-dev linux-base locales lsof ncurses-base ncurses-term ntpdate patchutils pkg-config pv qemu-user-static rsync swig u-boot-tools udev uuid-dev zlib1g-dev file tree expect colorized-logs unzip zip pigz xz-utils pbzip2 lzop zstd parted gdisk fdisk aria2 curl wget axel parallel python3-dev python3-distutils python3-setuptools python3-pip gcc-x86-64-linux-gnu gcc-aarch64-linux-gnu gcc-arm-linux-gnueabihf gcc-arm-linux-gnueabi gcc-riscv64-linux-gnu debian-archive-keyring libc6-amd64-cross g++-aarch64-linux-gnu g++ btrfs-progs cryptsetup openssh-client f2fs-tools nilfs-tools xfsprogs zerofree qemu-utils qemu-utils libudev-dev libusb-1.0-0-dev dh-autoreconf build-essential gcc-arm-linux-gnueabi gcc-or1k-elf qemu-utils
[🔨]   #6 0.996 --> CACHE MISS IN DOCKERFILE: apt packages.
[🔨]   #6 49.05 Ign:1 http://deb.debian.org/debian bookworm InRelease
[🔨]   #6 97.10 Ign:2 http://deb.debian.org/debian bookworm-updates InRelease
[🔨]   #6 145.2 Ign:3 http://deb.debian.org/debian-security bookworm-security InRelease
...
[🔨]   #6 481.4   Temporary failure resolving 'deb.debian.org'
[🔨]   #6 529.4 Err:2 http://deb.debian.org/debian bookworm-updates InRelease
[🔨]   #6 529.4   Temporary failure resolving 'deb.debian.org'
[🔨]   #6 577.4 Err:3 http://deb.debian.org/debian-security bookworm-security InRelease
[🔨]   #6 577.4   Temporary failure resolving 'deb.debian.org'
[🔨]   #6 577.4 Reading package lists...
[🔨]   #6 577.8 W: Failed to fetch http://deb.debian.org/debian/dists/bookworm/InRelease  Temporary failure resolving 'deb.debian.org'
[🔨]   #6 577.8 W: Failed to fetch http://deb.debian.org/debian/dists/bookworm-updates/InRelease  Temporary failure resolving 'deb.debian.org'
[🔨]   #6 577.8 W: Failed to fetch http://deb.debian.org/debian-security/dists/bookworm-security/InRelease  Temporary failure resolving 'deb.debian.org'
[🔨]   #6 577.8 W: Some index files failed to download. They have been ignored, or old ones used instead.

 

as i'm not sure how to define that correctly, i patched lib/functions/host/docker.sh docker_cli_prepare()
 

declare -g DOCKER_ARMBIAN_BASE_IMAGE="${DOCKER_ARMBIAN_BASE_IMAGE:-"debian:bookworm"}"
# declare -g DOCKER_ARMBIAN_BASE_IMAGE="${DOCKER_ARMBIAN_BASE_IMAGE:-"ubuntu:jammy"}"

 

that managed to pick-up the Debian bookworm image, but the same resolving errors persists

 

 

 

Edited November 20, 20232 yr by ag123

The problem apparently is in the 

RUN apt-get -y update

step in the dockerfile, and it happens during the initial build.

It is observed after seperating the apt-get -y update step and the subsequent apt-get install step into 2 RUN commands.

 

For reasons unknown, the same step run in a systemd-nspawn container used for the build works perfectly fine and completes in less than 10 secs.

But that it fails while being run in the docker container during the build run by compile.sh

 

turns out there is various network issues in the build docker container driven by compile.sh, need to figure out how to fix that

 

there is no ping and ip command in the image, hence downloaded these

 

using apt-get download

• iproute2_5.15.0-1ubuntu2_amd64.deb - this is /usr/sbin/ip and friends
• iputils-ping_3%3a20211215-1_amd64.deb - this is ping

and dependencies

• libcap2-bin_1%3a2.44-1ubuntu0.22.04.1_amd64.deb
• libmnl0_1.0.4-3build2_amd64.deb
• libbpf0_1%3a0.5.0-1ubuntu22.04.1_amd64.deb
• libxtables12_1.8.7-1ubuntu5.1_amd64.de

and use 

docker cp, copy them into the container

https://docs.docker.com/engine/reference/commandline/cp/

from there run a shell (/bin/bash) and use dpkg -i to install them all

docker run -it [armbian-ubuntu-jammy-latest image ID] /bin/bash
root@cee6f1b4e249:/armbian# dpkg -i *deb

then docker commit

https://docs.docker.com/engine/reference/commandline/commit/

now there is a new image with /usr/sbin/ip and /usr/bin/ping

 

docker network is 'hairy' (lots of pitfalls), edit

lib/functions/host/docker.sh

https://github.com/armbian/build/blob/main/lib/functions/host/docker.sh#L361

run docker build with --network host, to use the host network to work around some issues

in addition use --add-host to work around some DNS issues.

well partly as my ipv6 setup is goofed, ipv6 won't route properly in my setup, most likely a cause of the problems.

 

 

Edited November 21, 20232 yr by ag123

got that NO_PUBKEY error again

 

tried patching

RUN DEBIAN_FRONTEND=noninteractive apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 871920D1991BC93C

Into the Dockerfile

(not directly but in the script https://github.com/armbian/build/blob/main/lib/functions/host/docker.sh#L276)

(^a possible improvement may be: check if the Dockerfile exists, if not then generate it. This may help in cases of Dockerfile experiments, e.g. user edits, but that it is deemed 'do not edit')

and run compile.sh again

[3/9] RUN DEBIAN_FRONTEND=noninteractive apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 871920D1991BC93C:
[🔨]   0.279 Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
[🔨]   0.286 Executing: /tmp/apt-key-gpghome.gobsOrrUwl/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys 871920D1991BC93C
[🔨]   6.301 gpg: connecting dirmngr at '/tmp/apt-key-gpghome.gobsOrrUwl/S.dirmngr' failed: End of file

 

apparently, keyserver.ubuntu.com is down, does not respond to pings whatever, try again later.
 

Edited November 21, 20232 yr by ag123

I have built multiple images in the last two days. Haven't faced the same. I will suggest you to check your network. Try using different dns server. Or try using different network if possible.

A colleague just pointed out to me that you are doing docker in docker. May be just run ./compile.sh PREFER_DOCKER=no to avoid spawning docker instance within docker?

hi Gunjan Gupta,

thanks much!

 

PREFER_DOCKER=no

works in the systemd-nspawn Ubuntu Jammy 22.04 container that I created to run compile.sh

 

for reasons I couldn't figure out, I did not manage to resolve the NO PUBKEY errors running in a docker container.

but that the run proceed significantly without issues doing apt-get update, apt-get install and all with PREFER_DOCKER=no

 

Edited November 21, 20232 yr by ag123

@Gunjan Gupta

Thanks, and a new guide/tutorial

 

Building Armbian using Ubuntu (jammy) in a systemd-nspawn container

https://gist.github.com/ag88/05245121cce37cb8f029424a20752e35

oops, building from within systemd-nspawn-container, at the end of it

[🌱] Unmounting [ /home/armbian/build/.tmp/rootfs-7074b0dd-8f8a-4d2a-9eb5-2064fe133932 ]
[🌿] Actual rootfs size [ 1638MiB ]
[🌱] Preparing image file for rootfs [ orangepizero2 jammy ]
[🌱] Current rootfs size [ 1638 MiB ]
[🌱] Creating blank image for rootfs [ truncate: 2136 MiB ]
[🌱] Creating partitions [ root: ext4 ]
[🔨]   Checking that no-one is using this disk right now ... OK
[🔨]   
[🔨]   Disk /home/armbian/build/.tmp/rootfs-7074b0dd-8f8a-4d2a-9eb5-2064fe133932.raw: 2.09 GiB, 2239758336 bytes, 4374528 sectors
[🔨]   Units: sectors of 1 * 512 = 512 bytes
[🔨]   Sector size (logical/physical): 512 bytes / 512 bytes
[🔨]   I/O size (minimum/optimal): 512 bytes / 512 bytes
[🔨]   
[🔨]   >>> Script header accepted.
[🔨]   >>> Created a new DOS disklabel with disk identifier 0x6335c2bb.
[🔨]   /home/armbian/build/.tmp/rootfs-7074b0dd-8f8a-4d2a-9eb5-2064fe133932.raw1: Created a new partition 1 of type 'Linux' and of size 2.1 GiB.
[🔨]   /home/armbian/build/.tmp/rootfs-7074b0dd-8f8a-4d2a-9eb5-2064fe133932.raw2: Done.
[🔨]   
[🔨]   New situation:
[🔨]   Disklabel type: dos
[🔨]   Disk identifier: 0x6335c2bb
[🔨]   
[🔨]   Device                                                                    Boot Start     End Sectors  Size Id Type
[🔨]   /home/armbian/build/.tmp/rootfs-7074b0dd-8f8a-4d2a-9eb5-2064fe133932.raw1       8192 4374527 4366336  2.1G 83 Linux
[🔨]   
[🔨]   The partition table has been altered.
[🔨]   Syncing disks.
losetup: cannot find an unused loop device: No such device
[💥] error! [ Unable to find free loop device  ]
[💥] Exiting with error 43 [ at /home/armbian/build/lib/functions/logging/traps.sh:1
                exit_with_error() --> lib/functions/logging/traps.sh:1
             prepare_partitions() --> lib/functions/image/partitioning.sh:218
                do_with_logging() --> lib/functions/logging/section-logging.sh:81
         build_rootfs_and_image() --> lib/functions/main/rootfs-image.sh:80
   full_build_packages_rootfs_and_image() --> lib/functions/main/default-build.sh:36
          do_with_default_build() --> lib/functions/main/default-build.sh:42
         cli_standard_build_run() --> lib/functions/cli/cli-build.sh:25
        armbian_cli_run_command() --> lib/functions/cli/utils-cli.sh:136
                 cli_entrypoint() --> lib/functions/cli/entrypoint.sh:176
                           main() --> compile.sh:50
 ]
[💥] Cleaning up [ please wait for cleanups to finish ]
[🌿] Unmounting recursively [ SDCARD - be patient ]
[🌿] Unmounting recursively [ MOUNT - be patient ]
[🌿] ANSI log file built; inspect it by running: [ less -RS output/logs/log-build-7074b0dd-8f8a-4d2a-9eb5-2064fe133932.log.ans ]
[🌿] Share log manually (or SHARE_LOG=yes): [ curl --data-binary @output/logs/log-build-7074b0dd-8f8a-4d2a-9eb5-2064fe133932.log.ans https://paste.next.armbian.com/log ]
armbian@snoopy1:~/build$ ls /dev
console  fd    hugepages  log     net   ptmx  random  stderr  stdout  urandom
core     full  initctl    mqueue  null  pts   shm     stdin   tty     zero

 

well, this is an artifact / limitations of the systemd-nspawn container

but that reaching here is pretty good, just that the last few steps (probably) needs to be manually done

Old Thread, but I was stuck with a similar Issue related to

 

Temporary failure resolving 'security.ubuntu.com'

 

Turns out in my Case I had to specify a custom NAMESERVER Build Switch.

 

Looking at the Reference Guide, the NAMESERVER Environment Variable defaults to 1.0.0.1.

https://docs.armbian.com/Developer-Guide_Build-Switches/#hidden-options-for-advanced-users-default-values-are-marked-bold

 

In my Case however, a DNS Block is enforced on my Main Router for all direct DNS Requests to any Nameserver (except the approved local DNS Servers, which are the only ones that can forward DNS Requests to Upstream DNS Servers).

 

The Solution was quite easy once that was figured out.

Just pass a local DNS Server as part as the Build Command 😃:

 

compile.sh \
build \
NAMESERVER=192.168.1.3 \
...

 

I only came across the Reference Guide by Chance. It was tricky because the Docker Container used for the build Process could resolve any host successfully (e.g. curl -4 ifconfig.me or even curl -6 ifconfig.me), but I guess that either another Container (Docker inside Docker) or systemd-nspawn does something weird of its own.

use system resolver instead of default:https://github.com/armbian/build/blob/60e869c42c03af0428d9983750eb766a7a846d43/config/templates/config-example.conf.template#L25