Speed is too slow. Is there any way to speed up?

[yjcn]

I try to build qBittorrent. It works well, but speed is very slow. Travis CI only uses 9 min, github action uses 1h18m
Github Action arm64

Travis CI arm64

[uraimo]

Thanks for pointing this out with real numbers @yjcn , since the general slowness is a problem for me too in other projects.
Right now github provides only x86_64 vms to run CI jobs and this projects can only encapsulate your jobs in an aarch64 emulated environment (via QEmu) to provide something similar to a real arm64 system.

On the other hand, Travis CI runs the jobs on real arm64 hardware (servers with a 32 cores X-Gene 3 APM883832-X3 cpu) and that's the reason for the difference you see right now.

It could be interesting to see if they have dedicated machines for ppc64le and s390x too, and if not to compare the numbers and see if, on similar underlying hosts, there is space for improvements tweaking some parameters/options in QEmu.

Refs:
https://travis-ci.community/t/apt-addon-does-not-work-for-armhf-on-arm64/6374
https://travis-ci.community/c/environments/multi-cpu-arch
https://docs.travis-ci.com/user/multi-cpu-architectures/

[uraimo]

Just for future reference, the 18.04 distro is using 2.11+dfsg-1ubuntu7.20 that should include a performance fix for 1740219 (should be included in 2.11+dfsg-1ubuntu6, included for sure in 2.12 onwards), that affects every release of qemu after 2.10.

References:
multiarch/qemu-user-static#35
https://forums.gentoo.org/viewtopic-t-1077144-start-0-postdays-0-postorder-asc-highlight-.html
https://bugs.launchpad.net/qemu/+bug/1740219
https://gist.github.com/Liryna/10710751
https://wiki.ubuntu.com/ARM/RootfsFromScratch/QemuDebootstrap
https://wiki.gentoo.org/wiki/Crossdev_qemu-static-user-chroot

[elijahr]

@yjcn we recently released some changes that speed up builds. The architectures are still emulated, but if some of your build time is spent installing dependencies you can look into using the new install and githubToken input parameters.

As for faster compilation, you could try installing ccache in the container and persisting ccache's outputs between builds using actions/cache@v2, something like this:

name: Build

on:
  push:
    branches: [ '*' ]
    tags: [ '*' ]
  pull_request:
    branches: [ '*' ]

jobs:
  # Build for Linux aarch64, ppc64le, etc
  build_linux_other_archs:
    name: Build ubuntu-18.04
    runs-on: ubuntu-latest

    strategy:
      matrix:
        include:
          - arch: aarch64
          - arch: ppc64le

    steps:
      - name: Setup cache
        uses: actions/cache@v2
        with:
          path: |
            ~/.ccache
          key: build_ubuntu-18.04_${{ matrix.arch }}

      - uses: actions/checkout@v2.1.0

      - name: Build
        id: build
        uses: uraimo/run-on-arch-action@v2.0.4
        with:
          arch: ${{ matrix.arch }}
          distro: ubuntu18.04

          # Upload docker image to your project's public package registry
          githubToken: ${{ github.token }}

          # Create cached/volume directories on host
          setup: |
            mkdir -p ~/.ccache

          # Mount cached directories in the container for faster builds
          dockerRunArgs: |
            --volume "${HOME}/.ccache:/root/.ccache"

          # Install ccache and gcc/g++ symlinks in the container
          install: |
            apt-get update -q -y
            apt-get install build-essential ccache -q -y
            /usr/sbin/update-ccache-symlinks
            echo 'export PATH="/usr/lib/ccache:$PATH"' | tee -a ~/.bashrc

          # compile things in the container
          run: |
            ...

I can't guarantee this will give a performance gain, but maybe.

[bradbell]

I am trying to run on ppc64le using this package; see
/~https://github.com/coin-or/CppAD/blob/master/.github/workflows/uraimo_arch_linux.yml

The current state of the job is

 Build on alpine_latest ppc64le
Started 46m 46s ago
Waiting for a runner to pick up this job...

It this just slow, or is there some other problem ?

I cancelled the job with it got to the following state:

 Build on alpine_latest ppc64le
Started 9h 23m 5s ago
Waiting for a runner to pick up this job...

[uraimo]

@bradbell github issue, thanks for reporting, I'll keep an eye out to understand if it's becoming more frequent for other projects too.

[Dan-wanna-M]

The same problem happened to me as well: /~https://github.com/Dan-wanna-M/kbnf/actions/runs/9588165976/job/26439674105.

[uraimo]

If it's the free tier you are using, the waiting times are pretty large and completely random at times. I've seen jobs being stuck for days and some other times failing after a few hours of waiting. Extremely unpredictable.

[Dan-wanna-M]

I think my jobs are actually being run since they quickly exit queued status? The build time is also stable between different triggering 获取Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: uraimo ***@***.***> Sent: Thursday, June 20, 2024 1:01:23 AM To: uraimo/run-on-arch-action ***@***.***> Cc: Huanghe ***@***.***>; Comment ***@***.***> Subject: Re: [uraimo/run-on-arch-action] Speed is too slow. Is there any way to speed up? (#4) If it's the free tier you are using, the waiting times are pretty large and completely random at times. I've seen jobs being stuck for days and some other times failing after a few hours of waiting. Extremely unpredictable. ― Reply to this email directly, view it on GitHub<#4 (comment)>, or unsubscribe</~https://github.com/notifications/unsubscribe-auth/ANIMAX6NCJHUJRETQ7ATDILZIJV3HAVCNFSM6AAAAABJSX3LQ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZZHA4DANJTHA>. You are receiving this because you commented.Message ID: ***@***.***>

[uraimo]

Yes, that's what happened in this case, overall very unpredictable.

[hipersayanX]

@uraimo I'm conducting the tests in my projects, I get the following build times:

	armv7	aarch64
Debian	1h 40m	1h 40m
Ubuntu	1h 40m	> 6h

I perfectly understand that the build time will be much longer than running in the host architecture, so the expected time in my case is 1h 40m, but look that Ubuntu aarch64 is the only one giving a painfully slow build time, there seems to be that specific image that is having some problem there, note that in all cases I'm installing the same packages with the same environment so the build times are expected to be more or less the same, its maybe some problem with that image and not in your side.