JDK 21 preview updatecli manifest is failing

[lemeurherve]

Jenkins and plugins versions report

N/A

What Operating System are you using (both controller, and any agents involved in the problem)?

N/A

Reproduction steps

Observe updatecli actions logs in this repository.

Expected Results

No error in logs.

Actual Results

Error in logs:

SOURCES
=======

getLatestJDK21PreviewVersion
----------------------------
Searching for version matching pattern ".*-ea-.*"
✔ Git tag "jdk-21.0.4+1-ea-beta" found matching pattern ".*-ea-.*" of kind "regex"


CONDITIONS:
===========

checkIfReleaseIsAvailable
-------------------------
The shell 🐚 command "/bin/sh /var/folders/ly/77mv7l9968g068w9_fwzwggw0000gp/T/updatecli/bin/d3ffbc3fbbe15c1cb14dacbe20856423b65a42ac0add7e5e19d1b29ad3fe138d.sh" exited on error (exit code 1) with the following output:
----
ERROR: the following URL is NOT available: /~https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.4%2B1-ea-beta/OpenJDK21U-jdk_x64_linux_hotspot_21.0.4_1-ea.tar.gz.

Anything else?

No response

Are you interested in contributing a fix?

No response

[lemeurherve]

Not sure why it's failing on main, on #796 but not on #786, not on #799 and not locally 🤔

[gounthar]

updatecli version maybe?

[dduportal]

The error is clearly related to the shell script:

<...> exited on error (exit code 1) with the following output:

=> worth executing it in debug over there

[gounthar]

The script is failing, but the URL is valid. 🤔

[lemeurherve]

The script is failing, but the URL is valid. 🤔

/~https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.4%2B1-ea-beta/OpenJDK21U-jdk_x64_linux_hotspot_21.0.4_1-ea.tar.gz was returning a 404 error yesterday 🤔

Maybe a GitHub intermittent issue that would explain why it succeeded sometimes?

[lemeurherve]

FTR, the URL from #772 isn't the same cf https://ci.jenkins.io/job/Packaging/job/docker-agent/job/PR-772/16/pipeline-console/?selected-node=102

- /~https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.4%2B1-ea-beta/OpenJDK21U-jdk_x64_linux_hotspot_21.0.4_1-ea.tar.gz
+ /~https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.4%2B1-ea-beta/OpenJDK21U-jdk_arm_linux_hotspot_ea_21-0-4-1.tar.gz

[dduportal]

Is the error reproducible locally?
And if yes, is running the shell script (without updatecli) allows to reproduce the error?

[lemeurherve]

Is the error reproducible locally?

No (cf #798 (comment))

Not reproduced in CI anymore and version correctly bumped by updatecli in #772.
The manifest is valid in the sense of even if it returns a wrong link, at least it doesn't fail the updatecli pipeline.

Important note, this jdk21 preview manifest isn't using the adoptium scripts to retrieve the JDK download link (it was introduced before them): /~https://github.com/jenkinsci/docker-agent/blob/master/updatecli/updatecli.d/jdk21-preview.yaml

The adoptium scripts need additional logic if we want to use them for that as this preview version is an "EA" one, not a "GA".
The Adoptium API logic to retrieve data is not the same. (From what I recall and if I'm not mistaken, multiple EA releases can correspond to a JDK version).

I'd suggest fixing this jdk21 preview manifest to return the correct URL for now instead of increasing the complexity of the adoptium scripts immediately.

[lemeurherve]

Important note, this jdk21 preview manifest isn't using the adoptium scripts to retrieve the JDK download link

Nor the jdk21 preview Dockerfile cf

docker-agent/debian/preview/Dockerfile

Lines 15 to 27 in d473399

	# hadolint ignore=DL3008
	RUN set -x; apt-get update \
	&& apt-get install --no-install-recommends -y \
	ca-certificates \
	jq \
	wget \
	&& DASHED_JAVA_VERSION=$(echo "$JAVA_VERSION" | sed -e 's/\./-/g' -e 's/+/-/' -e 's/21-35/21-0-35/') \
	&& JAVA_MAJOR_VERSION=$(echo "$JAVA_VERSION" | cut -d'+' -f1 | cut -d'.' -f1) \
	&& ENCODED_JAVA_VERSION=$(echo "$JAVA_VERSION" | jq "@uri" -jRr) \
	&& CONVERTED_ARCH=$(arch | sed -e 's/x86_64/x64/' -e 's/armv7l/arm/') \
	&& wget --quiet /~https://github.com/adoptium/temurin"${JAVA_MAJOR_VERSION}"-binaries/releases/download/jdk-"${ENCODED_JAVA_VERSION}"-ea-beta/OpenJDK"${JAVA_MAJOR_VERSION}"U-jdk_"${CONVERTED_ARCH}"_linux_hotspot_ea_"${DASHED_JAVA_VERSION}".tar.gz -O /tmp/jdk.tar.gz \
	&& tar -xzf /tmp/jdk.tar.gz -C /opt/ \
	&& rm -f /tmp/jdk.tar.gz

Last URL from https://ci.jenkins.io/job/Packaging/job/docker-agent/job/PR-772/25/pipeline-console/?selected-node=94:

/~https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.4%2B2-ea-beta/OpenJDK21U-jdk_arm_linux_hotspot_ea_21-0-4-2.tar.gz (404 not found)

I'd suggest fixing this jdk21 preview manifest

or the jdk21 Dockerfile logic.

[lemeurherve]

21.0.4+2 is absent from /~https://github.com/adoptium/temurin21-binaries/releases 🤔

[dduportal]

21.0.4+2 is absent from /~https://github.com/adoptium/temurin21-binaries/releases 🤔

/~https://github.com/adoptium/temurin21-binaries/releases/tag/jdk-21.0.4%2B2-ea-beta

[dduportal]

Is the error reproducible locally?

No (cf #798 (comment))

Not reproduced in CI anymore and version correctly bumped by updatecli in #772.

I don't understand what the problem is then. The updatecli manifest is made to retrieve the "latest" version matching a certain pattern as source, and the condition verify that this version has the proper binaries available (because between creating a tag and having all binary released, a lot of time can pass).

The partial log output you've added in the body message shows that the condition failed, which is usually because the binaries are NOT available (yet).

The manifest is valid in the sense of even if it returns a wrong link, at least it doesn't fail the updatecli pipeline.

Yes, that is and always has been the whole puprose of running the check in an "update condition".

=> What is the problem we want to solve, considering all of this thread? I'm asking because it looks like the error was "transient" (which happens on each Temurin release as they don't release everything immediately) so the issue might be closeable but I might be missing something?

[lemeurherve]

This issue can be closed but the jdk21 Dockerfile needs a fix as the final URL called in the image build /~https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.4%2B2-ea-beta/OpenJDK21U-jdk_arm_linux_hotspot_ea_21-0-4-2.tar.gz is invalid.