🐛 Cannot get the details of the consumed artifacts TypeError: Cannot read properties of null (reading 'dataProviders')

[FH-Inway]

Azure DevOps Extensions

Generate Release Notes (Node Cross Platform)

Platform

Azure DevOps Services

Azure DevOps Server (TFS) Version

No response

Extension Version

No response

Describe the bug

When running the release notes creation either locally with the tester or from a YAML pipeline, I'm getting the following message:

Get the artifacts consumed by the build
##vso[task.issue type=warning;]Cannot get the details of the consumed artifacts TypeError: Cannot read properties of null (reading 'dataProviders')

Both locally and in the pipeline, the task is executed on Windows.

When I directly call the API to get the consumed artifacts, it works. See https://gist.github.com/FH-Inway/cd6c717489f3a97709050219423d6c69

The message originates from

AzurePipelines/Extensions/XplatGenerateReleaseNotes/XplatGenerateReleaseNotesTask/src/ReleaseNotesFunctions.ts

Line 703 in 6c6cd2d

tl.warning(`Cannot get the details of the consumed artifacts ${err}`);

Any idea what causes the error or how to prevent it?

Repo Steps

Run the release notes creation e.g. with the build-consumed-artifact-table.md template.

Expected Behavior

No warning and data from the consumed artifacts is available and used to populate the template.

Logging Information

Get the artifacts consumed by the build
##vso[task.issue type=warning;]Cannot get the details of the consumed artifacts TypeError: Cannot read properties of null (reading 'dataProviders')

The log with system.debug=true does not seem to include any further debug information that could help with this issue.

[rfennell]

That is a strange one, other than the API call to ' ${tpcUri}/_apis/Contribution/HierarchyQuery/project/${teamProject}?api-version=5.1-preview,' there is nothing in the method, so it is hard to see what more debugging can be done.

In the test harness, you could maybe dump out the value of payload and make sure it is identical to your GIST.

Is there anything special about the artifact you are calling?

[FH-Inway]

Yeah, I was stumped, too. As far as I can tell, my GIST does the same as the call in getConsumedArtifactsForBuild. I already checked the payload from the test harness, but it just has an empty array for the consumed artifacts: "consumedArtifacts": [],
Or are you referring to the payload that gets used by the request call? How would I dump that using the test harness?

As far as I can tell, there is nothing special about the artifacts, just a bunch of other Git repositories that are in the same ADO organization and in GitHub.

I updated my GIST with the result: result.json

Maybe some additional error handling could be added in case the response from the call does not have the expected structure? And then do a debug output of the received response?

[rfennell]

My think had been that payload sent to the URL somehow differs to what you expect, so best to check it.

So on your local PC where you are running the local test harness (which you said showed the same issue), edit the ReleaseNotesFunctions.ts around Ln 675 to add a tl.debug(payload) to write it to the log e.g.

   var payload = {
                "contributionIds": [
                    "ms.vss-build-web.run-consumed-artifacts-data-provider"
                ],
                "dataProviderContext": {
                    "properties": {
                        "buildId": `${buildid}`,
                        "sourcePage": {
                            "url": `${tpcUri}/${teamProject}/_build/results?buildId=${buildid}&view=results`,
                            "routeId": "ms.vss-build-web.ci-results-hub-route",
                            "routeValues": {
                                "project": `${teamProject}`,
                                "viewname": "build-results",
                                "controller": "ContributedPage",
                                "action": "Execute"
                            }
                        }
                    }
                }
            };
            
             tl.debug(payload);
            
            let response = await restClient.create(
                `${tpcUri}/_apis/Contribution/HierarchyQuery/project/${teamProject}?api-version=5.1-preview`,
                payload);
            var result = response.result;

Hopefully it shows something strange, like an unexpected escaped character. You might have to fiddle with the formatting of the object when you log it out, as the default .tostring() might not render it that well

Problem I really can't see how any the inject parameters could cause a problem, they are all ones commonly used elsewhere

[FH-Inway]

I added this debug information: FH-Inway@b3d435a

Which gives me this output:

Get the artifacts consumed by the build
##vso[task.debug]{"contributionIds":["ms.vss-build-web.run-consumed-artifacts-data-provider"],"dataProviderContext":{"properties":{"buildId":"20979","sourcePage":{"url":"https://dev.azure.com/inwaysystems/KD Max Würstelmacher/_build/results?buildId=20979&view=results","routeId":"ms.vss-build-web.ci-results-hub-route","routeValues":{"project":"KD Max Würstelmacher","viewname":"build-results","controller":"ContributedPage","action":"Execute"}}}}}
##vso[task.debug]response: {"statusCode":404,"result":null,"headers":{}}
##vso[task.debug]response.result: null
##vso[task.issue type=warning;]Cannot get the details of the consumed artifacts TypeError: Cannot read properties of null (reading 'dataProviders')
##vso[task.issue type=warning;]API call result was null

So it is a permission issue, which is very curious, because I'm running both the test harness and my PowerShell script with the same PAT that has full access.
Any idea why the request from the test harness would have less permissions? It can retrieve all the other stuff, just not the consumed artifacts.
The request payload is identical to the one I use with the PowerShell script. Just to be sure, I also took the request from the output and used that as payload for my PowerShell request. Same result as before.

[rfennell]

Well that makes sense, permissions causing an error.

I agree hard to see why the same PAT used for the tester and your script give different results.

However it could be a platform issue i.e Node and Powershell?

Maybe the handling of the name ü in Max Würstelmacher?

[FH-Inway]

Good instinct. I added html encoding for the project part in the url and finally got the artifacts: FH-Inway@59f83cb

There seems to be another bug in the test harness when running it with a local payload.json. Although the json contains the consumed artifacts, they are not recognized and the debug log says

##vso[task.debug] Consumed Artifacts: 0

When running it without the payload.json file against the ADO API, the consumed artifacts are recognized:

##vso[task.debug] Consumed Artifacts: 3

[rfennell]

Great that got you unblocked, there are been a few issues over the years with encoding.

On the payload.json front, is there possibly a similar encoding issue left over from a previous run?

Maybe the actual if serialising the data to the file is the issue?

[FH-Inway]

Well, we figured out the issue. To unblock me, the extension in the marketplace would need to be updated. I will try to work on a pull request for the fix.
I wonder though if the html encoding should not be handled in a more general way?

Regarding payload.json, I will give it another try, but I'm pretty sure I took the latest payload.json file after a successful run. Otherwise, the consumed artifacts would not appear in the file.

[FH-Inway]

Started #2022 for the main issue.

I was looking into a more general solution, as there seem to be a number of cases where the project name is used in urls without encoding. Since this would touch a lot of files and require some testing as to whether this actually is an issue in the other cases, I decided to not do this as part of this pr.

[FH-Inway]

Tested this successfully with the 4.22.4 version.