User Info fetcher: Improve http request error output

[NickLarsenNZ]

Problem

Currently, we raise an error if the response to an HTTP request is 4xx or 5xx. We only know which part of the code the User Info Fetcher failed on. We have no HTTP response body in the error to give a hint to the problem. Eg: Is it a WAF or proxy issue, or are we reaching the idP but some configuration is wrong?

opa-operator/rust/user-info-fetcher/src/backend/keycloak.rs

Lines 11 to 14 in a4a6f44

	#[derive(Snafu, Debug)]
	pub enum Error {
	#[snafu(display("failed to get access_token"))]
	AccessToken { source: reqwest::Error },

The difficulty in adding the extra indfoamtion is that the Reqwest library consumes the response when calling methods like .text() (or .json()), or error_for_status(), and we need a combination of both methods.

Suggested fix

Don't make use of error_for_status() (which simply checks for 4xx or 5xx errors) and instead manually check the status code, and then either response with the json(), or produce a Snafu error containing the text() (which is fallible, so consideration should be made how to handle that).

Acceptance criteria

• Snafu errors for util::send_json_request
• 4xx and 5xx still produce an error, but also contain any body output
  • Consider what happens if the decoding fails, or is very long.

[NickLarsenNZ]

@fhennig, here was my Snafu that I made for util.rs

#[derive(Snafu, Debug)]
pub enum Error {
    #[snafu(display("failed to execute request"))]
    HttpRequest { source: reqwest::Error },

    #[snafu(display("failed to parse json response"))]
    ParseJson { source: reqwest::Error },

    #[snafu(display("response was an http error"))]
    HttpErrorResponse { source: reqwest::Error },

    #[snafu(display("response was an http error: {text}"))]
    Something { text: String },
}

That last one was the one most relevant to this issue. The rest would be necessary to return Snafu's regardless of this issue being solved.)

❌ A failed attempt

I got this far before realising it won't work, and we should just use do response.status() rather than using response.error_for_status()

pub async fn send_json_request3<T: DeserializeOwned>(req: RequestBuilder) -> Result<T> {
    let response = req.send().await.context(HttpRequestSnafu)?;

    match response.error_for_status() {
        Ok(response) => {
            let json = response.json().await.context(ParseJsonSnafu)?;
            Ok(json)
        }
        Err(error) => {
            let text = response
                .text() // try bytes
                .await
                .unwrap_or("unable to decode response body as text".to_owned());

            Err(Error::Something { text })

            // Err(Error::Something {
            //     text: "blah".to_owned(),
            // })
        }
    }
}