NIFI-10831 enable JWT realm authentication with Elasticsearch

[ChrisSamo632]

Summary

NIFI-10831 enable JWT realm authentication with Elasticsearch.

Also allow for setting of runas_user in the Elasticsearch Controller Service, which may be of use to impersonate a user within Elasticsearch from NiFi, without having the user's credentials (but where the connecting NiFi user/token is granted permission to impersonate the alternate user).

Other changes include:

• removing unused dependencies from Elasticsearch component POMs

Testing:

• Attempted to verify changes using LogTo.io as an OIDC and JWT IdP, but it turns out that LogTo provides Access Token JWTs with a typ of at+jwt, which Elasticsearch cannot currently handle (Support at+jwt Types in JWT Realm elastic/elasticsearch#119370)
• Verified using Keycloak instead (with a machine-to-machine/service-account only Client setup to provide Access Tokens)
• Configure Elasticsearch to allow JWT realm authentication, an example might be like:

    jwt.keycloak_test_jwt:
      order: 3
      token_type: access_token
      client_authentication.type: shared_secret
      allowed_issuer: https://localhost:9443/realms/elasticsearch-test
      allowed_audiences: [account]
      allowed_subjects: [e61503d0-8fab-473e-9f70-95d91b085f52]
      allowed_signature_algorithms: [ES384]
      pkc_jwkset_path: https://keycloak:9443/realms/elasticsearch-test/protocol/openid-connect/certs
      claims.principal: preferred_username
      ssl.certificate_authorities:
        [/usr/share/elasticsearch/config/step/certs/root_ca.crt]

Note that NiFi requires the IdP to be running over HTTPS (for which SmallStep CA was used to generate certificates locally).

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
  • JDK 21

Licensing

• [ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
• [ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• [ ] Documentation formatting appears as expected in rendered files

[exceptionfactory]

Thanks for introducing JWT support and describing the details @ChrisSamo632.

Of particular note, since the new properties depend on the JWT scheme, they can be marked as required, which removes the need for custom validation and conditional checking.

On a related note, with the introduction of Run As User support here, does that remove one of the potential use cases for custom HTTP request headers in the related pull request?

[exceptionfactory]

Thanks for the updates @ChrisSamo632, just one additional note on the jackson-databind dependency, and then this looks ready to go.

[exceptionfactory]

Thanks for making the adjustments and for adding this new feature @ChrisSamo632! +1 merging