[CORE-7152] Admin: Introduce `GET /v1/features/enterprise` endpoint #23314

oleiman · 2024-09-13T00:41:43Z

This PR introduces an Admin API endpoint to expose

Whether or not each enterprise licensed feature is enabled
The current license status of the cluster (valid, expired, or not present)
Whether the current set of enabled enterprise features violates the license
- i.e. violation = (any enterprise feature enabled) AND (license status != valid)

Includes integration tests for the endpoint and a "feature report" representation for config state.

Fixes CORE-7152

Backports Required

Release Notes

Features

Adds admin API endpoint for enterprise feature info GET /v1/features/enterprise

michael-redpanda

looks really good

michael-redpanda · 2024-09-16T20:14:45Z

tests/rptest/tests/enterprise_features_license_test.py

+        violation = rsp.get('violation', None)
+        assert type(
+            violation) == bool, f"Ill-formed violation flag {type(violation)}"
+        assert not violation, "Config unexpectedly in violation"


fix/test: will probably in FIPS CDT as FIPS is enabled in the FIPS CDT environment

right, right, we talked about this the other day. per below, will give a run through CDT and see what happens.

added fips_ok_to_fail to avoid too much mucking around. lmk if you think it's worth refactoring the tests to account for this.

tests/rptest/tests/enterprise_features_license_test.py

michael-redpanda · 2024-09-16T20:18:58Z

Once initial CI completes let's run a "/cdt fips "

vbotbuildovich · 2024-09-16T20:23:31Z

new failures in https://buildkite.com/redpanda/redpanda/builds/54529#0191fc48-b238-4427-9ff6-97c8d35ff9a1:

"rptest.tests.enterprise_features_license_test.EnterpriseFeaturesTest.test_get_enterprise"

new failures in https://buildkite.com/redpanda/redpanda/builds/54529#0191fc4b-47f3-4c1f-8d94-c1278905b7e1:

"rptest.tests.enterprise_features_license_test.EnterpriseFeaturesTest.test_get_enterprise"

new failures in https://buildkite.com/redpanda/redpanda/builds/54545#0191fd0d-6347-47ad-a2f6-a8f7862da9cf:

"rptest.tests.delete_records_test.DeleteRecordsTest.test_delete_records_concurrent_truncations.cloud_storage_enabled=True.truncate_point=start_offset"

new failures in https://buildkite.com/redpanda/redpanda/builds/54570#0191feb1-8056-482d-8eb3-fac453fd76f5:

"rptest.tests.data_migrations_api_test.DataMigrationsApiTest.test_migrated_topic_data_integrity.transfer_leadership=True.params=TmtpdiParams.cancellation=None.use_alias=False"

oleiman · 2024-09-16T20:31:09Z

oh, whoops, my license tests are bugged - forgot to populate the env var on the last few runs. back to draft for a sec

vbotbuildovich · 2024-09-16T20:51:08Z

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/54529#0191fc48-b23b-4879-ae17-d30346b9eb12

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/54545#0191fd0d-634c-436d-b50e-9142b27aa051

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/54545#0191fd0d-6347-47ad-a2f6-a8f7862da9cf

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/54545#0191fd0d-6344-4217-bbc9-e463daf7dd26

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/54631#01920249-e915-46ff-aedf-03a23a2ca4c8

oleiman · 2024-09-16T21:16:02Z

force push contents:

CR comments (dead code, etc)
Fixed wonky license testing.

src/v/model/metadata.h

src/v/cluster/feature_manager.cc

dotnwat · 2024-09-17T02:34:59Z

src/v/redpanda/admin/server.cc

+    res.license_status = license_status;
+    res.violation = license_status != status::valid && report.any();
+    for (std::size_t i = 0; i < report.size(); ++i) {
+        auto feat = static_cast<model::license_required_feature>(i);


if the feature set were represented as a vector of enum then we could avoid this casting from integer to enum, right? is the size of the feature set important? it seems like right now it would be about 20 bytes max and one would only exist in memory for the lifetime of the request?

feature set were represented as a vector of enum

we want both enabled and disabled features, so I was choosing between bitset and map<enum,bool>, both of which are kinda smelly IMO. not really about size, just teasing out a "natural" representation for both the state of each feature and whether any enterprise feature is ON.

that said I think the answer is neither - 2x vector<enum> is 👌

oleiman · 2024-09-17T05:16:08Z

force push move report bits to v/features and clean up interface

oleiman · 2024-09-17T05:25:16Z

force push bazel flub

oleiman · 2024-09-17T15:51:06Z

force push another bazel flub

src/v/features/enterprise_features.h

dotnwat · 2024-09-17T16:58:28Z

src/v/features/enterprise_features.h

+    bool any() const { return !_enabled.empty(); }
+
+private:
+    vtype _enabled;
+    vtype _disabled;


dotnwat · 2024-09-17T16:59:25Z

src/v/cluster/feature_manager.cc


-    return cfg.audit_enabled || cfg.cloud_storage_enabled
-           || cfg.partition_autobalancing_mode
-                == model::partition_autobalancing_mode::continuous
-           || cfg.core_balancing_continuous() || has_gssapi() || has_oidc()
-           || has_schma_id_validation() || has_non_default_roles
-           || fips_enabled();
+    features::enterprise_feature_report report;
+    report.set(
+      features::license_required_feature::audit_logging, cfg.audit_enabled());
+    report.set(


awesome. this diff is great

dotnwat · 2024-09-17T17:38:49Z

src/v/redpanda/admin/server.cc

+          ss::httpd::features_json::enterprise_feature elt;
+          elt.name = fmt::format("{}", feat);
+          elt.enabled = enabled;


src/v/features/enterprise_features.cc

pgellert

lgtm, mainly nits and test-related suggestions (nit: the PR header is slightly out of date ~~as a bitset~~)

What's the reasoning behind only backporting this to v24.2.x?

tests/rptest/tests/enterprise_features_license_test.py

src/v/redpanda/admin/api-doc/features.json

oleiman · 2024-09-17T20:52:40Z

force push minor test improvements (formatting, fips_ok_to_fail) and some asserts in feature report

- features::license_required_feature Enumeration of redpanda features that require an enterprise license. - features::enterprise_feature_report A thin wrapper around a couple of sets to account for the status of enterprise features (usually based on cluster configs). Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

Also refactor feature_manager::license_required_feature_enabled to use the report rather than calculating feature status on its own. Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

Useful for testing enterprise features, since a condition for "RBAC enabled" is the presence of one or more non-default roles. Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

Includes scenarios both with and without a valid license loaded up. Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

oleiman · 2024-09-17T21:41:31Z

force push rebase dev to get skip_fips_mode
force push to fix bazel

oleiman · 2024-09-17T22:24:05Z

CI Failure:

bazel build failure looks transient? retrying

vbotbuildovich · 2024-09-18T14:16:08Z

/backport v24.2.x

vbotbuildovich · 2024-09-18T14:17:16Z

Failed to create a backport PR to v24.2.x branch. I tried:

git remote add upstream /~https://github.com/redpanda-data/redpanda.git
git fetch --all
git checkout -b backport-pr-23314-v24.2.x-820 remotes/upstream/v24.2.x
git cherry-pick -x bf972c04d80c297ed9322e2e6501cb8cda9ec63f 067df42f1df3d30b50230d1c3df34efaafc276ae 653d5a979aba1e50f37dee31b6cf231fb1fcafc8 d6e33cd177f87c52c847e7398a9384c73c6d34b3 5fdc1782506e85d43c05b06f4551f5627017494a b5c06fa59d7c149bdb5b723592f16c4d04a5db64 ebfdd7fe27a4f441e2379397501c639420724fc1

Workflow run logs.

oleiman · 2024-09-19T21:25:22Z

/cdt
tests/rptest/tests/enterprise_features_license_test.py

oleiman · 2024-10-11T21:22:10Z

/backport v24.1.x

vbotbuildovich · 2024-10-11T21:23:47Z

Failed to create a backport PR to v24.1.x branch. I tried:

git remote add upstream /~https://github.com/redpanda-data/redpanda.git
git fetch --all
git checkout -b backport-pr-23314-v24.1.x-779 remotes/upstream/v24.1.x
git cherry-pick -x bf972c04d8 067df42f1d 653d5a979a d6e33cd177 5fdc178250 b5c06fa59d ebfdd7fe27

Workflow run logs.

oleiman self-assigned this Sep 13, 2024

github-actions bot added the area/redpanda label Sep 13, 2024

oleiman force-pushed the license/core-7152/admin-features-endpoint branch 2 times, most recently from cc05a65 to 23b1634 Compare September 13, 2024 21:42

oleiman marked this pull request as ready for review September 16, 2024 18:04

oleiman requested a review from a team as a code owner September 16, 2024 18:04

oleiman requested review from a team, pgellert and michael-redpanda and removed request for a team September 16, 2024 18:04

michael-redpanda reviewed Sep 16, 2024

View reviewed changes

oleiman marked this pull request as draft September 16, 2024 20:31

oleiman force-pushed the license/core-7152/admin-features-endpoint branch from 23b1634 to 6943ee6 Compare September 16, 2024 21:14

oleiman marked this pull request as ready for review September 16, 2024 21:16

dotnwat reviewed Sep 17, 2024

View reviewed changes

oleiman force-pushed the license/core-7152/admin-features-endpoint branch from 6943ee6 to b0afb4b Compare September 17, 2024 05:14

github-actions bot added the area/build label Sep 17, 2024

oleiman force-pushed the license/core-7152/admin-features-endpoint branch from b0afb4b to 9b13050 Compare September 17, 2024 05:23

oleiman force-pushed the license/core-7152/admin-features-endpoint branch from 9b13050 to 77e64c4 Compare September 17, 2024 15:49

oleiman force-pushed the license/core-7152/admin-features-endpoint branch from 77e64c4 to f60fc59 Compare September 17, 2024 15:57

dotnwat reviewed Sep 17, 2024

View reviewed changes

pgellert reviewed Sep 17, 2024

View reviewed changes

oleiman requested review from pgellert, dotnwat and michael-redpanda September 17, 2024 20:54

dotnwat previously approved these changes Sep 17, 2024

View reviewed changes

oleiman dismissed dotnwat’s stale review via 44e8f6c September 17, 2024 21:40

oleiman force-pushed the license/core-7152/admin-features-endpoint branch from 689e482 to 44e8f6c Compare September 17, 2024 21:40

oleiman added 7 commits September 17, 2024 14:40

c/feature_manager: Generate an enterprise features report

067df42

Also refactor feature_manager::license_required_feature_enabled to use the report rather than calculating feature status on its own. Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

admin/api: Swagger for get_enterprise

653d5a9

Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

admin: get_enterprise handler

d6e33cd

Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

dt/admin: Add node param to list_roles

5fdc178

Useful for testing enterprise features, since a condition for "RBAC enabled" is the presence of one or more non-default roles. Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

dt/admin: Add get_enterprise

b5c06fa

Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

dt/enterprise: Tests for enterprise features endpoint

ebfdd7f

Includes scenarios both with and without a valid license loaded up. Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>

oleiman force-pushed the license/core-7152/admin-features-endpoint branch from 44e8f6c to ebfdd7f Compare September 17, 2024 21:40

dotnwat approved these changes Sep 18, 2024

View reviewed changes

pgellert approved these changes Sep 18, 2024

View reviewed changes

michael-redpanda approved these changes Sep 18, 2024

View reviewed changes

oleiman merged commit 6d6a87c into redpanda-data:dev Sep 18, 2024
18 checks passed

vbotbuildovich mentioned this pull request Sep 18, 2024

[v24.2.x] [CORE-7152] Admin: Introduce GET /v1/features/enterprise endpoint #23365

Closed

oleiman mentioned this pull request Sep 18, 2024

[v24.2.x] [CORE-7152] Admin: Introduce endpoint #23368

Merged

vbotbuildovich mentioned this pull request Oct 11, 2024

[v24.1.x] [CORE-7152] Admin: Introduce GET /v1/features/enterprise endpoint #23760

Closed

oleiman mentioned this pull request Oct 11, 2024

[v24.1.x] [CORE-7152] Admin: Introduce endpoint #23761

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CORE-7152] Admin: Introduce `GET /v1/features/enterprise` endpoint #23314

[CORE-7152] Admin: Introduce `GET /v1/features/enterprise` endpoint #23314

oleiman commented Sep 13, 2024 •

edited

Loading

michael-redpanda left a comment

michael-redpanda Sep 16, 2024

oleiman Sep 16, 2024

oleiman Sep 17, 2024

michael-redpanda commented Sep 16, 2024

vbotbuildovich commented Sep 16, 2024 •

edited

Loading

oleiman commented Sep 16, 2024

vbotbuildovich commented Sep 16, 2024 •

edited

Loading

oleiman commented Sep 16, 2024

dotnwat Sep 17, 2024

oleiman Sep 17, 2024

oleiman commented Sep 17, 2024

oleiman commented Sep 17, 2024

oleiman commented Sep 17, 2024

dotnwat Sep 17, 2024

dotnwat Sep 17, 2024

dotnwat Sep 17, 2024

pgellert left a comment

oleiman commented Sep 17, 2024

oleiman commented Sep 17, 2024

oleiman commented Sep 17, 2024

vbotbuildovich commented Sep 18, 2024

vbotbuildovich commented Sep 18, 2024

oleiman commented Sep 19, 2024

oleiman commented Oct 11, 2024

vbotbuildovich commented Oct 11, 2024

[CORE-7152] Admin: Introduce GET /v1/features/enterprise endpoint #23314

[CORE-7152] Admin: Introduce GET /v1/features/enterprise endpoint #23314

Conversation

oleiman commented Sep 13, 2024 • edited Loading

Backports Required

Release Notes

Features

michael-redpanda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michael-redpanda commented Sep 16, 2024

vbotbuildovich commented Sep 16, 2024 • edited Loading

oleiman commented Sep 16, 2024

vbotbuildovich commented Sep 16, 2024 • edited Loading

oleiman commented Sep 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleiman commented Sep 17, 2024

oleiman commented Sep 17, 2024

oleiman commented Sep 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pgellert left a comment

Choose a reason for hiding this comment

oleiman commented Sep 17, 2024

oleiman commented Sep 17, 2024

oleiman commented Sep 17, 2024

vbotbuildovich commented Sep 18, 2024

vbotbuildovich commented Sep 18, 2024

oleiman commented Sep 19, 2024

oleiman commented Oct 11, 2024

vbotbuildovich commented Oct 11, 2024

[CORE-7152] Admin: Introduce `GET /v1/features/enterprise` endpoint #23314

[CORE-7152] Admin: Introduce `GET /v1/features/enterprise` endpoint #23314

oleiman commented Sep 13, 2024 •

edited

Loading

vbotbuildovich commented Sep 16, 2024 •

edited

Loading

vbotbuildovich commented Sep 16, 2024 •

edited

Loading