OTA-1010: extract included manifests with net-new capabilities

[hongkailiu]

Before this pull, we enabled three 3 net-new capabilities for 4.13 clusters:

oc/pkg/cli/admin/release/extract_tools.go

Lines 1241 to 1252 in e005223

	// FIXME: eventually pull in GetImplicitlyEnabledCapabilities from /~https://github.com/openshift/cluster-version-operator/blob/86e24d66119a73f50282b66a8d6f2e3518aa0e15/pkg/payload/payload.go#L237-L240 for cases where a minor update would implicitly enable some additional capabilities. For now, 4.13 to 4.14 will always enable MachineAPI, ImageRegistry, etc..
	currentVersion := clusterVersion.Status.Desired.Version
	matches := regexp.MustCompile(`^(\d+[.]\d+)[.].*`).FindStringSubmatch(currentVersion)
	if len(matches) < 2 {
	return config, fmt.Errorf("failed to parse major.minor version from ClusterVersion status.desired.version %q", currentVersion)
	} else if matches[1] == "4.13" {
	build := configv1.ClusterVersionCapability("Build")
	deploymentConfig := configv1.ClusterVersionCapability("DeploymentConfig")
	imageRegistry := configv1.ClusterVersionCapability("ImageRegistry")
	config.Capabilities.EnabledCapabilities = append(config.Capabilities.EnabledCapabilities, configv1.ClusterVersionCapabilityMachineAPI, build, deploymentConfig, imageRegistry)
	config.Capabilities.KnownCapabilities = append(config.Capabilities.KnownCapabilities, configv1.ClusterVersionCapabilityMachineAPI, build, deploymentConfig, imageRegistry)
	}

Now the capabilities for the incoming release is calculated with the function from CVO based on the manifests from the current release and the ones from the incoming release.

To fit the current code that had TarEntryCallback already, the above logic is implemented via a ManifestReceiver that works between the upstream TarEntryCallback and the downstream manifestsCallback. With needEnabledCapabilities, it tells the receiver that the manifestsCallback is called with enabled capabilities computed. The price is that manifestsCallback is called only after it collects all the manifests from the upstream.

[openshift-ci-robot]

@hongkailiu: This pull request references OTA-1010 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: hongkailiu
Once this PR has been reviewed and has the lgtm label, please assign atiratree for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@hongkailiu: This pull request references OTA-1010 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

This pull adds a ManifestReceiver that works between the upstream TarEntryCallback and the downstream manifestsCallback. With needEnabledCapabilities, it tells the receiver that the manifestsCallback is called with enabled capabilities computed. The price is that manifestsCallback is called only after it collects all the manifests from the upstream.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@hongkailiu: This pull request references OTA-1010 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

Before this pull, we enabled three 3 net-new capabilities for 4.13 clusters:

oc/pkg/cli/admin/release/extract_tools.go

Lines 1241 to 1252 in e005223

	// FIXME: eventually pull in GetImplicitlyEnabledCapabilities from /~https://github.com/openshift/cluster-version-operator/blob/86e24d66119a73f50282b66a8d6f2e3518aa0e15/pkg/payload/payload.go#L237-L240 for cases where a minor update would implicitly enable some additional capabilities. For now, 4.13 to 4.14 will always enable MachineAPI, ImageRegistry, etc..
	currentVersion := clusterVersion.Status.Desired.Version
	matches := regexp.MustCompile(`^(\d+[.]\d+)[.].*`).FindStringSubmatch(currentVersion)
	if len(matches) < 2 {
	return config, fmt.Errorf("failed to parse major.minor version from ClusterVersion status.desired.version %q", currentVersion)
	} else if matches[1] == "4.13" {
	build := configv1.ClusterVersionCapability("Build")
	deploymentConfig := configv1.ClusterVersionCapability("DeploymentConfig")
	imageRegistry := configv1.ClusterVersionCapability("ImageRegistry")
	config.Capabilities.EnabledCapabilities = append(config.Capabilities.EnabledCapabilities, configv1.ClusterVersionCapabilityMachineAPI, build, deploymentConfig, imageRegistry)
	config.Capabilities.KnownCapabilities = append(config.Capabilities.KnownCapabilities, configv1.ClusterVersionCapabilityMachineAPI, build, deploymentConfig, imageRegistry)
	}

Now the capabilities for the incoming release is calculated with the function from CVO based on the manifests from the current release and the ones from the incoming release.

To fit the current code that had TarEntryCallback already, the above logic is implemented via a ManifestReceiver that works between the upstream TarEntryCallback and the downstream manifestsCallback. With needEnabledCapabilities, it tells the receiver that the manifestsCallback is called with enabled capabilities computed. The price is that manifestsCallback is called only after it collects all the manifests from the upstream.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[wking]

hmm, but with IncludeAllowUnknownCapabilities, aren't the ones that fail the ones we can ignore? I'd guess they meant "not even with all these new caps enabled will this manifest get included". The ones we'll implicitly enable caps for are the ones that fail to get included if we apply the current caps, but which do get included if we allow new caps, and which match SameResourceID.

[hongkailiu]

This function is refactored from this one in CVO: /~https://github.com/openshift/cluster-version-operator/blob/e90705bb5457b2b9d449b4377036c3de6617ebb6/pkg/payload/payload.go#L247-L252

My understanding is very weak on this function (although i have read it a couple of times already).

hmm, but with IncludeAllowUnknownCapabilities, aren't the ones that fail the ones we can ignore?

No. Passed ones (updateManErr == nil in the next line) are ignored. The manifests that are included on the cluster wont generate new implicitly enabled capabilities.

[openshift-ci]

@hongkailiu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/okd-scos-e2e-aws-ovn	27e03eb	link	false	/test okd-scos-e2e-aws-ovn
ci/prow/e2e-agnostic-ovn-cmd	27e03eb	link	true	/test e2e-agnostic-ovn-cmd
ci/prow/e2e-aws-ovn-serial	27e03eb	link	true	/test e2e-aws-ovn-serial

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.