Add certificate_map to compute_target_https_proxy

[MoinTom]

related to issue hashicorp/terraform-provider-google#11037

Add certificate_map option to compute_target_https_proxy in order to support new certificate-manager (with wildcard support).

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

compute: added `certificate_map` to `compute_target_https_proxy` resource

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

For more information, open the CLA check for this pull request.

[modular-magician]

Oops! It looks like you're using an unknown release-note type in your changelog entries:

• REPLACEME

Please only use the types listed in /~https://github.com/GoogleCloudPlatform/magic-modules/blob/master/.ci/RELEASE_NOTES_GUIDE.md.

[modular-magician]

Hello! I am a robot who works on Magic Modules PRs.

I've detected that you're a community contributor. @megan07, a repository maintainer, has been assigned to assist you and help review your changes.

❓ First time contributing? Click here for more details

---

Your assigned reviewer will help review your code by:

• Ensuring it's backwards compatible, covers common error cases, etc.
• Summarizing the change into a user-facing changelog note.
• Passes tests, either our "VCR" suite, a set of presubmit tests, or with manual test runs.

You can help make sure that review is quick by running local tests and ensuring they're passing in between each push you make to your PR's branch. Also, try to leave a comment with each push you make, as pushes generally don't generate emails.

If your reviewer doesn't get back to you within a week after your most recent change, please feel free to leave a comment on the issue asking them to take a look! In the absence of a dedicated review dashboard most maintainers manage their pending reviews through email, and those will sometimes get lost in their inbox.

---

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 23 insertions(+), 17 deletions(-))
Terraform Beta: Diff ( 2 files changed, 81 insertions(+), 17 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[megan07]

/gcbrun

[modular-magician]

Oops! It looks like you're using an unknown release-note type in your changelog entries:

• REPLACEME

Please only use the types listed in /~https://github.com/GoogleCloudPlatform/magic-modules/blob/master/.ci/RELEASE_NOTES_GUIDE.md.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 23 insertions(+), 17 deletions(-))
Terraform Beta: Diff ( 2 files changed, 81 insertions(+), 17 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[MoinTom]

@megan07 i already created a PR (hashicorp/terraform-provider-google-beta#4061) without realising that this repo is generated. @shuyama1 pointed out that i should make those changes here. However i wasn't able to recreate the changes i made by hand.

My main issue is that it's not possible to create a field with type ResourceRef without also defining the resource. Which makes sense, but is a bit unfortunate for the time being, since we don't have the Certificate Manager-API yet.
However i noticed that someone already implemented the CertificateManager, but it got removed: #4968
Maybe resurrecting this would be the smarter first move here.

[modular-magician]

Tests analytics

Total tests: 1993
Passed tests 1751
Skipped tests: 241
Failed tests: 1

Action taken

Triggering VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccFirebaserulesRelease_BasicRelease

[modular-magician]

Tests passed during RECORDING mode:
TestAccFirebaserulesRelease_BasicRelease[view]

All tests passed
View the build log or the debug log for each test

[modular-magician]

Tests analytics

Total tests: 1993
Passed tests 1752
Skipped tests: 241
Failed tests: 0

All tests passed in REPLAYING mode
View the build log

[modular-magician]

Oops! It looks like you're using an unknown release-note type in your changelog entries:

• REPLACEME

Please only use the types listed in /~https://github.com/GoogleCloudPlatform/magic-modules/blob/master/.ci/RELEASE_NOTES_GUIDE.md.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 24 insertions(+), 17 deletions(-))
Terraform Beta: Diff ( 2 files changed, 83 insertions(+), 17 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 1995
Passed tests 1754
Skipped tests: 241
Failed tests: 0

All tests passed in REPLAYING mode
View the build log

[megan07]

@MoinTom - I've opened a new PR to add certificate manager back in: #6004

[megan07]

@MoinTom - #6004 has now been merged. Thanks for your patience! Let me know when you're ready for me to review this PR.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 30 insertions(+), 23 deletions(-))
Terraform Beta: Diff ( 4 files changed, 87 insertions(+), 21 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[MoinTom]

@megan07 ok cool. That took care about DnsAuthorization and Certificate. However CertificateMap and CertificateMapEntry are still missing.

My PR works as is, however i assume it would be better to use ResourceRef instead of String. But i don't know if it's possible to use ResourceRef for resources defined outside of the current product.

[modular-magician]

Tests analytics

Total tests: 2018
Passed tests 1785
Skipped tests: 231
Failed tests: 2

Action taken

Triggering VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccFirebaserulesRelease_BasicRelease|TestAccCloudAssetFolderFeed_cloudAssetFolderFeedExample

[modular-magician]

Tests passed during RECORDING mode:
TestAccFirebaserulesRelease_BasicRelease[view]

Tests failed during RECORDING mode:
TestAccCloudAssetFolderFeed_cloudAssetFolderFeedExample[view]

Please fix these to complete your PR
View the build log or the debug log for each test

[megan07]

Hi @MoinTom ! Sure, ok, so if I understand correctly you would prefer to have the google_certificate_map and google_certificate_map_entry resources implemented prior to this PR, if possible - although this would work without it.

Do you want to include their implementations within this PR, or open an issue and associate them?

Regardless, would you please add a test for the functionality that you currently have added?

Thanks!

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 24 insertions(+), 17 deletions(-))
Terraform Beta: Diff ( 2 files changed, 83 insertions(+), 17 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[MoinTom]

Regardless, would you please add a test for the functionality that you currently have added?

Not sure how to write a functional test without a google_certificate_map resource, since a valid certificateMap would be required in order to use the new certificate_map field.

Do you want to include their implementations within this PR, or open an issue and associate them?

I am not sure how i would approach this. Unfortunately i am quite busy those days, so i am not sure if i find enough time to look into it.

[modular-magician]

Tests analytics

Total tests: 2022
Passed tests 1796
Skipped tests: 226
Failed tests: 0

All tests passed in REPLAYING mode
View the build log

[MoinTom]

The functionality is stable, so i decided to change the status to review. However a functional test would clearly be nice, but that means that this PR is stuck until CertificateMap and CertificateMapEntry resources are available.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 130 insertions(+), 17 deletions(-))
Terraform Beta: Diff ( 5 files changed, 189 insertions(+), 17 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 1 file changed, 12 insertions(+))

[modular-magician]

Tests analytics

Total tests: 2126
Passed tests 1888
Skipped tests: 226
Failed tests: 12

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccSqlUser_mysqlDisabled|TestAccFirebaserulesRelease_BasicRelease|TestAccContainerCluster_withMeshCertificatesConfig|TestAccComputeTargetHttpsProxy_update|TestAccComputeInstance_soleTenantNodeAffinities|TestAccCertificateManagerCertificateMapEntry_certificateManagerCertificateMapEntryFullExample|TestAccCGCSnippet_eventarcWorkflowsExample|TestAccActiveDirectoryPeering_activeDirectoryPeeringBasicExample|TestAccActiveDirectoryDomain_update|TestAccActiveDirectoryDomain_activeDirectoryDomainBasicExample|TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 203 insertions(+), 17 deletions(-))
Terraform Beta: Diff ( 5 files changed, 203 insertions(+), 17 deletions(-))
TF Validator: Diff ( 3 files changed, 13 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 1 file changed, 19 insertions(+))

[MoinTom]

Hi @megan07 i added a test and example. However the linter and VCR-test are failing. Unfortunately i am not able to see any details on why they are failing. Maybe u can point me into the right direction.
Thanks

[modular-magician]

Tests analytics

Total tests: 2126
Passed tests 1889
Skipped tests: 226
Failed tests: 11

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation|TestAccComputeTargetHttpsProxy_update|TestAccContainerCluster_withMeshCertificatesConfig|TestAccComputeInstance_soleTenantNodeAffinities|TestAccCertificateManagerCertificateMapEntry_certificateManagerCertificateMapEntryFullExample|TestAccCGCSnippet_eventarcWorkflowsExample|TestAccActiveDirectoryDomain_activeDirectoryDomainBasicExample|TestAccActiveDirectoryPeering_activeDirectoryPeeringBasicExample|TestAccActiveDirectoryDomain_update|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccSqlUser_mysqlDisabled

[modular-magician]

Tests passed during RECORDING mode:
TestAccSqlUser_mysqlDisabled[view]
TestAccContainerCluster_withMeshCertificatesConfig[view]
TestAccCGCSnippet_eventarcWorkflowsExample[view]

Tests failed during RECORDING mode:
TestAccActiveDirectoryDomain_activeDirectoryDomainBasicExample[view]
TestAccComputeTargetHttpsProxy_update[view]
TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation[view]
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view]
TestAccCertificateManagerCertificateMapEntry_certificateManagerCertificateMapEntryFullExample[view]
TestAccComputeInstance_soleTenantNodeAffinities[view]
TestAccActiveDirectoryPeering_activeDirectoryPeeringBasicExample[view]
TestAccActiveDirectoryDomain_update[view]

Please fix these to complete your PR
View the build log or the debug log for each test

[megan07]

Hi @MoinTom, the error is: Error when reading or editing SslCertificate: googleapi: Error 400: The ssl_certificate resource '.../sslCertificates/httpsproxy-test-cert1-x173lbq28h' is already being used by '.../targetHttpsProxies/httpsproxy-test-x173lbq28h', resourceInUseByAnotherResource

and seems to be coming after trying to update the google_compute_url_map call:

{
 "defaultService": ".../global/backendServices/httpsproxy-test-backend-x173lbq28h",
 "fingerprint": "...",
 "hostRules": [],
 "name": "httpsproxy-test-url-map-x173lbq28h"
}

If you're looking to test updating from ssl_certificates to certificate_map, we can roll with this and that's fine, but it'd also be fine to create a new test specifically for certificate_map. Totally up to you! I think you can also remove the addition to this test too. This one was failing because labels isn't an option on google_compute_url_map, but testing the basic functionality on a google_compute_target_https_proxy tests seems sufficient to me. What are your thoughts?

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 5 files changed, 209 insertions(+), 17 deletions(-))
Terraform Beta: Diff ( 5 files changed, 209 insertions(+), 17 deletions(-))
TF Validator: Diff ( 3 files changed, 13 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 1 file changed, 15 insertions(+))

[modular-magician]

Tests analytics

Total tests: 2123
Passed tests 1887
Skipped tests: 226
Failed tests: 10

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccFirebaserulesRelease_BasicRelease|TestAccComputeTargetHttpsProxy_certificateMap|TestAccComputeInstance_soleTenantNodeAffinities|TestAccCertificateManagerCertificateMapEntry_certificateManagerCertificateMapEntryFullExample|TestAccCGCSnippet_eventarcWorkflowsExample|TestAccCloudRunService_cloudRunServiceStaticOutboundExample|TestAccActiveDirectoryDomain_activeDirectoryDomainBasicExample|TestAccActiveDirectoryPeering_activeDirectoryPeeringBasicExample|TestAccActiveDirectoryDomain_update

[modular-magician]

Tests passed during RECORDING mode:
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view]
TestAccFirebaserulesRelease_BasicRelease[view]
TestAccCGCSnippet_eventarcWorkflowsExample[view]

Tests failed during RECORDING mode:
TestAccActiveDirectoryDomain_activeDirectoryDomainBasicExample[view]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[view]
TestAccComputeTargetHttpsProxy_certificateMap[view]
TestAccComputeInstance_soleTenantNodeAffinities[view]
TestAccCertificateManagerCertificateMapEntry_certificateManagerCertificateMapEntryFullExample[view]
TestAccActiveDirectoryPeering_activeDirectoryPeeringBasicExample[view]
TestAccActiveDirectoryDomain_update[view]

Please fix these to complete your PR
View the build log or the debug log for each test

[megan07]

TestAccComputeTargetHttpsProxy_certificateMap fails with:

Error: Error creating TargetHttpsProxy: googleapi: Error 400: Invalid value for field 'resource.certificateMap': 'projects.../locations/global/certificateMaps/certificatemap-test-ydzxpceh8s'. 'projects/.../locations/global/certificateMaps/certificatemap-test-ydzxpceh8s' is not a valid reference. It should have the format //certificatemanager.googleapis.com/projects/{project}/locations/{location}/certificateMaps/{resourceName}., invalid

I think we're going to need to create the name like this

certificate_map = "//certificatemanager.googleapis.com/${google_certificate_manager_certificate_map.map.id}"

TestAccCertificateManagerCertificateMapEntry_certificateManagerCertificateMapEntryFullExample fails with:

Error: Invalid combination of arguments

with google_compute_url_map.url-map,
on terraform_plugin_test.tf line 63, in resource "google_compute_url_map" "url-map":
63: resource "google_compute_url_map" "url-map" {

"default_url_redirect": one of
default_route_action.0.weighted_backend_services,default_service,default_url_redirect
must be specified

However, as mentioned, I'm not sure this test is necessary, its meant to test the functionality of the map entry creation and I think attaching it to google_compute_target_https_proxy is tested in the first test, so feel free to roll back the changes on this test.

Thanks!

[MoinTom]

Thanks for the help.

The resource URI being used for certificate maps is quite odd or rather unique compared to other resources within GCP. I kinda hopped that the needed prefix would be part of the google_certificate_manager_certificate_map.id, so that the end user can simply use it without needing to know this detail.
However for now i will add the needed prefix within the test. But maybe we want to consider adding the prefix to the google_certificate_manager_certificate_map.id field.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 3 files changed, 179 insertions(+), 17 deletions(-))
Terraform Beta: Diff ( 3 files changed, 179 insertions(+), 17 deletions(-))
TF Validator: Diff ( 3 files changed, 13 insertions(+), 3 deletions(-))

[megan07]

@MoinTom I was also disappointed to see we didn't have an output that had exactly what we needed there. Would you mind opening a different issue to address it and we can move forward with this PR for now? Thanks!

[modular-magician]

Tests analytics

Total tests: 2126
Passed tests 1890
Skipped tests: 226
Failed tests: 10

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccComputeTargetHttpsProxy_certificateMap|TestAccComputeInstance_soleTenantNodeAffinities|TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample|TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample|TestAccCloudFunctions2Function_fullUpdate|TestAccCloudRunService_cloudRunServiceStaticOutboundExample|TestAccActiveDirectoryDomain_activeDirectoryDomainBasicExample|TestAccActiveDirectoryPeering_activeDirectoryPeeringBasicExample|TestAccActiveDirectoryDomain_update

[modular-magician]

Tests passed during RECORDING mode:
TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample[view]
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view]
TestAccComputeTargetHttpsProxy_certificateMap[view]
TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample[view]
TestAccCloudFunctions2Function_fullUpdate[view]

Tests failed during RECORDING mode:
TestAccActiveDirectoryDomain_activeDirectoryDomainBasicExample[view]
TestAccComputeInstance_soleTenantNodeAffinities[view]
TestAccCloudRunService_cloudRunServiceStaticOutboundExample[view]
TestAccActiveDirectoryPeering_activeDirectoryPeeringBasicExample[view]
TestAccActiveDirectoryDomain_update[view]

Please fix these to complete your PR
View the build log or the debug log for each test

[megan07]

LGTM! Thanks so much!