kubernetes: unset configMapAndSecretChangeDetectionStrategy

[etungsten]

Issue number:
N/A

Description of changes:

    kubernetes: unset `configMapAndSecretChangeDetectionStrategy`
    
    We've carried this kubelet config setting since the beginning and it has
    become apparent that this causes significant pod start up delays.
    
    The default value for `configMapAndSecretChangeDetectionStrategy` is
    "watch" which seems appropriate for most use cases.

Testing done:
Applying a deployment of 300 nginx pods takes 1 min 30~ on two m5.8xlarge nodes. Where as before this change, it would have taken at least 3 mins:

$ time kubectl wait --for=condition=available --timeout=600s deployment/nginx-deployment
deployment.apps/nginx-deployment condition met

real    1m30.614s

Run some workloads with the following variants and their corresponding clusters

• aws-k8s-1.19
• aws-k8s-1.20
• aws-k8s-1.21
• aws-k8s-1.22
• vmware-k8s-1.20
• vmware-k8s-1.21
• vmware-k8s-1.22

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.