Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ability to disable nodePort Support #4727

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions charts/external-dns/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Fixed `provider.webhook.imagePullPolicy` behavior to correctly leverage pull policy ([#4643](/~https://github.com/kubernetes-sigs/external-dns/pull/4643)) _@kimsondrup_
- Add correct webhook metric port to `Service` and `ServiceMonitor` ([#4643](/~https://github.com/kubernetes-sigs/external-dns/pull/4643)) _@kimsondrup_

### Added
- Added support for `ignore-nodeports` argument ([#4727](/~https://github.com/kubernetes-sigs/external-dns/pull/4727)) _@jpiper_

## [v1.14.5] - 2023-06-10

### Added
Expand Down
31 changes: 16 additions & 15 deletions charts/external-dns/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,24 +67,24 @@ namespaces as `external-dns`.

The annotation `external-dns.alpha.kubernetes.io/endpoints-type: NodeExternalIP` is not supported.

If `namespaced` is set to `true`, please ensure that `sources` my only contains supported sources (Default: `service,ingress`).
If `namespaced` is set to `true`, please ensure that `sources` only contains supported sources (Default: `service,ingress`).

### Support Matrix

| Source | Supported | Infos |
|------------------------|------------|------------------------|
| `ingress` | ✅ | |
| `istio-gateway` | ✅ | |
| `istio-virtualservice` | ✅ | |
| `crd` | ✅ | |
| `kong-tcpingress` | ✅ | |
| `openshift-route` | ✅ | |
| `skipper-routegroup` | ✅ | |
| `gloo-proxy` | ✅ | |
| `contour-httpproxy` | ✅ | |
| `service` | ⚠️️ | NodePort not supported |
| `node` | ❌ | |
| `pod` | ❌ | |
| Source | Supported | Info |
|------------------------|------------|--------------------------------|
| `ingress` | ✅ | |
| `istio-gateway` | ✅ | |
| `istio-virtualservice` | ✅ | |
| `crd` | ✅ | |
| `kong-tcpingress` | ✅ | |
| `openshift-route` | ✅ | |
| `skipper-routegroup` | ✅ | |
| `gloo-proxy` | ✅ | |
| `contour-httpproxy` | ✅ | |
| `pod` | ⚠ | hostNetwork pods not supported |
| `service` | ⚠️️ | NodePort not supported |
| `node` | ❌ | |

## Values

Expand All @@ -105,6 +105,7 @@ If `namespaced` is set to `true`, please ensure that `sources` my only contains
| extraVolumeMounts | list | `[]` | Extra [volume mounts](https://kubernetes.io/docs/concepts/storage/volumes/) for the `external-dns` container. |
| extraVolumes | list | `[]` | Extra [volumes](https://kubernetes.io/docs/concepts/storage/volumes/) for the `Pod`. |
| fullnameOverride | string | `nil` | Override the full name of the chart. |
| ignoreNodePorts | bool | `false` | |
| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy for the `external-dns` container. |
| image.repository | string | `"registry.k8s.io/external-dns/external-dns"` | Image repository for the `external-dns` container. |
| image.tag | string | `nil` | Image tag for the `external-dns` container, this will default to `.Chart.AppVersion` if not set. |
Expand Down
30 changes: 15 additions & 15 deletions charts/external-dns/README.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -62,24 +62,24 @@ namespaces as `external-dns`.

The annotation `external-dns.alpha.kubernetes.io/endpoints-type: NodeExternalIP` is not supported.

If `namespaced` is set to `true`, please ensure that `sources` my only contains supported sources (Default: `service,ingress`).
If `namespaced` is set to `true`, please ensure that `sources` only contains supported sources (Default: `service,ingress`).

### Support Matrix

| Source | Supported | Infos |
|------------------------|------------|------------------------|
| `ingress` | ✅ | |
| `istio-gateway` | ✅ | |
| `istio-virtualservice` | ✅ | |
| `crd` | ✅ | |
| `kong-tcpingress` | ✅ | |
| `openshift-route` | ✅ | |
| `skipper-routegroup` | ✅ | |
| `gloo-proxy` | ✅ | |
| `contour-httpproxy` | ✅ | |
| `service` | ⚠️️ | NodePort not supported |
| `node` | ❌ | |
| `pod` | ❌ | |
| Source | Supported | Info |
|------------------------|------------|--------------------------------|
| `ingress` | ✅ | |
| `istio-gateway` | ✅ | |
| `istio-virtualservice` | ✅ | |
| `crd` | ✅ | |
| `kong-tcpingress` | ✅ | |
| `openshift-route` | ✅ | |
| `skipper-routegroup` | ✅ | |
| `gloo-proxy` | ✅ | |
| `contour-httpproxy` | ✅ | |
| `pod` | ⚠ | hostNetwork pods not supported |
| `service` | ⚠️️ | NodePort not supported |
| `node` | ❌ | |


{{ template "chart.requirementsSection" . }}
Expand Down
3 changes: 3 additions & 0 deletions charts/external-dns/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,9 @@ spec:
{{- if .Values.namespaced }}
- --namespace={{ .Release.Namespace }}
{{- end }}
{{- if or (eq .Values.ignoreNodePorts true) (eq .Values.namespaced true)}}
- --ignore-nodeports=true
{{- end }}
{{- range .Values.domainFilters }}
- --domain-filter={{ . }}
{{- end }}
Expand Down
3 changes: 3 additions & 0 deletions charts/external-dns/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,9 @@ domainFilters: []
## -- Intentionally exclude domains from being managed.
excludeDomains: []

## -- Disable nodePort support (for environments where access to Node resources is forbidden)
ignoreNodePorts: false

provider:
# -- _ExternalDNS_ provider name; for the available providers and how to configure them see [README](/~https://github.com/kubernetes-sigs/external-dns/blob/master/charts/external-dns/README.md#providers).
name: aws
Expand Down
1 change: 1 addition & 0 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,7 @@ func main() {
ResolveLoadBalancerHostname: cfg.ResolveServiceLoadBalancerHostname,
TraefikDisableLegacy: cfg.TraefikDisableLegacy,
TraefikDisableNew: cfg.TraefikDisableNew,
IgnoreNodePorts: cfg.IgnoreNodePorts,
}

// Lookup all the selected sources by names and pass them the desired configuration.
Expand Down
3 changes: 3 additions & 0 deletions pkg/apis/externaldns/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@ type Config struct {
TraefikDisableLegacy bool
TraefikDisableNew bool
NAT64Networks []string
IgnoreNodePorts bool
}

var defaultConfig = &Config{
Expand Down Expand Up @@ -351,6 +352,7 @@ var defaultConfig = &Config{
TraefikDisableLegacy: false,
TraefikDisableNew: false,
NAT64Networks: []string{},
IgnoreNodePorts: false,
}

// NewConfig returns new Config object
Expand Down Expand Up @@ -441,6 +443,7 @@ func (cfg *Config) ParseFlags(args []string) error {
app.Flag("traefik-disable-legacy", "Disable listeners on Resources under the traefik.containo.us API Group").Default(strconv.FormatBool(defaultConfig.TraefikDisableLegacy)).BoolVar(&cfg.TraefikDisableLegacy)
app.Flag("traefik-disable-new", "Disable listeners on Resources under the traefik.io API Group").Default(strconv.FormatBool(defaultConfig.TraefikDisableNew)).BoolVar(&cfg.TraefikDisableNew)
app.Flag("nat64-networks", "Adding an A record for each AAAA record in NAT64-enabled networks; specify multiple times for multiple possible nets (optional)").StringsVar(&cfg.NAT64Networks)
app.Flag("ignore-nodeports", "Disables the nodeport functionality. Needed when source is set to service or pod in clusters where external-dns does not have permissions to watch node resources, with the caveat that you will not be able to sync node IPs (optional)").Default(strconv.FormatBool(defaultConfig.IgnoreNodePorts)).BoolVar(&cfg.IgnoreNodePorts)

// Flags related to providers
providers := []string{"akamai", "alibabacloud", "aws", "aws-sd", "azure", "azure-dns", "azure-private-dns", "civo", "cloudflare", "coredns", "designate", "digitalocean", "dnsimple", "exoscale", "gandi", "godaddy", "google", "ibmcloud", "inmemory", "linode", "ns1", "oci", "ovh", "pdns", "pihole", "plural", "rdns", "rfc2136", "scaleway", "skydns", "tencentcloud", "transip", "ultradns", "webhook"}
Expand Down
7 changes: 7 additions & 0 deletions source/compatibility.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ package source
import (
"strings"

log "github.com/sirupsen/logrus"

v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/labels"

Expand Down Expand Up @@ -123,6 +125,11 @@ func legacyEndpointsFromDNSControllerService(svc *v1.Service, sc *serviceSource)
// It will use node role label to check if the node has the "node" role. This means control plane nodes and other
// roles will not be used as targets.
func legacyEndpointsFromDNSControllerNodePortService(svc *v1.Service, sc *serviceSource) ([]*endpoint.Endpoint, error) {
if sc.nodeInformer == nil {
log.Warnf("Unable to extract nodePort targets from service %s/%s as nodePort support is disabled", svc.Namespace, svc.Name)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the user explicitly disabled nodePort support then Info seems more appropriate than Warn.

Suggested change
log.Warnf("Unable to extract nodePort targets from service %s/%s as nodePort support is disabled", svc.Namespace, svc.Name)
log.Infof("Unable to extract nodePort targets from service %s/%s as nodePort support is disabled", svc.Namespace, svc.Name)

I've suggested Info level here (and elsewhere Warn is used) but I see other places in the PR where Debug is used.

return nil, nil
}

var endpoints []*endpoint.Endpoint

// Get the desired hostname of the service from the annotations.
Expand Down
43 changes: 28 additions & 15 deletions source/pod.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,23 +39,28 @@ type podSource struct {
}

// NewPodSource creates a new podSource with the given config.
func NewPodSource(ctx context.Context, kubeClient kubernetes.Interface, namespace string, compatibility string) (Source, error) {
func NewPodSource(ctx context.Context, kubeClient kubernetes.Interface, namespace string, compatibility string, disableNodeInformer bool) (Source, error) {
informerFactory := kubeinformers.NewSharedInformerFactoryWithOptions(kubeClient, 0, kubeinformers.WithNamespace(namespace))
podInformer := informerFactory.Core().V1().Pods()
nodeInformer := informerFactory.Core().V1().Nodes()
var nodeInformer coreinformers.NodeInformer

podInformer.Informer().AddEventHandler(
cache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
},
},
)
nodeInformer.Informer().AddEventHandler(
cache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
if disableNodeInformer {
log.Infoln("host information (host IP/hostname) for pods is disabled as the node informer is disabled")
} else {
nodeInformer = informerFactory.Core().V1().Nodes()
nodeInformer.Informer().AddEventHandler(
cache.ResourceEventHandlerFuncs{
AddFunc: func(obj interface{}) {
},
},
},
)
)
}
Copy link
Contributor

@szuecs szuecs Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we warn here in else instead of in Endpoints() ?
I guess it's expected to not flood all day the logs.

Same applies to service

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, I've dropped the log level in servive/pod to a debug level and then added warnings here.


informerFactory.Start(ctx.Done())

Expand Down Expand Up @@ -108,6 +113,10 @@ func (ps *podSource) Endpoints(ctx context.Context) ([]*endpoint.Endpoint, error
domainList := splitHostnameAnnotation(domainAnnotation)
for _, domain := range domainList {
if len(targets) == 0 {
if ps.nodeInformer == nil {
log.Debugf("Unable to determine node's ip address for pod %s/%s as node informer is disabled", pod.Namespace, pod.Name)
continue
}
node, _ := ps.nodeInformer.Lister().Get(pod.Spec.NodeName)
for _, address := range node.Status.Addresses {
recordType := suitableType(address.Address)
Expand All @@ -133,14 +142,18 @@ func (ps *podSource) Endpoints(ctx context.Context) ([]*endpoint.Endpoint, error
}

if domainAnnotation, ok := pod.Annotations[kopsDNSControllerHostnameAnnotationKey]; ok {
domainList := splitHostnameAnnotation(domainAnnotation)
for _, domain := range domainList {
node, _ := ps.nodeInformer.Lister().Get(pod.Spec.NodeName)
for _, address := range node.Status.Addresses {
recordType := suitableType(address.Address)
// IPv6 addresses are labeled as NodeInternalIP despite being usable externally as well.
if address.Type == corev1.NodeExternalIP || (address.Type == corev1.NodeInternalIP && recordType == endpoint.RecordTypeAAAA) {
addToEndpointMap(endpointMap, domain, recordType, address.Address)
if ps.nodeInformer == nil {
log.Debugf("Unable to determine node's hostname for pod %s/%s as node informer is disabled", pod.Namespace, pod.Name)
} else {
domainList := splitHostnameAnnotation(domainAnnotation)
for _, domain := range domainList {
node, _ := ps.nodeInformer.Lister().Get(pod.Spec.NodeName)
for _, address := range node.Status.Addresses {
recordType := suitableType(address.Address)
// IPv6 addresses are labeled as NodeInternalIP despite being usable externally as well.
if address.Type == corev1.NodeExternalIP || (address.Type == corev1.NodeInternalIP && recordType == endpoint.RecordTypeAAAA) {
addToEndpointMap(endpointMap, domain, recordType, address.Address)
}
}
}
}
Expand Down
Loading
Loading