Support sb topic subscriptions

[jsturtevant]

Allow for users of the metrics adapter to gather message counts from Azure Service Bus Topic Subscriptions to decide their scaling targets

Closes #42

This replaces #49 which I didn't have permission to push updates too. Thanks to @marc-sensenich for the initial implementation.

[jsturtevant]

cc: @lee0c

[tomkerkhove]

We might want to add queues as well in a new PR

[paaland]

Can the pipleline be fixed so we can get this? Subscription message count is what I'm waiting for....

[jsturtevant]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.

[tomkerkhove]

I don't like your motivation @azure-pipelines, try harder!

Also, it's not easy to configure this app correctly ☹️

[jsturtevant]

It seems that Azure devops does find the pipeline as it started to run. Have not figured out why it sends that message.

Which app is not easy to configure?

[tomkerkhove]

I find the Azure Pipelines GitHub app not easy to setup, but that might just be me..

[jsturtevant]

/azp run image-pipeline

[azure-pipelines]

No pipelines are associated with this pull request.

[aminebizid]

Any news guys

[aminebizid]

Hello,
I couldn't get the metrics using this PR on my 1.13 K8S:

My External Metric

apiVersion: azure.com/v1alpha2
kind: ExternalMetric
metadata:
  name: example-external-metric-service-bus-subscription
spec:
  type: servicebussubscription
  azure:
    resourceGroup: RG-TEST
    serviceBusNamespace: ttalogs
    serviceBusTopic: testk8s
    serviceBusSubscription: testk8ssub
  metric:
    # This would default to activeMessageCount, but could be updated to one of the counts from /~https://github.com/Azure/azure-sdk-for-go/blob/master/services/servicebus/mgmt/2017-04-01/servicebus/models.go#L1116
    metricName: activeMessageCount

My HPA

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
 name: consumer-scaler
spec:
 scaleTargetRef:
   apiVersion: extensions/v1beta1
   kind: Deployment
   name: consumer
 minReplicas: 1
 maxReplicas: 10
 metrics:
  - type: External
    external:
      metricName: activeMessageCount
      targetValue: 30

Testing

kubectl -n hpa get hpa consumer-scaler -w
consumer-scaler   Deployment/consumer   <unknown>/30   1         10        1          20m

k -n azure-metrics logs -f azure-metrics-azure-k8s-metrics-adapter-6bb89f96fb-nbv8v
I0505 22:01:57.976022       1 provider_external.go:24] Received request for namespace: hpa, metric name: activemessagecount, metric selectors:
I0505 22:01:57.976060       1 metric_cache.go:41] metric not found ExternalMetric/hpa/activemessagecount
I0505 22:01:57.976082       1 metricrequest.go:104] Successfully parsed AzureMetric


kubectl get --raw /apis/azure.com/v1alpha2/namespaces/hpa/externalmetrics/example-external-metric-service-bus-subscription| jq

{
  "apiVersion": "azure.com/v1alpha2",
  "kind": "ExternalMetric",
  "metadata": {
    "annotations": {
      "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"azure.com/v1alpha2\",\"kind\":\"ExternalMetric\",\"metadata\":{\"annotations\":{},\"name\":\"example-external-metric-service-bus-subscription\",\"namespace\":\"hpa\"},\"spec\":{\"azure\":{\"resourceGroup\":\"RG-TEST\",\"serviceBusNamespace\":\"ttalogs\",\"serviceBusSubscription\":\"testk8ssub\",\"serviceBusTopic\":\"testk8s\"},\"metric\":{\"metricName\":\"ActiveMessageCount\"},\"type\":\"servicebussubscription\"}}\n"
    },
    "creationTimestamp": "2019-05-05T20:59:42Z",
    "generation": 2,
    "name": "example-external-metric-service-bus-subscription",
    "namespace": "hpa",
    "resourceVersion": "3697203",
    "selfLink": "/apis/azure.com/v1alpha2/namespaces/hpa/externalmetrics/example-external-metric-service-bus-subscription",
    "uid": "b41c2510-6f78-11e9-8ce8-000d3a48b4d9"
  },
  "spec": {
    "azure": {
      "resourceGroup": "RG-TEST",
      "serviceBusNamespace": "ttalogs",
      "serviceBusSubscription": "testk8ssub",
      "serviceBusTopic": "testk8s"
    },
    "metric": {
      "metricName": "ActiveMessageCount"
    },
    "type": "servicebussubscription"
  }
}

The Field "value" is not found

How can I debug?

[jsturtevant]

@aminebizid You HPA metricName should reference the name of the External Metric which would be example-external-metric-service-bus-subscription instead of the metric name activeMessageCount

Additionally you should now check out the KEDA project for future work. Specifically topic subscriptions are supported.

[jsturtevant]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.