[TT 5843] v4 support

README.md

@@ -4,31 +4,54 @@ Tyk provides 3 different helm charts in this repo. Please visit the respective p
 - [Tyk Hybrid](/~https://github.com/TykTechnologies/tyk-helm-chart/tree/master/tyk-hybrid)
 - [Tyk Headless](/~https://github.com/TykTechnologies/tyk-helm-chart/tree/master/tyk-headless)
 
-## Redis and MongoDB
+## Redis and MongoDB or PostgreSQL
 - Redis is required for all of the Tyk installations it must be installed in the cluster or reachable from inside K8s.
-- MongoDB is only required for the tyk-pro installation and must be installed in the cluster, or reachable from inside K8s. If you are using the mongo pumps in the tyk-headless installation you will require mongo installed for that as well.
+- MongoDB or PostgreSQL are only required for the tyk-pro installation and must be installed in the cluster, or reachable from inside K8s. If you are using the MongoDB or SQL pumps in the tyk-headless installation you will require MongoDB or PostgreSQL installed for that as well.
 
-For Redis and MongoDB you can use these rather excellent charts provided by Bitnami:
+For Redis you can use these rather excellent charts provided by Bitnami:
+```bash
+helm repo add bitnami https://charts.bitnami.com/bitnami
+helm repo update
+kubectl create namespace tyk
 
-	helm repo add bitnami https://charts.bitnami.com/bitnami
-	helm repo update
-	kubectl create namespace tyk
-	helm install tyk-mongo bitnami/mongodb --set "replicaSet.enabled=true" -n tyk
-	(follow notes from the installation output to get connection details and update them in `values.yaml` file)
-	helm install tyk-redis bitnami/redis -n tyk
-	(follow notes from the installation output to get connection details and update them in `values.yaml` file)
+helm install tyk-redis bitnami/redis -n tyk
+(follow notes from the installation output to get connection details and update them in `values.yaml` file)
+```
+
+For Mongo or PostgreSQL you can use these rather excellent charts provided by Bitnami:
+```bash
+helm install tyk-mongo bitnami/mongodb --version {HELM_CHART_VERSION} --set "replicaSet.enabled=true" -n tyk
+(follow notes from the installation output to get connection details and update them in `values.yaml` file)
+```
+
+>**_NOTE_**: [Here](https://tyk.io/docs/planning-for-production/redis-mongodb/#supported-versions) is list of supported mongo versions. Please make sure you are installing mongo helm chart that matches these version.
+
+```bash
+helm install tyk-postgres bitnami/postgresql --set "auth.database=tyk_analytics" -n tyk
+(follow notes from the installation output to get connection details and update them in `values.yaml` file)
+```
 
 *Important Note regarding MongoDB:* This helm chart enables the PodDisruptionBudget for MongoDB with an arbiter replica-count of 1.  If you intend to perform system maintenance on the node where the MongoDB pod is running and this maintenance requires for the node to be drained, this action will be prevented due the replica count being 1.  Increase the replica count in the helm chart deployment to a minimum of 2 to remedy this issue.
 
 Another option for Redis and MongoDB, if you want to get started quickly is to use our simple charts. **Please note that these provided charts must not ever be used in production and for anything but a quick start evaluation only, use external DBs or Official Helm charts for MongoDB and Redis in any other case.**
 We provide these charts so you can quickly have Tyk running however they are not meant for long term storage of data for example.
 
-	kubectl create namespace tyk
-	helm repo add tyk-helm https://helm.tyk.io/public/helm/charts/
-	helm repo update
-	helm install redis tyk-helm/simple-redis -n tyk
-	helm install mongo tyk-helm/simple-mongodb -n tyk
+```bash
+kubectl create namespace tyk
+helm repo add tyk-helm https://helm.tyk.io/public/helm/charts/
+helm repo update
+```
+
+#### Redis
+
+```bash
+helm install redis tyk-helm/simple-redis -n tyk
+```
 
+#### MongoDB
+```bash
+helm install mongo tyk-helm/simple-mongodb -n tyk
+```
 
 ## TLS
 You can turn on the tls option under the gateway section in the values.yaml files which will make the gateways listen on port 443 and load up a dummy certificate. You can set your own default certificate by replacing the files in the certs/ folder.

tyk-headless/README.md

@@ -1,5 +1,5 @@
 ## Tyk OSS Gateway
-This chart deploys the open source Tyk Gateway.
+This chart deploys the open source Tyk Gateway and Pump.
 
 ---
 
@@ -13,8 +13,26 @@ Redis should already be installed or accessible by the gateway.
 Check [Tyk OSS Helm chart Installation](https://tyk.io/docs/tyk-oss/ce-helm-chart/) for the full details.
 
 ## Installation
+```bash
+helm repo add tyk-helm https://helm.tyk.io/public/helm/charts/
+helm repo update
+kubectl create namespace tyk
+helm install tyk-headless tyk-helm/tyk-headless -n tyk
+```
 
-	helm repo add tyk-helm https://helm.tyk.io/public/helm/charts/
-	helm repo update
-	kubectl create namespace tyk
-	helm install tyk-ce tyk-helm/tyk-headless -n tyk --create-namespace
+## Pump Installation
+By default pump installation is disabled. You can enable it by setting `pump.enabled` to `true` in `values.yaml` file.
+Alternatively, you can use `--set pump.enabled=true` while doing helm install.
+
+## Pump Configuration
+### Mongo Pump
+
+To configure mongo pump, do following changings in `values.yaml` file:
+1. Set `backend` to `mongo`.
+2. Set connection string in `mongo.mongoURL`.
+
+
+### Postgres Pump
+To configure postgres pump, do following changings in `values.yaml` file:
+1. Set `backend` to `postgres`.
+2. Set connection string parameters in `postgres` section.

tyk-headless/configs/pump.conf

@@ -1,35 +1,16 @@
 {
+  "log_level": "info",
   "analytics_storage_type": "redis",
   "analytics_storage_config": {
     "type": "redis",
-    "enable_cluster": false,
-    "hosts": {},
+    "hosts": null,
     "username": "",
-    "password": "",
-    "database": 0,
     "optimisation_max_idle": 2000,
     "optimisation_max_active": 4000
   },
-  "purge_delay": 4,
-  "pumps": {
-    "mongo": {
-      "name": "mongo",
-      "meta": {
-        "collection_name": "tyk_analytics_headless",
-        "mongo_url": ""
-      }
-    },
-    "mongo-pump-aggregate": {
-      "name": "mongo-pump-aggregate",
-      "meta": {
-        "mongo_url": "",
-        "use_mixed_collection": true
-      }
-    }
-  },
+  "dont_purge_uptime_data": false,
+  "purge_delay": 2,
   "uptime_pump_config": {
-    "collection_name": "tyk_uptime_analytics_headless",
-    "mongo_url": ""
-  },
-  "dont_purge_uptime_data": false
-}
+    "collection_name": "tyk_uptime_analytics_headless"
+  }
+}

tyk-headless/configs/tyk_mgmt.conf

@@ -1,7 +1,5 @@
 {
     "listen_port": 8080,
-    "secret": "$sharedsecret",
-    "node_secret": "$sharedsecret",
     "template_path": "/opt/tyk-gateway/templates",
     "tyk_js_path": "/opt/tyk-gateway/js/tyk.js",
     "middleware_path": "/mnt/tyk-gateway/middleware",

tyk-headless/templates/_helpers.tpl

@@ -57,3 +57,23 @@ mongodb://{{ .Values.mongo.host }}:{{ .Values.mongo.port }}/tyk_analytics
 mongodb://mongo.{{ .Release.Namespace }}.svc.cluster.local:27017/tyk_analytics
 {{- end -}}
 {{- end -}}
+
+{{- define "tyk-headless.pg_connection_string" -}}
+{{- if .Values.postgres -}}
+{{- range $key, $value := .Values.postgres }}{{ print $key "=" $value " " }}{{- end }}
+{{- end -}}
+{{- end -}}
+
+{{- define "tyk-headless.backend" -}}
+{{- if .Values.backend -}}
+{{- if eq "postgres" .Values.backend -}}
+postgres
+{{- else if eq "mongo" .Values.backend -}}
+mongo
+{{- end -}}
+{{- else if .Values.mongo.enabled -}}
+mongo
+{{- else -}}
+none
+{{- end -}}
+{{- end -}}

tyk-headless/templates/deployment-pmp.yaml

@@ -48,22 +48,77 @@ spec:
             drop:
               - all
         env:
-          # Lagacy support for Redis Cluster driver. Driver dropped in v3.0.0.
+          # Legacy support for Redis Cluster driver. Driver dropped in v3.0.0.
           - name: REDIGOCLUSTER_SHARDCOUNT
             value: "128"
-          {{- if .Values.mongo.enabled }}
-          - name: PMP_MONGO_MONGOURL
-            value: {{ include "tyk-headless.mongo_url" . | quote }}
-          - name: PMP_MONGO_MONGOUSESSL
+          {{- if eq "postgres" (include "tyk-headless.backend" .) }}
+          # SQL Pump
+          - name: TYK_PMP_PUMPS_SQL_TYPE
+            value: "sql"
+          - name: TYK_PMP_PUMPS_SQL_META_TYPE
+            value: "postgres"
+          - name: TYK_PMP_PUMPS_SQL_META_CONNECTIONSTRING
+            valueFrom:
+              secretKeyRef:
+                name: {{ if .Values.secrets.useSecretName }} {{ .Values.secrets.useSecretName }} {{ else }} secrets-{{ include "tyk-headless.fullname" . }} {{ end }}
+                key: pgConnectionString
+
+          # SQL Aggregate Pump
+          - name: TYK_PMP_PUMPS_SQLAGGREGATE_TYPE
+            value: "sql_aggregate"
+          - name: TYK_PMP_PUMPS_SQLAGGREGATE_META_TYPE
+            value: "postgres"
+          - name: TYK_PMP_PUMPS_SQLAGGREGATE_META_CONNECTIONSTRING
+            valueFrom:
+              secretKeyRef:
+                name: {{ if .Values.secrets.useSecretName }} {{ .Values.secrets.useSecretName }} {{ else }} secrets-{{ include "tyk-headless.fullname" . }} {{ end }}
+                key: pgConnectionString
+
+          # SQL Uptime Pump
+          - name: TYK_PMP_UPTIMEPUMPCONFIG_UPTIMETYPE
+            value: "sql"
+          - name: TYK_PMP_UPTIMEPUMPCONFIG_TYPE
+            value: "postgres"
+          - name: TYK_PMP_UPTIMEPUMPCONFIG_CONNECTIONSTRING
+            valueFrom:
+              secretKeyRef:
+                name: {{ if .Values.secrets.useSecretName }} {{ .Values.secrets.useSecretName }} {{ else }} secrets-{{ include "tyk-headless.fullname" . }} {{ end }}
+                key: pgConnectionString
+          {{ else }}
+          # Mongo Pump
+          - name: TYK_PMP_PUMPS_MONGO_META_COLLECTIONNAME
+            value: "tyk_analytics_headless"
+          - name: TYK_PMP_PUMPS_MONGO_META_MONGOUSESSL
             value: "{{ default "false" .Values.mongo.useSSL }}"
-          - name: PMP_MONGOAGG_MONGOURL
+          - name: TYK_PMP_PUMPS_MONGO_META_MONGOURL
             valueFrom:
               secretKeyRef:
-                name: {{ if .Values.secrets.useSecretName }} {{ .Values.secrets.useSecretName }} {{ else }} secrets-{{ include "tyk-headless.fullname" . }} {{ end}}
+                name: {{ if .Values.secrets.useSecretName }} {{ .Values.secrets.useSecretName }} {{ else }} secrets-{{ include "tyk-headless.fullname" . }} {{ end }}
                 key: mongoURL
-          - name: PMP_MONGOAGG_MONGOUSESSL
+
+          # Mongo Aggregate Pump
+          - name: TYK_PMP_PUMPS_MONGOAGGREGATE_TYPE
+            value: "mongo-pump-aggregate"
+          - name: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOUSESSL
             value:  "{{ default "false" .Values.mongo.useSSL }}"
-          {{- end }}
+          - name: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOURL
+            valueFrom:
+              secretKeyRef:
+                name: {{ if .Values.secrets.useSecretName }} {{ .Values.secrets.useSecretName }} {{ else }} secrets-{{ include "tyk-headless.fullname" . }} {{ end }}
+                key: mongoURL
+          - name: TYK_PMP_PUMPS_MONGOAGGREGATE_META_COLLECTIONNAME
+            value: "tyk_analytics_headless"
+          - name: TYK_PMP_PUMPS_MONGOAGGREGATE_META_USEMIXEDCOLLECTION
+            value: "true"
+
+          # Mongo Uptime Pump
+          - name: TYK_PMP_UPTIMEPUMPCONFIG_MONGOURL
+            valueFrom:
+              secretKeyRef:
+                name: {{ if .Values.secrets.useSecretName }} {{ .Values.secrets.useSecretName }} {{ else }} secrets-{{ include "tyk-headless.fullname" . }} {{ end }}
+                key: mongoURL
+          {{ end }}
+
           - name: TYK_PMP_ANALYTICSSTORAGECONFIG_ADDRS
             {{- if and .Values.redis.host .Values.redis.port }}
             value: "{{ .Values.redis.host }}:{{ .Values.redis.port }}"

tyk-headless/templates/secrets.yaml

@@ -13,5 +13,6 @@ stringData:
   redisPass: "{{ .Values.redis.pass }}"
   redisSentinelPass: "{{ .Values.redis.sentinelPass }}"
   mongoURL: {{ include "tyk-headless.mongo_url" . | quote }}
+  pgConnectionString: {{ include "tyk-headless.pg_connection_string" . }}
   APISecret: "{{ .Values.secrets.APISecret }}"
 {{- end }}

tyk-headless/values.yaml

@@ -60,13 +60,9 @@ redis:
   storage:
     database: 0
 
+# Choose the pumps backend for Tyk. [ "", "mongo", "postgres" ]
+backend: ""
 mongo:
-  # If you want to collect analytics through the mongo pumps you can turn this
-  # option on. Once on, Tyk Pump will assume that MongoDB is avaibale at
-  # mongo.tyk.svc.cluster.local:27017 if it is not, please change the mongoURL
-  # value below.
-  enabled: false
-
   # The mongoURL value will allow you to set your MongoDB address.
   # Default value: mongodb://mongo.{{ .Release.Namespace }}.svc.cluster.local:27017/tyk_analytics
   # mongoURL: mongodb://mongo.tyk.svc.cluster.local:27017/tyk_analytics
@@ -75,7 +71,16 @@ mongo:
 
   # Enables SSL for MongoDB connection. MongoDB instance will have to support that.
   # Default value: false
-  # useSSL: true
+  useSSL: false
+
+# Postgres connection string parameters.
+postgres:
+  host: tyk-postgres-postgresql.tyk.svc.cluster.local
+  port: 5432
+  user: postgres
+  password:
+  database: tyk_analytics
+  sslmode:
 
 gateway:
   # The hostname to bind the Gateway to.
@@ -88,7 +93,7 @@ gateway:
   containerPort: 8080
   image:
     repository: docker.tyk.io/tyk-gateway/tyk-gateway
-    tag: v3.2.1
+    tag: v4.0.0
     pullPolicy: IfNotPresent
   service:
     type: NodePort
@@ -145,13 +150,13 @@ gateway:
 # possible pump configs can be found here:
 # /~https://github.com/TykTechnologies/tyk-pump#configuration
 pump:
-  # Determines whither or not the pump component should be installed.
+  # Determines whether or not the pump component should be installed.
   enabled: false
 
   replicaCount: 1
   image:
     repository: docker.tyk.io/tyk-pump/tyk-pump
-    tag: v1.4.0
+    tag: v1.5.1
     pullPolicy: IfNotPresent
   annotations: {}
   resources: {}

tyk-hybrid/values.yaml

@@ -42,7 +42,7 @@ redis:
   # Default value: false
   # useSSL: true
 
-  # The enableCluster value will allow you to indicate to Tyk whither you are running
+  # The enableCluster value will allow you to indicate to Tyk whether you are running
   # a Redis cluster or not.
   # Default value: false
   # enableCluster: true
@@ -96,7 +96,7 @@ gateway:
   containerPort: 8080
   image:
     repository: tykio/tyk-gateway
-    tag: v3.2.1
+    tag: v4.0.0
     pullPolicy: Always
   service:
     type: NodePort

tyk-pro/README.md

@@ -9,7 +9,7 @@ and MDCB (Multi Data-centre Bridge).
 ---
 
 ## Prerequisites
-Redis and MongoDB should already be installed or accessible by the gateway.
+Redis and MongoDB or PostgreSQL should already be installed or accessible by the gateway.
 
 Check [Tyk Self-managed Helm chart Installation](https://tyk.io/docs/tyk-self-managed/tyk-helm-chart/) for the full details.