Add doc on how to provision kubernetes on aws #71

liufuyang · 2018-08-31T11:10:03Z

Hi there,

I would like to try to add a doc on how to provision a k8s on AWS via EKS and Terraform code.

Do you think it would be a good idea to add a doc like this here?

And do you have any suggestion where to put the terraform code?
In this repo as well or somewhere else? I am thinking about using some of the terraform code from here:
/~https://github.com/wardviaene/terraform-course/tree/master/eks-demo
But not sure whether it will work out of the box or not.

Also I hope it is okay to open this work-in-progress PR, as I am in the hope someone else is trying the same and perhaps can help out with writing the doc as well :)

CLAassistant · 2018-08-31T11:10:12Z

All committers have signed the CLA.

gregwebs · 2018-08-31T13:50:36Z

Yes, we would like to have a AWS doc! I prefer using terraform. However, in our google tutorial, for the sake of fewer dependencies, we use cli commands, and we would likely tend to do the same for AWS (or maybe use cloudformation). But it would be good to also have a terraform alternative, so if you write your doc, that will help us figure out how to organize terraform documentation.

I actually made some terraform for this 2 months ago, but its basically just a copy of what was here at the time.

liufuyang · 2018-08-31T19:31:11Z

oh cool, thanks for the info. Definitely it will help. I will try compare those repos and see if I can make one that works for tidb. Hopefully they just work out of the box :D

morgo · 2018-09-15T00:50:32Z

Hi @liufuyang, your tutorial is looking good! I'm happy to test it out and provide feedback when you are ready.

liufuyang · 2018-09-17T19:26:13Z

@morgo Thank you for the support. I am a bit busy these days, will try to find sometime to complete it and it would be very helpful if you could test it out as well :)

(Just a note to myself here: it seems AWS has just released some EKS optimised AMIs, might need to check out if they can be used for our cases. https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html)

morgo · 2018-09-17T20:15:43Z

@liufuyang Google has something similar: Container Optimized OS (cos).

It is a best practice to use a minimal OS so the surface area for vulnerabilities is lower. It should work just fine :-)

liufuyang · 2018-09-22T10:40:18Z

@morgo @gregwebs Today I have successfully deployed tidb onto an AWS EKS cluster, and update my doc here, it still needs a bit of further polishing but now if you want to try out, you can try test it out. With terraform, it seems fairly simple to deploy a cluster onto AWS.

Let me know if you have any questions or ideas on improve this doc.

morgo

Amazing work - thank you so much!

Sorry for my delay in reviewing, I had a project I had to finish off before attempting this.

morgo · 2018-10-25T18:36:57Z

docs/aws-eks-tutorial.md

+
+## Introduction
+
+This tutorial is designed to be run locally with tools like [AWS Command Line Interface](https://aws.amazon.com/cli/) and [Terraform](https://www.terraform.io/). The Terraform code will ultilize a relatively new AWS service called [Amazon Elastic Container Service for Kubernetes (Amazon EKS)](https://aws.amazon.com/eks).


Just a suggestion: did you want to make it tailored for running on a stock Amazon Linux 2 AMI? That way you can include pastes for exact commands.

I am not exactly sure what do you mean here. I didn't tailor it for anything. I am simply trying to use AWS EKS service, and it seems the easiest solution is just to use the default AMI they suggest to use?
https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html

I meant: many people deploy on AWS with a Bastian host running Amazon Linux. You could simplify the instructions if instead of running locally (with instructions for both Mac OS X + Linux) you provide instructions to run on the Bastian host. In this way, it would be similar to the GKE tutorial with Cloud Shell always being Debian.

(This is independent of the OS the EKS deploy uses.)

morgo · 2018-10-25T18:38:30Z

docs/aws-eks-tutorial.md

+belongs to. With more detailed permissions, you will have to be sure you also have
+`AmazonEKSClusterPolicy` and `AmazonEKSServicePolicy` for this user.
+
+Then generate a pair of access keys and keep them safe locally. Now we can continue about using Terraform to provision a Kubernetes on AWS.


If you do chose to make it Amazon Linux 2 EC2 based - please also suggest assigning a role to an instance instead here.

Hmmm, I am not that a guru of AWS IAM yet, mind if I ask why we need a role here? I think the worker instances are all managed by the EKS service, and this EKS services is started by a user who has AmazonEKSClusterPolicy and AmazonEKSServicePolicy .

Or if we have a role, I am not exactly sure how to configure the terraform code to use certain roles.

This is about assigning a role to the Bastian/Cloud Shell. It is not required - just a suggestion to avoid saving credentials on the machine.

morgo · 2018-10-25T18:46:40Z

docs/aws-eks-tutorial.md

+
+## A bit more about Terraform
+
+Information about using Terraform with EKS can be found [here](https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html).


"We will use Terraform templates to deploy EKS. Please install terraform using the steps described in the terraform manual. For example, on Linux:

wget https://releases.hashicorp.com/terraform/0.11.10/terraform_0.11.10_linux_amd64.zip unzip terraform* sudo mv terraform /usr/local/bin/

(Optional) You can confirm that Terraform is installed correctly by deploying the most simple infrastructure provision. From the Terraform manual:

.. paste example ..

In our next step, we will be deploying infrastructure based on the Terraform EKS tutorial.

morgo · 2018-10-25T18:47:23Z

docs/aws-eks-tutorial.md

+the most simplest infrastructure provision (e.g [this example](https://www.terraform.io/intro/getting-started/build.html#configuration)) before you
+continue.
+
+## Launch a 3-node Kubernetes cluster


This should say "Install Terraform", since you do not launch a 3-node Kubernetes cluster until the next step.

morgo · 2018-10-25T18:49:51Z

docs/aws-eks-tutorial.md

+For now we will follow the configs from the [last link](/~https://github.com/liufuyang/terraform-course/tree/master/eks-demo).
+As it should be more easier and has more related info.
+
+## Setting up AWS EKS (Hosted Kubernetes)


I believe this should be "Clone Tutorial Repo" (or similar) ?

morgo · 2018-10-25T19:51:08Z

docs/aws-eks-tutorial.md

+
+
+### Destroy
+At the end of the demo, please make sure all the resources created by Kubernetes are removed (LoadBalancers, Security groups), and issue:


Please move this to the very end of the tutorial.

morgo · 2018-10-25T19:51:44Z

docs/aws-eks-tutorial.md

+
+**Now you can switch back to the `tidb-operator` folder to follow the guide below.**
+
+Then from this stage, you should be able to follow the [Tidb-Operator guide](/~https://github.com/pingcap/tidb-operator/blob/master/docs/google-kubernetes-tutorial.md) from section `Install Helm`, `Deploy TiDB Operator` and so on.


We should include helm install steps here. They can be a direct paste:

curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > get_helm.sh chmod 700 get_helm.sh && ./get_helm.sh # clone repo git clone /~https://github.com/pingcap/tidb-operator.git

morgo · 2018-10-25T19:56:26Z

docs/aws-eks-tutorial.md

+If you already have helm installed, you can continue here:
+```
+kubectl create serviceaccount tiller --namespace kube-system &&
+kubectl apply -f ./manifests/tiller-rbac.yaml &&


either cd to tidb-operator in the commands:

cd tidb-operator kubectl apply -f ./manifests/tiller-rbac.yaml

or type full command:

kubectl apply -f ./tidb-operator/manifests/tiller-rbac.yaml

FYI: The weird && is for Google Cloud Shell. The paste commands must be chained. You can remove it for AWS if you want.

morgo · 2018-10-25T20:05:43Z

docs/aws-eks-tutorial.md

+Then from this stage, you should be able to follow the [Tidb-Operator guide](/~https://github.com/pingcap/tidb-operator/blob/master/docs/google-kubernetes-tutorial.md) from section `Install Helm`, `Deploy TiDB Operator` and so on.
+
+### Deploy tidb-operator
+


Explain what TIDB Operator does briefly.

morgo · 2018-10-25T20:11:24Z

docs/aws-eks-tutorial.md

+
+Or just:
+```
+kubectl -n tidb port-forward demo-tidb-0 4000:4000 &>/tmp/port-forward.log &


liufuyang · 2018-10-27T12:22:40Z

@morgo I have updated the doc based on your input, please take a look again :)

morgo

LGTM. A couple of small comments left in the document, but very good work!

morgo · 2018-10-27T13:08:23Z

docs/aws-eks-tutorial.md

+### Deploy Kubernetes Dashboard
+
+#### Step 1: Deploy the Dashboard
+


Maybe add (optional) here too. If they do deploy on a bastian host, it will be harder to get the dashboard to work.

liufuyang · 2018-10-28T08:39:24Z

@gregwebs and @morgo, I squashed and rebased onto mater, plus a few final touches on the doc.

tennix

LGTM

morgo · 2018-10-29T04:03:35Z

@liufuyang one more suggestion. Did you want to push the contents of terraform-course/eks-demo in this PR? That will simplify the repo cloning for users.

tennix · 2018-10-29T05:30:33Z

I would suggest do it in another PR. I'm merging it now.

liufuyang · 2018-10-29T22:28:26Z

oh cool, thanks for merging it. I will try find time to create a new PR to move the terraform code here 😄

weekface changed the title ~~Add doc on how to provision kubernetes on aws [WIP]~~ [WIP] Add doc on how to provision kubernetes on aws Aug 31, 2018

liufuyang mentioned this pull request Sep 9, 2018

SSD with AWS EKS Kubernetes #91

Closed

liufuyang force-pushed the aws-eks-tidb branch from f992187 to 7f65887 Compare September 22, 2018 10:36

liufuyang changed the title ~~[WIP] Add doc on how to provision kubernetes on aws~~ Add doc on how to provision kubernetes on aws Sep 22, 2018

tennix requested review from gregwebs and morgo October 18, 2018 09:18

liufuyang force-pushed the aws-eks-tidb branch from 7f65887 to 4ce96eb Compare October 23, 2018 08:57

morgo suggested changes Oct 25, 2018

View reviewed changes

morgo previously approved these changes Oct 27, 2018

View reviewed changes

liufuyang dismissed morgo’s stale review via 507b8f6 October 28, 2018 08:24

liufuyang force-pushed the aws-eks-tidb branch from 3dde23d to 507b8f6 Compare October 28, 2018 08:24

Add doc on how to provision kubernetes on aws

ff586e9

liufuyang force-pushed the aws-eks-tidb branch from 507b8f6 to ff586e9 Compare October 28, 2018 08:34

tennix approved these changes Oct 29, 2018

View reviewed changes

morgo approved these changes Oct 29, 2018

View reviewed changes

QueenyJin self-requested a review October 29, 2018 04:10

QueenyJin approved these changes Oct 29, 2018

View reviewed changes

tennix merged commit 705f006 into pingcap:master Oct 29, 2018

liufuyang deleted the aws-eks-tidb branch October 29, 2018 22:27

queenliuxx pushed a commit to queenliuxx/tidb-operator that referenced this pull request Dec 19, 2018

Add doc on how to provision kubernetes on aws (pingcap#71)

5194571

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add doc on how to provision kubernetes on aws #71

Add doc on how to provision kubernetes on aws #71

liufuyang commented Aug 31, 2018

CLAassistant commented Aug 31, 2018 •

edited

Loading

gregwebs commented Aug 31, 2018

liufuyang commented Aug 31, 2018 •

edited

Loading

morgo commented Sep 15, 2018

liufuyang commented Sep 17, 2018

morgo commented Sep 17, 2018

liufuyang commented Sep 22, 2018

morgo left a comment

morgo Oct 25, 2018

liufuyang Oct 27, 2018

morgo Oct 27, 2018

morgo Oct 25, 2018

liufuyang Oct 27, 2018

morgo Oct 27, 2018

morgo Oct 25, 2018

morgo Oct 25, 2018

morgo Oct 25, 2018

morgo Oct 25, 2018

morgo Oct 25, 2018

morgo Oct 25, 2018

morgo Oct 25, 2018

morgo Oct 25, 2018

liufuyang commented Oct 27, 2018

morgo left a comment

morgo Oct 27, 2018

liufuyang commented Oct 28, 2018 •

edited

Loading

tennix left a comment

morgo commented Oct 29, 2018

tennix commented Oct 29, 2018

liufuyang commented Oct 29, 2018


		## Introduction

		This tutorial is designed to be run locally with tools like [AWS Command Line Interface](https://aws.amazon.com/cli/) and [Terraform](https://www.terraform.io/). The Terraform code will ultilize a relatively new AWS service called [Amazon Elastic Container Service for Kubernetes (Amazon EKS)](https://aws.amazon.com/eks).


		## A bit more about Terraform

		Information about using Terraform with EKS can be found [here](https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html).



		### Destroy
		At the end of the demo, please make sure all the resources created by Kubernetes are removed (LoadBalancers, Security groups), and issue:


		Now you can switch back to the `tidb-operator` folder to follow the guide below.

		Then from this stage, you should be able to follow the [Tidb-Operator guide](/~https://github.com/pingcap/tidb-operator/blob/master/docs/google-kubernetes-tutorial.md) from section `Install Helm`, `Deploy TiDB Operator` and so on.

		Then from this stage, you should be able to follow the [Tidb-Operator guide](/~https://github.com/pingcap/tidb-operator/blob/master/docs/google-kubernetes-tutorial.md) from section `Install Helm`, `Deploy TiDB Operator` and so on.

		### Deploy tidb-operator

		### Deploy Kubernetes Dashboard

		#### Step 1: Deploy the Dashboard

Add doc on how to provision kubernetes on aws #71

Add doc on how to provision kubernetes on aws #71

Conversation

liufuyang commented Aug 31, 2018

CLAassistant commented Aug 31, 2018 • edited Loading

gregwebs commented Aug 31, 2018

liufuyang commented Aug 31, 2018 • edited Loading

morgo commented Sep 15, 2018

liufuyang commented Sep 17, 2018

morgo commented Sep 17, 2018

liufuyang commented Sep 22, 2018

morgo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liufuyang commented Oct 27, 2018

morgo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liufuyang commented Oct 28, 2018 • edited Loading

tennix left a comment

Choose a reason for hiding this comment

morgo commented Oct 29, 2018

tennix commented Oct 29, 2018

liufuyang commented Oct 29, 2018

CLAassistant commented Aug 31, 2018 •

edited

Loading

liufuyang commented Aug 31, 2018 •

edited

Loading

liufuyang commented Oct 28, 2018 •

edited

Loading