Capstone project for Udacity's Cloud DevOps Nanodegree

*Pods that run a single container. The “one-container-per-Pod” model is the most common Kubernetes use case; in this case, you can think of a Pod as a wrapper around a single container, and Kubernetes manages the Pods rather than the containers directly.
*Pods that run multiple containers that need to work together. A Pod might encapsulate an application composed of multiple co-located containers that are tightly coupled and need to share resources. These co-located containers might form a single cohesive unit of service–one container serving files from a shared volume to the public, while a separate “sidecar” container refreshes or updates those files. The Pod wraps these containers and storage resources together as a single manageable entity.

Finally, we will introduce another Kubernetes framework called Stateful Set, used to manage stateful applications such as databases. In this link I found the best definition of them:

StatefulSets provides the capabilities of stable unique network hostnames and stable dedicated network storage volume mappings, essential for a database cluster to function properly and for data to exist and outlive the lifetime of inherently ephemeral containers.

For any quick modifications to app.py, we can test locally by running a postgres Docker container:

1. Run docker Postgres: docker run -p 5432:5432 --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
2. Test to run a query with sh post_query.sh localhost 5000 "<my_query>"

• Configuration file for the application and scripts, can be found in scripts/config.sh.
• Scripts are prepared to be run from the main repo directory.

Note that disabled ignore-modules=flask_sqlalchemy on pylintrc to disable E1101 errors due to methods only being available at runtime. Those can't be picked up by pylint and thus it throws ghost errors.

To test the application in Kubernetes (e.g., minikube), first get the IP of the cluster with minikube ip and run the following commands:

• sh scripts/deploy.sh
• sh scripts/post_query.sh 192.168.99.100 31234 "create table account (id_user serial PRIMARY KEY, username VARCHAR(50) NOT NULL)" should return OK.
• sh scripts/post_query.sh 192.168.99.100 31234 "insert into account (username) values ('pmbrull')" should return OK.
• sh scripts/post_query.sh 192.168.99.100 31234 "select * from account" should return

{
  "result": [
    {
      "id_user": 1, 
      "username": "pmbrull"
    }
  ]
}

We enabled a rolling update on the flask app deployment to ensure that we always have available instances, even if they still do not have the new code versions. To do so, in kubernetes/flask-deployment.yaml:

spec:
  replicas: 2
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 50%
      maxSurge: 1

We have two replicas and at maximum, only one will be down updating the images.

In the Jenkins EC2 instance we need to install:

• awscli and configure with the kops user.

export AWS_ACCESS_KEY_ID=$(aws configure get aws_access_key_id)
export AWS_SECRET_ACCESS_KEY=$(aws configure get aws_secret_access_key)

• kops - to create the cluster and deploy our app.

• kubectl - to interact with the cluster-

• Register a domain in Route 53. For that, I registered pmbrull-k8.com so that I can reach my cluster by name.

• Create an S3 bucket to store the cluster state & enable versioning.

• Export cluster variables

export NAME=pmbrull-k8.com
export KOPS_STATE_STORE=s3://pmbrull-k8-com-state-store

• Create ssh key for the server if we don't have one: ssh-keygen.
• SSH public key must be specified when running with AWS (create with kops create secret --name pmbrull-k8.com sshpublickey admin -i ~/.ssh/id_rsa.pub)
• Then, create the kubernetes cluster with kops:

kops create cluster \
    --zones us-west-2a \
    ${NAME}

• Now we can review the cluster configuration.
• Finally, build the cluster: kops update cluster ${NAME} --yes
• It will need a few minutes to validate the DNS from the created Route53 domain. Then, we can validate the cluster: kops validate cluster.
• Handshake Jenkins server with the cluster so that we can execute kubectl commands directly to pmbrull-k8.com:

sudo mkdir -p /var/lib/jenkins/.kube
sudo cp ~/.kube/config /var/lib/jenkins/.kube/
cd /var/lib/jenkins/.kube/
sudo chown jenkins:jenkins config
sudo chmod 750 config

• In Jenkins, store the Docker Hub password as Secret text to upload the image.
• Once we're all finished, delete the cluster kops delete cluster --name=${NAME} --state=${KOPS_STATE_STORE} --yes

Fix docker agent in Jenkinsfile permission error: sudo usermod -aG docker Jenkins

• kops-validate-cluster.png: Shows that after creating the cluster, everything is correctly set and available through the Jenkins server.
• kops-cluster.png: To check that we do have instances running as master and nodes.
• jenkins-success.png: With all the steps of the pipeline succeeding.
• lint-step.png: With the details of the linting step, to check both Python and Dockerfiles.
• k8s-services: With the result of the correct deployment of the kubernetes templates.
• load-balancer-service: Showing the details of the flask service created as a Load Balancer.
• test-pmbrull-k8: Testing the deployed application.

• Nirmata series on Kubernetes
• Testdriven: Flask + Vue + Kubernetes
• Flask Postgres Kubernetes Workshop
• Kops
• How to Create a Kubernetes Cluster on AWS in Few Minutes
• CI/CD with Jenkins, Docker and Kubernetes on AWS