Civo's documentation on how to Secure your Kubernetes services with Let's Encrypt and cert-manager

Get a TLS Certificate For Your Application With JetStack's Cert-Manager

• If you have not, please complete the Civo Environment Setup procedure
• It is highly recommended to monitor each installation step with kubernetes-dashboard

1. Confirm you are in the build directory:

   cd build

2. Run the following command to install cert-manager only if you did not install it in the Civo Setup Steps:

   civo kubernetes applications add cert-manager --cluster cultivate-finance

3. Confirm cert manager is running on it's own namespace entitled 'cert-manager':

   kubectl get pods --namespace cert-manager
   # you should see 3 entries:
   # 1. cert-manager-cainjector-<uid>
   # 2. cert-manager-<uid>
   # 3. cert-manager-webhook-<uid>

We are going to use ClusterIssuers because we are going to be creating certificates in the 'default' namespace using issuers in the cert-manager namespace.

1. Create the cert issuer:

   kubectl apply -f k8s/ssl-certificate/cert-issuer.yml

2. Confirm your cert-issuers exist in the cert-manager namespace:

   kubectl get secrets --namespace cert-manager
   # you should see 2 entries:
   # letsencrypt-prod
   # letsencrypt-staging

Civo's default DNS is a uuid followed by .k8s.civo.com, or looks something like 798a2f3d-a352-40aa-a882-ef5000d3653a.k8s.civo.com. We need change the entries in k8s/certificates-staging.yml and k8s/certificates-prod.yml to match our DNS name.

1. Get the uuid that matches our dns:

   civo kubernetes show cultivate-finance -o custom -f "DNSEntry" | sed "s/\..*//"

2. Edit k8s/certificates-staging.yml:

   apiVersion: cert-manager.io/v1
   kind: Certificate
   metadata:
     name: <UUID GOES HERE>-staging-cert
     namespace: default
   spec:
     commonName: '<UUID GOES HERE>.k8s.civo.com'
     dnsNames:
     - '<UUID GOES HERE>.k8s.civo.com'
     issuerRef:
       kind: ClusterIssuer
       name: letsencrypt-staging 
     secretName: <UUID GOES HERE>-staging-cert
    
   ### EXAMPLE 
   # apiVersion: cert-manager.io/v1
   # kind: Certificate
   # metadata:
   #   name: 798a2f3d-a352-40aa-a882-ef5000d3653a-staging-cert
   #   namespace: default
   # spec:
   #   commonName: '798a2f3d-a352-40aa-a882-ef5000d3653a.k8s.civo.com'
   #   dnsNames:
   #   - '798a2f3d-a352-40aa-a882-ef5000d3653a.k8s.civo.com'
   #   issuerRef:
   #     kind: ClusterIssuer
   #     name: letsencrypt-staging 
   #   secretName: 798a2f3d-a352-40aa-a882-ef5000d3653a-staging-cert

3. Do the same with k8s/certificates-prod.yml:

   apiVersion: cert-manager.io/v1
   kind: Certificate
   metadata:
     name: <UUID GOES HERE>-prod-cert
     namespace: default
   spec:
     commonName: '<UUID GOES HERE>.k8s.civo.com'
     dnsNames:
     - '<UUID GOES HERE>.k8s.civo.com'
     issuerRef:
       kind: ClusterIssuer
       name: letsencrypt-prod 
     secretName: <UUID GOES HERE>-prod-cert

4. Do the same with our ingress controller:

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: cultivate-finance-ingress-base-dns
     namespace: default
     labels:
       name: cultivate-finance-ingress-base-dns
     annotations:
       kubernetes.io/ingress.class: traefik
   spec:
     tls:
     - hosts:
       - '<UUID GOES HERE>.k8s.civo.com'
       secretName: <UUID GOES HERE>-staging-cert
     rules:
     - host: '<UUID GOES HERE>.k8s.civo.com'
       http:
         paths:
         - path: "/"
           pathType: Prefix
           backend:
             service:
               name: webapp-service 
               port: 
                 number: 8662

Staging certificates are not functional, but since we are using letsencrypt's api, we want to make sure our certificate creation is working as expected before creating real certificates.

1. Create the staging certificates:

   kubectl apply -f k8s/ssl-certificate/certificates-staging.yml

2. Confirm you see the certificates in your cluster:

   kubectl get certificate --namespace default
   ## You should see 3 entries
   # cultivate-finance-staging-www-cert
   # cultivate-finance-staging-base-cert
   # <UUID from earlier>-staging-cert

3. These certs will not be considered valid until their READY status is True. You can repeat the command over and over again to check their ready status, but you can also check the event log in kubernetes dashboard
4. This section can only be considered complete when all 3 of the certificates are marked as READY

Now that we have our test certificates, we need to smoke test them by confirming they are attached to our webapp. In order to do that, we must confirm our ingress and webapp pods are deployed by doing the following:

1. Follow the instructions for deploying our ingress: Deploy Ingress
2. Follow the instructions for deploying our webapp: Deploying the Webapp

Once those are deployed you can confirm they are using our testing certificates from earlier:

1. Test our www domain:

   openssl s_client -connect www.cultivatefinance.org:443
    
   ## Confirm you see something along the lines of:
   # depth=1 C=US, O=(STAGING) Let's Encrypt, CN=(STAGING) Wannabe Watercress R11
   # issuer=C=US, O=(STAGING) Let's Encrypt, CN=(STAGING) Wannabe Watercress R11
   # 
   # The important part is O=(STAGING) Let's Encrypt, CN=(STAGING)

2. Repeat the process with our non-www domain:

   openssl s_client -connect cultivatefinance.org:443

3. Repeat the process with our civo domain:

   openssl s_client -connect $(civo kubernetes show cultivate-finance -o custom -f "DNSEntry"):443

* NOTE: ONLY CONTINUE TO THESE STEPS AFTER YOU HAVE SUCCESSFULLY COMPLETED THE PREVIOUS ONES WITHOUT ISSUE.

Now that we have confirmed our application is running with testing tls certificates, we can do the real thing.

1. Generate the production certificates:

   kubectl apply -f k8s/ssl-certificate/certificates-prod.yml

2. Confirm the certificates have been properly issued:

   kubectl get certificate --namespace default
   ## You should see 3 new entries
   # cultivate-finance-prod-www-cert
   # cultivate-finance-prod-base-cert
   # <UUID from earlier>-prod-cert

3. Wait until the certificates are in the READY state. It could take up to 30 minutes.
4. Delete the staging ingress:

   kubectl delete -f k8s/ingress-traefik-staging.yml

5. Deploy the production ingress:

   kubectl apply -f k8s/ingress-traefik-prod.yml

6. Use the links to confirm the connection is secure:
   1. https://cultivatefinance.org
   2. https://www.cultivatefinance.org