According to CNCF surveys, security is always among the top concerns for Kubernetes practitioners and platform engineers. Percona Everest — an open source cloud-native database platform – is now in Alpha stage, but it is important to set it up properly from day 0. In this blog post, we will explain how to run Everest UI in Kubernetes behind ingress and valid SSL certificate.
The design
The Percona Everest frontend is a web-based application. The regular design is to have ingress as a single point of entry, where certificates are provided by cert-manager and Let’s Encrypt.
You can find the manifests that were used in this blog post in the GitHub repository blog-data/everest-ingress-ssl.
Prerequisites
- Kubernetes cluster
- Percona Everest is up and running
- Ingress Controller
– I will deliberately use an open-source ingress-nginx controller to demonstrate the generic case.
– Most cloud providers already have their own ingress solutions in place, so you can just use them instead. Please consult with the corresponding documentation, but I do not expect drastic differences in the manifests. It should just work.
– Domain name to have a proper certificate and to point it to Percona Everest instance.
Action
Install Percona Everest
Before exposing Everest, we need to have it up and running. A quick way here would be to download the CLI tool everestctl and install Everest with it.
Find how to download everestctl in our documentation. For example, for MacOS with an ARM chip, do the following:
1 2 3 | curl -sSL -o everestctl-darwin-arm64 https://github.com/percona/percona-everest-cli/releases/latest/download/everestctl-darwin-arm64 sudo install -m 555 everestctl-darwin-arm64 /usr/local/bin/everestctl rm everestctl-darwin-arm64 |
Now install Pecona Everest:
1 | everestctl install |
Ingress
Start with deploying an ingress-nginx controller. Helm is most probably the easiest way:
1 2 3 | helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace |
This will deploy nginx-ingress controller and expose it with a LoadBalancer service. If your Kubernetes cluster does not support load balancers, then please read through the ingress controller documentation on how to customize the deployment.
To verify that the ingress controller works as expected, we will create an ingress resource pointing to Everest and exposing it on port 80, no TLS for now. By default, Percona Everest is deployed in the percona-everest namespace and exposed through a ClusterIP service everest on port 8080.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: everest-ingress namespace: percona-everest annotations: kubernetes.io/ingress.allow-http: "true" spec: ingressClassName: nginx rules: - host: everest.percona.com http: paths: - path: / pathType: Prefix backend: service: name: everest port: number: 8080 |
1 | kubectl apply -f 01-ingress.yaml |
01-ingress.yaml will create an ingress resource. Check if it is there and get the public IP-address:
1 2 3 4 | % kubectl -n percona-everest get ingress NAME CLASS HOSTS ADDRESS PORTS AGE everest-ingress nginx everest.percona.com 35.232.162.56 80 106s |
Now if you point your domain name to the ADDRESS, you should be able to open it in your browser. In my case, it is http://everest.percona.com; it is also set in the rules section:
1 2 | rules: - host: everest.percona.com |
Secure
Let’s Encrypt is the easiest way to get a free TLS certificate. To get it, we will use cert-manager. Install it by following the documentation.
Staging issuer
Let’s Encrypt API has a strict rate-limit. It is strongly recommended to try to get a staging certificate first.
Create an Issuer for Let’s Encrypt staging:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | apiVersion: cert-manager.io/v1 kind: Issuer metadata: name: letsencrypt-staging namespace: percona-everest spec: acme: server: https://acme-staging-v02.api.letsencrypt.org/directory email: <YOUR_EMAIL> # ❗ Replace this with your email address privateKeySecretRef: name: letsencrypt-staging solvers: - http01: ingress: ingressClassName: nginx |
1 | kubectl apply -f 02-issuer-staging.yaml |
Reconfigure ingress with TLS
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | --- 01-ingress.yaml 2024-02-01 16:48:44 +++ 03-ingress-tls-staging.yaml 2024-02-01 16:56:16 @@ -5,8 +5,14 @@ namespace: percona-everest annotations: kubernetes.io/ingress.allow-http: "true" + cert-manager.io/issuer: letsencrypt-staging + acme.cert-manager.io/http01-edit-in-place: "true" spec: ingressClassName: nginx + tls: + - secretName: everest-ssl + hosts: + - everest.percona.com rules: - host: everest.percona.com http: |
1 | kubectl apply -f 03-ingress-tls.yaml |
It might take a couple of minutes for the certificate to be provisioned. Verify it by trying to connect to https endpoint of your domain for Everest:
1 2 3 4 5 6 7 8 | % curl -v --insecure https://everest.percona.com ... * Server certificate: * subject: CN=everest.percona.com * start date: Feb 1 13:54:59 2024 GMT * expire date: May 1 13:54:58 2024 GMT * issuer: C=US; O=(STAGING) Let's Encrypt; CN=(STAGING) Artificial Apricot R3 |
Production issuer
Now that everything is working with the Let’s Encrypt staging server, we can switch to the production server and get a trusted SSL certificate for your domain.
Create production Issuer:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | apiVersion: cert-manager.io/v1 kind: Issuer metadata: name: letsencrypt-production namespace: percona-everest spec: acme: server: https://acme-v02.api.letsencrypt.org/directory email: <YOUR_EMAIL> privateKeySecretRef: name: letsencrypt-production solvers: - http01: ingress: ingressClassName: nginx |
1 | kubectl apply -f 04-issuer-prod.yaml |
Upgrade ingress controller to use production Issuer:
1 2 3 4 5 6 7 8 9 10 11 12 13 | % diff -u 03-ingress-tls-staging.yaml 05-ingress-tls-prod.yaml --- 04-ingress-tls-staging.yaml 2024-02-01 12:25:39 +++ 06-ingress-tls-prod.yaml 2024-02-01 12:25:01 @@ -5,7 +5,7 @@ namespace: percona-everest annotations: kubernetes.io/ingress.allow-http: "true" - cert-manager.io/issuer: letsencrypt-staging + cert-manager.io/issuer: letsencrypt-production acme.cert-manager.io/http01-edit-in-place: "true" spec: ingressClassName: nginx |
1 | kubectl apply -f 05-ingress-tls-prod.yaml |
After a few moments, the certificate will be updated. Now you can connect to your Percona Everest instance through HTTPs.
Troubleshooting
The issue that I faced, and it seems to be quite common (1, 2), is not getting the certificate applied and seeing the following error in cert-manager logs:
1 | Waiting for http-01 challenge propagation: did not get expected response when querying endpoint, expected "xxxxx" but got: <!doctype html> |
It depends on how routing is configured in your cluster and if cert-manager request leaves the Kubernetes cluster. The simplest fix here is to add acme.cert-manager.io/http01-edit-in-place annotation to your ingress objects.
1 2 3 4 5 6 | apiVersion: networking.k8s.io/v1 kind: Ingress metadata: ... annotations: acme.cert-manager.io/http01-edit-in-place: "true" |
Read more about this annotation in cert-manager documentation.
Conclusion
In conclusion, Percona Everest represents a significant advancement in cloud-native database solutions, offering a robust, scalable, and secure platform for managing databases in Kubernetes environments. By leveraging the power of Percona Everest, organizations can enjoy the benefits of an open-source database platform designed to meet the demands of modern applications, with the added advantages of cloud-native technologies for enhanced performance, flexibility, and reliability.
We encourage you to take the first step towards a more secure and efficient database management experience by trying out Percona Everest. Visit our documentation to get started, and join our community to share your experiences and learn from others. Embrace the future of database management with Percona Everest, and take your applications to new heights with confidence.
