DNS-01 challenge: Certificats wildcard TLS sur Kubernetes

When deploying applications on a Kubernetes cluster, we often expose them outside using ingress controllers, and using cert-manager to manage TLS certificates with Let’s Encrypt. When requesting certificates, the most common way is to use the HTTP-01 challenge, that checks if you can host content on a specific URL.

Sometimes you can use another challenge type, called DNS-01: Let’s Encrypt checks a specific DNS entry, to be sure you own the domain you requested the certificate for. This can be useful for some use cases:

Applications that are not exposed to Internet (Let’s Encrypt cannot connect to the URL content) Wildcard certificates, as there is no default URL to store content You can read more about it here: https://letsencrypt.org/docs/challenge-types/

To request such a certificate in a Kubernetes cluster, you need to automatically update the DNS records from your provider. Only major cloud providers are natively supported by default, but you can extend it using webhooks.

In this article, we will see how to quickly set up a k8s cluster, configuring cert-manager to handle DNS-01 challenge and get wildcard certificates.

Setup & Prerequisites

For this setup, I will use the following platform and tools:

• Scaleway Kubernetes Kapsule to bootstrap a k8s cluster
• Scaleway DNS, as we can manage it with cert-manager webhooks
• Terraform to quickly setup everything
• k8s Lens to easily view the k8s resources

I will not cover Terraform installation or other kubectl & helm tools, you can find good resources somewhere else.

Cluster provisioning

Using Terraform, you can provision a Kapsule cluster. We just use a scaleway_k8s_cluster and a scaleway_k8s_pool resource:

 1terraform {
 2  required_providers {
 3    scaleway = {
 4      source = "scaleway/scaleway"
 5      version = "2.1.0"
 6    }
 7  }
 8  required_version = ">= 0.13"
 9}
10
11provider "scaleway" {
12  zone            = "fr-par-1"
13  region          = "fr-par"
14}
15
16resource "scaleway_k8s_cluster" "k8s_cluster" {
17  name = var.k8s_cluster_name
18  version = "1.22.2"
19  cni = "cilium"
20}
21
22resource "scaleway_k8s_pool" "k8s_pool" {
23  cluster_id = scaleway_k8s_cluster.k8s_cluster.id
24  name = var.k8s_pool_name
25  node_type = "DEV1-M"
26  size = 2
27  autohealing = true
28}
29
30resource "local_file" "kubeconfig" {
31  content = scaleway_k8s_cluster.k8s_cluster.kubeconfig[0].config_file
32  filename = "${path.module}/${scaleway_k8s_cluster.k8s_cluster.name}-kubeconfig"
33  file_permission = "0600"
34}

Now we can provision the cluster:

1$ terraform init
2$ terraform plan -out k8s_scaleway.out
3$ terraform apply k8s_scaleway.out

Wait few minutes and we will have a functional cluster with Nginx as an ingress controller, with a load-balancer, and a cert-manager configured. We will also find a kubeconfig file ready for playing with the cluster!

DNS Setup

We can configure DNS entry to link the load balancer IP address to our cluster domain. I could do it automatically, but there is no Terraform resource for Scaleway right now (planned for the next release, see here).

For this setup, my domain is scw.vrchr.fr, so any URL ending with it will be redirect to the k8s cluster through the load balancer.

Edit: the cluster just created does not exists anymore as you are reading this, as it was only online for this article. So don’t try to resolve the following FQDN ;)

Wildcard Certificate with DNS-01

Scaleway webhook

Remember, for some providers we need to use cert manager webhooks to manage the DNS entries. So we install it:

1$ git clone https://github.com/scaleway/cert-manager-webhook-scaleway.git
2$ cd cert-manager-webhook-scaleway
3$ helm install scaleway-webhook deploy/scaleway-webhook --set secret.accessKey=changeme --set secret.secretKey=changeme --set certManager.serviceAccountName=jetstack-cert-manager --namespace=cert-manager

Be careful of the serviceAccountName which has to be the same as the one created with the cert-manager Helm chart.

So now we can see, using our preferred tool, the deployments and all the cert-manager configs and custom resources:

Issuer and Certificates

Next, we have to create a certificate issuer, which will be responsible to request the TLS certificates. This is a Custom Resource, with the property “dns01”. We create a ClusterIssuer instead of Issuer to be able to manage certificates cluster-wide:

 1---
 2apiVersion: cert-manager.io/v1
 3kind: ClusterIssuer
 4metadata:
 5  name: scaleway-issuer-prod
 6spec:
 7  acme:
 8    email: name@company.com
 9    server: https://acme-v02.api.letsencrypt.org/directory
10    privateKeySecretRef:
11      name: scaleway-private-key-secret
12    solvers:
13    - dns01:
14        webhook:
15          groupName: acme.scaleway.com
16          solverName: scaleway

1$ kubectl apply -f cert_issuer.yaml

We can add another issuer for staging certificates if needed.

Now, Let’s create a certificate request! We’ll make request one for “*.scw.vrchr.fr”, which will be stored in a secret called “wildcard-scw-vrchr-fr-tls”:

 1apiVersion: cert-manager.io/v1
 2kind: Certificate
 3metadata:
 4  name: wildcard-scw-vrchr-fr-tls
 5  namespace: default
 6spec:
 7  dnsNames:
 8  - "*.scw.vrchr.fr"
 9  issuerRef:
10    name: scaleway-issuer-prod
11    kind: ClusterIssuer
12  secretName: wildcard-scw-vrchr-fr-tls

What happens now?

Let’s Encrypt detects a Certificate Request, and makes the request to the servers It uses the webhook to create a TXT DNS entry, here it’s : _acme-challenge.scw.vrchr.fr The server will check the DNS entry, and validate the request. Be patient, DNS propagation can be long… The certificate is now created, and a secret is now in you cluster!

Application deployment

OK! We have now a wildcard certificate automatically managed, so let’s use it!

We create a simple deployment, expose the application, and create an ingress rule. The important configuration is in the ingress, which specify which TLS certificate to use. We don’t need to add specific annotation for the ingress controller, as the wildcard certificate is already created and managed by the previous CRD. As we have created a wildcard, every FQDN matching the wildcard will be valid:

1[...]
2spec:
3  tls:
4  - hosts:
5    - "*.scw.vrchr.fr" 
6    secretName: wildcard-scw-vrchr-fr-tls
7[...]

Here is the complete deployment, using the jpetazzo/webcolor application to expose different colors.

Deployment & Service example:

 1---
 2apiVersion: apps/v1
 3kind: Deployment
 4metadata:
 5  name: green
 6spec:
 7  selector:
 8    matchLabels:
 9      app: green
10  replicas: 1
11  template:
12    metadata:
13      labels:
14        app: green
15    spec:
16      containers:
17      - image: jpetazzo/webcolor
18        name: webcolor
19
20---
21apiVersion: v1
22kind: Service
23metadata:
24  name: green
25spec:
26  ports:
27  - port: 8000
28    targetPort: 8000
29  selector:
30    app: green

Ingress:

 1---
 2apiVersion: networking.k8s.io/v1
 3kind: Ingress
 4metadata:
 5  name: color-ingress
 6  annotations:
 7    kubernetes.io/ingress.class: nginx
 8spec:
 9  tls:
10  - hosts:
11    - "*.scw.vrchr.fr" 
12    secretName: wildcard-scw-vrchr-fr-tls
13  rules:
14  - host: green.scw.vrchr.fr
15    http:
16      paths:
17      - path: /
18        pathType: Prefix
19        backend:
20          service:
21            name: green
22            port:
23              number: 8000
24  - host: purple.scw.vrchr.fr
25    http:
26      paths:
27      - path: /
28        pathType: Prefix
29        backend:
30          service:
31            name: purple
32            port:
33              number: 8000
34  - host: yellow.scw.vrchr.fr
35    http:
36      paths:
37      - path: /
38        pathType: Prefix
39        backend:
40          service:
41            name: yellow
42            port:
43              number: 8000

1$ kubectl apply -f dep_green.yaml
2$ kubectl apply -f wildcard_tls_ingress.yaml

Now, accessing to https://green.scw.vrchr.fr would present a wildcard certificate!

Conclusion and Remarks

We have seen here how to:

• Deploy a Kubernetes Cluster on Scaleway using Terraform
• Deploy Cert Manager and Nginx using Helm chart in Terraform
• Deploy Scaleway DNS Webhook
• Request a Wildcard Certificate using the DNS-01 protocol
• Deploy an application in a subdomain, using a ingress rule presenting the wildcard DNS

Now you can deploy many sub-application as we want, all using the same wildcard TLS certificates. For the comprehensive read, I deployed applications in the same namespace, please feel free to read cert-manager docs if you want to share certificates across multiples NS.

Note that I could use another cloud provider, but I like experiment french ones ;)

Finally, You can find code example in my GitLab repository, here : https://gitlab.com/rverchere/vrchr-k8s-dns-demo

And remember, it’s always DNS!

References

These resources were very useful to understand these concepts and write my article:

• https://particule.io/blog/scaleway-cert-manager/
• https://www.youtube.com/watch?v=OvxSk5YVvII&t=1366s
• https://cert-manager.io
• https://registry.terraform.io/providers/scaleway/scaleway/latest