technical,
work,
kubernetes,
security,
docker. haproxy21:08 -0500
A while ago, I blogged about
automatic Let's Encrypt certificate renewal with nginx. Since then, I've also set up renewal in our Kubernetes cluster.
Like with nginx, I'm using
acme-tiny to do the renewals. For Kubernetes, I created a
Docker image. It reads the Let's Encrypt secret key from /etc/acme-tiny/secrets/account.key, and CSR files from /etc/acme-tiny/csrs/{name}.csr. In Kubernetes, these can be set up by mounting a Secrets store and a ConfigMap, respectively. It also reads the current certificates from /etc/acme-tiny/certs/{name}, which should also be set up by mounting a ConfigMap (called certificates), since that is where the container will put the new certificates.
Starting an acme-tiny pod will start an nginx server to store the .well-known directory for the Acme challenge. Running /opt/acme-tiny-utils/renew in the pod will renew the certificate if it will expire within 20 days (running it with the -f option will disable the check). Of course, we want the renewal to be automated, so we want to set up a sort of cron task. Kubernetes has cron jobs since 1.4, but at the time I was setting this up, we were still on 1.3. Kubernetes also does cron jobs by creating a new pod, whereas the way I want this to work is to run a program in an existing pod (though it could be set up to work the other way too). So I have another
cron Docker image, which I have set up to run
kubectl exec `kubectl get pods --namespace=lb -l role=acme -o name | cut -d / -f 2` --namespace=lb ./renew sbscalculus.com
every day. That command finds the acme-tiny pod and runs the renew command, telling it to renew the sbscalculus.com certificate.
Now in order for the Acme challenge to work, HTTP requests to /.well-known/acme-challenge/ get redirected to acme-tiny rather than to the regular pods serving those services. Our services are behind
our HAProxy image. So I have a 0acmetiny entry (the 0 causes it to be sorted before all other entries) in the services ConfigMap for HAProxy that reads:
{
"namespace": "lb",
"selector": {
"role": "acme"
},
"hostnames": ["^.*"],
"path": "^/\\.well-known/acme-challenge/.*$",
"ports":
[80]}
This causes HAProxy to all the Acme challeges to the acme-tiny pod, while leaving all the other requests alone.
And that's how we have our certificates automatically renewed from Let's Encrypt.
0 Comments 
