Question

Trouble getting cert-manager on hosted Kubernetes with Drone helm chart.

Posted on November 9, 2018
Let's Encrypt CI/CD Load Balancing Kubernetes
Asked by Tony Apuzzo

I am trying to get Drone CI/CD (v0.8.6) running in hosted Kubernetes (1.12) with SSL provisioned by Let’s Encrypt cert-manager.

The standard unencrypted install works fine, I specify a service of LoadBalancer and then DO notices that and creates a LoadBalancer that forwards to the ingress. It works fine with HTTP.

But when I introduce cert-manager, there are 2 problems:

while it seems to mostly work, the http01 challenge is not mapped correctly. I see that there are routes created in the ingress for the challenge but they don’t get routed by the LoadBalancer and consequently no certificates are issued.
The load balancer doesn’t get updated to forward 443 through to the ingress for termination – I"m guessing this is just a limitation of the automaticly deployed LB in DO.

I’m guessing that the helm chart needs to have some additional annotations passed through to the ingress (or manually applied) to make everything sync up in the DO environment – but I am new to cert-manager and don’t have a working environment to use as a reference.

So with all that, here are my questions:

Does anyone have cert-manager working with a Load Balancer and ingress in hosted Kubernetes? It doesn’t look to me like this is currently working with the standard install of cert-manager via helm.
would it be better to use dns01 challenges? it seemed like that was going to be harder to implement in DO but since the kubernetes networking is out of the loop it might be easier.
Has anyone gotten Drone running in hosted kubernetes on DO with cert-manager?

Advice appreciated, thank you!

BTW, currently I took the lazy way and am using non-SSL deployment with a manually applied SSL termination in the Load Balancer and that is working OK, although with that approach I cannot allocate a certificate in a sub-domain and had to create a whole new TLD, but that is a topic for another thread.

Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

myyra • November 13, 2018

Accepted Answer

I have that exact setup working with ingress-nginx, in addition to external-dns chart adding the DNS entries automatically. You didn’t mention having nginx as the ingress so if you don’t I suggest you install that.

ingress-nginx will create the LoadBalancer automatically and forward ports 80 and 443 as TCP to the Droplets (so the TLS termination happens in the cluster). Nginx will also be cheaper than using services since only one LoadBalancer is needed for the whole cluster.

When installing ingress-nginx through Helm, set controller.publishService.enabled: true so that the Ingress objects will get the IP of the LoadBalancer instead of the Droplet they reside on. I don’t think this matters if you’re not using external-dns, but at least to me it improves readability.

With this setup the cert-manager will add a path for the http01 challenge to the Ingress object automatically.

These are my values for the drone ingress:

ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: 'true'
  hosts:
    - drone.example.com
  tls:
    - secretName: drone-tls
      hosts:
        - drone.example.com

naarani • November 11, 2018

hi! for these points:

it works for me however i started with ONE domain and one service and one ssl cert, (server PRIME and domanin www.prime.xyz) then i moved to differents domanins to differents services and different ssl cert (server PRIME and domanin www.prime.xyz and service SECONDARY with domain splash.secondary.xyz ) now i’m trying to have differents domains with multplice ssl cert and i’m not able to make it work actually (server PRIME and domanin www.prime.xyz and service SECONDARY with domain splash.secondary.xyz AND service TROUBLE with domain trouble.secondary.xyz )
i’m using http01
i’m not using drone…