AWS EKS service EXTERNAL-IP of load balancer is Pending
I was working on a staging cluster for my application, it required around 12 load balancers for my services definition. All of 12 looked pretty much the same:
apiVersion: v1 kind: Service metadata: labels: app: my-app-api name: my-app-api namespace: default annotations: service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http service.beta.kubernetes.io/aws-load-balancer-ssl-cert: <some aws cert name> service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https" spec: externalTrafficPolicy: Cluster ipFamilies: - IPv4 ipFamilyPolicy: SingleStack ports: - name: http port: 80 protocol: TCP targetPort: 5001 - name: https port: 443 protocol: TCP targetPort: 5001 selector: app: my-app-api sessionAffinity: None type: LoadBalancer
After that I went on creating production cluster with the same setup. After I have created it and deployed k8s manifests: deployments, services, I was not able to get LoadBalancer Ingress with kubectl describe service
command. I noticed the following picture:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE <some-name> LoadBalancer <some_ip> ****.us-west-1.elb.amazonaws.com 80:30339/TCP,443:32754/TCP 1m <some-name> LoadBalancer <some_ip> ****.us-west-1.elb.amazonaws.com 80:31538/TCP,443:32061/TCP 1m <some-name> LoadBalancer <some_ip> ****.us-west-1.elb.amazonaws.com 80:30976/TCP,443:31323/TCP 1m <some-name> LoadBalancer <some_ip> ****.us-west-1.elb.amazonaws.com 80:30288/TCP,443:32073/TCP 1m <some-name> LoadBalancer <some_ip> ****.us-west-1.elb.amazonaws.com 80:32270/TCP,443:31159/TCP 1m <some-name> LoadBalancer <some_ip> ****.us-west-1.elb.amazonaws.com 80:31966/TCP,443:30944/TCP 1m kubernetes ClusterIP <some_ip> <none> 443/TCP 1m <some-name> LoadBalancer <some_ip> PENDING 80:31901/TCP,443:30444/TCP 1m <some-name> LoadBalancer <some_ip> PENDING 80:31510/TCP,443:30393/TCP 1m <some-name> LoadBalancer <some_ip> PENDING 80:32613/TCP,443:32616/TCP 1m <some-name> LoadBalancer <some_ip> PENDING 80:32069/TCP,443:30320/TCP 1m <some-name> LoadBalancer <some_ip> PENDING 80:31667/TCP,443:32194/TCP 1m <some-name> LoadBalancer <some_ip> PENDING 80:31943/TCP,443:32081/TCP 1m
Answer
After troubleshooting the reason of above behaviour, I have made the following conclusions about LoadBalancers(LB):
service.beta.kubernetes.io/aws-load-balancer-ssl-cert
directly depends on AWS load balancer, and if certificate is not signed in a region where LB is created, LB will not be added to the k8s cluster.- My problem was because k8s cluster could not automatically provision AWS LB for newly created service, as default LB limit for single region in AWS account is 20
I requested quota increase LB limit from AWS, but since it took more time, I moved my production cluster to a different AWS region. After that LBs created as expected and I could get my ingresses.