Annotation reference

Every annotation SSLB3's ingress controller reads, what it accepts, and what it does when you leave it alone. All of them, not a selection.

Put these on an Ingress, each prefixed sslb3.ingress.hacking.hu/. Every one of them is optional: an Ingress with no annotations at all is a valid Ingress, and it is the one you should start from.

An annotation under the prefix that I do not recognise is logged and ignored. A value that will not parse is logged and dropped rather than quietly becoming a default, because a setting that silently did nothing is worse than one that was refused. Another controller's annotations — anything under nginx.ingress.kubernetes.io/, say — are not mine to interpret and are left alone.

Applied to the serverfarm

These are enforced by the balancer itself, on the farm behind the rule. The important consequence is in the note under the table: a farm is one object per service, so two Ingresses pointing at the same service share these settings whether they meant to or not.

AnnotationAcceptsUnsetWhat it does

load-balance

round-robin, weighted, least-connections, lua

balancer default

How a backend is chosen for this farm. lua hands the choice to a script.

connect-timeout

seconds

balancer default

How long to wait dialling a backend before giving up on it.

response-timeout

seconds, 0 waits forever

balancer default (60)

How long a backend has to start answering before the request is given up on with a 504. The failure a connect timeout cannot see: a pod that accepts the connection and then never replies holds the request forever. Raise it on a long-poll or SSE service; upgrades such as WebSockets are exempt already.

idle-timeout

seconds, 0 disables

balancer default

How long a connection may go silent before it is closed.

max-connection-duration

seconds, 0 disables

balancer default

How long a connection may last at all. The one worth setting: an idle timeout does nothing against a connection that trickles a byte just often enough to look busy.

serverfarm-sni

a server name

plaintext to backends

Speak TLS to the backends, presenting this name in the handshake.

serverfarm-ssl-verify

true, false

balancer default

Set false to accept the backend's certificate without checking it. Only relevant once the farm is speaking TLS.

affinity

cookie, none

none

Send a returning client back to the backend that served it. Sessions live in the balancer process, so with more than one replica a client reaching a different pod is balanced afresh.

session-cookie-name

a cookie name

balancer default

The affinity cookie's name. Only meaningful once affinity is on.

session-cookie-max-age

seconds

balancer default

How long a session survives unused, and the cookie's Max-Age.

threat-detection

off, detect, block

balancer default

What the intrusion detector does about this service. block refuses, detect counts and reports without refusing anyone, off does not look. This is what makes an IPS rollout possible one service at a time.

impersonation-enforcement

true, false

on

Set false to stop refusing clients that claim a browser in their User-Agent while sending none of a browser’s headers. The signal is still counted and reported; only the refusal stops. For the app or SDK that spoofs a browser User-Agent on purpose.

Examples

A fast failure when a backend will not answer, rather than waiting on the balancer’s default:

connect.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/connect-timeout: "3"

Timeouts for a service with long-lived connections — a websocket endpoint, say, where an idle timeout would be wrong but an unbounded connection is worse:

timeouts.yaml

metadata:
  annotations:
    # Silence is expected here, so do not close on it.
    sslb3.ingress.hacking.hu/idle-timeout: "0"
    # But nothing gets to live forever. This is the one worth setting: an idle
    # timeout does nothing against a connection trickling a byte to look busy.
    sslb3.ingress.hacking.hu/max-connection-duration: "3600"

Cookie affinity, for a backend that keeps per-user state in memory. Worth knowing before you reach for it: sessions live in the balancer's own process, so with more than one replica a client that reaches a different pod is balanced afresh whatever the cookie says.

affinity.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/affinity: cookie
    sslb3.ingress.hacking.hu/session-cookie-name: shop_backend
    sslb3.ingress.hacking.hu/session-cookie-max-age: "3600"

Talking TLS to the backends, for a service that terminates its own certificate. The verify switch is there for the common case of an internal certificate nothing in the cluster trusts:

backend-tls.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/serverfarm-sni: shop.internal
    sslb3.ingress.hacking.hu/serverfarm-ssl-verify: "false"

And the rollout this design exists for. Leave the balancer on block with the serverfarm default off, then move services across one at a time — watch first, enforce when the numbers look right:

threat-rollout.yaml

# The service you are still watching. Counted and reported, nobody refused.
metadata:
  annotations:
    sslb3.ingress.hacking.hu/threat-detection: detect
---
# The service you are confident about.
metadata:
  annotations:
    sslb3.ingress.hacking.hu/threat-detection: block
---
# The service with a mobile SDK that spoofs a browser User-Agent on purpose.
# Enforcing everywhere except that one signal, which would turn it away.
metadata:
  annotations:
    sslb3.ingress.hacking.hu/threat-detection: block
    sslb3.ingress.hacking.hu/impersonation-enforcement: "false"

Health checks

Off unless you turn one on. Kubernetes already removes an unready pod from the endpoints SSLB3’s backends are built from, so a probe from SSLB3 as well is a second opinion rather than the only one — worth having where readiness is too coarse, or where a pod can be running and still not answering.

Written per service, like the farm settings above, and resolved the same way when two Ingresses disagree. But they are not sent on the farm: the controller stamps the whole check onto every backend of the service, because a health check belongs to a backend and that is where the balancer runs it.

AnnotationAcceptsUnsetWhat it does

health-check

true, false

off

Have SSLB3 probe the backends itself, as well as whatever Kubernetes already thinks of the pods. None of the settings below do anything until this is on.

health-check-type

tcp, http, https, smtp, noop

balancer default

What kind of probe. tcp only connects; http and https send a GET and compare the status. lua is refused here — see below.

health-check-port

1–65535

each backend’s own

Where the check connects, for a health endpoint on a separate management port.

health-check-path

a path starting with /

/

What an http or https check requests.

health-check-host

a host name

the address checked

The Host header sent. For https it is also the server name presented, unless the service sets serverfarm-sni.

health-check-status

codes and ranges

200-399

Status codes that pass, comma separated: 200,401 or 200-399. For an application that answers its health path with a 401 when called unauthenticated.

health-check-interval

seconds, at least 1

10

How often each backend is checked.

health-check-timeout

seconds, at least 1

3

How long one check may take before it counts as failed.

health-check-fail-tolerance

1–255

3

Consecutive failures before a backend is taken out of rotation. One success resets it.

Examples

A Spring Boot service moving across from the AWS Load Balancer Controller, whose liveness endpoint answers 401 to an unauthenticated caller — which is healthy:

health.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/health-check: "true"
    sslb3.ingress.hacking.hu/health-check-type: http
    sslb3.ingress.hacking.hu/health-check-path: /actuator/health/liveness
    sslb3.ingress.hacking.hu/health-check-status: "200,401"
    sslb3.ingress.hacking.hu/health-check-interval: "15"

The mapping from the AWS controller’s annotations, for an Ingress being moved:

alb.ingress.kubernetes.io/sslb3.ingress.hacking.hu/

healthcheck-path

health-check-path

success-codes

health-check-status

healthcheck-interval-seconds

health-check-interval

healthcheck-timeout-seconds

health-check-timeout

healthcheck-port

health-check-port

healthcheck-protocol

health-check-type

unhealthy-threshold-count

health-check-fail-tolerance

A service whose health endpoint is on a separate management port, checked over TLS with the name its certificate carries:

health-management-port.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/health-check: "true"
    sslb3.ingress.hacking.hu/health-check-type: https
    sslb3.ingress.hacking.hu/health-check-port: "8443"
    sslb3.ingress.hacking.hu/health-check-path: /healthz
    sslb3.ingress.hacking.hu/health-check-host: shop.internal

lua is refused as a type. A health check script is a file on the balancer’s own disk, chosen in its configuration; neither an Ingress nor the admin API can name one.

Applied per request

These are carried on the route and applied to each request as it is served, so unlike the farm settings above they really are per-Ingress.

AnnotationAcceptsUnsetWhat it does

serverfarm-vhost

a host name

client's Host

Replace the Host header sent to the backend.

force-ssl-redirect

true, false

false

Answer plaintext requests with a 308 to the same URL over https.

allowlist-source-range

CIDRs or addresses, comma separated

admit anything

Admit only these. Also spelled whitelist-source-range.

denylist-source-range

CIDRs or addresses, comma separated

deny nothing

Refuse these. Also spelled blocklist-source-range. Resolved against the allow list by specificity, with an exact tie going to deny.

auth-type

basic

no authentication

The only kind supported. Needs auth-secret alongside it or nothing is applied and the omission is logged.

auth-secret

name, or namespace/name

—

The Secret holding htpasswd credentials, from the Ingress's own namespace. A namespace/name in another namespace is refused unless controller.allowCrossNamespaceSecrets is on.

auth-realm

free text

Restricted

Shown in the browser's credential prompt.

auth-keep-authorization

true, false

false

Let the backend see the Authorization header too, rather than having it consumed here.

enable-access-log

true, false

false

Write one access record per request served by this route.

allowed-protocols

http1, http2, http3

all three

HTTP versions this route answers, comma separated. h1, h2, h3, http/1.1 and http/1.0 are accepted spellings. An unrecognised name leaves the whole list unapplied rather than refusing traffic nobody asked to refuse.

http3-advertise-port

1–65535

the listener’s

The port named in the Alt-Svc header for this Ingress. Unset, it is the balancer’s own advertised HTTP/3 port — the same one a natively served route names — so set it only when one Ingress is reached on a different port from the rest.

enable-cors

true, false

false

Answer preflights on this route and add the CORS headers to every response. The six cors- settings below do nothing until this is on.

cors-allow-origin

an origin

*

Access-Control-Allow-Origin.

cors-allow-methods

method list

GET, PUT, POST, DELETE, PATCH, OPTIONS

Access-Control-Allow-Methods.

cors-allow-headers

header list

the ingress-nginx set

Access-Control-Allow-Headers. The default matches ingress-nginx: DNT, Keep-Alive, User-Agent, X-Requested-With, If-Modified-Since, Cache-Control, Content-Type, Range, Authorization.

cors-expose-headers

header list

none

Access-Control-Expose-Headers. Omitted entirely when unset.

cors-allow-credentials

true, false

true

Access-Control-Allow-Credentials.

cors-max-age

seconds

1728000

Access-Control-Max-Age — how long a preflight may be cached.

request-headers

Name: value per line

none

Set on the request to the backend, replacing any header of the same name.

request-headers-remove

names, comma or line separated

none

Stripped from the request before it reaches the backend.

response-headers

Name: value per line

none

Set on the response to the client. A route that sets a CORS header itself this way keeps its own value.

response-headers-remove

names, comma or line separated

none

Stripped from the response before it reaches the client.

Examples

Basic auth, end to end. The Secret is an ordinary htpasswd file — the same shape ingress-nginx expects, so an existing one can be moved across unchanged. The Secret is read from the Ingress's own namespace.

shell

htpasswd -nbB alice 'correct horse battery staple' > auth
htpasswd -nbB bob 'hunter2' >> auth
kubectl -n shop create secret generic staging-users --from-file=auth

auth.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/auth-type: basic
    sslb3.ingress.hacking.hu/auth-secret: staging-users
    sslb3.ingress.hacking.hu/auth-realm: "Staging"
    # By default the header is consumed here. Set this if the backend
    # wants to see it too -- to know which user it is serving, say.
    sslb3.ingress.hacking.hu/auth-keep-authorization: "true"

Both halves are required. auth-secret on its own is logged and applied to nothing, rather than leaving a route that was meant to be protected quietly open — that is the worst outcome available here, so it is the one I made impossible.

A Secret in another namespace, auth-secret: infra/shared-users, is refused unless the chart sets controller.allowCrossNamespaceSecrets (SSLB_ALLOW_CROSS_NAMESPACE_SECRETS). The Ingress is then not served, and a NotServed Event on it names the setting. The controller reads Secrets cluster-wide and the balancer checks logins against whatever it read, so allowing the reference would let anyone who can create an Ingress point a route of their own at another team's Secret and test guesses at its values. Turn it on only where everyone who can create an Ingress may already read every Secret.

An internal-only service, admitting the private ranges and cutting out one host inside them. Note the specificity rule doing the work: the deny is narrower than the allow, so it wins without needing to be ordered first.

restricted.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/allowlist-source-range: "10.0.0.0/8,192.168.0.0/16"
    sslb3.ingress.hacking.hu/denylist-source-range: "10.1.2.3"

A browser API with CORS. Turning it on is the one annotation that matters; the six cors- settings only narrow what the headers say, and each has a working default.

cors.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/enable-cors: "true"
    sslb3.ingress.hacking.hu/cors-allow-origin: "https://app.example.com"
    sslb3.ingress.hacking.hu/cors-allow-methods: "GET, POST, OPTIONS"
    sslb3.ingress.hacking.hu/cors-expose-headers: "X-Request-Id"
    sslb3.ingress.hacking.hu/cors-max-age: "86400"

Leave the rest out and you get * for the origin, the ingress-nginx method and header lists, credentials allowed, and a twenty-day preflight cache.

A backend that routes on the Host header and does not know the public name, plus a couple of headers tidied on the way past:

vhost.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/serverfarm-vhost: shop.internal
    sslb3.ingress.hacking.hu/request-headers: |
      X-Forwarded-Host: shop.example.com
      X-Environment: production
    sslb3.ingress.hacking.hu/response-headers: |
      Strict-Transport-Security: max-age=31536000; includeSubDomains
    sslb3.ingress.hacking.hu/response-headers-remove: Server, X-Powered-By

Narrowing the HTTP versions a route answers. Rarely needed, but a client library that negotiates HTTP/2 badly is a real thing and this is cheaper than fixing it everywhere:

protocols.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/allowed-protocols: "http1,http2"
    # Only when this one Ingress is reached on a different HTTP/3 port from
    # the rest. Unset, it advertises the balancer's own HTTP/3 port.
    sslb3.ingress.hacking.hu/http3-advertise-port: "8443"

And redirect plus per-route logging, for a public site where you want a record of what was served without turning access logging on for everything:

public.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/force-ssl-redirect: "true"
    sslb3.ingress.hacking.hu/enable-access-log: "true"

How values are read

Booleans

Anywhere a table says true or false, I also accept yes/no, on/off and 1/0. That is partly convenience and partly so a manifest carried over from ingress-nginx, which writes on and off, does not need editing to keep working.

Header lists

The four header annotations take one Name: value per line. The -remove pair takes bare header names, one per line or comma separated.

headers.yaml

metadata:
  annotations:
    sslb3.ingress.hacking.hu/request-headers: |
      X-Forwarded-Host: shop.example.com
      X-Environment: production
    sslb3.ingress.hacking.hu/response-headers: |
      Strict-Transport-Security: max-age=31536000
      X-Frame-Options: DENY
    sslb3.ingress.hacking.hu/response-headers-remove: Server, X-Powered-By

Names with two spellings

The source range annotations each answer to two names. allowlist-source-range and whitelist-source-range are the same setting, as are denylist-source-range and blocklist-source-range. The second spelling of each is what ingress-nginx calls it, so a manifest moved across keeps working; the first is what I would write today.

Allow and deny are resolved together by specificity, not by list order: a rule about one host beats a rule about its network whichever list it came from, and an exact tie goes to deny. On a layer 4 service the same rules travel into the kernel as nftables rules with that ordering preserved — see the fast path.

One Ingress with all of it

The pieces above in a single object, so the shape is clear. This is more annotation than any real service needs — it is a reference, not a recommendation. A production Ingress usually carries two or three of these, and plenty carry none.

everything.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop
  namespace: shop
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt

    # --- serverfarm: shared by every Ingress pointing at this service ---
    sslb3.ingress.hacking.hu/load-balance: least-connections
    sslb3.ingress.hacking.hu/connect-timeout: "3"
    sslb3.ingress.hacking.hu/idle-timeout: "60"
    sslb3.ingress.hacking.hu/max-connection-duration: "3600"
    sslb3.ingress.hacking.hu/threat-detection: block

    # --- health check: per service, stamped onto every backend ---
    sslb3.ingress.hacking.hu/health-check: "true"
    sslb3.ingress.hacking.hu/health-check-type: http
    sslb3.ingress.hacking.hu/health-check-path: /healthz
    sslb3.ingress.hacking.hu/health-check-status: "200-299"

    # --- route: this Ingress only ---
    sslb3.ingress.hacking.hu/force-ssl-redirect: "true"
    sslb3.ingress.hacking.hu/enable-access-log: "true"
    sslb3.ingress.hacking.hu/allowlist-source-range: "10.0.0.0/8"
    sslb3.ingress.hacking.hu/serverfarm-vhost: shop.internal
    sslb3.ingress.hacking.hu/request-headers: |
      X-Forwarded-Host: shop.example.com
    sslb3.ingress.hacking.hu/response-headers-remove: Server
spec:
  ingressClassName: sslb3
  tls:
    - hosts: [shop.example.com]
      secretName: shop-tls
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: shop
                port: { number: 80 }

Deliberately absent

Three things people look for here and do not find, with the reason in each case. These are decisions, not gaps.

Not hereWhy

proxy-body-size

No body size limit exists. nginx has one because it buffers request bodies before forwarding, so the limit protects nginx itself. SSLB3 splices, so there is no buffer to protect. It could be added as policy rather than safety, but it would force the script to stay in the data path for the whole request instead of splicing, which costs 15–25% of throughput.

Rate limiting

A firewall or traffic shaper's job. What SSLB3 does have is intrusion detection, which judges behaviour rather than counting requests — threat-detection above turns it on per service.

Anything process-wide

Maximum connections, worker threads, shutdown timeout and the TLS handshake timeout are properties of the load balancer, not of one Ingress. An annotation for them would mean whichever Ingress reconciled last silently decided for everybody. They live in the configuration file and in the Helm values.

Layer 4 services are not annotated at all — they are a resource of their own, with the equivalent controls as schema fields. See the L4Ingress reference.