Un équilibreur de charge pour standardiser l'accès aux services web hétérogènes

Publié le dimanche 28 janvier 2024 à 07:40

Le besoin principal est de mettre en place un point d'entrée unique pour différents services web dont Apache et quelques autres sur des ports exotiques.

Comme tout nouveau projet, la mise en œuvre, comme d'habitude, a nécessité plus de temps que prévu initialement.

À cet effet, j'ai réalisé la mise en paquet du service HAProxy dans la distribution linux Mageia avec une configuration de proxy transparent par défaut.

HAProxy ne prend pas en charge la disposition de chemin des certificats utilisée sur les distributions dérivées de Red Hat, où clef publique et privée sont séparées en deux fichiers distincts dans deux répertoires différents.

HAProxy souffre d'un bug critique lors de l'usage de la sortie standard pour la journalisation qui reprend en début de fichier après un redémarage.

HAProxy est pénalisé par la désactivation du tampon sur sa sortie standard lorsqu'elle est utilisée pour la journalisation.

Trois correctifs ont été réalisé par mes soins à cet effet pour résoudre ces problèmes sur le chemin de mes besoins :

prise en charge de la disposition de chemin des certificats pour distribution Red Hat
ne recommence plus au début du fichier journal après un redémarage
ne plus désactiver le tampon de la sortie standard

De nouvelles fonctionnalités sont en cours de développement pour la prochaine version d'HAProxy qui pourraient rendre obsolètes mes choix et développements sur la journalisation.

Pour rendre possible la prise en charge du protocole HTTP/3 par HAProxy, il a été nécessaire d'intégrer la librairie quictls, une branche d'OpenSSL avec support du protocole QUIC.

Pour installer le serveur :

# urpmi haproxy haproxy-quic

Activer le service :

# systemctl enable haproxy.service

Démarrer le service :

# systemctl start haproxy.service

Afin d'éviter une valse de ports lors de l'intégration dans une architecture existante, l'indirection de ports dans Shorewall est proposée :

# Redirect tcp traffic from net on port 80 to 8000
REDIRECT    net 8000    tcp 80
# Redirect tcp traffic from net on port 443 to 8000
REDIRECT    net 8000    tcp 443
# Redirect udp traffic from net on port 443 to 8443
REDIRECT    net 8443    udp 443

Cela permet à HAProxy de capturer le trafic HTTP et HTTPS sur le port 8000/TCP et QUIC sur le port 8443/UDP.

La configuration par défaut capture le trafic HTTP et HTTPS sur le frontal tcp_default, puis le bascule sur les dorsaux tcp_http et tcp_https, ces derniers enverront les flux sur les frontaux http_default et https_default, en prenant soin d'ajouter un en-tête proxy pour transmettre l'adresse IP et le port d'origine. Cette configuration permet à d'autre services TCP (SSH, VPN, etc) d'être capturés et redistribués au besoin.

Le trafic est ensuite reçu respectivement par les frontaux http_default et https_default, l'adresse IP et le port d'origine est conservée via l'en-tête proxy reçu des dorsaux TCP.

Ils peuvent annoncer la prise en charge de HTTP/3, nettoyer certains en-têtes, refuser l'accès aux robots, transférer les adresses IP et ports d'origine via un en-tête Forwarded et distribuer à différents dorsaux en fonction des hôtes et chemins demandés.

Configurer Apache pour recevoir l'en-tête proxy est problématique, il est plus simple de transmettre l'en-tête Forwarded que de configurer le fraîchement publié mod_remoteip.

La prise en charge du protocole HTTP/3 est annoncée par l'envoi d'un en-tête alt-svc: h3=":"; ma= où host peut être vide, port=443 et timeout=3600 par exemple. Le certificat servi doit correspondre au nom d'hôte initial et non à celui du serveur de déchargement pour des raisons de sécurité.

Configurer les différents backends http se résumera à déclarer les serveurs de déchargement, les en-têtes supplémentaires à ajouter, la compression selon les types de contenus et la méthode pour tester leur santé.

Sans plus attendre la configuration à adapter à vos besoins :

# HAProxy configuration file

# Global config
global
    # Log to systemd
    log stdout format short daemon
    # Number of threads
    nbthread 8
    # Pid file
    pidfile /run/haproxy/haproxy.pid

    # Stat socket
    stats socket /run/haproxy/haproxy.sock mode 0660 level admin
    # Stat timeout
    stats timeout 10s
    # Max connections
    stats maxconn 10

    # Certificate base dir
    crt-base /etc/pki/tls/certs
    # Private key base dir
    key-base /etc/pki/tls/private
    # Don't load extra files
    ssl-load-extra-files none
    # Do not verify certificate
    ssl-server-verify none
    # Supported bind ciphersuites
    #XXX: https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended-configurations
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    # Disable SSL-v3 TLSv1.0 TLSv1.1 and TLSv1.2 without TLS tickets
    ssl-default-bind-options ssl-min-ver TLSv1.3

    # SSL/TLS session cache size
    tune.ssl.cachesize 20000
    # SSL/TLS session life time in cache
    tune.ssl.lifetime 300
    # SSL/TLS layer maximum passed bytes at a time
    tune.ssl.maxrecord 0
    # Diffie-Hellman ephemeral keys max size
    tune.ssl.default-dh-param 2048
    # Buffer size
    tune.bufsize 16384
    # Reserved buffer size
    tune.maxrewrite 1024
    # Max number of headers
    tune.http.maxhdr 101

# Default config
defaults
    # Use global log
    log global

    # Set balance mode
    balance random
    # Set http mode
    mode http
    # Set http keep alive mode
    #XXX: https://cbonte.github.io/haproxy-dconv/2.3/configuration.html#4
    option http-keep-alive
    # Dont log empty line
    option dontlognull
    # Dissociate client from dead server
    option redispatch

    # Number of retries on connection failure
    retries 3
    # Max concurrent connections
    maxconn 30000
    # Max pending connections
    backlog 30000

    # Max time for a connection attempt
    timeout connect 5s
    # Max inactivitiy time on client side
    timeout client 60s
    # Max inactivitiy time on server side
    timeout server 60s
    # Max inactivitiy time on client and server for tunnels
    timeout tunnel 3600s
    # Max time for new request
    timeout http-keep-alive 1s
    # Max time for a complete http request
    timeout http-request 15s
    # Max time for a free slot
    timeout queue 5s
    # Duration for tarpitted connections
    timeout tarpit 5s

    # Error documents
    errorfile 400 /usr/share/doc/haproxy/error/400.http
    errorfile 403 /usr/share/doc/haproxy/error/403.http
    errorfile 408 /usr/share/doc/haproxy/error/408.http
    errorfile 500 /usr/share/doc/haproxy/error/500.http
    errorfile 502 /usr/share/doc/haproxy/error/502.http
    errorfile 503 /usr/share/doc/haproxy/error/503.http
    errorfile 504 /usr/share/doc/haproxy/error/504.http

# Default tcp frontend
frontend tcp_default
    # Set tcp mode
    mode tcp
    # Bind to 8000 port
    bind :::8000
    # Log disabled
    no log
    # Set tcp log
    #option tcplog
    # Set inspect delay
    tcp-request inspect-delay 5s
    # Wait for extension detection
    #tcp-request content accept if { req.proto_http } or { req.ssl_hello_type 1 } or { req.ssl_ec_ext 1 }
    tcp-request content accept if { req.proto_http } or { req.ssl_hello_type 1 }

    # Send to https tcp backend
    use_backend tcp_https if { req.ssl_hello_type 1 }

    # Send to ec tcp backend
    #use_backend tcp_ec if { req.ssl_ec_ext 1 }

    # Send to OpenVPN backend
    #acl openvpn payload(0,2) -m bin 003c
    #tcp-request content accept if openvpn
    #use_backend tcp_openvpn if openvpn

    # Send to OpenSSH backend
    #XXX: https://jonnyzzz.com/blog/2017/05/24/ssh-haproxy/
    #XXX: https://issues.apache.org/jira/browse/SSHD-656
    #acl ssh payload(0,7) -m str SSH-2.0
    #tcp-request content accept if ssh
    #use_backend tcp_ssh if ssh

    # Send to http tcp backend (if { req.proto_http })
    default_backend tcp_http

# Http tcp backend
backend tcp_http
    # Set tcp mode
    mode tcp
    # Send to localhost without ssl with v2 proxy header
    server haproxy 127.0.0.1:8080 no-ssl verify none send-proxy-v2

# Https tcp backend
backend tcp_https
    # Set tcp mode
    mode tcp
    # Send to localhost without ssl with v2 proxy header
    server haproxy 127.0.0.1:8443 no-ssl verify none send-proxy-v2-ssl

# Default http frontend
frontend http_default
    # Bind to 8080 port
    bind :::8080 accept-proxy
    # Insert X-Forwarded-For header
    option forwardfor
    # Set http log format
    option httplog
    # Log enabled
    log global

    # Check if acme challenge
    acl acme_challenge path_beg /.well-known/acme-challenge/

    # Add X-Backend header
    #http-response add-header X-Backend %[haproxy.backend_name]

    # Advertise QUIC
    #http-response add-header alt-svc 'h3=":443"; ma=3600'
    #http-after-response add-header alt-svc 'h3=":443"; ma=3600'

    # Remove server and x-powered-by headers
    #http-after-response del-header server
    #http-after-response del-header x-powered-by

    # Redirect to https scheme when unsecure and not acme challenge
    http-request redirect scheme https code 302 unless { ssl_fc } || acme_challenge

    # Check if denied path
    #XXX: use ,url_dec like in https://serverfault.com/questions/754752/block-specific-url-in-haproxy-url-encoding
    #acl denied_path path_reg ^/(\.env|login|admin/|wp-login\.php|\.git/config)$
    # Deny access on denied path
    #http-request deny if denied_path

    # Check if protected path
    #acl protected_path path_reg ^/(contact|register)$
    # Deny access on protected path
    #http-request deny deny_status 503 if protected_path { method 'POST' } { req.ver '1.0' }

    # Store origin variable as txn
    http-request set-var(txn.origin) req.hdr(Origin)
    # Store host variable as txn
    http-request set-var(txn.host) req.hdr(Host),field(1,:),lower
    # Store proto variable as txn
    http-request set-var(txn.proto) ssl_fc,iif(https,http)

    # Set forwarded proto
    http-request set-header X-Forwarded-Proto %[var(txn.proto)]
    # Set forwarded port
    http-request set-header X-Forwarded-Port %[dst_port]
    # Set forwarded for
    #http-request set-header X-Forwarded-For %[src]
    # Set forwarded by
    http-request set-header X-Forwarded-By %[dst]

    # Set forwarded
    #http-request set-header Forwarded by=%[dst]:%[dst_port];for=%[src]:%[src_port];host=%[var(txn.host)];proto=%[var(txn.proto)]
    http-request set-header Forwarded by=%[dst]:%[dst_port];for=%[src]:%[src_port];proto=%[var(txn.proto)]

    # Check if host is cdn.example.com
    acl cdn var(txn.host) -m str cdn.example.com
    # Check if cdn css path
    acl cdn_css path_beg /css
    # Check if cdn js path
    acl cdn_js path_beg /js
    # Check if haproxy status path
    acl haproxy_status path_beg /haproxy-status
    # Check if debug path
    acl debug path_beg /debug

    # Send to css backend if path start with /css
    use_backend http_css if cdn cdn_css
    # Send to js backend if path start with /js
    use_backend http_js if cdn cdn_js
    # Send to status backend if path start with /haproxy-status
    use_backend http_status if haproxy_status
    # Send to debug backend if path start with /debug
    use_backend http_debug if debug

    # Send to https backend
    use_backend https_default if { ssl_fc }

    # Send to default backend
    default_backend http_default

# Default https frontend
#XXX: copy of upper one, just done to skip logs here
frontend https_default
    # Bind to 8443 tcp port as ssl
    bind :::8443 ssl crt haproxy.pem alpn h2,http/1.1,http/1.0 accept-proxy
    # Bind to 8443 udp port as ssl
    #bind quic6@:::8443 ssl crt haproxy.pem alpn h3
    # Insert X-Forwarded-For header
    option forwardfor
    # Set http log format
    option httplog
    # Log enabled
    log global

    # Check if acme challenge
    acl acme_challenge path_beg /.well-known/acme-challenge/

    # Add X-Backend header
    #http-response add-header X-Backend %[haproxy.backend_name]

    # Advertise QUIC
    #http-response add-header alt-svc 'h3=":443"; ma=3600'
    #http-after-response add-header alt-svc 'h3=":443"; ma=3600'

    # Remove server and x-powered-by headers
    #http-after-response del-header server
    #http-after-response del-header x-powered-by

    # Redirect to https scheme when unsecure and not acme challenge
    http-request redirect scheme https code 302 unless { ssl_fc } || acme_challenge

    # Check if denied path
    #XXX: use ,url_dec like in https://serverfault.com/questions/754752/block-specific-url-in-haproxy-url-encoding
    #acl denied_path path_reg ^/(\.env|login|admin/|wp-login\.php|\.git/config)$
    # Deny access on denied path
    #http-request deny if denied_path

    # Check if protected path
    #acl protected_path path_reg ^/(contact|register)$
    # Deny access on protected path
    #http-request deny deny_status 503 if protected_path { method 'POST' } { req.ver '1.0' }

    # Store origin variable as txn
    http-request set-var(txn.origin) req.hdr(Origin)
    # Store host variable as txn
    http-request set-var(txn.host) req.hdr(Host),field(1,:),lower
    # Store proto variable as txn
    http-request set-var(txn.proto) ssl_fc,iif(https,http)

    # Set forwarded proto
    http-request set-header X-Forwarded-Proto %[var(txn.proto)]
    # Set forwarded port
    http-request set-header X-Forwarded-Port %[dst_port]
    # Set forwarded for
    #http-request set-header X-Forwarded-For %[src]
    # Set forwarded by
    http-request set-header X-Forwarded-By %[dst]

    # Set forwarded
    #http-request set-header Forwarded by=%[dst]:%[dst_port];for=%[src]:%[src_port];host=%[var(txn.host)];proto=%[var(txn.proto)]
    http-request set-header Forwarded by=%[dst]:%[dst_port];for=%[src]:%[src_port];proto=%[var(txn.proto)]

    # Check if host is cdn.example.com
    acl cdn var(txn.host) -m str cdn.example.com
    # Check if cdn css path
    acl cdn_css path_beg /css
    # Check if cdn js path
    acl cdn_js path_beg /js
    # Check if haproxy status path
    acl haproxy_status path_beg /haproxy-status
    # Check if debug path
    acl debug path_beg /debug

    # Send to css backend if path start with /css
    use_backend http_css if cdn cdn_css
    # Send to js backend if path start with /js
    use_backend http_js if cdn cdn_js
    # Send to status backend if path start with /haproxy-status
    use_backend http_status if haproxy_status
    # Send to debug backend if path start with /debug
    use_backend http_debug if debug

    # Send to https backend
    use_backend https_default if { ssl_fc }

    # Send to default backend
    default_backend http_default

# Debug http backend
backend http_debug
    # Check if trusted
    acl trusted src 127.0.0.0/8 ::1
    # Allow access from trusted only
    http-request deny unless trusted
    # Server without ssl or check
    server debug 127.0.0.1:8090 no-ssl verify none

# Default http backend
backend http_default
    # Enable check
    option httpchk
    # User server default
    http-check connect default
    # Send HEAD on / with protocol HTTP/1.1 for host example.com
    http-check send meth HEAD uri / ver HTTP/1.1 hdr Host example.com
    # Expect return code between 200 and 399
    http-check expect status 200-399

    # Insert header X-Server: apache
    #http-response add-header X-Server apache

    # Set compression algorithm
    #compression algo gzip
    # Enable compression for html, plain and css text types
    #compression type text/html text/plain text/css

    # Server with ssl and check without certificate verification
    server apache 127.0.0.1:80 no-ssl verify none check #cookie apache

# Default https backend
backend https_default
    # Enable check
    option httpchk
    # User server default
    http-check connect default
    # Send HEAD on / with protocol HTTP/1.1 for host example.com
    http-check send meth HEAD uri / ver HTTP/1.1 hdr Host example.com
    # Expect return code between 200 and 399
    http-check expect status 200-399

    # Insert header X-Server: apache
    #http-response add-header X-Server apache

    # Force HSTS for 5 minutes on domain and all subdomains
    #http-response set-header Strict-Transport-Security max-age=300#;\ includeSubDomains#;\ preload

    # Set compression algorithm
    #compression algo gzip
    # Enable compression for html, plain and css text types
    #compression type text/html text/plain text/css

    # Server with ssl and check without certificate verification
    server apache 127.0.0.1:443 ssl verify none check #cookie apache

# Css http backend
backend http_css
    # Enable check
    option httpchk
    # User server default
    http-check connect default
    # Send GET on /css/empty.css with protocol HTTP/1.1 for host cdn.example.com
    http-check send meth GET uri /css/empty.css ver HTTP/1.1 hdr Host cdn.example.com
    # Expect return code between 200 and 399
    http-check expect status 200-399

    # Server with check without ssl and certificate verification
    server css 127.0.0.1:80 no-ssl verify none check

# Js http backend
backend http_js
    # Enable check
    option httpchk
    # User server default
    http-check connect default
    # Send HEAD on /js/missing.js with protocol HTTP/1.1 for host cdn.example.com
    http-check send meth HEAD uri /js/missing.js ver HTTP/1.1 hdr Host cdn.example.com
    # Expect return code 404
    http-check expect status 404

    # Check if txn.origin start with https://cdn.example.com
    acl cdn_origin var(txn.origin) -m beg https://cdn.example.com
    # Send origin as ACAO
    http-response set-header Access-Control-Allow-Origin %[var(txn.origin)] if cdn_origin
    # Set ACMA for one day
    http-response set-header Access-Control-Max-Age 86400 if cdn_origin

    # Server with check without ssl and certificate verification
    server js 127.0.0.1:80 no-ssl verify none check

# Status user list
userlist status
    # Add user admin
    user admin insecure-password ADMINPASSWORD
    # Add user operator
    user operator insecure-password OPERATORPASSWORD
    # Assign admin in admin group
    group admin users admin
    # Assign operator and admin in operator group
    group operator users operator,admin

# Status http backend
backend http_status
    # Add operator acl
    acl is_operator http_auth(status)
    # Add admin acl
    acl is_admin http_auth_group(status) admin
    # Check if trusted
    acl trusted src 127.0.0.0/8 ::1
    # Enable stats
    stats enable
    # Set stats hook on /haproxy-status
    stats uri /haproxy-status
    # Set refresh time
    stats refresh 10s
    # Display legends
    stats show-legends
    # Display node
    stats show-node
    # Allow access from trusted or authentified operator only
    #stats http-request auth unless trusted or is_operator
    stats http-request auth unless trusted
    # Activate admin interface from trusted or authentified admin only
    #stats admin if is_admin

Espérons que cet article vous a été utile.