Apache tomcat websocket download balance

Question

I am currently developing a websocket application that deploys on a Tomcat server. Due to the sheer number of users, I would like to spread the workload across multiple instances of Tomcat. I decided to use Apache for load balancing.

Now I have a problem with implementing Apache load balancing and sticky session for webcam requests. This is my Apache configuration:

ProxyRequests off SSLProxyEngine on RewriteEngine On <Proxy balancer://http-localhost/> BalancerMember https://mcsgest1.desy.de:8443/Whiteboard/ route=jvm1 keepalive=On smax=1 connectiontimeout=10 retry=600 timeout=900 ttl=900 BalancerMember https://mcsgest1.desy.de:8444/Whiteboard/ route=jvm2 keepalive=On smax=1 connectiontimeout=10 retry=600 timeout=900 ttl=900 ProxySet lbmethod=byrequests ProxySet stickysession=JSESSIONID|sid scolonpathdelim=On </Proxy> <Proxy balancer://ws-localhost/> BalancerMember wss://mcsgest1.desy.de:8443/Whiteboard/ route=jvm1 keepalive=On smax=1 connectiontimeout=10 retry=600 timeout=900 ttl=900 BalancerMember wss://mcsgest1.desy.de:8444/Whiteboard/ route=jvm2 keepalive=On smax=1 connectiontimeout=10 retry=600 timeout=900 ttl=900 ProxySet lbmethod=byrequests ProxySet stickysession=JSESSIONID|sid scolonpathdelim=On </Proxy> RewriteCond %{HTTP:Upgrade} =websocket RewriteRule /jddd/(.*) balancer://ws-localhost/$1 [P,L] ProxyPassReverse /jddd/ balancer://ws-localhost/ RewriteCond %{HTTP:Upgrade} !=websocket RewriteRule /jddd/(.*) balancer://http-localhost/$1 [P,L] ProxyPassReverse /jddd/ balancer://http-localhost/ 

The first https request is balanced for port 8443. The updated wss request is also redirected to 8443.

The second https request contains the session identifier of the first request: https: //...&sid=C28C13EEEC525D203F8CA4E827605E0B.jvm1

As I can see in the Apache log file, this sessionID is evaluated for stickySession:

... Found value C28C13EEEC525D203F8CA4E827605E0B.jvm1 for stickysession s.i.d.

... jvm1 route found

... balancing: // http-localhost: worker (htttps: //mcsgest1.desy.de: 8443 / Whiteboard /), rewritten to htttps: //mcsgest1.desy.de: 8443 / whiteboard // File = octocenter.xml & address = /// & s.i.d. = C28C13EEEC525D203F8CA4E827605E0B.jvm1

The second https request is still on port 8443, but after upgrading to the websocket protocol, ws-balancer does not evaluate the session identifier and overwrites the code 8444:

... balancing: // ws-localhost: worker (wss: //mcsgest1.desy.de: 8444 / Whiteboard /), rewritten to WSS: //mcsgest1.desy.de: 8444 / whiteboard // whiteboardendpoint

What do I need to change in Apache configuration to enable stickysession also for wss protocol? Do I really need two balancers (http and ws) to balance websockets?

Answer 1

You do not need a separate balancer for websokets, because the initial HTTP request already has an http cookie and belongs to the correct instance.

You just need to detect a connection update and manually request a route depending on the sticky part of the cookie

Make sure you download the proxy module for websokets - mod_proxy_wstunnel

eg

 SSLProxyEngine on RewriteEngine On <Proxy balancer://http-localhost/> BalancerMember https://mcsgest1.desy.de:8443/Whiteboard/ route=jvm1 keepalive=On smax=1 connectiontimeout=10 retry=600 timeout=900 ttl=900 BalancerMember https://mcsgest1.desy.de:8444/Whiteboard/ route=jvm2 keepalive=On smax=1 connectiontimeout=10 retry=600 timeout=900 ttl=900 ProxySet lbmethod=byrequests ProxySet stickysession=JSESSIONID|sid scolonpathdelim=On </Proxy> RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC] RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC] RewriteCond %{HTTP_COOKIE} ^.*(JSESSIONID|sid)=([^=]*)\.jvm1 [NC] RewriteRule .* wss://mcsgest1.desy.de:8443%{REQUEST_URI} [P,L] RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC] RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC] RewriteCond %{HTTP_COOKIE} ^.*(JSESSIONID|sid)=([^=]*)\.jvm2 [NC] RewriteRule .* wss://mcsgest1.desy.de:8444%{REQUEST_URI} [P,L] RewriteRule /jddd/(.*) balancer://http-localhost$1 [P,L] ProxyPreserveHost On ProxyRequests Off ProxyPass /jddd/ balancer://http-localhost ProxyPassReverse /jddd/ balancer://http-localhost 

Explanation:

 # if header upgrade = WebSocket RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC] # and header connection contains Upgrade (header may be like this: connection=keep-alive, upgrade) RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC] # and header cookie contains JSESSIONID or sid, ending with sticky part - .jvm1 in that case RewriteCond %{HTTP_COOKIE} ^.*(JSESSIONID|sid)=([^=]*)\.jvm1 [NC] #than we route request to application server via mod_proxy (P flag) and end rewrite rule check RewriteRule .* wss://mcsgest1.desy.de:8443%{REQUEST_URI} [P]