Optimizing SocketCluster for High Performance and Scalability
Introduction
SocketCluster is a powerful tool for building real-time applications, offering easy setup and horizontal scalability. While SocketCluster provides excellent features out of the box, there are several factors to consider for an optimal setup. In this article, we will explore some observations and strategies that can help you maximize the performance of your SocketCluster deployment.
Identifying Performance Issues:
When initially setting up SocketCluster on a single machine, in our use case we encountered high CPU and memory usage. Upgrading to a newer version, such as v17, may seem like a straightforward solution, thanks to its simpler code structure utilizing async/await. However, our experience revealed that there were challenges to overcome.
Limitations with TCP Connections:
One limitation we discovered was the maximum number of TCP connections a single machine can handle, which hindered using Nginx as a load balancer. For example, an EC2 instance typically has a limit of 64k TCP connections, to reach that you will need to change the number of open ports on the instance which by default is not at its maximum. If your goal is to fully utilize a machine, using Nginx as a load balancer becomes impractical due to this threshold of the tcp connections (as, if you have a central machine which routes requests through nginx that means limiting to the number of open ports). To work around this, we employed a routing strategy on AWS to distribute the load between workers.
Scaling with Clustering:
Despite these changes, we still faced difficulties establishing more than 25k connections, experiencing high load and event loop saturation during load testing. To address this, we implemented clustering by spinning up multiple processes on our 2-core system, assigning workers in Node.js to divide the load. Although this approach showed improvements, we were not entirely satisfied with the performance.
The Impact of Async/Await:
Upon deployment to production, we encountered event loop saturation once again, despite having around 2k connections and 5k messages published per second. Our suspicion fell on the use of async/await. As a result, we revisited older versions of SocketCluster and settled on v14.2, which relied on callback-based approaches. Despite the code becoming slightly more complex compared to v17, we couldn’t ignore the significant performance gains achieved through callbacks.
Choosing the Right Version:
To achieve maximum performance with minimum resource usage, we recommend using the callback approach in SocketCluster, especially when dealing with heavy load or frequent connections/disconnections. If you opt for newer versions, implementing clustering by spinning up multiple worker processes is crucial for better results. Newer versions excel in scenarios with less system load or fewer connection/disconnection events, whereas callbacks handle high reconnection rates more effectively.
In both the server setup we made changes to the npm package where ping pong mechanism was handled as we wanted it to send both #1 and empty data because our client libraries were not having a parity and were not sending a protocol version.
Achieving Scalability:
With our current setup, we can establish 5,000 parallel connections at once and accommodate around 40,000 connections on each instance within 10 minutes. It’s worth noting that wss (secure socket) connections are more CPU-intensive, limiting us to only 1,500 simultaneous connections. Make sure you do all the required changes on the machine and tune it in such a way that it allows maximum connections.
Conclusion:
SocketCluster offers remarkable features, but optimizing its performance requires careful consideration of factors like TCP connection limits, event loop saturation, and the choice between async/await and callbacks. By implementing clustering and selecting the appropriate version based on your specific use case, you can achieve an optimal SocketCluster setup for high performance and scalability. By incorporating these insights into your SocketCluster deployment, you can unlock the full potential of this technology and ensure a smooth and efficient real-time application experience.
Please note that the observations and recommendations mentioned throughout this article are based on our personal experiences. Every deployment and application may have unique requirements and considerations.
