//Karthik Srinivasan

High concurrency configuration for Node.js

June 25, 2017

After a lot of testing by generating users to create persistent connections to the server, was finally able to reach 320k connections in M3.xlarge EC2 instance. At this level of live persistent connections the CPU load is constantly at 100% on all CPUs on the server due to V8's (Node.js engine) garbage collection. The settings to reach the above load:

Node.js flags

Set the following flags to launch your node.js application:

node --nouse-idle-notification--expose-gc--max-new-space-size=2048--max-old-space-size=8192 ./server/userserver.js

–nouse-idle-notification

Turns of the idle garbage collection which makes the GC constantly run and is devastating for a realtime server environment. If not turned off the system will get a long hickup for almost a second once every few seconds.

–expose-gc

Use the expose-gc command to enable manual control of the GC from your code. I recommend to call GC once every 30 seconds.

–max-old-space-size=8192

Increases the limit for each V8 node process to use max 8Gb of heap memory instead of the 1,4Gb default on 64-bit machines(512Mb on a 32-bit machine).

–max-new-space-size=2048

Specified in kb and setting this flag optimizes the V8 for a stable allround environment with short pauses and ok high peak performance.

If this flag is not used the pauses will be a little bit longer but the machine will handle peaks a little bit better. What you need in this case depends on the project you are working on. My pick is to have an allround stable server instead of just handling peaks so I stick with this flag.

EC2 configuration

Set the “soft” and “hard” nofile limit to 1000000. Instead of using the “ulimit -n” as some people do I had to specify the “soft” and “hard” limits for both root and all other users, for some reason I had to specify them separately.

/etc/security/limits.d/custom.conf

root soft nofile 1000000
root hard nofile 1000000
* soft nofile 1000000
* hard nofile 1000000

Now set the amount of possible opened file handles and the size of the NAT ip connection tracking table.

/etc/sysctl.conf

fs.file-max = 1000000
 fs.nr_open = 1000000       
 net.ipv4.netfilter.ip_conntrack_max = 1048576
 net.nf_conntrack_max = 1048576

“fs.file-max”

The maximum file handles that can be allocated

“fs.nr_open”

Max amount of file handles that can be opened

“net.ipv4.netfilter.ip_conntrack_max”

Specifies how many connections the NAT can keep track of in the “tracking” table before it starts to drop packets and just break connections, this we totally want to avoid. The default value for this is 65536 so without this setting you wont be able to get more connections than that.