Overview
Both industry and academia are adopting the Node.js* runtime at an exponential rate. This is partly because Node.js is both lightweight and very efficient at handling massive numbers of simultaneous connections. Node.js is also highly scalable — which makes it extremely useful for network applications. Also, the Node.js ecosystem of open source libraries is one of the largest in the world1 and is continuously being improved. However, especially for server-side Node.js applications, network communication can still be a critical bottleneck.
I/O processing in Node.js applications
Node.js uses an asynchronous, event-driven, non-blocking I/O model to translate TCP/IP messages from kernel space into user space. For network communications, Node.js applications listen to requests that are sent on the Internet in messages. The faster these messages are received and processed from the Internet, the earlier a server application can respond.
Remember that in Node.js, I/O processing uses TCP/IP stack processing in the kernel. According to the Node.js Foundation, almost no function in Node.js directly performs I/O. Because of this, you can usually minimize communication bottlenecks by optimizing the Node.js runtime to increase I/O throughout.
Such optimizations are particularly important when you want your Node.js application to take advantage of user space via the Data Plane Development Kit (DPDK).2 Basically, the DPDK helps Node.js maximize the benefits of the user-space network stack. The DPDK is a Linux Foundation* project, to which Intel is a strong contributor. The kit is available as a free download from dpdk.org.
In this article, we’ll show you how to set up and run a Node.js application on top of the Linux Kernel Library (LKL)3 and the DPDK, in order to get better I/O throughput. For this experiment, we’ll use Security Compass Let’s Chat* as our sample application. In our solution, we’ll implement the binding between Node.js and the DPDK using the LKL.
What exactly does LKL do for a Node.js application?
For applications like Let’s Chat, LKL allows translation of computations from kernel space to user space. It does this by Intercepting TCP/IP messages via the LD_PRELOAD mechanism. Messages intercepted by LKL are redirected to a hijack library, which is a sub-module of LKL. It’s that hijack library which is used to bind LKL with the Node.js runtime.
As for DPDK, LKL interacts directly with the DPDK. In this article, we’ll describe the components of our solution, and explain the interactions between the Node.js application, LKL, and DPDK. We’ll also take a cursory look at performance gains to see how effective our solution could be.
Benefits of Using the LKL
What does all of this mean for you? Basically, with LKL, you can translate TCP/IP processing from kernel space to user space without necessarily having to modify the target application. LKL also gives you the option of using DPDK support, which provides fast TCP/IP stack processing. Both of these capabilities help minimize bottlenecks in network communication.
There are a couple other advantages of using LKL. First, the LKL code is very clean. Second, the LKL code closely follows the coding guidelines of the Linux kernel. Obviously, these benefits help with keeping your application code readable and easier to maintain.
Solution Components and Interactions
Let's look at the key software components on which we built this solution.
Figure 1. Main components of a typical Node.js* (Node.js) application running on Linux Foundation Linux*. The left half of the figure shows how Node.js applications are run directly on top of Linux. The right half shows our enhanced solution with the new, enhanced Linux kernel library (LKL) and the Data Plane Development Kit (DPDK). The LKL was written by Octavian Purdila, Ph.D.2,3
Node.js is a JavaScript runtime. Its non-blocking model is implemented by a component called libUV, which is a multi-threaded library. We mention libUV here, because we do have to modify a couple of files in libUV in order for our solution to work.
libUV is a multi-platform support library with a focus on asynchronous I/O. When used in the Node.js runtime, it is driven by four working threads. For our solution, because of the NICs we used in our test setup, we had to modify two of the Node.js libUV files.4 The changes let us experiment with using LKL with and without DPDK. We submitted the mods as pull requests to the Node.js community’s public repository on github. One set of mods has been accepted (pull request 366), and the other is a work in progress (pull request 369). These mods are described in the configuration section of the code walk-through in this blog. Basically, the mods change the way system calls are triggered, in order to make sure that TCP/IP messages are actually intercepted by LKL.
Let's Chat is a self-hosted chat application for small teams. A JavaScript server app, Let’s Chat runs on top of Node.js and MongoDB*, and lets users open chat rooms and communicate via messages. Access to rooms is password-controlled, and messages are encrypted. Messages and rooms are stored in a persistent database implemented via MongoDB.
MongoDB is a non-SQL (structured query language) database that tracks chats in different chat rooms, and stores that data in a persistent database. It’s important to note that the default settings for the database's host and port are not appropriate for our solution, and must be changed during setup. We show these steps in the solution walk-through.
DPDK is a set of libraries and drivers for fast packet processing. Specifically, DPDK deals with packet processing in user space, using the CPU.2
LKL is a forked version of the Linux kernel that allows you to run operations from the kernel in user space.3 Our solution uses a sub-component of the LKL: the hijack library. This hijack library redirects the system calls of an application that is running. Using this type of interception implies that you are also using LD_PRELOAD mechanisms to redirect messages towards the hijack library, which is actually a shared object.
Hijack Library. The LKL hijack library allows you to transfer the processing of TCP/IP messages from kernel space to user space. Remember that, while a process is running, system calls are generated for network communications. So, using our Let’s Chat example, Let's Chat is an application that runs on top of Node.js, and Node.js employs libUV. The libUV library then generates the system calls. The LKL hijack library intercepts those calls. From this point, the TCP/IP processing passes through user space. Note that you can speed up this processing by calling DPDK functions.
Theory versus reality
LKL was developed for several reasons. One is that developers were translating kernel TCP/IP computations from kernel to user space by copying pieces of software out of the kernel in their own applications. This is problematic especially in terms of maintainability. Also, the developers who can do this successfully must usually be highly skilled.
The advantage of LKL is that it lets you translate TCP/IP processing from kernel space to user space without needing to know much about the target application. In other words, applications running on top of LKL do not need to be modified — theoretically!
Theoretically, you don’t need to modify any application that runs on top of LKL. Theoretically, you only need to pre-load the hijack library by setting LD_PRELOAD.
In reality, you will have to modify some applications. A simple example of this is modifying an application to enable ping. Enabling ping can significantly increase the number of applications that can benefit from LKL. You can then decide which systems calls to intercept, in order to tailor Node.js more specifically for your use case. For example, in our walk-through, we are interested in system calls related to communication.
A Short Analysis of Performance
When it comes to TCP/IP message processing, even small gains can have a significant impact on performance. To get an idea of how effective our solution might be, we performed a short analysis5 of the performance of Let’s Chat in monolithic mode. Our metric is requests processed per second. After establishing a baseline for our application, we ran Let’s Chat with LKL. That showed a significant 5 percent improvement in performance over the baseline. We then ran Let’s Chat with both LKL and DPDK, which demonstrated an incremental 2 percent improvement.
Admittedly, this was a very quick analysis — a minimal number of experimental runs — just to get an idea of the gains we might see if we optimized for LKL and/or DPDK. We would need to perform comprehensive tests with a variety of applications to get a more accurate idea of the potential gains of this optimization. Also, remember that we used inexpensive NICs, and did not fully optimize our software in this preliminary study. If we used better NICs, optimized our test parameters, and improved the software, we suspect that performance gains could be much more significant.
Our test configuration, setup, and libUV modifications are described in the walk-through.
Conclusion
Intel often works with software developer communities to help optimize applications. Here, our goal is to minimize I/O bottlenecks so that Node.js applications see better performance.
What we’ve shown here is just one way to enable your Node.js applications (such as Let’s Chat) to run on top of LKL and DPDK. A key aspect of our solution is optimizing to bring TCP/IP stack processing more efficiently from the kernel into user space. Basically, we use LKL to enable application-agnostic translation of TCP/IP messages into user space, since translated messages can be processed faster.
Although we performed only rudimentary testing of our solution, the results show solid and/or incremental performance gains. We suspect our model, when optimized, could significantly help you minimize communication bottlenecks, especially on the server side. Cluster operations for similar solutions might be another area to explore for future optimizations.
For now, check out the code walk-through of our solution.
Code Walk-through
What we’ll show you here is a complete example of setting up a Node.js application to run on top of LKL and DPDK.
Note that we do use absolute paths in this walk-through — and we recommend that you use absolute paths to duplicate this specific solution. Obviously, however, you will need to change the paths for your own use cases and applications.
Test configuration
Our test configuration includes software, hardware, low-end NICs, and the setup of Mongo and Let’s Chat. We also had to make changes to two of the Node.js libUV files; those changes are described in the specific sections for each type of Let’s Chat test run: LKL alone or LKL and DPDK.
If you’d like to measure the performance of the Let's Chat server we recommend writing an http request generator similar to runspec.py in https://github.com/Node-DC/Node-DC-EIS/tree/master/Node-DC-EIS-client. The http request generator we used was written by Uttam Pawar in Python Software Foundation Python*.
Software elements
- Red Hat Fedora* 23: 4.2.3-300.fc23.x86_64
- Node.js v7.4.0
- A modified version of LKL with commit number c5969210b701c84abb513dcd5153a47420fde79a
- DPDK version 17.02
Hardware components
- Intel® Core™ i7-3960X processor 3.30GHz; with 12 logical cores on 6 physical cores
NIC configuration
Network interface card (NIC) configuration:
- 00:19.0: Intel® 82579V Gigabit Ethernet Controller (rev 06)
- 02:00.0: Intel® I350 Gigabit Ethernet Controller (rev 01)
- 02:00.1: Intel® I350 Gigabit Ethernet Controller (rev 01)
- 02:00.2: Intel® I350 Gigabit Ethernet Controller (rev 01)
- 02:00.3: Intel® I350 Gigabit Ethernet Controller (rev 01)
- 03:00.0: Intel® 82574L Gigabit Ethernet Controller
In our setup, we connected the 00:19.0 Ethernet controller to a local area network (LAN) socket.
Also note that the 02:00.0, 02:00.1, 02:00.2, and 02:00.3 LANs rely on the same physical NIC. The 03:00.0 is a separate NIC.
We used relatively low-end NICs in our setup. Because this is a demo, we wanted to provide a walk-through that was cost-effective, easy to set up, and as easy as possible to run and repeat. Using these NICs did require that we modify the libUV files mentioned earlier. We’ll explain those mods in the setup section for each test run.
Cables and connectivity
We used an inverted internet cable to connect 02:00.0 and 03:00.0.
Setting up MongoDB
- In the file /etc/mongod.conf (or in the file responsible for defining variables for MongoDB), set:
bind_ip = 192.168.209.1 - When calling
sudo mongod --fork --config /etc/mongod.conf
set Internet protocol (IP) 192.168.209.1 as appropriate, depending on whether you are using LKL by itself, or LKL along with DPDK. Otherwise the MongoDB process will not start. If MongoDB fails to start, it will fail quietly and could take some time to troubleshoot; so perform this setup carefully.
You can use the sudo ethtool command to check the settings available for your own logical interfaces.
Setting up Let's Chat
For this walk-through, here’s how we set up Let’s Chat.;
In the file lets-chat/defaults.yml, set:
http: ... host: 192.168.209.39 ... https: ... host: 192.168.209.39 ... database: uri: mongodb://192.168.209.1/letschat
Walk-through of Let’s Chat with LKL
Here, we’ll show you the specific mods we had to make to libUV in order to run Let’s Chat with LKL. We’ll then show you how to set up Let’s Chat and run the application with LKL.
Modifications to the Node.js libUV library
In order to use our application with LKL, we had to make several changes to the libUV file tools/lkl/lib/hijack/hijack.c. We needed these particular mods because without them, the functions were not being intercepted by LKL via hijacking. These particular changes were adopted by the Node.js community in pull request 363.4
For convenience, here are the approved changes (in git diff format):
+HOOK_CALL_USE_HOST_BEFORE_START(accept4);
+int accept4(int fd, struct sockaddr *addr, socklen_t *addrlen, int flags)
+{
+ return lkl_call(__lkl__NR_accept4, 4, fd, addr, addrlen, flags);
+}
+
+
+HOOK_CALL_USE_HOST_BEFORE_START(pipe2);
+int pipe2(int pipefd[2], int flags)
+{
+ return lkl_call(__lkl__NR_pipe2, 2, pipefd, flags);
+}
...
+int epoll_create(int flags)
+{
+ int res;
+
+ if (!lkl_running)
+ res = host_epoll_create(flags);
+ else
+ res = lkl_call(__lkl__NR_epoll_create, 1, flags);
+
+ return res;
+}
+
+HOOK_CALL_USE_HOST_BEFORE_START(epoll_create1);
+int epoll_create1(int flags)
+{
+ return lkl_call(__lkl__NR_epoll_create1, 1, flags);
+}
+
Run Let’s Chat with LKL
With the approved mods in place, we’re ready to set up the interfaces and try Let’s Chat with LKL.
- For the interfaces, we set the tap logical interfaces with 192.168.209.1 this way:
sudo ip tuntap del dev tap0 mode tap sudo ip tuntap add dev tap0 mode tap user $USER sudo ip link set dev tap0 up sudo ip addr add dev tap0 192.168.209.1/24 ip link show ip addr show
The important point to remember here is making sure to set the tap0 interface for communications.
- Make sure you’ve executed the previous step so that tap0 is set. Only then can you successfully start MongoDB. So now... start MongoDB.
- Now check that LKL is actually intercepting TCP/IP messages. Ping is a good check for this, so for our solution, we enabled ping and used it to make sure LKL was working. TCP/IP message interception is verified if you get a successful ping. Use this command to verify interception:
LKL_HIJACK_NET_IFTYPE=tap LKL_HIJACK_NET_IFPARAMS=tap0 LKL_HIJACK_NET_IP=192.168.209.39 LKL_HIJACK_NET_NETMASK_LEN=24 LKL_HIJACK_NET_GATEWAY=192.168.209.1 ./bin/lkl-hijack.sh ping 192.168.209.1
- Next, let's try our sample application with LKL:
LKL_HIJACK_NET_IFTYPE=tap LKL_HIJACK_NET_IFPARAMS=tap0 LKL_HIJACK_NET_IP=192.168.209.39 LKL_HIJACK_NET_NETMASK_LEN=24 LKL_HIJACK_NET_GATEWAY=192.168.209.1 ./bin/lkl-hijack.sh /home/octavian/Programs/Node.jss/Node.js_system_calls/Node.js/install/usr/local/bin/Node.js /home/octavian/Octavian/LetsChat/2017/06June/28/ssg_dcst_rt-Node.jsjs-letschat_system_calls/lets-chat/app.js
If the command is successful, the Let’s Chat server will initialize, and Let’s Chat will be ready to process requests. You should see the Let’s Chat image displayed:
Debugging
For debugging, use LKL_HIJACK_DEBUG=0x400 as a prefix to the command.
Walk-through of Let’s Chat with LKL and DPDK
Now that we’ve tried our application with LKL, let’s try it with both LKL and DPDK. First we’ll explain the mods we had to make to libUV in order to run Let’s Chat with LKL and DPDK. Then we’ll explain how to set up and execute the application with LKL and DPDK coupled.
Github procedure
Github has posted a short procedure for coupling LKL with DPDK. The procedure is relatively clear, but we hope it will be updated with more detail. We hope to work with the Node.js community to help with this, especially considering the results of our experiment.
Modifications to the Node.js libUV library
As with the previous test run, running the application with both LKL and DPDK required some changes to libUV. We submitted the mods as pull requests, and they are currently a work in progress (pull request 369).4
The mods we made to the file virtio_net_dpdk.c were necessary because of the NICs we used in our configuration. Other values might be needed for other NICs.
Here are the essential changes (in git diff format) that we’re proposing for the file virtio_net_dpdk.c:
... #define MEMPOOL_CACHE_SZ 0 /* for TSO pkt */ -#define MAX_PACKET_SZ (65535 \ +//For ixgbe and vmxnet3 drivers +/*#define MAX_PACKET_SZ (65535 \*/ +/* - (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM))*/ +//For Intel i350 NIC (igb driver) +#define MAX_PACKET_SZ (16383 \ - (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)) -#define MBUF_NUM (512*2) /* vmxnet3 requires 1024 */ +//#define MBUF_NUM (512*2) /* vmxnet3 requires 1024 */ +#define MBUF_NUM 4096 /* Intel i350 NIC (igb driver): 4096 */
Set up and run Let’s Chat with both LKL and DPDK
With the proposed mods, we’re ready to set up the interfaces and try Let’s Chat with LKL and DPDK coupled.
- When binding the NICs, we used this setup:
Network devices using DPDK-compatible driver ============================================ 0000:02:00.0 'I350 Gigabit Network Connection' drv=igb_uio unused=igb,vfio-pci,uio_pci_genericNetwork devices using kernel driver =================================== 0000:00:19.0 '82579V Gigabit Network Connection' if=eno1 drv=e1000e unused=igb_uio,vfio-pci,uio_pci_generic *Active* 0000:02:00.1 'I350 Gigabit Network Connection' if=enp2s0f1 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic 0000:02:00.2 'I350 Gigabit Network Connection' if=enp2s0f2 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic 0000:02:00.3 'I350 Gigabit Network Connection' if=enp2s0f3 drv=igb unused=igb_uio,vfio-pci,uio_pci_generic 0000:03:00.0 '82574L Gigabit Network Connection' if=enp3s0 drv=e1000e unused=igb_uio,vfio-pci,uio_pci_generic
- We then set up the interfaces:
ifconfig enp3s0 192.168.209.1 netmask 255.255.255.0 up
Note that you can install ifconfig on Fedora. We use it here for convenience.
- At this point, you should once again make sure that LKL is intercepting TCP/IP messages. Here, we again use Ping to make sure LKL is working. TCP/IP message interception is verified if you get a successful ping.
LKL_HIJACK_NET_IFTYPE=dpdk LKL_HIJACK_NET_IFPARAMS=dpdk0 LKL_HIJACK_NET_IP=192.168.209.39 LKL_HIJACK_NET_NETMASK_LEN=24 LKL_HIJACK_NET_GATEWAY=192.168.209.1 ./bin/lkl-hijack.sh ping 192.168.209.1
- Now we’re ready to try Let’s Chat with LKL and DPDK:
LKL_HIJACK_NET_IFTYPE=dpdk LKL_HIJACK_NET_IFPARAMS=dpdk0 LKL_HIJACK_NET_IP=192.168.209.39 LKL_HIJACK_NET_NETMASK_LEN=24 LKL_HIJACK_NET_GATEWAY=192.168.209.1 ./bin/lkl-hijack.sh /home/octavian/Programs/Node.jss/Node.js_system_calls/Node.js/install/usr/local/bin/Node.js /home/octavian/Octavian/LetsChat/2017/06June/28/ssg_dcst_rt-Node.jsjs-letschat_system_calls/lets-chat/app.js
If the command is successful, the Let’s Chat server will initialize, and the application will be ready to process requests. You should see the Let’s Chat image displayed:
Debugging
Again, if you’re debugging, use LKL_HIJACK_DEBUG=0x400 as a prefix to the LKL command.
For more information
You’ve now seen two examples of using a Node.js application with LKL to minimize I/O bottlenecks. For more information about LKL, DPDK, and our proposed mods, visit github. You might also want to check out Dr. Octavian Purdila’s excellent paper on LKL.3
For more information about DPDK, please visit /content/www/cn/zh/develop/networking/dpdk.html.
To download the DPDK, visit DPDK.org.
Acknowledgments
I’d like to thank Dr. Monica Ene-Pietrosanu in particular. I also appreciate the contributions of Uttam C. Pawar, Dr. Hajime Tazaki, Dr. Octavian Purdila, and Dr. Dong-Yuan Chen. Finally, my thanks to an anonymous friend who helped with the article and provided valuable advice.
References
1. Source: Node.js Foundation, https://Node.jsjs.org/en/
2. LKL was designed by Octavian Purdila, Ph. D. Hajime Tazaki, Ph. D., then connected LKL with DPDK. For information about using LKL and DPDK together, refer to Dr. Tazaki’s DPDK Howto page on github: https://github.com/libos-nuse/lkl-linux/wiki/DPDK-Howto-(tmp).
3. LKL was created by Octavian Purdila, Ph. D; it was his dissertation thesis. LKL: The Linux Kernel Library, Octavian Purdila, Lucian Adrian Grijincu, Nicolae Tapus, The 9th RoEduNet IEEE International Conference, pp. 328-333, June 24-26, 2010.
4. When initiating pull requests, please report requested changes first to https://github.com/libos-nuse/lkl-linux/tree/fix-dpdk-virtio, which is the section of the Node.js community that handles issues regarding communications with NICs.
5. The performance analysis was conducted using Red Hat Fedora* 23: 4.2.3-300.fc23.x86_64; Joyent Node.js* version v7.4.0; a modified version of the Linux Foundation Linux* Kernel Library (LKL) with commit number c5969210b701c84abb513dcd5153a47420fde79a; Data Plane Development Kit (DPDK) version 17.02; and an Intel® Core™ i7 processor 3960X CPU @ 3.30GHz with 12 logical cores on 6 physical cores. The testing was done by Octavian Soldea, Intel.
Notices
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as Business Applications Performance Corporation SYSmark* and Business Applications Performance Corporation MobileMark*, are measured using specific computer systems, components, software, operations, and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.
Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance.
"