Much to my surprise, Appeal's JRockit has arrived on the scene to post the best network scalability scores of any Java virtual machine I've tested. In fact, it's the only Java virtual machine I've found which is able to support 4,000 connections on Windows NT or Windows 2000 without modification.
Appeal has made Java threads as lightweight as they're advertised to be, mapping their thin threads onto native threads as required. JRockit seems to be particularly suited for pure Java servers which must handle many dedicated network connections. Like TowerJ, though, it's not free. Let's hope for a Linux port of JRockit!
For background information and download locations, please see the Java Benchmarks page on Volano's Web site. Since the server side of the VolanoMark benchmark is our VolanoChat server, these test results help our customers choose the best Java virtual machine for their environment.
VolanoMark is a pure Java server benchmark characterized by long-lasting network connections and high thread counts. In this context, long-lasting means the connections last several minutes or longer, rather than just a few seconds. The VolanoMark benchmark creates client connections in groups of 20 and measures how long it takes for the clients to take turns broadcasting their messages to the group. At the end of the test, it reports a score as the average number of messages transferred by the server per second.
Bug Id 4075058, "java.io: Add support for non-blocking I/O," explains why you need lots of threads in order to handle dedicated connections to a pure Java server. JSR 51, "New I/O APIs for the Java Platform," explains how Sun plans to fix that.
The tests presented in this report look at the performance and network scalability of 18 Java virtual machines on 7 operating systems using a common Intel hardware platform running VolanoMark™ 2.1. VolanoMark attempts to answer two questions about Java virtual machines used in network server environments -- especially those environments involving a large number of dedicated socket connections:
See the Environments section for details on the hardware platform, operating systems, and Java virtual machine environments, along with a table showing the compiler and threading model of each Java platform.
The performance test was executed with the following commands on a local loop-back connection, using the heap size options shown below where possible:
java -ms8m -mx64m COM.volano.Main
java -ms8m -mx64m COM.volano.Mark -count 100
See the COM.volano.Mark command synopsis for a complete description of all options.
The operating system was rebooted before each set of tests for a particular Java virtual machine. The server side was not restarted before each run of the client benchmark. The client benchmark ran four times, with the final score as the average of the last three results.
Scores are the throughput of the server in messages per second. Bigger numbers are faster. See the Environments section for details on the hardware platform, operating systems, and Java virtual machine environments.
Java Platform | Score |
---|---|
Tower TowerJ 3.1.4 Linux (a) IBM JDK 1.1.8 Windows NT IBM JDK 1.1.8 OS/2 Microsoft VM 3234 Windows NT Microsoft VM 3229 Windows 2000 Appeal JRockit 1.0 Windows 2000 (a) IBM JDK 1.1.8 Linux Sun HotSpot Server 2.0 Windows NT Sun JDK 1.3 RC1 Windows NT Sun JDK 1.2.2-001 Windows NT Sun JDK 1.2.1_04 Solaris Novell JDK 1.1.7 NetWare (b) Sun/Inprise JDK 1.2.2 Linux Blackdown/Inprise JDK 1.2.2 Linux Blackdown JDK 1.2.2 RC4 Linux Transvirtual Kaffe 1.0.5 Linux (c) Blackdown JDK 1.1.7 Linux JDK 1.1.8 FreeBSD |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Java Platform | Operating System | Results | Score |
---|---|---|---|
![]() |
Red Hat Linux 6.0 Intel | 2303, 2313, 2313, 2300 | 2309 |
![]() |
Windows NT Workstation 4.0 | 2244, 2267, 2267, 2263 | 2266 |
![]() |
OS/2 Warp Server for e-business | 2145, 2141, 2118, 2150 | 2136 |
![]() |
Windows NT Workstation 4.0 | 2034, 2040, 2030, 2018 | 2029 |
![]() |
Windows 2000 Server Release Candidate 2 | 1948, 1965, 1976, 1968 | 1970 |
![]() |
Windows 2000 Server | 1910, 1926, 1925, 1931 | 1927 |
![]() |
Red Hat Linux 6.0 Intel | 1317, 1823, 1766, 1752 | 1780 |
![]() |
Windows NT Workstation 4.0 | 1638, 1664, 1668, 1675 | 1669 |
![]() |
Windows NT Workstation 4.0 | 1570, 1593, 1578, 1605 | 1592 |
![]() |
Windows NT Workstation 4.0 | 1477, 1481, 1479, 1471 | 1477 |
![]() |
Solaris 7 Desktop Intel Platform Edition | 1409, 1408, 1400, 1389 | 1399 |
![]() |
NetWare 5 | 1227, 1232, 1229, 1226 | 1229 |
![]() |
Red Hat Linux 6.0 Intel | 1035, 1039, 1042, 1039 | 1040 |
![]() |
Red Hat Linux 6.0 Intel | 1032, 1037, 1039, 1037 | 1038 |
![]() |
Red Hat Linux 6.0 Intel | 951, 952, 953, 950 | 952 |
![]() |
Red Hat Linux 6.0 Intel | 906, 905, 909, 909 | 908 |
![]() |
Red Hat Linux 6.0 Intel | 284, 285, 286, 283 | 285 |
![]() |
FreeBSD 3.2-RELEASE | 169, 173, 173, 172 | 173 |
The network scalability test was executed with the following commands over an isolated 10-Mbps Ethernet connection, using the heap and stack size options shown below where possible:
java -ms8m -mx128m -ss32k COM.volano.Main
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 50
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 100
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 150
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 200
Note: IBM JDK 1.1.8 on Linux required an initial heap size of
64 MB (using the option -ms64m
) in order to complete the
network scalability tests shown in Table 2 below, so you should judge
its relative performance instead by the loopback scores in Table 1
above.
See the COM.volano.Mark command synopsis for a complete description of all options.
The client test driver was executed under the Sun JDK 1.2.1_03 Production Release for Solaris using Solaris 7 on a dual 200-MHz Sun UltraSPARC with 256 megabytes of RAM. The operating systems on both sides were rebooted before each set of tests for a particular Java virtual machine. The VolanoMark server was not restarted between client test runs.
Scores are the throughput of the server in messages per second based on the total number of concurrent connections. Bigger numbers are faster. See the Environments section for details on the hardware platform, operating systems, and Java virtual machine environments.
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
|||
1000 | 2000 | 3000 | 4000 | |||
---|---|---|---|---|---|---|
|
Java Platform | 1000 | 2000 | 3000 | 4000 | Notes |
---|---|---|---|---|---|
![]() |
2486 | 1357 | 739 | 421 | No errors! |
![]() |
2571 | 1755 | 1072 | ---- | "java.lang.OutOfMemoryError" at 3508/4000 connections. |
![]() |
1521 | ---- | ---- | ---- | "java.lang.OutOfMemoryError" at 1098/2000 connections. |
![]() |
1695 | 979 | 535 | ---- | "java.lang.OutOfMemoryError" at 3796/4000 connections. |
![]() |
1408 | 708 | 406 | ---- | "java.lang.OutOfMemoryError" at 3785/4000 connections. |
![]() |
2354 | 1901 | 1237 | 688 | No errors! |
![]() |
722 | 556 | ---- | ---- | "java.lang.OutOfMemoryError: cannot create any more threads" at 2007/3000 connections. Required an initial heap size of 64 MB instead of 8 MB. |
![]() |
---- | ---- | ---- | ---- | "HotSpot Virtual Machine Error, EXCEPTION_ACCESS_VIOLATION, Error ID: 4F533F57494E13120E43505002D9" at 880/1000 connections. |
![]() |
1521 | 871 | 591 | ---- | "java.lang.OutOfMemoryError" at 3476/4000 connections. |
![]() |
---- | ---- | ---- | ---- | "java.lang.OutOfMemoryError" at 926/1000 connections. |
![]() |
1821 | 1060 | 695 | ---- | Hangs with disk thrashing at 3940/4000 connections. "Segmentation Fault - core dumped" when client test driver is killed to stop the test. |
![]() |
1575 | ---- | ---- | ---- | Hangs with disk thrashing at 1960/2000 connections. |
![]() |
1174 | ---- | ---- | ---- | "SIGSEGV 11* segmentation violation" at just over 1000 connections. See the second of the two bugs placed into Sun Bug Id 4298809. Only the first bug has been fixed. |
![]() |
1177 | 718 | 473 | 310 | No errors! |
![]() |
1141 | 666 | 452 | ---- | "java.lang.StackOverflowError" after a total of 6816 cumulative connections, causing it to fail 816 connections into the 4000-connection test. See Sun Bug Id 4299454. |
![]() |
---- | ---- | ---- | ---- | Hangs with zero percent CPU usage after starting, even at just 500 connections. Dumps core when the client side is killed to stop the test. |
![]() |
165 | ---- | ---- | ---- | Dumps core at just over 1000 connections. |
![]() |
37 | ---- | ---- | ---- | Client begins failing at 1017 connections with "java.net.NoRouteToHostException: Connection timed out". |
All tests ran identical copies of VolanoMark 2.1.2 on identical hardware -- a 200-MHz Intel Pentium Pro processor with a 256-kilobyte L2 cache and 256 megabytes of RAM on an Intel VS440FX motherboard running American Megatrends AMIBIOS Version 1.00.18.CS1 dated "08/28/98-13:59". The network card is a 3Com Etherlink III PCI Bus-Master Adapter Model 3C590.
"version 3.1.4.0 x86-linux"
TowerJ_3_1_4_0_x86_linux.class
(12,913,370 bytes).
-Dtowerj.poll
for the
network scalability test.
Main.tj
and
Mark.tj
.
"JDK 1.1.8 IBM build n118p-20000108 (JIT enabled: ibmjitc V3.5-IBMJDK1.1-20000108)"
ibm-jdk-n118p-win32-x86.zip
(10,824,610 bytes).
ibmjitc
just-in-time compiler.
14.039F_UNI
)
"JDK 1.1.8 IBM build o118-19990728 (JIT enabled: javax V3.5-IBMJDK1.1-19990728)"
javainuf.exe
(20,805,936 bytes) and
javaintk.exe
(15,273,452 bytes).
javax
just-in-time compiler.
CONFIG.SYS
file to increase the thread
limit from 1024 to 4095 (THREADS=4095)
and the initial swap
file size from 2 MB to 32 MB
(SWAPPATH=C:\OS2\SYSTEM 2048 32768)
.
"1.1.4"
5.00.3229
5.0.3234.0
msjavx86.exe
(5,449,664 bytes).
"1.1.4"
5.00.3229
"1.2"
"JRockit JVM 1.2 version 1.0 build excelsior45"
-Xgc:sc
and -Xexpandstacks
as recommend by
Appeal. Did not use the -Xss32k
stack size option on the
network test, since -Xexpandstacks
was used instead.
"JDK 1.1.8 IBM build l118-19991221 (JIT enabled: jitc)"
ibm-jdk-l118-linux-x86.tgz
(10,560,355 bytes).
jitc
just-in-time compiler.
"Classic VM (build JDK-1.2.2-001, native threads, symcjit)"
"Java HotSpot(TM) Server VM (2.0rc1, mixed mode, build I)"
jdk1_2_2-001-win.exe
(20,449,862 bytes)
and hotspot2_0rc1-win.exe
(1,305,731 bytes).
"X"
.
"1.3.0rc1, Java(TM) 2 Runtime Environment,
Standard Edition (build 1.3.0rc1-T), Java HotSpot(TM) Client VM (build
1.3.0rc1-S, mixed mode)"
j2sdk1_3_0rc1-win.exe
(30,847,507 bytes).
"X"
.
"Classic VM (build JDK-1.2.2-001, native threads, symcjit)"
jdk1_2_2-001-win.exe
(20,449,862 bytes).
symcjit
Symantec
just-in-time compiler.
"X"
.
"Solaris VM (build Solaris_JDK_1.2.1_04, native threads, sunwjit)"
Solaris_JDK_1.2.1_04_i386.bin
(18,591,552 bytes) and
1.2.1_04_patches_i386_5.7.tar
(4,011,008 bytes).
sunwjit
just-in-time compiler.
"X"
.
rlim_fd_max
variable in /etc/system
.
"1.1.7 B"
jvm.exe
(18,015,477 bytes).
symcjit
just-in-time compiler
(Symantec Java! JustInTime Compiler Version 3.10.106(x) for JDK 1.1.x).
java.cfg
file to enable the just-in-time
compiler (JAVA_COMPILER=symcjit)
.
"1.2.2", "Classic VM (build 1.2.2-L, green threads, javacomp)"
jdk1_2_2-linux-i386.tar.gz
(21,580,986 bytes) and
javacomp-1.2.13-glibc2.1.tar.gz
(202,058 bytes).
javacomp
just-in-time compiler.
"X"
.
"1.2.2", "Classic VM (build Linux_JDK_1.2.2_RC4, green threads, javacomp)"
jdk-1.2.2-RC4-linux-i386-glibc-2.1.2.sh
(22,485,129 bytes) and
libjavacomp.so
(118,468 bytes).
javacomp
just-in-time compiler.
"-green"
option for user-level threads and the
"-Djava.compiler=javacomp"
option for the Inprise
just-in-time compiler.
"X"
.
"1.2.2", "Classic VM (build Linux_JDK_1.2.2_RC4, green threads, sunwjit)"
jdk-1.2.2-RC4-linux-i386-glibc-2.1.2.sh
(22,485,129 bytes).
sunwjit
just-in-time
compiler.
"-green"
option for user-level threads.
"X"
.
"Engine: Just-in-time v3 Version: 1.0.5 Java Version: 1.1"
kaffe-1.0.5.tar.gz
(3,145,844 bytes).
kaffe.jit
just-in-time compiler.
"Linux_JDK_1.1.7B_v3_green_threads"
jdk_1.1.7-v3-glibc-x86.tar.bz2
(10,130,082 bytes).
"jdk1.1.8-FreeBSD:1999/7/19"
jdk1.1.8_ELF.V99-7-19.tar.gz
(11,337,773 bytes).
sysctl
to modify the kern.maxfilesperproc
and
kern.maxfiles
variables in /etc/rc.local
.
Java Platform | Operating System | Compiler | Thread Model |
---|---|---|---|
![]() |
Red Hat Linux 6.0 Intel | static | many-to-one |
![]() |
Windows NT Workstation 4.0 | mixed mode | one-to-one |
![]() |
OS/2 Warp Server for e-business | mixed mode | one-to-one |
![]() |
Windows NT Workstation 4.0 | JIT | one-to-one |
![]() |
Windows 2000 Server Release Candidate 2 | JIT | one-to-one |
![]() |
Windows 2000 Server | mixed mode | many-to-many |
![]() |
Red Hat Linux 6.0 Intel | JIT | one-to-one |
![]() |
Windows NT Workstation 4.0 | mixed mode | one-to-one |
![]() |
Windows NT Workstation 4.0 | mixed mode | one-to-one |
![]() |
Windows NT Workstation 4.0 | JIT | one-to-one |
![]() |
Solaris 7 Desktop Intel Platform Edition | JIT | many-to-many |
![]() |
NetWare 5 | JIT | many-to-one |
![]() |
Red Hat Linux 6.0 Intel | JIT | many-to-one |
![]() |
Red Hat Linux 6.0 Intel | JIT | one-to-one or many-to-one |
![]() |
Red Hat Linux 6.0 Intel | JIT | one-to-one or many-to-one |
![]() |
Red Hat Linux 6.0 Intel | JIT | many-to-one |
![]() |
Red Hat Linux 6.0 Intel | none | many-to-one |
![]() |
FreeBSD 3.2-RELEASE | none | many-to-one |