Much to my surprise, Appeal's JRockit has arrived on the scene to post the best network scalability scores of any Java virtual machine I've tested. In fact, it's the only Java virtual machine I've found which is able to support 4,000 connections on Windows NT or Windows 2000 without modification.
Appeal has made Java threads as lightweight as they're advertised to be, mapping their thin threads onto native threads as required. JRockit seems to be particularly suited for pure Java servers which must handle many dedicated network connections. Like TowerJ, though, it's not free. Let's hope for a Linux port of JRockit!
For background information and download locations, please see the Java Benchmarks page on Volano's Web site. Since the server side of the VolanoMark benchmark is our VolanoChat server, these test results help our customers choose the best Java virtual machine for their environment.
VolanoMark is a pure Java server benchmark characterized by long-lasting network connections and high thread counts. In this context, long-lasting means the connections last several minutes or longer, rather than just a few seconds. The VolanoMark benchmark creates client connections in groups of 20 and measures how long it takes for the clients to take turns broadcasting their messages to the group. At the end of the test, it reports a score as the average number of messages transferred by the server per second.
Bug Id 4075058, "java.io: Add support for non-blocking I/O," explains why you need lots of threads in order to handle dedicated connections to a pure Java server. JSR 51, "New I/O APIs for the Java Platform," explains how Sun plans to fix that.
The tests presented in this report look at the performance and network scalability of 18 Java virtual machines on 7 operating systems using a common Intel hardware platform running VolanoMark™ 2.1. VolanoMark attempts to answer two questions about Java virtual machines used in network server environments -- especially those environments involving a large number of dedicated socket connections:
See the Environments section for details on the hardware platform, operating systems, and Java virtual machine environments, along with a table showing the compiler and threading model of each Java platform.
The performance test was executed with the following commands on a local loop-back connection, using the heap size options shown below where possible:
java -ms8m -mx64m COM.volano.Main
java -ms8m -mx64m COM.volano.Mark -count 100
See the COM.volano.Mark command synopsis for a complete description of all options.
The operating system was rebooted before each set of tests for a particular Java virtual machine. The server side was not restarted before each run of the client benchmark. The client benchmark ran four times, with the final score as the average of the last three results.
Scores are the throughput of the server in messages per second. Bigger numbers are faster. See the Environments section for details on the hardware platform, operating systems, and Java virtual machine environments.
Java Platform | Score |
---|---|
Tower TowerJ 3.1.4 Linux (a) IBM JDK 1.1.8 Windows NT IBM JDK 1.1.8 OS/2 Microsoft VM 3234 Windows NT Microsoft VM 3229 Windows 2000 Appeal JRockit 1.0 Windows 2000 (a) IBM JDK 1.1.8 Linux Sun HotSpot Server 2.0 Windows NT Sun JDK 1.3 RC1 Windows NT Sun JDK 1.2.2-001 Windows NT Sun JDK 1.2.1_04 Solaris Novell JDK 1.1.7 NetWare (b) Sun/Inprise JDK 1.2.2 Linux Blackdown/Inprise JDK 1.2.2 Linux Blackdown JDK 1.2.2 RC4 Linux Transvirtual Kaffe 1.0.5 Linux (c) Blackdown JDK 1.1.7 Linux JDK 1.1.8 FreeBSD |
2309 2266 2136 2029 1970 1927 1780 1669 1592 1477 1399 1229 1040 1038 952 908 285 173 |
Java Platform | Operating System | Results | Score |
---|---|---|---|
Tower TowerJ 3.1.4 Linux (a) | Red Hat Linux 6.0 Intel | 2303, 2313, 2313, 2300 | 2309 |
IBM JDK 1.1.8 Windows NT | Windows NT Workstation 4.0 | 2244, 2267, 2267, 2263 | 2266 |
IBM JDK 1.1.8 OS/2 | OS/2 Warp Server for e-business | 2145, 2141, 2118, 2150 | 2136 |
Microsoft VM 3234 Windows NT | Windows NT Workstation 4.0 | 2034, 2040, 2030, 2018 | 2029 |
Microsoft VM 3229 Windows 2000 | Windows 2000 Server Release Candidate 2 | 1948, 1965, 1976, 1968 | 1970 |
Appeal JRockit 1.0 Windows 2000 (a) | Windows 2000 Server | 1910, 1926, 1925, 1931 | 1927 |
IBM JDK 1.1.8 Linux | Red Hat Linux 6.0 Intel | 1317, 1823, 1766, 1752 | 1780 |
Sun HotSpot Server 2.0 Windows NT | Windows NT Workstation 4.0 | 1638, 1664, 1668, 1675 | 1669 |
Sun JDK 1.3 RC1 Windows NT | Windows NT Workstation 4.0 | 1570, 1593, 1578, 1605 | 1592 |
Sun JDK 1.2.2-001 Windows NT | Windows NT Workstation 4.0 | 1477, 1481, 1479, 1471 | 1477 |
Sun JDK 1.2.1_04 Solaris | Solaris 7 Desktop Intel Platform Edition | 1409, 1408, 1400, 1389 | 1399 |
Novell JDK 1.1.7 NetWare (b) | NetWare 5 | 1227, 1232, 1229, 1226 | 1229 |
Sun/Inprise JDK 1.2.2 Linux | Red Hat Linux 6.0 Intel | 1035, 1039, 1042, 1039 | 1040 |
Blackdown/Inprise JDK 1.2.2 Linux | Red Hat Linux 6.0 Intel | 1032, 1037, 1039, 1037 | 1038 |
Blackdown JDK 1.2.2 RC4 Linux | Red Hat Linux 6.0 Intel | 951, 952, 953, 950 | 952 |
Transvirtual Kaffe 1.0.5 Linux (c) | Red Hat Linux 6.0 Intel | 906, 905, 909, 909 | 908 |
Blackdown JDK 1.1.7 Linux | Red Hat Linux 6.0 Intel | 284, 285, 286, 283 | 285 |
JDK 1.1.8 FreeBSD | FreeBSD 3.2-RELEASE | 169, 173, 173, 172 | 173 |
The network scalability test was executed with the following commands over an isolated 10-Mbps Ethernet connection, using the heap and stack size options shown below where possible:
java -ms8m -mx128m -ss32k COM.volano.Main
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 50
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 100
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 150
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 200
Note: IBM JDK 1.1.8 on Linux required an initial heap size of
64 MB (using the option -ms64m
) in order to complete the
network scalability tests shown in Table 2 below, so you should judge
its relative performance instead by the loopback scores in Table 1
above.
See the COM.volano.Mark command synopsis for a complete description of all options.
The client test driver was executed under the Sun JDK 1.2.1_03 Production Release for Solaris using Solaris 7 on a dual 200-MHz Sun UltraSPARC with 256 megabytes of RAM. The operating systems on both sides were rebooted before each set of tests for a particular Java virtual machine. The VolanoMark server was not restarted between client test runs.
Scores are the throughput of the server in messages per second based on the total number of concurrent connections. Bigger numbers are faster. See the Environments section for details on the hardware platform, operating systems, and Java virtual machine environments.
1000 | 2000 | 3000 | 4000 | |||
---|---|---|---|---|---|---|
|
Java Platform | 1000 | 2000 | 3000 | 4000 | Notes |
---|---|---|---|---|---|
Tower TowerJ 3.1.4 Linux | 2486 | 1357 | 739 | 421 | No errors! |
IBM JDK 1.1.8 Windows NT | 2571 | 1755 | 1072 | ---- | "java.lang.OutOfMemoryError" at 3508/4000 connections. |
IBM JDK 1.1.8 OS/2 | 1521 | ---- | ---- | ---- | "java.lang.OutOfMemoryError" at 1098/2000 connections. |
Microsoft VM 3234 Windows NT | 1695 | 979 | 535 | ---- | "java.lang.OutOfMemoryError" at 3796/4000 connections. |
Microsoft VM 3229 Windows 2000 | 1408 | 708 | 406 | ---- | "java.lang.OutOfMemoryError" at 3785/4000 connections. |
Appeal JRockit 1.0 Windows 2000 | 2354 | 1901 | 1237 | 688 | No errors! |
IBM JDK 1.1.8 Linux | 722 | 556 | ---- | ---- | "java.lang.OutOfMemoryError: cannot create any more threads" at 2007/3000 connections. Required an initial heap size of 64 MB instead of 8 MB. |
Sun HotSpot Server 2.0 Windows NT | ---- | ---- | ---- | ---- | "HotSpot Virtual Machine Error, EXCEPTION_ACCESS_VIOLATION, Error ID: 4F533F57494E13120E43505002D9" at 880/1000 connections. |
Sun JDK 1.3 RC1 Windows NT | 1521 | 871 | 591 | ---- | "java.lang.OutOfMemoryError" at 3476/4000 connections. |
Sun JDK 1.2.2-001 Windows NT | ---- | ---- | ---- | ---- | "java.lang.OutOfMemoryError" at 926/1000 connections. |
Sun JDK 1.2.1_04 Solaris | 1821 | 1060 | 695 | ---- | Hangs with disk thrashing at 3940/4000 connections. "Segmentation Fault - core dumped" when client test driver is killed to stop the test. |
Novell JDK 1.1.7 NetWare | 1575 | ---- | ---- | ---- | Hangs with disk thrashing at 1960/2000 connections. |
Sun/Inprise JDK 1.2.2 Linux | 1174 | ---- | ---- | ---- | "SIGSEGV 11* segmentation violation" at just over 1000 connections. See the second of the two bugs placed into Sun Bug Id 4298809. Only the first bug has been fixed. |
Blackdown/Inprise JDK 1.2.2 Linux | 1177 | 718 | 473 | 310 | No errors! |
Blackdown JDK 1.2.2 RC4 Linux | 1141 | 666 | 452 | ---- | "java.lang.StackOverflowError" after a total of 6816 cumulative connections, causing it to fail 816 connections into the 4000-connection test. See Sun Bug Id 4299454. |
Transvirtual Kaffe 1.0.5 Linux | ---- | ---- | ---- | ---- | Hangs with zero percent CPU usage after starting, even at just 500 connections. Dumps core when the client side is killed to stop the test. |
Blackdown JDK 1.1.7 Linux | 165 | ---- | ---- | ---- | Dumps core at just over 1000 connections. |
JDK 1.1.8 FreeBSD | 37 | ---- | ---- | ---- | Client begins failing at 1017 connections with "java.net.NoRouteToHostException: Connection timed out". |
All tests ran identical copies of VolanoMark 2.1.2 on identical hardware -- a 200-MHz Intel Pentium Pro processor with a 256-kilobyte L2 cache and 256 megabytes of RAM on an Intel VS440FX motherboard running American Megatrends AMIBIOS Version 1.00.18.CS1 dated "08/28/98-13:59". The network card is a 3Com Etherlink III PCI Bus-Master Adapter Model 3C590.
"version 3.1.4.0 x86-linux"
TowerJ_3_1_4_0_x86_linux.class
(12,913,370 bytes).
-Dtowerj.poll
for the
network scalability test.
Main.tj
and
Mark.tj
.
"JDK 1.1.8 IBM build n118p-20000108 (JIT enabled: ibmjitc V3.5-IBMJDK1.1-20000108)"
ibm-jdk-n118p-win32-x86.zip
(10,824,610 bytes).
ibmjitc
just-in-time compiler.
14.039F_UNI
)
"JDK 1.1.8 IBM build o118-19990728 (JIT enabled: javax V3.5-IBMJDK1.1-19990728)"
javainuf.exe
(20,805,936 bytes) and
javaintk.exe
(15,273,452 bytes).
javax
just-in-time compiler.
CONFIG.SYS
file to increase the thread
limit from 1024 to 4095 (THREADS=4095)
and the initial swap
file size from 2 MB to 32 MB
(SWAPPATH=C:\OS2\SYSTEM 2048 32768)
.
"1.1.4"
5.00.3229
5.0.3234.0
msjavx86.exe
(5,449,664 bytes).
"1.1.4"
5.00.3229
"1.2"
"JRockit JVM 1.2 version 1.0 build excelsior45"
-Xgc:sc
and -Xexpandstacks
as recommend by
Appeal. Did not use the -Xss32k
stack size option on the
network test, since -Xexpandstacks
was used instead.
"JDK 1.1.8 IBM build l118-19991221 (JIT enabled: jitc)"
ibm-jdk-l118-linux-x86.tgz
(10,560,355 bytes).
jitc
just-in-time compiler.
"Classic VM (build JDK-1.2.2-001, native threads, symcjit)"
"Java HotSpot(TM) Server VM (2.0rc1, mixed mode, build I)"
jdk1_2_2-001-win.exe
(20,449,862 bytes)
and hotspot2_0rc1-win.exe
(1,305,731 bytes).
"X"
.
"1.3.0rc1, Java(TM) 2 Runtime Environment,
Standard Edition (build 1.3.0rc1-T), Java HotSpot(TM) Client VM (build
1.3.0rc1-S, mixed mode)"
j2sdk1_3_0rc1-win.exe
(30,847,507 bytes).
"X"
.
"Classic VM (build JDK-1.2.2-001, native threads, symcjit)"
jdk1_2_2-001-win.exe
(20,449,862 bytes).
symcjit
Symantec
just-in-time compiler.
"X"
.
"Solaris VM (build Solaris_JDK_1.2.1_04, native threads, sunwjit)"
Solaris_JDK_1.2.1_04_i386.bin
(18,591,552 bytes) and
1.2.1_04_patches_i386_5.7.tar
(4,011,008 bytes).
sunwjit
just-in-time compiler.
"X"
.
rlim_fd_max
variable in /etc/system
.
"1.1.7 B"
jvm.exe
(18,015,477 bytes).
symcjit
just-in-time compiler
(Symantec Java! JustInTime Compiler Version 3.10.106(x) for JDK 1.1.x).
java.cfg
file to enable the just-in-time
compiler (JAVA_COMPILER=symcjit)
.
"1.2.2", "Classic VM (build 1.2.2-L, green threads, javacomp)"
jdk1_2_2-linux-i386.tar.gz
(21,580,986 bytes) and
javacomp-1.2.13-glibc2.1.tar.gz
(202,058 bytes).
javacomp
just-in-time compiler.
"X"
.
"1.2.2", "Classic VM (build Linux_JDK_1.2.2_RC4, green threads, javacomp)"
jdk-1.2.2-RC4-linux-i386-glibc-2.1.2.sh
(22,485,129 bytes) and
libjavacomp.so
(118,468 bytes).
javacomp
just-in-time compiler.
"-green"
option for user-level threads and the
"-Djava.compiler=javacomp"
option for the Inprise
just-in-time compiler.
"X"
.
"1.2.2", "Classic VM (build Linux_JDK_1.2.2_RC4, green threads, sunwjit)"
jdk-1.2.2-RC4-linux-i386-glibc-2.1.2.sh
(22,485,129 bytes).
sunwjit
just-in-time
compiler.
"-green"
option for user-level threads.
"X"
.
"Engine: Just-in-time v3 Version: 1.0.5 Java Version: 1.1"
kaffe-1.0.5.tar.gz
(3,145,844 bytes).
kaffe.jit
just-in-time compiler.
"Linux_JDK_1.1.7B_v3_green_threads"
jdk_1.1.7-v3-glibc-x86.tar.bz2
(10,130,082 bytes).
"jdk1.1.8-FreeBSD:1999/7/19"
jdk1.1.8_ELF.V99-7-19.tar.gz
(11,337,773 bytes).
sysctl
to modify the kern.maxfilesperproc
and
kern.maxfiles
variables in /etc/rc.local
.
Java Platform | Operating System | Compiler | Thread Model |
---|---|---|---|
Tower TowerJ 3.1.4 Linux | Red Hat Linux 6.0 Intel | static | many-to-one |
IBM JDK 1.1.8 Windows NT | Windows NT Workstation 4.0 | mixed mode | one-to-one |
IBM JDK 1.1.8 OS/2 | OS/2 Warp Server for e-business | mixed mode | one-to-one |
Microsoft VM 3234 Windows NT | Windows NT Workstation 4.0 | JIT | one-to-one |
Microsoft VM 3229 Windows 2000 | Windows 2000 Server Release Candidate 2 | JIT | one-to-one |
Appeal JRockit Windows 2000 | Windows 2000 Server | mixed mode | many-to-many |
IBM JDK 1.1.8 Linux | Red Hat Linux 6.0 Intel | JIT | one-to-one |
Sun HotSpot Server 2.0 Windows NT | Windows NT Workstation 4.0 | mixed mode | one-to-one |
Sun JDK 1.3 RC1 Windows NT | Windows NT Workstation 4.0 | mixed mode | one-to-one |
Sun JDK 1.2.2-001 Windows NT | Windows NT Workstation 4.0 | JIT | one-to-one |
Sun JDK 1.2.1_04 Solaris | Solaris 7 Desktop Intel Platform Edition | JIT | many-to-many |
Novell JDK 1.1.7 NetWare | NetWare 5 | JIT | many-to-one |
Sun/Inprise JDK 1.2.2 Linux | Red Hat Linux 6.0 Intel | JIT | many-to-one |
Blackdown/Inprise JDK 1.2.2 Linux | Red Hat Linux 6.0 Intel | JIT | one-to-one or many-to-one |
Blackdown JDK 1.2.2 RC4 Linux | Red Hat Linux 6.0 Intel | JIT | one-to-one or many-to-one |
Transvirtual Kaffe 1.0.5 Linux | Red Hat Linux 6.0 Intel | JIT | many-to-one |
Blackdown JDK 1.1.7 Linux | Red Hat Linux 6.0 Intel | none | many-to-one |
JDK 1.1.8 FreeBSD | FreeBSD 3.2-RELEASE | none | many-to-one |