Azul Soups Up Java For Financial Trading 

The financial services sector adopted Java as a programming language because being able to recode applications and algorithms constantly is a competitive advantage and Java is easier to program than C, C++, or other alternatives. But Java has its issues. Or more precisely, the Java virtual machine, the construct of an idealized system that Java runs on top of, has issues and many of them make programmers do all kinds of unnatural things with their systems.

Azul Systems has just tackled one particularly pesky issue – the Java warm-up problem – that affects financial trading systems. This issue also affects the risk analysis and fraud detection systems that feed into trading platforms and that increasingly need to keep pace with those trading operations.

The warm-up problem is a complex one and it bears some explanation. The Java virtual machine employed in servers was never designed to be instantly ready. The idea was that Java would be interpreted at first and then be complied "just-in-time" later with optimizations to make it run faster.

Unlike C or C++, which have static compilation that converts the high-level code down to machine language that runs on the bare metal, modern Java has a two-step process. First, application code running inside the Java virtual machine runs in a relatively inefficient interpreted mode, and then, over time as an application runs and responds to incoming data and transactions, it gradually gets tuned and compiled down, running at its full performance.

After running transactions for a day or two, you would think that Java applications would be highly optimized by this dynamic compilation that is constantly going on. But it is not that simple. First of all, because of memory leaks in applications and all kinds of digital cruft that can clog up systems that have been hammered on, financial services firms reboot their systems every night to prepare for the next day's trading. If you reboot a JVM, you lose the Java application's optimizations. Moreover, as market conditions change and programmers create new trading strategies, they are changing the code as well, which means it would need to be optimized again. Finally, each day's trading on the financial markets is unique, with its own code paths and live data, and so Java optimization will necessarily be different from day to day.

To try to get their code warmed up for the day's trading after a system reboot, usually around midnight, financial services firms often run a set of fake transactions through their trading platforms, but this can be particularly dangerous because there is always a risk that fake trades will seep through protective barriers under test conditions and hit the actual market. This is what happened to Knight Capital in August 2012, where its test code got onto the real markets and it booked a $461 million loss, knocking it out of business.

The typical day's trading curve for the Dow Jones Industrial Average

The problem, explains Scott Sellers, founder and CEO at Azul, is that financial markets make most of their money at the market open and market close, or when dramatic events happen in the middle of the day causing trading spikes. All of these spikes cause application and data patterns to change and then the Java virtual machine kicks into dynamic code optimization mode. As part of this process, the JVM falls back to what is called "deopt" mode, short for de-optimization, and in this case code that has been previously optimized for one set of data flows falls back to interpreted mode to adapt to the new conditions.

"You have to take a step back to take two steps forward," as Sellers puts it.

This deopt mode can take several seconds to run, and there are several points in a trading system where everything will completely stall for several milliseconds. In financial trading, at the market close and the market open, several seconds is everything.

The performance differences are staggering between interpreted and dynamically compiled mode. In two minutes, the optimized code lets the application run about twice as fast as the interpreted code, Sellers says.

The new ReadyNow feature of Azul's Zing Java virtual machine allows for companies to change the behavior of the dynamic compilation process. ReadyNow does two things. First, rather than just let deopt run wild when a JVM encounters new data streams that require re-optimization of the code, ReadyNow says don't do that.

"This is a real problem at the market open, when you need peak performance, because the depot process actually makes the performance worse," says Sellers. "We go through a lot of decisions and basically force deopts to not occur or have a low probability that they will."

The ReadyNow feature also stores a lot of data from the prior running of applications, looking at code paths and data streams and correlating that to the compiler decisions that were made in the past so these can be applied immediately when a Java application boots up. Think of it as a cheat sheet for the just-in-time compiler, and this data is injected into the Zing JVM compiler as it tunes new code. Companies can use data from one or more trading days to create these compiler hints.

"Even though yesterday's trading was not the same as today's, a lot of the compilation decisions that were made were mostly right," Sellers explains. "So instead of capturing the output of the compiled app, we capture the decisions that the compiler made and the state of the system when those decisions were made and feed them into the compiler for today's trading. Now the JVM can get to that fully optimized state faster than if it had to relearn."

ReadyNow includes a bunch of other features, and they are all included in the Zing 5.9 JVM that Azul started shipping a few weeks ago, as it turns out. Sellers says that with ReadyNow, financial trading systems can be at 95 percent of the throughput and low-latency levels of optimized code as the market opens without having to pre-warm their systems by churning fake transactions through them for hours. And then Zing will continue to learn from there as the Java code runs throughout the day, as usual. "And we will continue to improve the technology to close that gap even more," he adds.

OptionsCity, a maker of trading systems based in Chicago, has been putting ReadyNow through the paces. The company was founded in 2006 and launched its Metro end-to-end electronic trading and risk management platform a year later. In 2011, the company launched Freeway, a low-latency algorithmic trading platform to speed up the process of building, testing, and executing multi-asset trading strategies. The Freeway system also includes AlgoStore, a marketplace where companies can buy and sell trading strategies that run against the Freeway platform. The company's customers include proprietary traders, market makers, trading arcades, hedge funds, institutional, brokers, and even producers. These customers use OptionsCity applications to trade in interest rates, agricultural products, metals, energy, and indexes across multiple geographies.

In an email interview, Victor Glava, CTO at OptionsCity, tells EnterpriseTech that his company went Java from the start precisely to have speed to market with its code, and in 2012 it started using the Zing JVM because it wanted to have the pause-free garbage collection that Zing offers. (Oracle's HotSpot, the OpenJDK open source JVM, and IBM's own JVM do not have pause free garbage collection, and they therefore are subject to jitter when garbage collection routines run.) OptionsCity is evaluating ReadyNow, and concurs that it has in fact eliminated the latencies added by the JVM compiler in the past – on the order of tens of milliseconds in its applications – and has also reduced de-optimization as well.

Glava has a wish list of other features that he would like to see in the JVM.

"As we push our platform ever-increasing performance, we are getting to areas where we find benefit from increasingly granular control of the pattern of execution of threads," Glava says. "This includes things like thread priority and scheduling definition. Along this line, we are working with Azul to provide the ability to manually control thread safe-pointing. This will allow us to specify when we want the threads performing critical operations like processing market data, refreshing orders and quotes at the exchange to be interrupted by operations that include garbage collections."

"Another future development could be allowing the compiled code to be persisted between runs of the application, in order to take advantage of the JIT compiler knowledge at runtime," he adds.

The ReadyNow feature is available now in Zing 5.9, and it does not carry an incremental charge above and beyond the licensing of the Zing JVM. Azul does not provide pricing for Zing, but has said previously that the price is consistent with what one would pay for a commercially supported JVM from IBM or Oracle.

The other interesting thing to contemplate is how ReadyNow might be deployed in other Java applications where being instantly on and tuned is important.