Please, help Haiti: http://bit.ly/8A5r8W

Damon

Late in 2009, we released LiveCycle Data Services 3.0, which included model-driven development, the new Edge Server functionality, enhanced high performance messaging and other great functionality requested by our customers.

It wasn't long before the new Edge Server and advanced high performance messaging features in LCDS 3.0 caught the attention of major financial institutions looking to do Flex-based applications for foreign-exchange trading and other financial services and trading applications. Almost immediately, we got a number of serious queries about the real performance potential of the new messaging architecture for these types of real-time, mission critical applications, and in particular, the messaging streaming performance. Streaming messaging performance is critical to trading desktop-type applications, since literally tens of millions of dollars can be lost if price updates are received even milliseconds later than expected. One very large financial customer in particular gave us a specific set of requirements for a particular application that we set about verifying that we could meet (and we did, successfully).

However, it was an interesting exercise, and the assets are generic and reusable, so I thought I'd share them here for everyone to see with instructions for reproducing the results if you're so inclined. If anyone is setting up LCDS 3.0 in a similar environment, or needs this type of world-class performance for real-time applications, it's simple to setup and you will likely find this information useful.

The table below shows the actual requirements from the customer:



They were very specific about how the latency was to be measured. In particular, the message generator, bolted onto the backend of LCDS, simulates movement in FX market rates, and publishes updates, with an appropriate timestamp to LCDS. A test client is then connected to the LCDS, and inspects the timestamp on each message received, comparing it to the prevailing time on arrival.

The load test harness (links below) then calculate maximum, minimum, average and standard deviation of latencies which can then be compared with the desired performance level in the table above. In essence, LCDS 3.0 was required to deliver 50,000 messages per second (20 live streams of data to each of 500 clients simultaneously) with no individual message latency taking more than 50ms, maximum. Message latency was to be measured as per below:



Happily, we easily met the metric with no changes to the LCDS 3.0 product, but we did tweak our load testing tool and JVM environment settings. See the "REASDME (PDF)" document link below for details.

Below is actual load testing tool output (showing a max latency of 37ms) from an actual LCDS 3.0 14-hour test run:

  [INFO] [NioAmfStreamTest] Virtual Consumer avg receive rate: 107.99 msg/s
  [INFO] [NioAmfStreamTest] Virtual Consumer min receive rate: 107.99 msg/s 
  [INFO] [NioAmfStreamTest] Virtual Consumer max receive rate: 107.99 msg/s 
  [INFO] [NioAmfStreamTest] Virtual Consumer avg latency: 0.28 ms
  [INFO] [NioAmfStreamTest] Virtual Consumer min latency: 0 ms
  [INFO] [NioAmfStreamTest] Virtual Consumer max latency: 37 ms
  [INFO] [NioAmfStreamTest] Virtual consumer std latency: 0.71 ms

Below are the assets needed to reproduce these results for yourself. Start with the README (PDF). A descripton of the machine and enviornmental settings are contained as well.

1. README (PDF)
2. load-test-tool.zip (LCDS load testing tool)
3. perf.zip (LCDS performance testing web app)
4. Perf_Reqs.doc.pdf

We're still working on the LCDS 3.0 Performance Brief, but I thought this might be interesting to share until that's officially available.

Enjoy!

Damon