In an industry where speed of execution is integral to trading success the ability to detect and consistently refine potential latency sources is fundamental.
However, as trading and exchange architectures increasingly consolidate into multi-core servers running client gateways, trading or matching engines, less inter-process communication is happening on the network. Unless developers are able to see inside these processes, they can very quickly become major blind spots.
By applying network monitoring techniques at the server’s ingress and egress points, users can identify if latency is occurring within the application. But, it is only by instrumenting inside the application that developers are able to gain precise visibility into the inner-processes and pinpoint latency sources between different components, for instance:
Having identified which component is causing the delay, developers can then more efficiently allocate engineering effort to the issue’s resolution.
Application Tap offers a patented, high precision approach to in-application instrumentation, with minimal performance overhead. Providing very accurate insight inside application processes, Application Tap offers a fine-grained view of the particular components introducing performance issues, enabling real-time system developers to become significantly more effective in quickly identifying the exact source of potential problems.
Introducing Application Tap
With Application Tap users can not only determine which component is causing the delay, but also independently validate whether the latency lies within the application, or by measuring across network boundaries, if the delay actually resides on the network stack or within the operating system.
This ability to gain a hop-by-hop understanding of the performance levels being achieved inside the application and across network/application boundaries would be impossible to comprehend using network only monitoring techniques.
Application Tap enables users to tap into an application and gain a deep, accurate understanding of what is going on, so the problem’s source can be isolated, in both testing and production environments. This enables burst and sustained performance throughput rates to be accurately calculated with minimal overhead.
Offering a very lightweight means of accessing data from within application processes, Application Tap also enables firms to achieve many additional goals as the instrumented data can then be fed into other centralised systems where it can be correlated and examined to meet client specific analysis needs
How it works
Application Tap delivers a view inside systems by placing minimally invasive instrumentation hooks into the application at customisable points of interest. For instance, developers may wish to precisely instrument the time of order matching or when a pre-trade risk check is passed. How Application Tap can instrument a process is demonstrated by the following diagram.
With Application Tap developers can signal events by simply using the user mode API to write to the Application Tap memory. When deployed as hardware, Application Tap is able to limit the overhead to a memory write operation avoiding CPU cycles being chewed up unnecessarily on instrumentation.
The code passes the instrumentation metadata to the Application Tap, which time stamps events to 10 nanoseconds resolution, synchronised to PTP, providing a wall clock time. The application tap then publishes the instrumented data onto the network, where TipOff or Velocimetrics can recreate a unified, correlated view of the complete stream of network and application data as demonstrated by the following diagram.
This is possible even across distributed environments, enabling users to rapidly detect performance issues impacting real-time financial systems and networks, or to feed the data to client specific analysis systems supporting for instance risk or regulatory needs.
Application Tap is a very easy to implement API that enables precisely timed software instrumentation with minimal performance overhead.
Developing to Application Tap follows the same process regardless of whether the user wishes to do so in software, where it is most commonly deployed in testing environments or on hardware where, due to its lightweight approach to instrumentation, it proves very popular in production.
Application Tap is available for C/C++ and Java, and is supported on Linux. Once implemented Application Tap can be then deployed throughout an environment in software or on hardware multiple times. This is possible without the need to actually set up a server with a physical Application Tap for each developer requiring access to perform instrumentation tasks.