Independently tracking all market data ticks, trades and payments moving across complex business critical processes, and providing detailed, completely customisable, live analysis at every stage, VMX EndToEnd delivers a real-time view of exactly what’s happening across your business, as it’s happening.
VMX EndToEnd is Velocimetrics’ strategic technology platform. VMX EndToEnd offers the full spectrum of advanced functionality required to continuously track and analyse the real-time status of all market data ticks, trades and payments being processed in a single, integrated solution.
To effectively track these items from the very moment they enter a process to their point of completion, VMX EndToEnd intelligently identifies all data elements that influence how the trade or payment is moving through the environment. By instantaneously chaining together all related data, the complete flow of events is examined in real-time, accurately assessing the data’s quality – its timeliness, correctness and completeness, and how the business is currently performing compared to client SLAs, regulatory commitments, thresholds and trends over time.
The result: Users are alerted in real-time to any changes to the way in which market data ticks, trades and payments are flowing that indicate emerging performance, operational risk, client experience or regulatory concerns.
Providing instant access to predictive analytics and powerful actionable insights, users can make quick and effective decisions. Enabling, for instance:
In doing so, avoiding many issues developing to start with and those that do to be swiftly and effectively brought under control from a technical, business and client perspective.
Employ a choice of network, application and middleware data capture techniques for complete operational oversight
To enable VMX EndToEnd to effectively track business items as they move across complex processes, VMX EndToEnd offers an unrivalled range of low-touch data capture methods. Allowing clients to mix-and-match approaches, the optimum capture solution to meet each client’s exact requirements can be implemented. These methods include, but are not limited to:
- High speed, passive, network packet capture to disk at a sustained rate of more than 40Gbps
- Capturing messages as they traverse middleware
- Monitoring files as they arrive at landing sites
- High precision in-application instrumentation
By capturing data at multiple points, and using a combination of system and network data capture techniques, clients can achieve real-time visibility across end-to-end business processes, regardless of how complex or dynamic these processes may be. This includes the ability to gain a very detailed understanding of how data is flowing between the individual, and often tightly coupled, components inside an application, which wouldn’t be possible by only capturing data on the network. It also means that with VMX EndToEnd, it is possible to comprehend exactly what’s happening to the data relating to a trade as it moves through blade servers, when using hardware to gain this visibility isn’t a viable option.
Conducting accurate time stamping, to a resolution of 10 nanoseconds when data is captured on the network or from inside applications, users can generate hop-by-hop and end-to-end latency and performance measurements with unparalleled accuracy. This is possible because the time stamps applied by both network and application capture techniques are accurately synced, enabling the complete journey taken by a piece of data to be precisely tracked at every possible stage. The following diagram provides just an example of the many hops that VMX EndToEnd could monitor within a trading flow.
Furthermore, should a business item such as a trade or payment go missing, or an issue occur, regardless of whether this happens on the network or inside an application, because the user has uncompromised visibility across the entire process, the issue or business item’s exact location can be immediately pinpointed. In doing so, considerably reducing the problem’s time to resolution.
Perform, real-time, end-to-end trade or payment flow tracking and analysis
VMX EndToEnd instantly sends all data captured, at the different points throughout the business process, to the VMX EndToEnd in-memory cache for correlation.
The data is examined as the business item it represents, such as a market data tick, order, trade or payment. By applying correlation techniques, VMX EndToEnd is able to identify that the same market data tick, for instance, was seen at three different capture points and reconstruct the exact path taken by that particular piece of market data.
Next real-time association techniques are applied to determine all of the different data elements contributing to a complete end-to-end flow, identifying, for example, the cause and effect relationships that exist between:
- A market data tick that generated an automated trading decision
- Which led to an order being executed
- That resulted in a trade
VMX EndToEnd then chains together, in real-time, all related information, such as the market data tick, order, trade and acknowledgement. This is possible even in highly complex environments where, for instance, parent and child trading relationships exist or where a unique ID fails to survive throughout the flow.
Accomplishing all these tasks in real-time, VMX EndToEnd is able to form complete end-to-end flows, providing a detailed understanding of the end-user’s experience. Furthermore, by storing the data as complete flows the relationships between different data elements can be more easily understood should future investigation be required.
Rapid payment or trade reconstruction
By storing data as complete trading chains, VMX EndToEnd is able to rapidly reconstruct the exact path taken by any monitored business item that the user may wish to examine, for instance a payment or trade, clearly identifying the contributing factors at every stage of its lifecycle.
This is even possible when the transaction takes place over an extended period of time enabling payments that complete over multiple days to be rapidly reconstructed with full lifecycle timings.
Users can search for any business item, for example by type, date, latency, trade / payment ID or a multitude of other attributes, and reconstruct the complete flow of events.
Furthermore, because VMX EndToEnd ties together all related data as it’s captured, in accordance with the business rules in place at the time, when users examine the stored data at a later date they don’t need to apply these rules retrospectively. This reduces the potential margin for error in future analysis.
Real-time alerting as issues emerge, facilitating preventative action
VMX EndToEnd enables users to track all business items as they move across complex business processes, and set customisable alerts for a range of business and technical metrics. Applying live, statistical distribution modelling techniques, using standard deviation and percentile calculations, VMX EndToEnd is able to instantly identify performance or value outliers as they are generated.
Unlike disk driven solutions, VMX EndToEnd uses an in-memory cache to rapidly process all captured data, applying correlation and association techniques and raising immediate alerts should unusual behaviours be detected or a threshold breached, whilst simultaneously capturing outliers for later analysis and detailed investigation.
As VMX EndToEnd captures business and technical data as well as timing information, users can apply a wide range of fully customisable alerts, notifying users of changing conditions and presenting opportunities to reduce risk or financial loss. Once an alert is raised, users can then immediately access detailed and powerful actionable insights, essential for effective decision-making.
This feature offers users many benefits, for instance in the post-trade environment, because VMX EndToEnd can assess the settlement requirements of each individual trade, alerts can be raised for trades approaching settlement dates that risk failing to complete all processes on time. Users can also be alerted, for example, if they are at risk of breaching client SLAs, regulatory commitments or approaching levels that could generate capacity challenges.
These alerts enable users to take proactive action, where possible entirely avoiding issues that would have otherwise developed and to bring problems under control in a swift and effective manner.
Gain completely customisable analysis to effectively support different user requirements
Once data has been processed through the correlation and association engines, innovative aggregation techniques are applied enabling large volumes of captured data to be sliced and diced in a multitude of different ways, with the desired intelligence displayed through highly configurable charts.
With this approach different users within the same business can analyse captured data from a variety of perspectives, enabling for instance technical management, application groups, operational staff and client support teams to all access the exact information they require. For example, trades could be analysed on a market-by-market, asset type, instrument or individual client basis, or payment instructions examined by individual client, type, batch, settlement date or notional value. Alternatively, performance issues could be analysed at an individual network packet and nanosecond level.
So when an issue occurs where and why it has happened can be quickly determined, its impact rapidly quantified and the clients affected swiftly identified. By quickly contacting and working with these clients, users can proactively manage their experience and where possible minimise the issue’s impact on their business.
Furthermore, when the volume of aggregated data being analysed is large users can locate figures in very deep or broad aggregations with ease. Users can select the search criteria using a filter bar to sort the data as they wish and then simply request a priority order for the information to be displayed, for example asking that the worst 10 performers by a particular metric are initially retrieved, followed by the next 10 etc. This enables large volumes of data to be meaningfully analysed in an easily manageable way.
With VMX EndToEnd users can also set-up and save a query with pre-defined query and output format selections. Other approved users, in a single click, can then quickly activate these queries. For instance users may be interested in identifying all trades that took place within the previous 10 minutes that took more than a specific number of microseconds to execute. Promptly retrieving these trades, users can then select any particular trade of interest and reveal hop-by-hop performance metrics.
Comprehend market data quality and timeliness
VMX EndToEnd is able to continually assess market data quality in real-time, alerting users to degradation issues so they can instantly:
- Understand if data is ticking as fast as expected, identify latency variations and intelligently assess data timeliness on an instrument-by-instrument basis. Compare absolute thresholds, content timestamps, historical norms and statistical distributions.
- Comprehend the impact that stale data or periodic congestion may be having on an instrument-by-instrument basis and fine-tune detection thresholds.
- Detect market data sequence gaps and market data tick absence at the feed, market, instrument or field level to identify on a live feed, if certain trading venues or individual instruments may be missing.
- Determine whether feed A or B, or the direct exchange versus consolidated feed is performing more effectively.
- Compare current price movements to a 5 or 30 day moving average to understand if pricing ticks are moving as expected compared to historical norms and receive alerts of sudden price jumps.
For further information about how VMX EndToEnd can assess the quality of market data being employed into trading decisions visit our page focused on VMX EndToEnd’s Market Data Quality capabilities
Benefit from automatic protocol discovery
With VMX EndToEnd’s automatic protocol discovery capabilities, users can accurately detect exactly which market data streams they are consolidating, distributing or consuming in their environment and identify market data sequence gaps.
Categorising and decoding the protocols against an extensive and well-maintained library of decoders, sequence gaps can be quickly detected and message rate statistics generated at a sustained speed of more than 1.2million packets per second, per feed.
In addition to notifying users that the market data being received may be incomplete, this feature can also provide a detailed understanding of which market data feeds users are actually subscribing to. It can prove beneficial in terms of capacity planning, enabling users to understand the bandwidth and volume of their feeds.
Furthermore, by identifying retransmission requests users can identify if certain applications may be struggling with current data volumes.
Compare data captured at geographically dispersed monitoring locations
The VMX EndToEnd architecture separates the data capture and analysis processes. This original approach enables cost-effective data capture agents to be applied at multiple points, across a distributed architecture. In doing so, comparisons can be made between data captured at different monitoring locations.
This technique can present significant cost savings compared to traditional monitoring solutions, which conduct both data capture and analysis activities at the same location, requiring the deployment of expensive technology at multiple points.
With VMX EndToEnd, the data is time-stamped at the point of capture and it is then sent to a central analysis server, or servers, for correlation, association and aggregation processing. By time-stamping data at source, the VMX EndToEnd analysis server is able to easily manage all data received, even when it is received out-of-order.
Conduct effective capacity planning
VMX EndToEnd enables users to gain a detailed understanding of how business items such as orders or payments are flowing across complex and / or dynamic environments.
This insight alongside VMX EndToEnd’s detailed network analysis capabilities enables system under or over utilisation to be determined and capacity levels to be more effectively managed.
Deploy a scalable solution able to support extremely complex environments processing high volumes of very fast data
To achieve maximum efficiency in high volume situations, instead of sending captured data to a single central analysis server, that can only track and analyse a finite number of events at any one time, with VMX EndToEnd a cluster of interactive servers in constant communication can be employed.
Using a clustered architecture the relationships that exist between data being processed by different partitions of the cluster can be accurately correlated, enabling for instance a market data tick being processed by one server, that led to the decision to execute a trade being processed by another server, to be correlated together. In doing so, users can continue benefiting from detailed real-time operational oversight; instant issue alerting and access to decision-making analytics as market data and trading volumes continue to expand in exceedingly complicated trading situations.
This innovative approach divides up complex correlation and tracking workloads, allowing near-linear scaling to cope with increasing volume, whilst retaining the capability to provide a unified view across the trading operation. Using proprietary distributed calculation schemes, the approach enables aggregated and statistical reporting to be delivered effectively at high volume in real-time.
Additionally, individual items can be tracked and alerts raised (for instance because an item has gone missing), distribution latencies across all trades processed in a period can be quickly formulated, highlighting outliers and enabling full order and trade life-cycles to be reconstructed.
With this scalable solution, VMX EndToEnd is able to effectively monitor complex environments such as intricate payment processing plants or equity trading environments and deliver rich analytics remarkably quickly, even when faced with extremely high volumes. This is also possible when the related information may be received over wide time windows, such as an order that is filled over a period of a day as multiple child orders.
The data is then rapidly persisted to disk in a scalable manner using the clustered architecture because each server writes to its own independent physical disk and database instance. This enables the full transaction detail be recorded, which can then be quickly reconstructed on demand enabling an exchange, for instance, to rapidly reconstruct any participant order and all resultant trades to quickly address and resolve client queries.
Creating a more agile environment, this scalable approach will enable your firm to continue effectively monitoring and controlling its environment, as market conditions evolve and volumes rise.
Enrich monitored data with supplementary information
VMX EndToEnd is an extensible solution that enables additional information to be incorporated with the monitored data to provide customisable business insight. Offering an Application Programming Interface (API) and Software Development Kits (SDKs), extensions can be incorporated by the VMX EndToEnd professional services implementation team or by a client’s own technical development team.
The ability to enrich tracked data offers a whole host of opportunities to VMX EndToEnd users, for instance if a user wishes to understand the notional value of all orders trading live, but they are displayed in different currencies, a foreign exchange calculator could be applied to demonstrate the equivalent value in a selected currency.
Easily integrate alerts into existing systems
VMX EndToEnd has been designed to easily integrate with existing client infrastructures and processes, for example VMX EndToEnd’s detection alerts can be incorporated within SNMP systems, enabling users to view a consolidated alerting feed.
This feature enables data to be streamed to other applications, such as, the user’s in-house or third party analysis tools, which might include enterprise infrastructure monitoring solutions. Because VMX EndToEnd is an open solution, both raw and statistically analysed data can be fed to a client’s applications, enabling for instance, time-stamped and normalised data, or time-stamped, normalised, correlated and analysed data to be sent.
Map system and network changes quickly and efficiently
VMX EndToEnd can be quickly configured to track business items as they move across an infrastructure. Designed to deliver forward compatibility, following a small amount of training, users can independently update VMX EndToEnd to map system changes, such as the addition of new networks or applications.
This approach significantly reduces the long-term dependence on external consultants as infrastructure changes are introduced to accommodate adjustments in trading practices.
Benefit from completely configurable real-time intra-day and scheduled reporting
Through the user-friendly monitoring console, VMX EndToEnd enables snapshot and scheduled reports to be generated on demand.
Users can configure the reports to meet their individual needs, determining what they would like the reports to look like and the selection criteria. For instance, users in payment processing environments may wish to generate a real-time report detailing the total number and value of payments currently in flight. Alternatively, in a trading environment receiving a list of the worse performing trades within a period could be of significant value.
Reports can be created in PDF, or Excel-friendly CSV format; the former suitable for sending to pre-loaded lists for email distribution on an automated schedule, whilst the latter offers a straight forward means of data extraction for client-specific further analysis and presentation.
Once created these reports can be saved and re-run as desired, enabling any user to simply select and generate the report from the menu option.
Additionally, VMX EndToEnd’s reporting capabilities can also be used to compare current activities, such as the performance of a specific link or latency levels across particular environments, to pre-recorded historical statistics using extensive charting capabilities. These capabilities can prove very useful in pre-production, as well as live environments to facilitate effective, automated performance testing enabling the performance of upgraded technology to be compared to existing functionality.
Access VMX EndToEnd’s powerful analytics from anywhere via a laptop, desktop or mobile device
VMX EndToEnd can be accessed using the VMX EndToEnd application or a secure browser-based user interface. Whilst the application is used by system administrators and Velocimetrics’ dedicated professional services team to initially configure captured data, firms can enable end-users to retrieve VMX EndToEnd’s rich analytics from anywhere with the browser interface.
This means adopters can provide an unlimited number of end-users with direct access to VMX EndToEnd’s powerful, actionable insights without incurring additional cost or implementation effort as there is no need to physically install the application on each user’s device. Furthermore, because the data available via the web-based interface is pre-configured, users with minimal training can very easily attain the exact information they require.