Empowering Efficient Trading Strategies - A Guide for Historical Market Data Buyers
High quality historical data can significantly enhance trading strategies by providing insights into market trends, liquidity, and volatility. By analysing historical data, traders can identify patterns, predict market movements, and optimise their strategies for better trading performance. However, it's critical to choose the right historical datasets and the right provider to meet your organisation’s needs. Before you invest, here are the most important considerations:
1. CHOOSE DATA THAT IS READY FOR USE
A high quality data vendor will provide data that has been collected, curated and engineered ready for use.
A good historical data provider will fully manage the data collection and curation process. For highly granular data, these processes must be robust with quality checks and documentation methods in place to ensure the data is complete. The provider should have a single harmonised data schema across all venues and asset classes containing every unique field from each venue to ensure no information loss.
Importantly, the vendor should provide all licences required for use of the product. There should be no need
for additional licences.
Select a vendor that has done the heavy lifting so that your team can quickly focus on gaining value.
2. FLEXIBLE AND SCALABLE DELIVERY OPTIONS TO SUIT YOU
The data provider you choose should be able to offer a broad range of delivery mechanisms to suit your specific needs. For example, you may wish to have a feed that delivers the data directly into your workflow. You may wish to access the data via your own Snowflake account, or receive the data via S3, API or SFTP.
For deeper analysis, a Jupyter sandbox research environment may be a requirement for your team, or for non-quants, powerful data visualisation tools can also be considered.
Whatever your requirements, the data should be delivered in a way that suits your specific set-up.
3. DOES THE VENDOR USE ITS OWN DATA?
If the historical data provider offers harmonised data and reference mapping, easy analytics computation
across all venues at a millisecond level can be expected.
A good vendor will offer a broad range of daily and intraday analytics so that you can quickly derive insight.
Examples of daily analytics include Market Quality, which allows you to compare venue quality across fragmented markets in order to optimise smart order routers and assess regulatory change; Venue Performance to understand venue liquidity over time; and Market Microstructure to understand how the market truly behaves to generate new sources of alpha.
4. DOES THE VENDOR HAVE A GOOD NORMALISATION PROCESS?
Trading practitioners and quants need access to high quality data that captures all information, but importantly is also consistent and easy to use. Put simply, they need ‘normalised’ data. Vendors that put strong emphasis on true normalisation of trades and order book data, and the integration between full research and production, enable their clients to spend less time on data formatting, and more time on their business.
The term ‘normalisation’ in the context of data simply means, organising and structuring the data so that it’s easy to query, analyse and understand. A good normalisation process enhances the data for ease of use and none of the raw data attributes are lost in the process.
Normalisation is often seen as problematic in the market data world, viewed as an issue that users either have to work around, or avoid altogether. This is primarily due to two reasons.
- The first is legacy data products, which often have inconsistent fields that vary over time, and are unrecoverable since the original source data has long since been lost.
- The second (and far worse) is that the process of normalisation is often a low priority, done at the convenience of engineering teams rather than for the benefit of end users. This leads to a myriad of classic problems - normalisation design choices that vary by region (or developer), endless lists of new fields that are added to a normalisation scheme, rather than a strict, clear model, and the inability of support teams to explain the normalisation approach. This leads to data science and quant teams spending their valuable time “re-normalising” vendor market data, rather than using the data to find value.
A good normalisation model means that data users can understand each market, and the nuances of what’s in the data, without having to go through the details of each market.
Vendors that put strong emphasis on true normalisation of trades and order book data, and the integration between full research and production, enable their clients to spend less time on data formatting, and more time on their business.
5. DATA DICTIONARY, REFERENCE GUIDES, DOCUMENTATION AND CONTINUOUS IMPROVEMENT
A good historical market data provider will provide a data dictionary, a complete reference guide to navigate and explore the full depth order book products. Definitions and calculations of the metrics should come as standard. A process for capturing every unique field from each venue without information loss from packet capture data, plus continuous improvement of the quality of the data should be in place. Additionally, a clear roadmap for new features, along with a consistent methodology for time stamps, and trade identification codes is critical. Documentation should be included, with easy to understand schemas, clear audit trails around assumptions and methodologies.
5. LOCATION OF YOUR CHOICE
The provider should leverage the efficient data storage capacity of the cloud, for example they should be able to store and access three different versions of the underlying data: the raw data direct from exchanges; a curated Level 3 version of the same data; and a harmonised version of data from a wide number of exchanges that allows cross-venue application. This enables forensic-level quality control by making it possible to refer to the original raw data, as well as a normalised view of the data to remove the burden of data engineering. With a high quality provider, the integrity of the data is never lost, due to complete Level 3 engineering applied to Level 1 and Level 2 datasets.
7. WELL SUPPORTED PRODUCT DELIVERY
When evaluating a data provider, it's important to ensure they have a track record of working with the leading trading firms in the world, ranging from Global Exchange Groups, T1 Investment Banks, Market Makers, Hedge Funds, the Academic Community and the Regulators. The vendor should offer world-class level support, with a team of client-facing data scientists, with quant or researcher experience in the capital markets.
NO COMPROMISE
A world class historical data supplier, such as BMLL, will provide the same quality and depth of information across all the major trading venues. It will also continue to invest in global coverage.
Further, with BMLL, the ordeal of working with data that is low quality or not ready for use is over.
BMLL has an unrivalled focus on historical data. It ensures that accurate and timely historical market data can be easily accessed and consumed. The integrity of the data is never lost, due to complete data engineering processes so that your team’s time can be focused on your business, and not on the time-consuming tasks associated with data licensing, collection, curation and formatting. Your team can focus on generating insights, strategy building, back testing and launching new products.
