2024-07-12 –, Method (1.5)
What do financial markets look like up close? TotalView-ITCH is a unique data source that allows researchers to explore limit order activity on the Nasdaq exchange down to nanosecond precision. Working with this data, however, poses a challenge to researchers due to its volume (over 100 million messages per day) and its distinct binary format. This lightning talk describes a new package built to parse and process this deluge of data in Julia.
Nasdaq TotalView-ITCH (“TotalView”) is a data feed used by professional traders to maintain a real-time view of market conditions. TotalView disseminates all quote and order activity for securities traded on the Nasdaq exchange—over 100 million message per day—allowing users to reconstruct the limit order book for any security up to arbitrary depth with nanosecond precision. It is a unique data source for financial economists and engineers examining topics such as information flows through lit exchanges, optimal trading strategies, and the development of macro-level indicators from micro-level signals (e.g., a market turbulence warning).
While TotalView data is provided at no charge to academic researchers via the Historical TotalView-ITCH offering, the historical data offering uses a binary file specification that poses challenges for researchers. TotalViewITCH.jl is a pure Julia package developed to efficiently process historical data files for academic research purposes. The package consists of: (1) a core module to parse Historical TotalView binary file format messages (i.e., deserialization), (2) a module to reconstruct limit order books from parsed messages, and (3) a module to store processed data into a research-friendly format.
Colin Swaney is a Senior Research Software Engineer at Princeton University's Data-Driven Social Science Initiative. His research and development interests revolve around simulation, generative models and the application of machine learning to large social datasets.