Earlier posts in this series looked at the institutional side of that process, the exchanges, clearinghouses, and FCMs. This blog focuses on what happens next: the last mile of the data chain. Every company handles this step differently, and it is often where problems start to appear.
Exchange connectivity
Should you use your trading system’s built-in exchange connectivity? The answer depends on what you need. For some firms that trade on a limited number of exchanges with straightforward workflows, the built-in connectivity may be adequate as it is already integrated, tested, and supported by your vendor. However, as your trading operation grows more sophisticated, the built-in solution can become a constraint. If you trade across multiple venues, need ultra-low latency execution, or rely on custom order types or routing logic, your TRM system’s built in connectors may not give you enough flexibility. They can also make it harder to switch trading systems without rebuilding connectivity. In many cases, this setup becomes a limitation rather than an advantage.
The choice is not always obvious, and it is worth understanding the trade-offs before you find yourself locked into an approach that doesn't scale with your business.
Connecting trade data
- Do other systems need this trade data? Risk, quants, margin, position limits, and compliance all need raw trade data.
- How will those systems receive data, through your E/CTRM, or directly from the data feed?
- Do you trust your E/CTRM team, maybe it’s an internal team, or vendor to manage the trade feed properly?
The trade feed is the backbone of the trading operation. If it breaks, everything downstream feels it. Many firms use a E/CTRM system at the core and surround it with smaller, specialized tools. This setup provides scale without creating bottlenecks.
A key step is using generic trade feeds, ones that can connect to multiple systems, not just your Trading and Risk Management (TRM) systems. Feeds that only connect to a single system create what we call a walled garden.
Locked-in data
Despite the risks, many firms still plug trades directly into a large TRM system. It often starts with good intentions:
- Executives: “We need a TRM to raise capital, scale, and stay compliant.”
- Operations: “The TRM will handle everything — load it with data.”
- IT: “No time to plan; we need this working now.”
- Traders: “Where are my trades?”
In the rush to get data flowing, teams forget that trade data serves more than one purpose. It is the lifeblood of a trading firm, supporting analytics, risk, compliance, and more.
Once it goes into a TRM, it becomes hard to extract. TRMs transform and store trade data in their own format, making it difficult to use for other needs.
As the firm grows, this becomes a problem.
- Your quant teams need historical trade data to build and backtest models, but extracting years of structured positions, prices, and risk factors turns into a multi-week data engineering project.
- Compliance teams need raw trade details for reports or audits, yet getting granular data in the right format means navigating complex query tools or waiting for IT to run custom scripts.
- Operations teams need to verify margin calculations or credit limits in real-time, but the TRM’s reporting layer does not expose the underlying data they need, forcing them to reconcile across multiple screens or exports.
- IT needs to debug production issues safely, but they cannot easily spin up a test environment with realistic data without risking the live system or violating data governance policies.
- Finally, when the firm wants to evaluate a new TRM or risk system, they cannot properly test it because migrating even a subset of historical data for proof-of-concept becomes a major undertaking in itself.
If all your trade data lives inside one system, these tasks become painful or impossible.
Replacing a trading and risk management system is hard
Replacing a TRM system is not just a software upgrade. Your TRM has been tailored to your specific workflows, and all that institutional knowledge lives in custom fields, reports, and logic that would need to be recreated. The data migration alone is extraordinarily complex, involving millions of historical trades, positions, and valuations with intricate relationships where one mistake means your P&L does not reconcile.
The operational disruption impacts everyone: traders, risk managers, and operations all need retraining, and during cutover, you are essentially trading blind.
The regulatory risk is significant because your audit trail must remain intact, and regulators do not accept “we were mid-migration” as an excuse for gaps or inconsistencies. The cost and timeline are daunting – multi-million dollar budgets and multi-year projects represent serious executive career risk. On top of all this, your TRM connects to numerous other systems including market data, clearing, settlements, accounting, and credit, and all of these integrations need re-wiring.
This is how the walled garden is formed
The walled garden
So firms make the rational decision: stick with what we have. However, over time, the lock-in deepens as more data accumulates, more custom workflows get built, and switching costs continue to rise. Technical debt accumulates as teams create workarounds upon workarounds, and shadow IT emerges as people find ways to route around the system’s limitations. Innovation stalls because you cannot adopt new technologies that cannot access your data. Your quants and analysts become frustrated, they want to work with modern tools, not spend time fighting with legacy systems. Meanwhile, nimbler competitors are leveraging AI, real-time analytics, and integrated workflows while you are running overnight batch processes.
Building data bridges
Everything comes down to how you handle trade data. If you can access raw trade data easily, you can adapt, analyze, and grow faster.
The solution is not necessarily ripping out your TRM, it is building bridges. Start by storing all raw trade data in a datamart/datastore/datalake before sending it to the TRM. Better yet, use a setup where both the TRM and datastore receive the data at the same time. With this approach, you can test new systems in parallel without interrupting operations. Other teams can also access the data they need without touching production systems.
Modern cloud data stores such as Snowflake and BigQuery offer strong analytical tools and scale well. Kafka can also be used for real-time trade data delivery and messaging. Given their flexibility and cost efficiency, these are practical choices for any trading firm.
A key takeaway: Keep your trade data accessible. The more freely it moves, the stronger and more adaptable your trading operation becomes.
