Squarepoint Capital
London
I spent six months at Squarepoint Capital, an investment management firm whose trading decisions are made by computer programs rather than by people. Those programs are only as good as the information they receive, so the firm runs infrastructure that collects price data from exchanges around the world and delivers it to them in a fraction of a microsecond. My placement was about making that infrastructure faster, and the work had two sides. The first was everyday speed: how long a normal piece of price data takes to travel through the system, which is what almost all of the traffic experiences. The second was the rare exception, where very occasionally a single update would arrive seconds or even minutes late. Rare events like that matter more than they sound, because a trading program acting on out of date prices is making decisions about a market that no longer exists. Both sides needed the same thing first, which was visibility. You cannot make a system faster if you cannot see where it currently spends its time, and you cannot catch something that happens a handful of times a day by sitting and watching for it. Much of my six months went into building ways to see inside the system while it was running. Alongside that I helped bring a new source of exchange data into production.
I joined the market data engineering team as a software engineering intern, working on the systems that receive exchange data and pass it on to the firm's trading programs. Most of my own contribution was to everyday speed. I built measurement into the system so that a slow piece of data could be traced to the exact stage that held it up, and used what that showed to change how the system was put together. On our production monitoring, the time taken to handle an ordinary update fell by roughly 87 percent at the midpoint, and by between 74 and 93 percent across the rest of the range. The rare delays were a harder and more collaborative problem. I read through the whole codebase until I could account for every point at which a delay was possible, then followed the evidence out of our own software and down into the Linux operating system underneath it, which narrowed a very large set of possible causes to a small one. My manager recognised the pattern in what came back and implemented the fix, and I worked with him on validating it. Those delays then stopped appearing. I also took a new data feed from configuration through conformance testing to a staged release in production.
Linux systems performance: working out what an application is really doing at the level of the operating system, using SystemTap for dynamic kernel instrumentation alongside procfs, kernel counters and standard performance tooling. Measurement and instrumentation: designing structured logging and timestamping that can attribute a delay to a specific stage, and treating latency as a distribution rather than an average, since rare slow events are invisible in a mean. Reading large unfamiliar codebases: building an accurate end to end model of a mature production system written by other people. Networking and market data: how exchange data is disseminated over multicast, and how a feed handler decodes, normalises and publishes it. Production engineering: staged rollouts, validating correctness against reference data, and deciding in advance what counts as failure and how to roll back.
Working with uncertainty: staying with a problem for weeks with nothing to show for it, and judging when to keep going and when to ask for help. Communicating to two audiences: giving engineers the mechanism and the limits of the evidence, and giving researchers and traders the consequence, meaning what they can rely on now and what is still open. Being honest about what I know: replacing optimistic progress updates with calibrated ones that say what has been assumed and what has not yet been checked. Working in a team: I did the groundwork on the main problem and my manager made the final connection. It taught me that a contribution is often evidence good enough for someone else to reason from, rather than the answer itself. Taking feedback: reading a critical code review as information about the work rather than as a judgement on me. Writing things down, so that what I learned does not stay in one person's head.