The problem
On 15 May 2026, between 13:00 and 20:00 UTC, WTI crude futures traded 12,890 times on CME Globex and Brent traded 4,305 times. The median gap between WTI trades was 0.12 seconds; for Brent it was about one second. No two of those 17,195 timestamps coincide. Sampled every five minutes, the two contracts show a correlation near 0.87, which is what anyone who watches oil markets would expect. Sampled every two seconds, from exactly the same trades, the estimate drops to 0.37. More than half the measured dependence is gone.
Nothing about the relationship between the two oils changes across that figure. Only the sampling clock does. The collapse is the Epps effect, documented in equity data since 1979, and it is the visible symptom of a deeper problem: the data live on two irregular clocks, and the question "how correlated are they?" is usually answered by pretending they live on one.
The obvious fix, and what it costs
The standard approach picks a grid, five minutes, one second, whatever the application demands, and carries the last observed price of each asset forward onto it. This is previous-tick sampling, and interpolation and binning are its close relatives. The appeal is real: once both series sit on a shared grid, every tool of classical multivariate statistics applies.
The cost is bias with a direction. A grid interval that contains a trade in one asset but not the other contributes a stale price, and stale prices push the measured covariance toward zero. Tighten the grid and the share of stale intervals grows, so the estimate collapses exactly when the data are richest. Widen the grid and the bias fades, but so does the sample: a day of thousands of trades becomes 84 five-minute returns. The practitioner is left choosing a point on a bias-variance curve that the method itself created.
Estimators that need no shared clock
This part has been solved. The Hayashi–Yoshida estimator sums products of returns over every pair of overlapping observation intervals, works directly on the raw asynchronous timestamps, and is unbiased with no synchronization step at all. Around it sits a family of refinements: weighted versions, refresh-time corrections, pre-averaged and kernel variants built to withstand market microstructure noise.
The unsolved part is choice. On any given day, on any given pair, which member of that family should you trust? The literature compares them asymptotically: each estimator's variance is known in a limit of ever-finer sampling. However, a trading day is not a limit. It is one realized set of timestamps, and two estimators with identical asymptotics can differ materially on the day you actually have. Until now the honest answer to "which estimator?" has been folklore, simulation studies, and habit.
An exact yardstick
Our approach conditions on the observation times actually seen, the way regression conditions on its design matrix. Fix the two sets of timestamps and treat them as the design of the experiment. In our doctoral work at Oxford we showed that, conditional on that design, the finite-sample variance of every estimator in the weighted Hayashi–Yoshida class is an exact polynomial in the observed interval overlaps. Not an approximation, not a limit: a formula you can evaluate on today's timestamps before the market closes.
That turns estimator choice from folklore into a computation. For any window, any pair, any candidate estimator, we can state its variance exactly and compare candidates with error bars that are themselves exact. It also yields the best weighting directly: minimizing the exact variance over the weight class is a solvable problem, and the solution adapts to the day's actual trading pattern rather than to an idealized one. Of course, exactness is conditional: the formulas hold under a model for the prices and given the observed clocks. Yet that is precisely the conditioning practitioners already accept in regression, and it is what makes same-day, same-data comparison possible at all.
The study under way
We are running this comparison at scale: seven estimators, from plain Hayashi–Yoshida through refresh-time, pre-averaged, kernel, and optimally-weighted variants, on four CME cross-asset pairs, gold with silver, 10-year with 30-year Treasuries, WTI with natural gas, euro with sterling futures, across eleven market regimes from 2010 to 2026, at full market-by-order resolution. Every window carries its exact error bars, so the output is not a single winner but a map: which estimator, at which liquidity ratio, under which noise regime. The deliverable is the table we wish had existed when we started trading.
The same machinery feeds forward. A correlation matrix with exact per-entry variances is the natural input to network methods, clustering, ranking, lead-lag detection, that currently assume every edge is equally trustworthy. That is problem 03.
Further reading
The exact-variance theory is developed in Correlation methods in the statistical analysis of financial trading data, DPhil thesis, University of Oxford, 2016. The estimator comparison is a working paper in preparation; if you would like to hear when it is available, write to us.