How AMD, Supermicro and myrtle.ai set a machine learning speed record for finance

By closing the gap on machine learning latency, the three companies broke the 2-microsecond barrier.

For the finance industry, the full potential of machine learning has been blocked by a technical issue known as AI latency. That’s the delay between sending a request and receiving a response. And as far as financiers are concerned, it’s been way too long.

Until now, that is.

Supermicro, AMD and a company called myrtle.ai have jointly solved this latency problem. In the process, the three have also set a new world record.

The machine that broke the record is Supermicro’s CloudDC server, AS -2015CS-TNR, powered by a single AMD EPYC 9575F 64-core processor and equipped with AMD networking and AI accelerator technology.

According to the results of the STAC-ML Benchmark test, Supermicro’s server ran the myrtle.ai VOLLO acceleration software stack fast enough to break the 2-microsecond barrier for the 99th percentile LSTM inference.

In other words, this system first received financial market data, next determined the best way to act on it, and finally issued a trading command. And it did all three steps almost instantly.

Why does Big Finance need the world’s fastest AI server?

Major players in the finance industry don’t like to take unnecessary chances. If they can stack the odds in their favor and outperform the market, they’ll do it. And they’ll do it as quickly and as often as possible.

To accomplish this, they rely on what are known as quant strategies. These are investment approaches based on complex math, statistics and algorithms.

Common examples of quant strategies include:

    1. Factor investing: Systematically trading stocks and other financing instruments with measurable traits like cheap valuations and low volatility.
    2. Statistical arbitrage: Very quickly exploiting tiny price discrepancies between two or more related assets.
    3. High-frequency trading: Executing thousands of trades per second based on micro-patterns.

These techniques all require high levels of both speed and accuracy. The more a trader can bring to bear, the better their chances of achieving “alpha”—returns that beat the market average.

Traders’ go-to quant strategies recently gained a powerful new variation: machine learning models. The rapid development of AI-centric software and hardware made it possible for them to task machines with combing massive data sets. In this way, the systems can find market signals that give traders an edge.

The rest is history. Once major financial institutions realized that faster computers could produce more profits, the race was on to develop a new generation of AI-enabled high-performance computers that would give them an edge over the competition.

What’s under the hood of a financial AI supercomputer?

Using machine learning to make lightning-fast trading decisions requires a lot of computing power. Doing it faster than any other computer in the history of the world requires even more.

Supermicro started with its CloudDC 2U rack-mount AI server optimized for “tick-to-trade” performance. That means the system is designed to bring in market data (the tick) and use it to create a transaction (the trade) with fantastically low latency.

The AI server used for the benchmark was equipped with a single AMD EPYC 9005 Series processor with 192GB of DDR5 memory.

The server also included hardware acceleration in the form of a Silicon Engineering Artena PCIe 5.0 expansion card featuring the AMD Versal Adaptive SoC, or system on a chip. The SoC’s ultra-high bandwidth enabled it to deal with simulated market-data surges.

To feed these components with data, Supermicro also needed a networking solution fast enough to prevent data bottlenecks. That came courtesy of AMD’s Solarflare X4 Ethernet Adapters engineered for sub-microsecond latency.

All that advanced hardware underpins a software stack designed by Myrtle.ai. The software, called VOLLO, enables users to compile machine learning models that can be used with a standard library. In other worse, no programming expertise is required.

Using ML for finance in the real world

Breaking a world record was an important step in proving the value of machine learning in the finance industry. But what financial movers and shakers really want to know is how they can use advanced AI servers to increase profits.

The answer can be broken into 3 points:

And that’s how Supermicro, AMD and myrtle.ai eliminated the inference gap in ML for trading decisions.

Do more: