Looking for AI's ROI? Try purpose-fitting

Delivering an AI return on investment can be challenging. A new IDC white paper offers a solution: leverage infrastructure to the use case.

Companies can build a strong return on investment (ROI) for their AI projects—but only if they understand how to leverage different infrastructure solutions for different AI use cases. In other words, they need to know how to do purpose-fitting.

That’s the case argued in a new IDC white paper sponsored by AMD and Supermicro.

The paper’s two co-authors are Peter Rutten, research VP in IDC’s worldwide infrastructure research group and global research lead of the firm’s performance-intensive computing practice; and Madhumitha Sathish, research manager for high-performance computing at IDC and lead of the firm’s AI infrastructure research.

Rutten and Sathish find not all is well in the world of AI. In a survey conducted by IDC this past September, fewer than half of companies worldwide said their AI-related projects have delivered any measurable business outcomes. And only about one in 10 companies (11.4%) said they’re obtaining measurable business results from more than 75% of their AI projects.

What’s blocking AI progress? According to the IDC survey, these are the top reasons:

• Competition for resources: cited by 34% of survey respondents
• Resistance to process change: cited by 30%
• Difficulty quantifying AI’s ROI: 28%
• Regulatory uncertainty: also 28% (multiple responses were allowed)

“Cost continues to be a major hurdle,” the authors write.

And the biggest cost? Over 60% of companies surveyed by IDC said it’s around developing and deploying AI is specialized infrastructure.

Four Questions

It doesn’t have to be this way, the IDC authors argue. Instead, AI-using organizations can build a strong ROI for their projects with purpose-fitting.

To do this, managers should ask (and answer) 4 important questions:

• Who decides what is your relevant AI use case? A separate IDC survey finds that fewer than one in three organizations involve IT during an AI initiative’s conceptual stage.
• What kind of AI model do you need? There are many, including machine learning, GenAI, agentic AI, deep neural network, etc. Not all require major capital expenditures.
• How will you obtain this AI model? Each approach involves trade-offs. For example, most businesses fine-tune or customize an existing commercial model. But this approach involves both licensing costs and training costs.
• Have you considered the biggest factors that impact AI infrastructure needs? These factors include AI model types, number of parameters, volume of training data, query response times, and query size.

By taking these factors into account, the authors say, enterprises can develop AI options that match their AI use case, creating a purpose-built infrastructure solution.

Spectrum Choices

To contain AI infrastructure costs, the IDC authors recommend that managers develop what they call a “spectrum of options” based on 7 factors: Complexity, parameter count, data volume, model accuracy, time to value, query response latency, and query size.

When these factors are low or small, an AI project is in the blue zone, which implies lower costs. As these factors become higher or larger, the project moves into the green and red zones, which imply higher costs, as shown in the IDC chart below.

Hardware system requirements can vary by spectrum, too.

Blue zone projects, those with the lowest infrastructure costs, can be run on CPU-based, air-cooled systems, or even a PC or workstation.

Green zone projects, those with intermediate infrastructure costs, can run on systems powered by CPUs with built-in accelerators and lighter co-processors.

And red zone projects, those with the highest infrastructure zones, require rack-scale systems with high-end CPUs, GPUs and liquid-cooling.

But wait, there’s more. The IDC authors point to several additional considerations:

• Is there more than one AI use case in development? Typically, there are. If that’s the case, then that will need to be built into the needs projection.
• How rapidly will the AI use case evolve over time? For example, if the number of users is projected to grow substantially, then the accounting must consider new infrastructure that will be required.
• How often will the AI model require generational updates? Many models are constantly being improved, expanded and retrained, and these updates will deliver infrastructure impacts.

Better Together

The IDC authors say AI-using companies would do well to consider AMD-powered Supermicro systems. The two suppliers work with a vast ecosystem of partners to offer alternatives and options.

AMD and Supermicro demystify complexity, helping companies plan their AI projects faster and better. And they offer reliable, high-performance platforms that support AI workloads across a wide range of deployment scales.

“AMD and Supermicro,” the IDC authors write, “have developed some of the most versatile, powerful and well-tailored solutions available today.”

Do More:

• Read the full IDC white paper: For a Solid Return on Investment with AI, Consider the Many Ways to Purpose-Fit Your Infrastructure
• Check out Supermicro’s AMD-powered systems for enterprise AI
• Explore AMD Instinct GPUs