The Channel Company

Sponsored by:

Visit AMD Visit Supermicro

Capture the full potential of IT

Research Roundup: AI and data centers, cybersec spending, AI for competitive advantage & sales

Featured content

Research Roundup: AI and data centers, cybersec spending, AI for competitive advantage & sales

Catch up on the latest IT industry market research and surveys. 

Learn More about this topic
  • Applications:

The rapid adoption of artificial intelligence is putting new stress on data centers. Cybersecurity spending is growing faster than expected. Business leaders say AI is a competitive advantage. And AI could even help salespeople meet their quotas.

That’s some of the latest from top IT research and polling organizations. And here’s your roundup.

AI Needs More Juice

So much AI, so few data centers. That’s one of the more surprising side effects of the AI explosion.

Demand for data centers is rising. Also rising are data centers’ electric bills, says market watcher IDC.

All data centers use a lot of electric power. Add AI to the mix, and demand for juice rockets even higher.

That’s important because electricity already accounts for nearly half (46%) of the average enterprise data center’s total operational cost, and even more (60%) for the average service-provider’s data center, IDC says.

IDC now predicts that AI data center energy consumption will rise by a compound average growth rate (CAGR) of nearly 45% from now through 2028, when it will reach a global total of 146.2 terawatt hours.

Further, IDC expects overall global data center electricity consumption to more than double between 2023 and 2028, reaching 847 terawatt hours. That’s equivalent to a five-year CAGR of 19.5%.

Cybersec Spending: Up!

Cybersecurity spending rose nearly 10% in the second quarter of this year, reaching a worldwide total of $21.1 billion, according to industry analysts Canalys.

That fast rate of growth left Canalys surprised. It had expected both closer scrutiny of cyber budgets and slower contracts signings due to uncertainty about the economy.

Instead, vendors focused on cross-selling their platforms. Canalys says the top 12 cyber providers collectively accounted for more than half (53.2%) of total spending in Q2.

“Vendors are positioning their cybersecurity platforms to reduce customers’ complexity by consolidating redundant and legacy point products, says Canalys chief analyst Matthew Ball. “But this also reduces organizations’ resilience by increasing dependency on fewer vendors.”

Looking ahead, Canalys expects even bigger growth in spending on cyber services (as opposed to cyber technology). For the full year 2024, Canalys predicts cyber-services spending to grow by nearly 13% year-on-year, reaching a global total of $163.3 billion.

AI: The New Competitive Advantage

Nearly 7 in 10 business leaders (68%) say their organizations’ competitive advantage now depends on making the best use of artificial intelligence. So finds a new poll conducted by Forrester on behalf of credit-reporting site Experian.

In the survey, roughly 6 in 10 respondents (62%) also said their top AI use case is analyzing alternative data sources with Generative AI.

But business leaders are also looking for faster results. More than half the respondents (55%) said developing and deploying AI and machine-learning models takes them too much time.

The survey, conducted earlier this year, reached 1,320 business leaders in 10 countries across the EMEA and Asia-Pacific regions.

AI for Sales? Yes, Please

Add sales to the list of jobs that can be enhanced with AI. A new forecast from researchers at Gartner posits that sellers who partner effectively with AI tools are 3.7 times more likely to meet their quotas than are those who don’t use AI.

The forecast is based on Gartner’s recent survey of more than 1,025 B2B sellers.

Gartner also says that in response, senior sales officers will need to prepare their staff for a world with AI. That could include training salespeople with new AI skills, setting new sales priorities, and refining compensation and even career paths.

One possible snag: In Gartner’s survey, nearly three-quarters of the salespeople (72%) said they’re already overwhelmed by the number of skills required for their job. And fully half (50%) said they’re similarly overwhelmed by the amount of technology needed.

Watch the related video podcast:

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

The AMD Instinct MI300X Accelerator draws top marks from leading AI benchmark

Featured content

The AMD Instinct MI300X Accelerator draws top marks from leading AI benchmark

In the latest MLPerf testing, the AMD Instinct MI300X Accelerator with ROCm software stack beat the competition with strong GenAI inference performance. 

Learn More about this topic
  • Applications:
  • Featured Technologies:

New benchmarks using the AMD Instinct MI300X Accelerator show impressive performance that surpasses the competition.

This is great news for customers operating demanding AI workloads, especially those underpinned by large language models (LLMs) that require super-low latency.

Initial platform tests using MLPerf Inference v4.1 measured AMD’s flagship accelerator against the Llama 2 70B benchmark. This test is an indication for real-world applications, including natural language processing (NLP) and large-scale inferencing.

MLPerf is the industry’s leading benchmarking suite for measuring the performance of machine learning and AI workloads from domains that include vision, speech and NLP. It offers a set of open-source AI benchmarks, including rigorous tests focused on Generative AI and LLMs.

Gaining high marks from the MLPerf Inference benchmarking suite represents a significant milestone for AMD. It positions the AMD Instinct MI300X accelerator as a go-to solution for enterprise-level AI workloads.

Superior Instincts

The results of the LLaMA2-70B test are particularly significant. That’s due to the benchmark’s ability to produce an apples-to-apples comparison of competitive solutions.

In this benchmark, the AMD Instinct MI300X was compared with NVIDIA’s H100 Tensor Core GPU. The test concluded that AMD’s full-stack inference platform was better than the H100 at achieving high-performance LLMs, a workload that requires both robust parallel computing and a well-optimized software stack.

The testing also showed that because the AMD Instinct MI300X offers the largest GPU memory available—192GB of HBM3 memory—it was able to fit the entire LLaMA2-70B model into memory. Doing so helped to avoid network overhead by preventing model splitting. This, in turn, maximized inference throughput, producing superior results.

Software also played a big part in the success of the AMD Instinct series. The AMD ROCm software platform accompanies the AMD Instinct MI300X. This open software stack includes programming models, tools, compilers, libraries and runtimes for AI solution development on the AMD Instinct MI300 accelerator series and other AMD GPUs.

The testing showed that the scaling efficiency from a single AMD Instinct MI300X, combined with the ROCm software stack, to a complement of eight AMD Instinct accelerators was nearly linear. In other words, the system’s performance improved proportionally by adding more GPUs.

That test demonstrated the AMD Instinct MI300X’s ability to handle the largest MLPerf inference models to date, containing over 70 billion parameters.

Thinking Inside the Box

Benchmarking the AMD Instinct MI300X required AMD to create a complete hardware platform capable of addressing strenuous AI workloads. For this task, AMD engineers chose as their testbed the Supermicro AS -8125GS-TNMR2, a massive 8U complete system.

Supermicro’s GPU A+ Client Systems are designed for both versatility and redundancy. Designers can outfit the system with an impressive array of hardware, starting with two AMD EPYC 9004-series processors and up to 6TB of ECC DDR5 main memory.

Because AI workloads consume massive amounts of storage, Supermicro has also outfitted this 8U server with 12 front hot-swap 2.5-inch NVMe drive bays. There’s also the option to add four more drives via an additional storage controller.

The Supermicro AS -8125GS-TNMR2 also includes room for two hot-swap 2.5-inch SATA bays and two M.2 drives, each with a capacity of up to 3.84TB.

Power for all those components is delivered courtesy of six 3,000-watt redundant titanium-level power supplies.

Coming Soon: Even More AI power

AMD engineers continually push the limits of silicon and human ingenuity to expand the capabilities of their hardware. So it should come as little surprise that new iterations of the AMD Instinct series are expected to be released in the coming months. This past May, AMD officials said they plan to introduce AMD Instinct MI325, MI350 and MI400 accelerators.

Forthcoming Instinct accelerators, AMD says, will deliver advances including additional memory, support for lower-precision data types, and increased compute power.

New features are also coming to the AMD ROCm software stack. Those changes should include software enhancements including kernel improvements and advanced quantization support.

Are you customers looking for a high-powered, low-latency system to run their most demanding HPC and AI workloads? Tell them about these benchmarks and the AMD Instinct MI300X accelerators.

Do More:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

Developing AI and HPC solutions? Check out the new AMD ROCm 6.2 release

Featured content

Developing AI and HPC solutions? Check out the new AMD ROCm 6.2 release

The latest release of AMD’s free and open software stack for developing AI and HPC solutions delivers 5 important enhancements. 

Learn More about this topic
  • Applications:
  • Featured Technologies:

If you develop AI and HPC solutions, you’ll want to know about the most recent release of AMD ROCm software, version 6.2.

ROCm, in case you’re unfamiliar with it, is AMD’s free and open software stack. It’s aimed at developers of artificial intelligence and high-performance computing (HPC) solutions on AMD Instinct accelerators. It's also great for developing AI and HPC solutions on AMD Instinct-powered servers from Supermicro. 

First introduced in 2016, ROCm open software now includes programming models, tools, compilers, libraries, runtimes and APIs for GPU programming.

ROCm version 6.2, announced recently by AMD, delivers 5 key enhancements:

  • Improved vLLM support 
  • Boosted memory efficiency & performance with Bitsandbytes
  • New Offline Installer Creator
  • New Omnitrace & Omniperf Profiler Tools (beta)
  • Broader FP8 support

Let’s look at each separately and in more detail.

LLM support

To enhance the efficiency and scalability of its Instinct accelerators, AMD is expanding vLLM support. vLLM is an easy-to-use library for the large language models (LLMs) that power Generative AI.

ROCm 6.2 lets AMD Instinct developers integrate vLLM into their AI pipelines. The benefits include improved performance and efficiency.

Bitsandbytes

Developers can now integrate Bitsandbytes with ROCm for AI model training and inference, reducing their memory and hardware requirements on AMD Instinct accelerators. 

Bitsandbytes is an open source Python library that enables LLMs while boosting memory efficiency and performance. AMD says this will let AI developers work with larger models on limited hardware, broadening access, saving costs and expanding opportunities for innovation.

Offline Installer Creator

The new ROCm Offline Installer Creator aims to simplify the installation process. This tool creates a single installer file that includes all necessary dependencies.

That makes deployment straightforward with a user-friendly GUI that allows easy selection of ROCm components and versions.

As the name implies, the Offline Installer Creator can be used on developer systems that lack internet access.

Omnitrace and Omniperf Profiler

The new Omnitrace and Omniperf Profiler Tools, both now in beta release, provide comprehensive performance analysis and a streamlined development workflow.

Omnitrace offers a holistic view of system performance across CPUs, GPUs, NICs and network fabrics. This helps developers ID and address bottlenecks.

Omniperf delivers detailed GPU kernel analysis for fine-tuning.

Together, these tools help to ensure efficient use of developer resources, leading to faster AI training, AI inference and HPC simulations.

FP8 Support

Broader FP8 support can improve the performance of AI inferencing.

FP8 is an 8-bit floating point format that provides a common, interchangeable format for both AI training and inference. It lets AI models operate and perform consistently across hardware platforms.

In ROCm, FP8 support improves the process of running AI models, particularly in inferencing. It does this by addressing key challenges such as the memory bottlenecks and high latency associated with higher-precision formats. In addition, FP8's reduced precision calculations can decrease the latency involved in data transfers and computations, losing little to no accuracy.  

ROCm 6.2 expands FP8 support across its ecosystem, from frameworks to libraries and more, enhancing performance and efficiency.

Do More:

Watch the related video podcast:

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

Research Roundup, AI Edition: platform power, mixed signals on GenAI, smarter PCs

Featured content

Research Roundup, AI Edition: platform power, mixed signals on GenAI, smarter PCs

Catch the latest AI insights from leading researchers and market analysts.

Learn More about this topic
  • Applications:
  • Featured Technologies:

Sales of artificial intelligence platform software show no sign of a slowdown. The road to true Generative AI disruption could be bumpy. And PCs with built-in AI capabilities are starting to sell.

That’s some of the latest AI insights from leading market researchers, analysts and pollsters. And here’s your research roundup.

AI Platforms Maintain Momentum

Is the excitement around AI overblown? Not at all, says market watcher IDC.

“The AI platforms market shows no sign of slowing down,” says IDC VP Ritu Jyoti.

IDC now believes that the market for AI platform software will maintain its momentum through at least 2028.

By that year, IDC expects, worldwide revenue for AI software will reach $153 billion. If so, that would mark a five-year compound annual growth rate (CAGR) of nearly 41%.

The market really got underway last year. That’s when worldwide AI platform software revenue hit $27.9 billion, an annual increase of 44%, IDC says.

Since then, lots of progress has been made. Fully half the organizations now deploying GenAI in production have already selected an AI platform. And IDC says most of the rest will do so in the next six months.

All that has AI software suppliers looking pretty smart.

Mixed Signals on GenAI

There’s no question that GenAI is having a huge impact. The question is how difficult it will be for GenAI-using organizations to achieve their desired results.

GenAI use is already widespread. In a global survey conducted earlier this year by management consultants McKinsey & Co., 65% of respondents said they use GenAI on a regular basis.

That was nearly double the percentage from McKinsey’s previous survey, conducted just 10 months earlier.

Also, three quarters of McKinsey’s respondents said they expect GenAI will lead their industries to significant or disruptive changes.

However, the road to GenAI could be bumpy. Separately, researchers at Gartner are predicting that by the end of 2025, at least 30% of all GenAI projects will be abandoned after their proof-of-concept (PoC). 

The reason? Gartner points to several factors: poor data quality, inadequate risk controls, unclear business value, and escalating costs.

“Executives are impatient to see returns on GenAI investments,” says Gartner VP Rita Sallam. “Yet organizations are struggling to prove and realize value.”

One big challenge: Many organizations investing in GenAI want productivity enhancements. But as Gartner points out, those gains can be difficult to quantify.

Further, implementing GenAI is far from cheap. Gartner’s research finds that a typical GenAI deployment costs anywhere from $5 million to $20 million.

That wide range of costs is due to several factors. These include the use cases involved, the deployment approaches used, and whether an organization seeks to be a market disruptor.

Clearly, an intelligent approach to GenAI can be a money-saver.

PCs with AI? Yes, Please

Leading PC makers hope to boost their hardware sales by offering new, built-in AI capabilities. It seems to be working.

In the second quarter of this year, 8.8 million PCs—that’s 14% of all shipped globally in the quarter—were AI-capable, says market analysts Canalys.

Canalys defines “AI-capable” pretty simply: It’s any desktop or notebook system that includes a chipset or block for one or more dedicated AI workloads.

By operating system, nearly 40% of the AI-capable PC shipped in Q2 were Windows systems, 60% were Apple macOS systems, and just 1% ran ChromeOS, Canalys says.

For the full year 2024, Canalys expects some 44 million AI-capable PCs to be shipped worldwide. In 2025, the market watcher predicts, these shipments should more than double, rising to 103 million units worldwide. There's nothing artificial about that boost.

Do more:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

Why Lamini offers LLM tuning software on Supermicro servers powered by AMD processors

Featured content

Why Lamini offers LLM tuning software on Supermicro servers powered by AMD processors

Lamini, provider of an LLM platform for developers, turns to Supermicro’s high-performance servers powered by AMD CPUs and GPUs to run its new Memory Tuning stack.

Learn More about this topic
  • Applications:
  • Featured Technologies:

Generative AI systems powered by large language models (LLMs) have a serious problem: Their answers can be inaccurate—and sometimes, in the case of AI “hallucinations,” even fictional.

For users, the challenge is equally serious: How do you get precise factual accuracy—that is, correct answers with zero hallucinations—while upholding the generalization capabilities that make LLMs so valuable?

A California-based company, Lamini, has come up with an innovative solution. And its software stack runs on Supermicro servers powered by AMD CPUs and GPUs.

Why Hallucinations Happen

Here’s the premise underlying Lamini’s solution: Hallucinations happen because the right answer is clustered with other, incorrect answers. As a result, the model doesn’t know that a nearly right answer is in fact wrong.

To address this issue, Lamini’s Memory Tuning solution teaches the model that getting the answer nearly right is the same as getting it completely wrong. Its software does this by tuning literally millions of expert adapters with precise facts on top of any open-source LLM, such as Llama 3 or Mistral 3.

The Lamini model retrieves only the most relevant experts from an index at inference time. The goal is high accuracy, high speed and low cost.

More than Fine-Tuning

Isn’t this just LLM fine-tuning? Lamini says no, its Memory Tuning is fundamentally different.

Fine-tuning can’t ensure that a model’s answers are faithful to the facts in its training data. By contrast, Lamini says, its solution has been designed to deliver output probabilities that are not just close, but exactly right.

More specifically, Lamini promises its solution can deliver 95% LLM accuracy with 10x fewer hallucinations.

In the real world, Lamini says one large customer used its solution and raised LLM accuracy from 50% to 95%, and reduced the rate of AI hallucinations from an unreliable 50% to just 5%.

Investors are certainly impressed. Earlier this year Lamini raised $25 million from an investment group that included Amplify Partners, Bernard Arnault and AMD Ventures. Lamini plans to use the funding to accelerate its expert AI development and expand its cloud infrastructure.

Supermicro Solution

As part of its push to offer superior LLM tuning, Lamini chose Supermicro’s GPU server — model number AS -8125S-TNMR2 — to train LLM models in a reasonable time.

This Supermicro 8U system is powered by dual AMD EPYC 9000 series CPUs and eight AMD Instinct MI300X GPUs.

The GPUs connect with CPUs via a standard PCIe 5 bus. This gives fast access when the CPU issues commands or sends data from host memory to the GPUs.

Lamini has also benefited from Supermicro’s capacity and quick delivery schedule. With other GPUs makers facing serious capacity issues, that’s an important benefit for both Lamini and its customers.

“We’re thrilled to be working with Supermicro,” says Lamini co-founder and CEO Sharon Zhou.

Could your customers be thrilled by Lamini, too? Check out the “do more” links below.

Do More:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

Why CSPs Need Hyperscaling

Featured content

Why CSPs Need Hyperscaling

Today’s cloud service providers need IT infrastructures that can scale like never before.

Learn More about this topic
  • Applications:
  • Featured Technologies:

Hyperscaling IT infrastructure may be one of the toughest challenges facing cloud service providers (CSPs) today.

The term hyperscale refers to an IT architecture’s ability to scale in response to increased demand.

Hyperscaling is tricky, in large part because demand is a constantly moving target. Without much warning, a data center’s IT demand can increase exponentially due to a myriad of factors.

That could mean a public emergency, the failure of another CSP’s infrastructure, or simply the rampant proliferation of data—a common feature of today’s AI environment.

To meet this growing demand, CSPs have a lot to manage. That includes storage measured in exabytes, AI workloads of massive complexity, and whatever hardware is needed to keep system uptime as close to 100% as possible.

The hardware alone can be a real challenge. CSPs now oversee both air- and liquid-powered cooling systems, redundant power sources, diverse networking gear, and miles of copper and fiber-optic cabling. It’s a real handful.

Design with CSPs in Mind

To help CSPs cope with this seemingly overwhelming complexity, Supermicro offers purpose-built hardware designed to tackle the world’s most demanding workloads.

Enterprise-class servers like Supermicro’s H13 and A+ server series offer CSPs powerful platforms built to handle the rigors of resource-intensive AI workloads. They’ve been designed to scale quickly and efficiently as demand and data inevitably increase.

Take the Supermicro GrandTwin. This innovative solution puts the power and flexibility of multiple independent servers in a single enclosure.

The design helps lower operating expenses by enabling shared resources, including a space-saving 2U enclosure, heavy-duty cooling system, backplane and N+1 power supplies.

To help CSPs tackle the world’s most demanding AI workloads, Supermicro offers GPU server systems. These include a massive—and massively powerful—8U eight-GPU server.

Supermicro H13 GPU servers are powered by 4th-generation AMD EPYC processors. These cutting-edge chips are engineered to help high-end applications perform better and return faster.

To make good on those lofty promises, AMD included more and faster cores, higher bandwidth to GPUs and other devices, and the ability to address vast amounts of memory.

Theory Put to Practice

Capable and reliable hardware is a vital component for every modern CSP, but it’s not the only one. IT infrastructure architects must consider not just their present data center requirements but how to build a bridge to the requirements they’ll face tomorrow.

To help build that bridge, Supermicro offers an invaluable list: 10 essential steps for scaling the CSP data center.

A few highlights include:

  • Standardize and scale: Supermicro suggests CSPs standardize around a preferred configuration that offers the best compute, storage and networking capabilities.
  • Plan ahead for support: To operate a sophisticated data center 24/7 is to embrace the inevitability of technical issues. IT managers can minimize disruption and downtime when some-thing goes wrong by choosing a support partner who can solve problems quickly and efficiently.
  • Simplify your supply chain: Hyperscaling means maintaining the ability to move new infra-structure into place fast and without disruption. CSPs can stack the odds in their favor by choosing a partner that is ever ready to deliver solutions that are integrated, validated, and ready to work on day one.

Do More:

Hyperscaling for CSPs will be the focus of a session at the upcoming Supermicro Open Storage Summit ‘24, which streams live Aug. 13 - Aug. 29.

The CSP session, set for Aug. 20, will cover the ways in which CSPs can seamlessly scale their AI operations across thousands of GPUs while ensuring industry-leading reliability, security and compliance capabilities. The speakers will feature representatives from Supermicro, AMD, Vast Data and Solidigm.

Learn more and register now to attend the 2024 Supermicro Open Storage Summit.

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

You’re invited to attend the Supermicro Open Storage Summit ‘24

Featured content

You’re invited to attend the Supermicro Open Storage Summit ‘24

Join this free online event being held August 13 – 29.

Learn More about this topic
  • Applications:
  • Featured Technologies:

Into storage? Then learn about the latest storage innovations at the Supermicro Open Storage Summit ’24. It’s an online event happening over three weeks, August 13 – 29. And it’s free to attend.

The theme of this year’s summit is “enabling software-defined storage from enterprise to AI.” Sessions are aimed at anyone involved with data storage, whether you’re a CIO, IT support professional, or anything in between.

The Supermicro Open Storage Summit ’24 will bring together executives and technical experts from the entire software-defined storage ecosystem. They’ll talk about the latest developments enabling storage solutions.

Each session will feature Supermicro product experts along with leaders from both hardware and software suppliers. Together, these companies give a boost to the software-defined storage solution ecosystem.

Seven Sessions

This year’s Open Storage Summit will feature seven sessions. They’ll cover topics and use cases that include storage for AI, CXL, storage architectures and much more.

Hosting and moderating duties will be filled by Rob Strechay, managing director and principal analyst at theCUBE Research. His company provides IT leaders with competitive intelligence, market analysis and trend tracking.

All the Storage Summit sessions will start at 10 a.m. PDT / 1 p.m. EDT and run for 45 minutes. All sessions will also be available for on-demand viewing later. But by attending a live session, you’ll be able to participate in the X-powered Q&A with the speakers.

What’s On Tap

What can you expect? To give you an idea, here are a few of the scheduled sessions:

Aug. 14: AI and the Future of Media Storage Workflows: Innovations for the Entertainment Industry

Whether it’s movies, TV, or corporate videos, the post-production process including editing, special effects, coloring, and distribution requires both high-performance and large-capacity solutions. In this session, Supermicro, Quantum, AMD and Western Digital will discuss how primary and secondary storage is optimized for post-production workflows.

Aug. 20: Hyperscale AI: Secure Data Services for CSPs

Cloud services providers must seamlessly scale their AI operations across thousands of GPUs, while ensuring industry-leading reliability, security, and compliance capabilities. Speakers from Supermicro, AMD, VAST Data, and Solidigm will explain how CSPs can deploy AI models at an unprecedented scale with confidence and security.

There’s a whole lot more, too. Learn more about the Supermicro Open Storage Summit ’24 and register to attend now.

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

Tech Explainer: What is multi-tenant storage?

Featured content

Tech Explainer: What is multi-tenant storage?

Similar to the way an apartment building lets tenants share heat, hot water and other services, multitenancy lets users share storage resources for fast development and low costs.

Learn More about this topic
  • Applications:
  • Featured Technologies:

Multi-tenant storage—also referred to as multitenancy—helps organizations develop applications faster and more efficiently.

It does this by enabling multiple users to both share the resources of a centralized storage architecture and customize their storage environments without affecting the others.

You can think of multi-tenant storage as being like an apartment building. The building’s tenants share a common infrastructure and related services, such as heat, hot water and electricity. Yet each tenant can also set up their individual apartment to suit their unique needs.

When it comes to data storage, leveraging a multi-tenant approach also helps lower each user’s overhead costs. It does this by distributing maintenance fees across all users. Also, tenants can share applications, security features and other infrastructure.

Multitenancy for Cloud, SaaS, AI

Chances are, your customers are already using multi-tenant storage architecture to their advantage. Public cloud platforms such as Microsoft Azure, Amazon Web Services and Google Cloud all serve multiple tenants from a shared infrastructure.

Popular SaaS providers including Dropbox also employ multitenancy to offer millions of customers a unique experience based on a common user interface. Each user’s data store is available to them only, despite its being kept in a common data warehouse.

AI-related workloads will become increasingly common in multi-tenant environments, too. That includes the use of large language models (LLMs) to enable Generative AI. Also, certain AI and ML workloads may be more effective in situations in which they feed—and are fed by—multiple tenants.

In addition, all users in a multitenancy environment can contribute data for AI training, which requires enormous quantities of data. And because each tenant creates a unique data set, this process may offer a wider array of training data more efficiently compared to a single source.

What’s more, data flowing in the other direction—from the AI model to each tenant—also increases efficiency. By sharing a common AI application, tenants gain access to a larger, more sophisticated resource than they would with single tenancy.

Choosing the Right Solution

Whether your customers opt for single tenant, multi-tenant or a combination of the two, they must deploy hardware that can withstand rigorous workloads.

Supermicro’s ASG-1115S–NE3X12R storage server is just such a storage solution. This system offers eight front hot-swap E3.S 1T PCIe 5.0 x4 NVMe drive bays; four front fixed E3.S 2T PCIe 5.0 x8 CXL Type 3 drive bays; and two M.2 NVMe slots.

Processing gets handled by a single AMD EPYC 9004-series CPU. It offers up to 128 cores and 6TB of ECC DDR5 main memory.

Considering the Supermicro storage server’s 12 drives, eight heavy-duty fans and 1600W redundant Titanium Level power supply, you might assume that it takes up a lot of rack space. But no. Astonishingly, the entire system is housed in a single 1U chassis.

Do More:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

What you need to know about high-performance storage for media & entertainment

Featured content

What you need to know about high-performance storage for media & entertainment

To store, process and share their terabytes of data, media and entertainment content creators need more than your usual storage.

Learn More about this topic
  • Applications:
  • Featured Technologies:

Maintaining fast, efficient and reliable data storage in the age of modern media and entertainment is an increasingly difficult challenge.

Content creators ranging from independent filmmakers to major studios like Netflix and Amazon are churning out enormous amounts of TV shows, movies, video games, and augmented and virtual reality (AR/VR) experiences. Each piece of content must be stored in a way that ensures it’s easy to access, ready to share and fast enough to stream.

This becomes a monumental task when you’re dealing with petabytes of high-resolution footage and graphics. Operating at that scale can overwhelm even the most seasoned professionals.

Those pros must also ensure they have both primary and secondary storage. Primary storage is designed to deliver rapid data retrieval speeds. Secondary storage, on the other hand, provides slower access times and is used for long-term storage.

Seemingly Insurmountable Odds

For media and entertainment production companies, the goal is always the same: speed production and cut costs. That’s why fast, efficient and reliable data storage solutions have become a vital necessity for those who want to survive and thrive in the modern age of media and entertainment.

The amount of data created in a single media project can be staggering.

Each new project uses one or more cameras producing footage with a resolution as high as 8K. And content captured at 8K has 16 times more pixels per frame than traditional HD video. That translates to around 1 terabyte of data for every 1.5 to 2 hours of footage.

For large-scale productions, shooting can continue for weeks, even months. At roughly a terabyte for every 2 hours of shooting, that footage quickly adds up, creating a major data-storage headache.

But wait, there’s more: Your customer’s projects may also include both AR and VR data. High-quality AR/VR can contain hundreds of effects, textures and 3D models, producing data that measures not just in terabytes but petabytes.

Further complicating matters even more, AR/VR data often requires real-time processing, low-latency transfer and multiuser access.

Deploying AI adds yet another dimension. Generative AI (GenAI) now has the ability to create stunning additions to any multimedia project. These may include animated backgrounds, special effects and even virtual actors.

However, AI accounts for some of the most resource-intensive workloads in the world. To meet these stringent demands, not just any storage solution will do.

Extreme Performance Required

For media and entertainment content creators, choosing the right storage solution can be a make-or-break decision. Production companies that produce the highest rate of data must opt for something like the Supermicro H13 Petascale storage server.

The H13 Petascale storage server boasts extreme performance for data-intensive applications. For major content producers, that means high-resolution media editing, AR and VR creation, special effects and the like.

The H13 Petascale storage server is also designed to handle some of the tech industry’s most demanding workloads. These include AI and machine learning (ML) applications, geophysical modeling and big data.

Supermicro’s H13 Petascale storage server delivers up to 480 terabytes of high-performance storage via 16 hot-swap all-flash drives. The system is based on the Enterprise Data Center Standard Form Factor (EDSFF) E3 form factor NVMe storage to provide high-capacity scaling. The 2U Petascale version has double the storage bays and capacity.

Operating on the EDSFF standard also offers better performance with PCIe 5 connectivity and improved thermal efficiency.

Under the hood of this storage beast is a 4th generation AMD EPYC processor with up to 128 cores and 6TB of DDR5 memory. Combined with 128 lanes of PCIe 5 bandwidth, H13 delivers more than 200GB/sec. of bandwidth and more than 25 million input/output operations per second (IOPS).

Data’s Golden Age

Storing, sending and streaming massive amounts of data will continue to be a challenge for the media and entertainment industry.

Emerging formats will push the boundaries of resolution. New computer-aided graphics systems will become the industry standard. And consumers will continue to demand fully immersive AR and VR experiences.

Each of these evolutions will produce more and more data, forcing content creators to search for faster and more cost-effective storage methods.

Note: The media and entertainment industry will be the focus of a special session at the upcoming Supermicro Open Storage Summit ‘24, streaming live from Aug. 13 to Aug. 29. The M&E session, scheduled for Aug. 14 at 10 a.m. PDT / 1 p.m. EDT, will focus on AI and the future of media storage workflows. The speakers will represent Supermicro, AMD, Quantum and Western Digital. Learn more and register now to attend the 2024 Supermicro Open Storage Summit.

Do more:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

HBM: Your memory solution for AI & HPC

Featured content

HBM: Your memory solution for AI & HPC

High-bandwidth memory shortens the information commute to keep pace with today’s powerful GPUs.

Learn More about this topic
  • Applications:
  • Featured Technologies:

As AI powered by GPUs transforms computing, conventional DDR memory can’t keep up.

The solution? High-bandwidth memory (HBM).

HBM is memory chip technology that essentially shortens the information commute. It does this using ultra-wide communication lanes.

An HBM device contains vertically stacked memory chips. They’re interconnected by microscopic wires known as through-silicon vias, or TSVs for short.

HBM also provides more bandwidth per watt. And, with a smaller footprint, the technology can also save valuable data-center space.

Here’s how: A single HBM stack can contain up to eight DRAM modules, with each module connected by two channels. This makes an HBM implementation of just four chips roughly equivalent to 30 DDR modules, and in a fraction of the space.

All this makes HBM ideal for workloads that utilize AI and machine learning, HPC, advanced graphics and data analytics.

Latest & Greatest

The latest iteration, HBM3, was introduced in 2022, and it’s now finding wide application in market-ready systems.

Compared with the previous version, HBM3 adds several enhancements:

  • Higher bandwidth: Up to 819 GB/sec., up from HBM2’s max of 460 GB/sec.
  • More memory capacity: 24GB per stack, up from HBM2’s 8GB
  • Improved power efficiency: Delivering more data throughput per watt
  • Reduced form factor: Thanks to a more compact design

However, it’s not all sunshine and rainbows. For one, HBM-equipped systems are more expensive than those fitted out with traditional memory solutions.

Also, HBM stacks generate considerable heat. Advanced cooling systems are often needed, adding further complexity and cost.

Compatibility is yet another challenge. Systems must be designed or adapted to HBM3’s unique interface and form factor.

In the Market

As mentioned above, HBM3 is showing up in new products. That very definitely includes both the AMD Instinct MI300A and MI300X series accelerators.

The AMD Instinct MI300A accelerator combines a CPU and GPU for running HPC/AI workloads. It offers HBM3 as the dedicated memory with a unified capacity of up to 128GB.

Similarly, the AMD Instinct MI300X is a GPU-only accelerator designed for low-latency AI processing. It contains HBM3 as the dedicated memory, but with a higher capacity of up to 192GB.

For both of these AMD Instinct MI300 accelerators, the peak theoretical memory bandwidth is a speedy 5.3TB/sec.

The AMD Instinct MI300X is also the main processor in Supermicro’s AS -8125GS-TNMR2, an H13 8U 8-GPU system. This system offers a huge 1.5TB of HBM3 memory in single-server mode, and an even huger 6.144TB at rack scale.

Are your customers running AI with fast GPUs, only to have their systems held back by conventional memory? Tell them to check out HBM.

Do More:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Related Content

Pages