The Channel Company

Sponsored by:

Visit AMD Visit Supermicro

Performance Intensive Computing

Capture the full potential of IT

Research Roundup: Edge, channel sales, insider risk, AI security, wireless LANs

Featured content

Research Roundup: Edge, channel sales, insider risk, AI security, wireless LANs

Catch up on the latest IT market research, surveys and forecasts. 

Learn More about this topic
  • Applications:

Edge computing is strategic. The IT channel is huge. Insider cyber risks deserve more attention. AI can be used to oversee AI. And enterprises are buying more wireless LANs.

That’s the latest from top IT market research. And here’s your Performance Intensive Computing roundup.

All hail the edge

More than 8 in 10 C-level executives (83%) believe that to remain competitive in the future, their organizations will need to implement edge computing.

Nearly as many (81%) believe that if they fail to act quickly on edge computing, they could be locked out from enjoying the technology’s full benefits.

Those figures come from a new study by Accenture. The report is based on a poll, conducted by the consulting firm late last year, of 2,100 C-suite execs—including 250 CEOs—across 18 industries and 16 countries.

There’s plenty of room for progress on the edge, the Accenture poll finds. Just two-thirds (65%) of companies use edge today. And among these adopters, only half have integrated edge into their digital core.

Edge systems can be enhanced with the cloud. Indeed, Accenture finds that nearly 8 in 10 respondents (79%) say they’ll fully integrate edge with cloud in the next three years.

Channel rules

How important is the IT channel? Very, according to market watcher Canalys.

Canalys expects that this year, partner-delivered IT technologies and services worldwide will total more than $3.4 trillion, or about 70% of the global addressable IT market.

And the market is rising, despite ongoing economic issues. Canalys predicts the worldwide IT market will rise 3.5% this year, for a full-year total of $4.7 trillion.

Some of the biggest growth opportunities this year coming in cybersecurity (with sales forecast to rise 11%), network infrastructure (14%) and public cloud (7.5%), according to Canalys.

There are also big implications for the IT hardware, software and services suppliers that rely on the channel.

“Given the importance of the channel,” says Canalys chief analyst Matthew Ball, “the success of vendors will increasingly rely on their resell, co-sell, co-marketing, co-retention, co-development and co-innovation strategies.”

Insider risk rising

Here’s a new reason to worry: The average annual cost of an insider cyber risk has risen 40% over the last 4 years, reaching $16.2 million. And the average amount of time it takes to contain an insider incident is now a about 3 months (86 days).

That’s according to a new study conducted by the Ponemon Institute on behalf of Dtex Systems, a supplier of risk-management software. Their new joint report is based on a recent survey of 1,075 security and line-of-business professionals at nearly 310 organizations worldwide.

Despite this risk, the survey finds that most organizations are dedicating only about 8% of their overall cybersec budget—the equivalent of $200 per employee—to insider threats.

What’s more, about 90% of the insider-risk budget gets spent after an insider incident has occurred, the survey found. These after-incident costs include containment, remediation, investigation, incident response and escalation.

AI vs. AI?

AI-powered risks may be so stealthy, only another AI system can fight them off.

That’s the sentiment revealed by a new Gartner survey. The research firm finds that about 1 in 3 organizations (34%) now use AI application security tools to mitigate the risks of generative AI. Over half (56%) are exploring such approaches for the future.

These numbers come from Gartner’s most recent Peer Community survey, conducted in April. Gartner collected responses from 150 IT and cybersecurity leaders at organizations that use either GenAI or foundational models.

When asked which risks of GenAI worry them the most, nearly 6 in 10 respondents (57%) said leaked secrets in AI-generated code. About the same number (58%) said they’re concerned about AI generating incorrect or biased outputs.

“Organizations that don’t manage AI risk will witness their models not performing as intended,” says Gartner analyst Avivah Litan. “In the worst case [AI] can cause human or property damage.”

Enterprise wireless LAN heats up

Looking for a new growth market? Consider the enterprise segment of wireless local area networking. In this year’s second quarter, sales in this sector grew 43%, reaching a total of $3 billion, according to market intelligence firm IDC.

The growth rate was even higher in both the United States and Canada. In both countries, Q2 sales of wireless LANs to enterprises rose nearly 80% year-on-year, IDC says.

By contrast, the consumer end of the wireless LAN market declined by 14% year-on-year in Q2, according to IDC.

Driving the enterprise sales are a couple of factors, including an easing of both components shortages and supply-chain disruptions, says IDC researcher Brandon Butler. Another growth factor is the rapid adoption by enterprises of the new Wi-Fi 6 and 6E standards.

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Tech Explainer: What is the intelligent edge? Part 1

Featured content

Tech Explainer: What is the intelligent edge? Part 1

The intelligent edge moves compute, storage and networking capabilities close to end devices, where the data is being generated. Organizations gain the ability to process and act on that data in real time, and without having to first transfer that data to the a centralized data center.

Learn More about this topic
  • Applications:
  • Featured Technologies:

The term intelligent edge refers to remote server infrastructures that can collect, process and act on data autonomously. In effect, it’s a small, remote data center.

Compared with a more traditional data center, the intelligent edge offers one big advantage: It locates compute, storage and networking capabilities close to the organization’s data collection endpoints. This architecture speeds data transactions. It also makes them more secure.

The approach is not entirely new. Deploying an edge infrastructure has long been an effective way to gather data in remote locations. What’s new with an intelligent edge is that you gain the ability to process and act on that data (if necessary) in real time—without having to first transfer that data to the cloud.

The intelligent edge can also save an organization money. Leveraging the intelligent edge makes sense for organizations that spend a decent chunk of their operating budget transferring data from the edge to public and private data centers, which could be a cloud infrastructure (often referred to as “the core”). Reducing bandwidth in both directions and storage charges helps them control costs.

3 steps to the edge

Today, an intelligent edge typically gets applied in one of three areas:

  • Operational Technology (OT): Hardware and software used to monitor and control industrial equipment, processes and events.
  • Information Technology (IT): Digital infrastructure—including servers, storage, networking and other devices—used to create, process, store, secure and transfer data.
  • Internet of Things (IoT): A network of smart devices that communicate and can be controlled via the internet. Examples include smart speakers, wearables, autonomous vehicles and smart-city infrastructure.

The highly efficient edge

There’s yet another benefit to deploying intelligent edge tech: It can help an organization become more efficient.

One way the intelligent edge does this is by obviating the need to transfer large amounts of data. Instead, data is stored and processed close to where it’s collected.

For example, a smart lightbulb or fridge can communicate with the intelligent edge instead of contacting a data center. Staying in constant contact with the core is unnecessary for devices that don’t change much from minute to minute.

Another way the intelligent edge boosts efficiency is by reducing the time needed to analyze and act on vital information. This, in turn, can lead to enhanced business intelligence that informs and empowers stakeholders. It all gets done faster and more efficiently than with traditional IT architectures and operations.

For instance, imagine that an organization serves a large customer base from several locations. By deploying an intelligent edge infrastructure, the organization could collect and analyze customer data in real time.

Businesses that gain insights from the edge instead of from the core can also respond quickly to market changes. For example, an energy company could analyze power consumption and weather conditions at the edge (down to the neighborhood), then determine whether there's be a power outage.

Similarly, a retailer could use the intelligent edge to support inventory management and analyze customers’ shopping habits. Using that data, the retailer could then offer customized promotions to particular customers, or groups of customers, all in real time.

The intelligent edge can also be used to enhance public infrastructure. For instance, smart cities can gather data that helps inform lighting, public safety, maintenance and other vital services, which could then be used for preventive maintenance or the allocation of city resources and services as needed.

Edge intelligence

As artificial intelligence (AI) becomes increasingly ubiquitous, many organizations are deploying machine learning (ML) models at the edge to help analyze data and deliver insights in real time.

In one use case, running AI and ML systems at the edge can help an organization reduce the service interruptions that often come with transferring large data sets to and from the cloud. Intelligent Edge is able to keep things running locally, giving distant data centers a chance to catch up. This, in turn, can help the organization provide a better experience for the employees and customers who rely on that data.

Deploying AI at the edge can also help with privacy, security and compliance issues. Transferring data to and from the core presents an opportunity for hackers to intercept data in transit. Eliminating this data transfer deprives cyber criminals of a threat vector they could otherwise exploit.

Part 2 of this two-part blog series dives deep into the biggest, most popular use of the intelligent edge today—namely, the internet of things (IoT). We also look at the technology that powers the intelligent edge, as well as what the future may hold for this emerging technology.

Do more:

 

 

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Research Roundup: AI chip sales, AI data centers, sustainability services, manufacturing clouds, tech-savvy or not

Featured content

Research Roundup: AI chip sales, AI data centers, sustainability services, manufacturing clouds, tech-savvy or not

Learn More about this topic
  • Applications:

Sales of AI semiconductors are poised for big growth. AI is transforming the data center. Sustainability services are hot. Manufacturers are saving big money with cloud. And Americans are surprisingly lacking in tech understanding.

That’s some of the latest IT market research. And here’s your Performance Intensive Computing roundup.

AI chip sales to rise 21% this year

Sales of semiconductors designed to execute AI workloads will rise this calendar year by 20.9% over last year, reaching a worldwide total of $53.4 billion, predicts research firm Gartner.

Looking further ahead, Gartner expects worldwide sales of AI chips in 2024 to reach $67.1 billion, a 25% increase over the projected figure for this year.

And by 2027, Gartner forecasts, those sales will top $119 billion, or more than double this year’s market size.

What’s behind the rapid rise? Two main factors, says Gartner: Generative AI, and the spread of AI-based applications in data centers, edge infrastructure and endpoint devices.

AI transforming data centers

Generative AI is transforming the data center, says Lucas Beran, a market analyst with Dell’Oro Group. Last month, his research group predicted that AI infrastructure spending will propel the data center CapEx to over a half-trillion dollars by 2027, an increase of 15%. (That figure is larger than Gartner’s because it includes more than just chips.) Now Dell’Oro says AI is ushering in a new era for data center physical infrastructure.

Here’s some of what Beran of Dell’Oro expects:

  • Due the substantial power consumption of AI systems, end users will adopt intelligent rack power distribution units (PDUs) that can remotely monitor and manage power consumption and environmental factors.
  • Liquid cooling will come into its own. Some users will retrofit existing cooling systems with closed-loop assisted liquid cooling systems. These use liquid to capture heat generated inside the rack or server, then blow it into a hot aisle. By 2025, global sales of liquid cooling systems will approach $2 billion.
  • A lack of power availability could slow AI adoption. Data centers need more energy than utilities can supply. One possible solution: BYOP – bring your own power.

Sustainability services: $65B by 2027

Speaking of power and liquid cooling, a new forecast from market researcher IDC has total sales of environmental, social and governance (ESG) services rising from $37.7 billion this year to nearly $65 billion by 2027, for a compound annual growth rate (CAGR) of nearly 15%.

For its forecast, IDC looked at ESG services that include consulting, implementation, engineering and IT services.

These services include ESG strategy development and implementation, sustainable operations consulting, reporting services, circularity consulting, green IT implementation services, and managed sustainability performance services. What they all share is the common goal of driving sustainability-related outcomes.

Last year, nearly two-thirds of respondents surveyed by IDC said they plan to allocate more than half their professional-services spending on sustainability services. Looking ahead, IDC expects that to rise to 60% by 2027.

"Pressure for [ESG] change is more prescient than ever,” says IDC research analyst Dan Versace. “Businesses that fail to act face risk to their brand image, financial performance, and even their infrastructure due to the ever-present threat of extreme weather events and resource shortages caused by climate change.”

Manufacturers finally see the cloud

For manufacturers, IT is especially complicated. Unlike banks and other purely digital businesses, manufacturers have to tie IT systems and networks with physical plants and supply chains.

That’s one reason why manufacturers have been comparatively slow to adopt cloud computing. Now that’s changing. In part, as a new report from ABI Research points out, because manufacturers that switch to cloud-based systems can enjoy up to 60% reductions in overhead costs relating to data storage, says James Iversen, an ABI industry analyst.

Iversen predicts that industrial cloud platform revenue in manufacturing will enjoy a nearly 23% CAGR for the coming decade.

Another benefit for manufacturers: The cloud can eliminate the data fragmentation common with external data warehouses. “Cloud manufacturing providers are eliminating these concerns by interconnecting applications bi-directionally,” Iversen says, “leading to sharing and communication between applications and their data.”

How tech-savvy are your customers?

If they’re like most Americans, not very.

A Pew Research Center poll of about 5,100 U.S. adults, conducted this past spring and just made public, found that fewer than a third (32%) knew that large language models such as ChatGPT produce answers from data already published on the internet.

Similarly, only about one in five (21%) knew that U.S. websites are prohibited from collecting data on minors under the age of 13.

Fewer than half of those polled (42%) knew what a deepfake is. And only a similar minority (48%) could identify an example of two-factor authentication.

What tech info do they know? Well, 80% of respondents correctly identified Elon Musk as the boss of Tesla and Twitter (now X). And nearly as many (77%) knew that Facebook had changed its name to Meta.

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Tech Explainer: What’s the difference between Machine Learning and Deep Learning? Part 2

Featured content

Tech Explainer: What’s the difference between Machine Learning and Deep Learning? Part 2

In Part 1 of this 2-part Tech Explainer, we explored the difference between how machine learning and deep learning models are trained and deployed. Now, in Part 2, we’ll get deeper into deep learning to discover how this advanced form of AI is changing the way we work, learn and create.

Learn More about this topic
  • Applications:
  • Featured Technologies:

Where Machine Learning is designed to reduce the need for human intervention, Deep Learning—an extension of ML—removes much of the human element altogether.

If ML were a driver-assistance feature that helped you parallel park and avoid collisions, DL would be an autonomous, self-driving car.

The human intervention we’re talking about has much to do with categorizing and labeling the data used by ML models. Producing this structured data is both time-consuming and expensive.

DL shortens the time and lowers the cost by learning from unstructured data. This elimnates much of the data pre-processing performed by humans for ML.

That’s good news for modern businesses. Market watcher IDC estimates that as much as 90% of corporate data is associated with unstructured data.

DL is particularly good at processing unstructured data. That includes information coming from the edge, the core and millions of both personal and IoT devices.

Like a brain, but digital

Deep Learning systems “think” with a neural network—multiple layers of interconnected nodes designed to mimic the way the human brain works. A DL system processes data inputs in an attempt to recognize, classify and accurately describe objects within data.

The layers of a neural network are stacked vertically. Each layer builds on the work performed by the one below it. By pushing data through each successive layer, the overall system improves its predictions and categorizations.

For instance, imagine you’ve tasked a DL system to identify pictures of junk food. The system would quickly learn—on its own—how to differentiate Pringles from Doritos.

It might do this by learning to recognize Pringles’ iconic tubular packaging. Then the system would categorize Pringles differently than the family-size sack of Doritos.

What if you fed this hypothetical DL system with more pictures of chips? Then it could begin to identify varying angles of packaging, as well as colors, logos, shapes and granular aspects of the chips themselves.

As this example illustrates, the longer a DL system operates, the more intelligent and accurate it becomes.

Things we used to do

DL tends to be deployed when it’s time to pull out the big guns. This isn’t tech you throw at a mere spam filter or recommendation engine.

Instead, it’s the tech that powers the world’s finance, biomedical advances and law enforcement. For these verticals, failure is simply not an option.

For these verticals, here are some of the ways DL operates behind the scenes:

  • BioMed: DL helps healthcare staff analyze medical imaging such as X-rays and CT scans. In many cases, the technology is more accurate than well-trained physicians with decades of experience.
  • Finance: For those seeking a market edge (read: everyone), DL employs powerful, algorithmic-based predictive analytics. This helps modern-day robber barons manage their portfolios based on insights from data so vast, they couldn’t leverage it themselves. DL also helps financial institutions assess loans, detect fraud and manage credit.
  • Law Enforcement: In the 2002 movie “Minority Report,” Tom Cruise played a police officer who could arrest people before they committed a crime. With DL, this fiction could turn into an unsettling reality. DL can be used to analyze millions of data points, then predict who is most likely to break the law. It might even give authorities an idea of where, when and how it could happen.

The future…?

Looking into a crystal ball—which these days probably uses DL—we can see a long succession of similar technologies coming. Just as ML begat DL, so too will DL beget the next form of AI—and the one after that.

The future of DL isn’t a question of if, but when. Clearly, DL will be used to advance a growing number of industries. But just when each sector will come to be ruled by our new smarty-pants robots is less clear.

Keep in mind: Even as you read this, DL systems are working tirelessly to help data scientists make AI more accurate and able to provide more useful assessments of datasets for specific outcomes. And as the science progresses, neural networks will continue to become more complex—and more like human brains.

That means the next generation of DL will likely be far more capable than the current one. Future AI systems could figure out how to reverse the aging process, map distant galaxies, even produce bespoke food based on biometric feedback from hungry diners.

For example, the upcoming AMD Instinct MI300 accelerators promise to usher in a new era of computing capabilities. That includes the ability to handle large language models (LLMs), the key approach behind generative AI systems such as ChatGPT.

Yes, the robots are here, and they want to feed you custom Pringles. Bon appétit!

 

Do more:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Tech Explainer: What’s the difference between Machine Learning and Deep Learning? Part 1

Featured content

Tech Explainer: What’s the difference between Machine Learning and Deep Learning? Part 1

What’s the difference between machine learning and deep learning? That’s the subject of this 2-part Tech Explainer. Here, in Part 1, learn more about ML. 

Learn More about this topic
  • Applications:
  • Featured Technologies:

As the names imply, machine learning and deep learning are types of smart software that can learn. Perhaps not the way a human does. But close enough.

What’s the difference between machine and deep learning? That’s the subject of this 2-part Tech Explainer. Here in Part 1, we’ll look in depth at machine learning. Then in Part 2, we’ll look more closely at deep learning.

Both, of course, are subsets of artificial intelligence (AI). To understand their differences, it helps to first understand something of the AI hierarchy.

At the very top is overarching AI technology. It powers both popular generative AI models such as ChatGPT and less famous but equally helpful systems such as the suggestion engine that tells you which show to watch next on Netflix.

Machine learning is a subset of AI. It can perform specific tasks without first needing explicit instructions.

As for deep learning, it’s actually a subset of machine learning. DL is powered by so-called neural networks, multiple node layers that form a system inspired by the structure of the human brain.

Machine learning for smarties

Machine learning is defined as the use and development of computer systems designed to learn and adapt without following explicit instructions.

Instead of requiring human input, ML systems use algorithms and statistical models to analyze and draw inferences from patterns they find in large data sets.

This form of AI is especially good at identifying patterns from structured data. Then it can analyze those patterns to make predictions, usually reliable.

For example, let’s say an organization wants to predict when a particular customer will unsubscribe from its service. The organization could use ML to make an educated guess based on previous data about customer churn.

The machinery of ML

Like all forms of AI, machine learning uses lots of compute and storage resources. Enterprise-scale ML models are powered by data centers packed to the gills with cutting-edge tech. The most vital of these components are GPUs and AI data-center accelerators.

GPUs, though initially designed to process graphics, have become the preferred tool for AI development. They offer high core counts—sometimes numbering in the thousands—as well as massive parallel processes. That makes them ideally suited to process a vast number of simple calculations simultaneously.

As AI gained acceptance, IT managers sought ever more powerful GPUs. The logical conclusion was the advent of new technologies like AMD’s Instinct MI200 Series accelerators. These purpose-built GPUs have been designed to power discoveries in mainstream servers and supercomputers, including some of the largest exascale systems in use today.

AMD’s forthcoming Instinct MI300X will go one step further, combining a GPU and AMD EPYC CPU in a single component. It’s set to ship later this year.

State-of-the-art CPUs are important for ML-optimized systems. The CPUs need as many cores as possible, running at high frequencies to keep the GPU busy. AMD’s EPYC 9004 Series processors excel at this.

In addition, the CPUs need to run other tasks and threads of the application. When looking at a full system, PCIe 5.0 connectivity and DDR4 memory are important, too.

The GPUs that power AI are often installed in integrated servers that have the capacity to house their constituent components, including processors, flash storage, networking tech and cooling systems.

One such monster server is the Supermicro AS -4125GS-TNRT. It brings together eight direct attached, double-width, full-length GPUs; up to 6TB of RAM; and two dozen 2.5-inch solid-state drives (SSDs). This server also supports the AMD Instinct MI210 accelerator.

ML vs. DL

The difference between machine learning and deep learning begins with their all-important training methods. ML is trained using four primary methods: supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning.

Deep learning, on the other hand, requires more complex training methods. These include convolutional neural networks, recurrent neural networks, generative adversarial networks and autoencoders.

When it comes to performing real-world tasks, ML and DL offer different core competencies. For instance, ML is the type of AI behind the most effective spam filters, like those used by Google and Yahoo. Its ability to adapt to varying conditions allows ML to generate new rules based on previous operations. This functionality helps it keep pace with highly motivated spammers and cybercriminals.

More complex inferencing tasks like medical imaging recognition are powered by deep learning. DL models can capture intricate relationships within medical images, even when those relationships are nonlinear or difficult to define. In other words, deep learning can quickly and accurately identify abnormalities not visible to the human eye.

Up next: a Deep Learning deep dive

In Part 2, we’ll explore more about deep learning. You’ll find out how data scientists develop new models, how various verticals leverage DL, and what the future holds for this emerging technology.

Do more:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

What’s inside Supermicro’s new Petascale storage servers?

Featured content

What’s inside Supermicro’s new Petascale storage servers?

Supermicro has a new class of storage servers that support E3.S Gen 5 NVMe drives. They offer up to 256TB of high-throughput, low-latency storage in a 1U enclosure, and up to half a petabyte in a 2U.

Learn More about this topic
  • Applications:
  • Featured Technologies:

Supermicro has introduced a new class of storage servers that support E3.S Gen 5 NVMe drives. These storage servers offer up to 256TB of high-throughput, low-latency storage in a 1U enclosure, and up to half a petabyte in a 2U.

Supermicro has designed these storage servers to be used with large AI training and HPC clusters. Those workloads require that unstructured data, often in extremely large quantities, be delivered quickly to the system’s CPUs and GPUs.

To do this, Supermicro has developed a symmetrical architecture that reduces latency. It does so in 2 ways. One, by ensuring that data travels the shortest possible signal path. And two, by providing the maximum airflow over critical components, allowing them to run as fast and cool as possible.

1U and 2U for you 

Supermicro’s new lineup of optimized storage systems includes 1U servers that support up to 16 hot-swap E3.S drives. An alternate configuration could be up to eight E3.S drives, plus four E3.S 2T 16.8mm bays for CMM and other emerging modular devices.

(CMM is short for Chassis Management Module. These devices provide management and control of the chassis, including basic system health, inventory information and basic recovery operations.)

The E3.S form factor calls for a short and thin NVMe SSD drive that is 76mm high, 112.75mm long, and 7.5mm thick.

In the 2U configuration, Supermicro’s servers support up to 32 hot-swap E3.S drives. A single-processor system, it support the latest 4th Gen AMD EPYC processors.

Put it all together, and you can have a standard rack that stores up to an impressive 20 petabytes of data for high-throughput NVMe over fabrics (NVMe-oF) configurations.

30TB drives coming

When new 30TB drives become available—a move expected later this year—the new Supermicro storage servers will be able to handle them. Those drives will bring the storage total to 1 petabyte in a compact 2U server.

Two storage-drive vendors working closely with Supermicro are Kioxia America and Solidigm, both of which make E3.S solid-state drives (SSDs). Kioxia has announced a 30.72TB SSD called the Kioxia CD8P Series. And Solidigm says its D5-P5336 SSD will ship in an E3.S form factor with up to 30.72TB in the first half of 2024.

The new Supermicro Petascale storage servers are shipping now in volume worldwide.

Learn more about the Supermicro E3.S Petascale All-Flash NVMe Storage Systems.

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Can liquid-cooled servers help your customers?

Featured content

Can liquid-cooled servers help your customers?

Liquid cooling can offer big advantages over air cooling. According to a new Supermicro solution guide, these benefits include up to 92% lower electricity costs for a server’s cooling infrastructure, and up to 51% lower electricity costs for an entire data center.

Learn More about this topic
  • Applications:
  • Featured Technologies:

The previous thinking was that liquid cooling was only for supercomputers and high-end gaming PCs. No more.

Today, many large-scale cloud, HPC, analytics and AI servers combine CPUs and GPUs in a single enclosure, generating a lot of heat. Liquid cooling can carry away the heat that’s generated, often with less overall cost and more efficiently than air.

According to a new Supermicro solution guide, liquid’s advantages over air cooling include:

  • Up to 92% lower electricity costs for a server’s cooling infrastructure
  • Up to 51% lower electricity costs for the entire data center
  • Up to 55% less data center server noise

What’s more, the latest liquid cooling systems are turnkey solutions that support the highest GPU and CPU densities. They’re also fully validated and tested by Supermicro under demanding workloads that stress the server. And unlike some other components, they’re ready to ship to you and your customers quickly, often in mere weeks.

What are the liquid-cooling components?

Liquid cooling starts with a cooling distribution unit (CDU). It incorporates two modules: a pump that circulates the liquid coolant, and a power supply.

Liquid coolant travels from the CDU through flexible hoses to the cooling system’s next major component, the coolant distribution manifold (CDM). It’s a unit with distribution hoses to each of the servers.

There are 2 types of CDMs. A vertical manifold is placed on the rear of the rack, is directly connected via hoses to the CDU, and delivers coolant to another important component, the cold plates. The second type, a horizontal manifold, is placed on the front of the rack, between two servers; it’s used with systems that have inlet hoses on the front.

The cold plates, mentioned above, are placed on top of the CPUs and GPUs in place of their typical heat sinks. With coolant flowing through their channels, they keep these components cool.

Two valuable CDU features are offered by Supermicro. First, the company’s CDU has a cooling capacity of 100kW, which enables very high rack compute densities. Second, Supermicro’s CDU features a touchscreen for monitoring and controlling the rack operation via a web interface. It’s also integrated with the company’s Super Cloud Composer data-center management software.

What does it work on?

Supermicro offers several liquid-cooling configurations to support different numbers of servers in different size racks.

Among the Supermicro servers available for liquid cooling is the company’s GPU systems, which can combine up to eight Nvidia GPUs and AMD EPYC 9004 series CPUs. Direct-to-chip (D2C) coolers are mounted on each processor, then routed through the manifolds to the CDU. 

D2C cooling is also a feature of the Supermicro SuperBlade. This system supports up to 20 blade servers, which can be powered by the latest AMD EPYC CPUs in an 8U chassis. In addition, the Supermicro Liquid Cooling solution is ideal for high-end AI servers such as the company’s 8-GPU 8125GS-TNHR.

To manage it all, Supermicro also offers its SuperCloud Composer’s Liquid Cooling Consult Module (LCCM). This tool collects information on the physical assets and sensor data from the CDU, including pressure, humidity, and pump and valve status.

This data is presented in real time, enabling users to monitor the operating efficiency of their liquid-cooled racks. Users can also employ SuperCloud Composer to set up alerts, manage firmware updates, and more.

Do more:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Meet Supermicro’s Petascale Storage, a compact rackmount system powered by the latest AMD EPYC processors

Featured content

Meet Supermicro’s Petascale Storage, a compact rackmount system powered by the latest AMD EPYC processors

Supermicro’s H13 Petascale Storage Systems is a compact 1U rackmount system powered by the AMD EPYC 97X4 processor (formerly codenamed Bergamo) with up to 128 cores.

 

 

Learn More about this topic
  • Applications:
  • Featured Technologies:

Your customers can now implement Supermicro Petascale Storage, an all-Flash NVMe storage system powered by the latest 4th gen AMD EPYC 9004 series processors.

The Supermicro system has been specifically designed for AI, HPC, private and hybrid cloud, in-memory computing and software-defined storage.

Now Supermicro is offering the first of these systems. It's the Supermicro H13 Petascale Storage System. This compact 1U rackmount system is powered by an AMD EPYC 97X4 processor (formerly codenamed Bergamo) with up to 128 cores.

For organizations with data-storage requirements approaching petascale capacity, the Supermicro system was designed with a new chassis and motherboard that support a single AMD EPYC processor, 24 DIMM slots for up to 6TB of main memory, and 16 hot-swap ES.3 slots. That's the Enterprise and Datacenter Standard Form Factor (EDSFF), part of the E3 family of SSD form factors designed for specific use cases. ES.3 is short and thin. It uses 25W and 7.5mm-wide storage media designed with a PCIe 5.0 interface.

The Supermicro Petascale Storage system can deliver more than 200 GB/sec. bandwidth and over 25 million input-output operations per second (IOPS) from a half-petabyte of storage.

Here's why 

Why might your customers need such a storage system? Several reasons, depending on what sorts of workloads they run:

  •  Training AI/ML applications requires massive amounts of data for creating reliable models.
  • HPC projects use and generate immense amounts of data, too. That's needed for real-world simulations, such as predicting the weather or simulating a car crash.
  • Big-data environments need susbstantial datasets. These gain intelligence from real-world observations ranging from sensor inputs to business transactions.
  • Enterprise applications need to locate large amounts of data close to computing over NVMe-over-Fabrics (NVMeoF) speeds.

Also, the Supermicro H13 Petascale Storage System offers significant performance, capacity, throughput and endurance--all while keeping excellent power efficiencies.

Do more:

Featured videos

Events

Find AMD & Supermicro Elsewhere

Interview: How German system integrator SVA serves high performance computing with AMD and Supermicro

Featured content

Interview: How German system integrator SVA serves high performance computing with AMD and Supermicro

In an interview, Bernhard Homoelle, head of the HPC competence center at German system integrator SVA, explains how his company serves customers with help from AMD and Supermicro. 

Learn More about this topic
  • Applications:
  • Featured Technologies:
  • Featured Companies:
  • SVA System Vertrieb Alexander GmbH

SVA System Vertrieb Alexander GmbH, better known as SVA, is among the leading IT system integrators of Germany. Headquartered in Wiesbaden, the company employs more than 2,700 people in 27 branch offices. SVA’s customers include organizations in automotive, financial services and healthcare.

To learn more about how SVA works jointly with Supermicro and AMD on advanced technologies, PIC managing editor Peter Krass spoke recently with Bernhard Homoelle, head of SVA’s high performance computing (HPC) competence center (pictured above). Their interview has been lightly edited.

For readers outside of Germany, please tell us about SVA?

First of all, SVA is an owner-operated system integrator. We offer high-quality products, we sell infrastructure, we support certain types of implementations, and we offer operational support to help our customers achieve optimum solutions.

We work with partners to figure out what might be the best solution for our customers, rather than just picking one vendor and trying to convince the customer they should use them. Instead, we figure out what is really needed. Then we go in the direction where the customer can really have their requirements met. The result is a good relationship with the customer, even after a particular deal has been closed.

Does SVA focus on specific industries?

While we do support almost all the big industries—automotive, transportation, public sector, healthcare and more—we are not restricted to any specific vertical. Our main business is helping customers solve their daily IT problems, deal with the complexity of new IT systems, and implement new things like AI and even quantum computing. So we’re open to new solutions. We also offer training with some of our partners.

Germany has a robust auto industry. How do you work with these clients?

In general, they need huge HPC clusters and machine learning. For example, autonomous driving demands not only more computing power, but also more storage. We’re talking about petabytes of data, rather than terabytes. And this huge amount of data needs to be stored somewhere and finally processed. That puts pressure on the infrastructure—not just on storage, but also on the network infrastructure as well as on the compute side. For their way into cloud, some these customers are saying, “Okay, offer me HPC as a Service.”

How do you work with AMD and Supermicro?

It’s a really good relationship. We like working with them because Supermicro has all these various types of servers for individual needs. Customers are different, and therefore they have their own requirements. Figuring out what might be the best server for them is difficult if you have limited types of servers available. But with Supermicro, you can get what you have in mind. You don’t have to look for special implementations because they have these already at hand.

We’re also partnering with AMD, and we have access to their benchmark labs, so we can get very helpful information. We start with discussions with the customer to figure out their needs. Typically, we pick up an application from the customer and then use it as a kind of benchmark. Next, we put it on a cluster with different memory, different CPUs, and look for the best solution in terms of performance for their particular application. Based on the findings, we can recommend a specific CPU, number of cores, memory type and size, and more.

With HPC applications, core memory bandwidth is almost as important as the number of cores. AMD’s new Genoa-X processors should help to overcome some of these limitations. And looking ahead, I’m keen to see what AMD will offer with the Instinct MI300.

Are there special customer challenges you’re solving with Supermicro and AMD solutions?

With HPC workloads, our academic customers say, “This is the amount of money available, so how many servers can you really give us for this budget?” Supermicro and AMD really help here with reasonable prices. They’re a good choice for price/performance.

With AI and machine learning, the real issue is software tools. It really depends what kinds of models you can use and how easy it is to use the hardware with those models.

This discussion is not easy, because for many of our customers today, AI means Nvidia. But I really recommend alternatives, and AMD is bringing some alternatives that are great. They offer a fast time to solution, but they also need to be easy to switch to.

How about "green" computing? Is this an important issue for your customers now?

Yes, more and more we’re seeing customers ask for this green computing approach. Typically, a customer has a thermal budget and a power-price budget. They may say, “In five years, the expenses paid for power should not exceed a certain limit.”

In Europe, we also have a supply-chain discussion. Vendors must increasingly provide proof that they’re taking care in their supply chain with issues including child labor and working conditions. This is almost mandatory, especially in government calls. If you’re unable to answer these questions, you’re out of the bid.

With green computing, we see that the power needed for CPUs and GPUs is going up and up. Five years ago, the maximum a CPU could burn was 200W, but now even 400W might not be enough. Some GPUs are as high as 700W, and there are super-chips beyond even that.

All this makes it difficult to use air-cooled systems. Customers can use air conditioning to a certain extent, but there’s only so much air you can press through the rack. Then you need either on-chip water cooling or some kind of immersion cooling. This can help in two dimensions: saving energy and getting density — you can put the components closer together, and you don’t need the big heat sink anymore.

One issue now is that each vendor offers a different cooling infrastructure. Some of our customers run multi-vendor data centers, so this could create a compatibility issue. That’s one reason we’re looking into immersion cooling. We think we could do some of our first customer implementations in 2024.

Looking ahead, what do you see as a big challenge?

One area is that we want to help customers get easier access to their HPC clusters. That’s done on the software side.

In contrast to classic HPC users, machine learning and AI engineers are not that interested in Linux stuff, compiler options or any other infrastructure details. Instead, they’d like to work on their frameworks. The challenge is getting them to their work as easily as possible—so that they can just log in, and they’re in their development environment. That way, they won’t have to care about what sort of operating system is underneath or what kind of scheduler, etc., is running.

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

How AMD and Supermicro are working together to help you deliver AI

Featured content

How AMD and Supermicro are working together to help you deliver AI

AMD and Supermicro are jointly offering high-performance AI alternatives with superior price and performance.

Learn More about this topic
  • Applications:
  • Featured Technologies:

When it comes to building AI systems for your customers, a certain GPU provider with a trillion-dollar valuation isn’t the only game in town. You should also consider the dynamic duo of AMD and Supermicro, which are jointly offering high-performance AI alternatives with superior price and performance.

Supermicro’s Universal GPU systems are designed specifically for large-scale AI and high-performance computing (HPC) applications. Some of these modular designs come equipped with AMD’s Instinct MI250 Accelerator and have the option of being powered by dual AMD EPYC processors.

AMD, with a newly formed AI group led by Victor Peng, is working hard to enable AI across many environments. The company has developed an open software stack for AI, and it has also expanded its partnerships with AI software and framework suppliers that now include the PyTorch Foundation and Hugging Face.

AI accelerators

In addition, AMD’s Instinct MI300A data-center accelerator is due to ship in this year’s fourth quarter. It’s the successor to AMD’s MI200 series, based on the company’s CDNA 2 architecture and first multi-die CPU, which powers some of today’s fastest supercomputers.

The forthcoming Instinct MI300A is based on AMD’s CDNA 3 architecture for AI and HPC workloads, which uses 5nm and 6nm process tech and advanced chiplet packaging. Under the MI300A’s hood, you’ll find 24 processor cores with Zen 4 tech, as well as 128GB of HBM3 memory that’s shared by the CPU and GPU. And it supports AMD ROCm 5, a production-ready, open source HPC and AI software stack.

Earlier this month, AMD introduced another member of the series, the AMD Instinct MI300X. It replaces three Zen 4 CPU chiplets with two CDNA 3 chiplets to create a GPU-only system. Announced at AMD’s recent Data Center and AI Technology Premier event, the MI300X is optimized for large language models (LLMs) and other forms of AI.

To accommodate the demanding memory needs of generative AI workloads, the new AMD Instinct MI300X also adds 64GB of HBM3 memory, for a new total of 192GB. This means the system can run large models directly in memory, reducing the number of GPUs needed, speeding performance, and reducing the user’s total cost of ownership (TCO).

AMD also recently introduced the AMD Instinct Platform, which puts eight MI300X systems and 1.5TB of memory in a standard Open Compute Project (OCP) infrastructure. It’s designed to drop into an end user’s current IT infrastructure with only minimal changes.

All this is coming soon. The AMD MI300A started sampling with select customers earlier this quarter. The MI300X and Instinct Platform are both set to begin sampling in the third quarter. Production of the hardware products is expected to ramp in the fourth quarter.

KT’s cloud

All that may sound good in theory, but how does the AMD + Supermicro combination work in the real world of AI?

Just ask KT Cloud, a South Korea-based provider of cloud services that include infrastructure, platform and software as a service (IaaS, PaaS, SaaS). With the rise of customer interest in AI, KT Cloud set out to develop new XaaS customer offerings around AI, while also developing its own in-house AI models.

However, as KT embarked on this AI journey, the company quickly encountered three major challenges:

  • The high cost of AI GPU accelerators: KT Cloud would need hundreds of thousands of new GPU servers.
  • Inefficient use of GPU resources in the cloud: Few cloud providers offer GPU virtualization due to overhead. As a result, most cloud-based GPUs are visible to only 1 virtual machine, meaning they cannot be shared by multiple users.
  • Difficulty using large GPU clusters: KT is training Korean-language models using literally billions of parameters, requiring more than 1,000 GPUs. But this is complex: Users would need to manually apply parallelization strategies and optimizations techniques.

The solution: KT worked with Moreh Inc., a South Korean developer of AI software, and AMD to design a novel platform architecture powered by AMD’s Instinct MI250 Accelerators and Moreh’s software.

The entire AI software stack was developed by Moreh from PyTorch and TensorFlow APIs to GPU-accelerated primitive operations. This overcomes the limitations of cloud services and large AI model training.

Users do not need to insert or modify even a single line of existing source code for the MoAI platform. They also do not need to change the method of running a PyTorch/TensorFlow program.

Did it work?

In a word, yes. To test the setup, KT developed a Korean language model with 11 billion parameters. Training was then done on two machines: one using Nvidia GPUs, the other being the AMD/Moreh cluster equipped with AMD Instinct MI250 accelerators, Supermicro Universal GPU systems, and the Moreh AI platform software.

Compared with the Nvidia system, the Moreh solution with AMD Instinct accelerators showed 116% throughput (as measured by tokens trained per second), and 2.05x higher cost-effectiveness (measured as throughput per dollar).

Other gains are expected, too. “With cost-effective AMD Instinct accelerators and a pay-as-you-go pricing model, KT Cloud expects to be able to reduce the effective price of its GPU cloud service by 70%,” says JooSung Kim, VP of KT Cloud.

Based on this test, KT built a larger AMD/Moreh cluster of 300 nodes—with a total of 1,200 AMD MI250 GPUs—to train the next version of the Korean language model with 200 billion parameters.

It delivers a theoretical peak performance of 434.5 petaflops for fp16/bf16 (a native 16-bit format for mixed-precision training) matrix operations. That should make it one of the top-tier GPU supercomputers in the world.

Do more:

 

Featured videos

Events

Find AMD & Supermicro Elsewhere

Pages