Sponsored by:
Performance Intensive Computing
Capture the full potential of IT
The Perfect Combination: The Weka Next-Gen File System, Supermicro A+ Servers and AMD EPYC™ CPUs
Featured content
The Perfect Combination: The Weka Next-Gen File System, Supermicro A+ Servers and AMD EPYC™ CPUs
Weka’s file system, WekaFS, unifies your entire data lake into a shared global namespace where you can more easily access and manage trillions of files stored in multiple locations from one directory.
One of the challenges of building machine learning (ML) models is managing data. Your infrastructure must be able to process very large data sets rapidly as well as ingest both structured and unstructured data from a wide variety of sources.
That kind of data is typically generated in performance-intensive computing areas like GPU-accelerated applications, structural biology and digital simulations. Such applications typically have three problems: how to efficiently fill a data pipeline, how to easily integrate data across systems and how to manage rapid changes in data storage requirements. That’s where Weka.io comes into play, providing higher-speed data ingestion and avoiding unnecessary copies of your data while making it available across the entire ML modeling space.
Weka’s file system, WekaFS, has been developed just for this purpose. It unifies your entire data lake into a shared global namespace where you can more easily access and manage trillions of files stored in multiple locations from one directory. It works across both on-premises and cloud storage repositories and is optimized for cloud-intensive storage so that it will provide the lowest possible network latencies and highest performance.
This next-generation data storage file system has several other advantages: it is easy to deploy, entirely software-based, plus it is a storage solution that provides all-flash level performance, NAS simplicity and manageability, cloud scalability and breakthrough economics. It was designed to run on any standard x86-based server hardware and commodity SSDs or run natively in the public cloud, such as AWS.
Weka’s file system is designed to scale to hundreds of petabytes, thousands of compute instances and billions of files. Read and write latency for file operations against active data is as low as 200 microseconds in some instances.
Supermicro has produced its own NVMe Reference Architecture that supports WekaFS on some of its servers, including the Supermicro A+ AS-1114S-WN10RT and AS-2114S-WN24RT using the AMD EPYC™ 7402P processors with at least 2TB of memory, expandable to 4TB. Both servers support hot-swappable NVMe storage modules for ultimate performance. Also check out the Supermicro WekaFS A/I and HPC Solution Bundle.
Featured videos
Events
Find AMD & Supermicro Elsewhere
Related Content
Eliovp Increases Blockchain-Based App Performance with Supermicro Servers
Featured content
Eliovp Increases Blockchain-Based App Performance with Supermicro Servers
Eliovp, which brings together computing and storage solutions for blockchain workloads, rewrote its code to take full advantage of AMD’s Instinct™ MI100 and MI250 GPUs. As a result, Eliovp’s blockchain calculations run up to 35% faster than what it saw on previous generations of its servers.
When you’re building blockchain-based applications, you typically need a lot of computing and storage horsepower. This is the niche that Belgium-based Eliovp fills. They have developed a line of extremely fast cloud-based servers designed to run demanding blockchain workloads.
Eliovp has been recognized as the top Filecoin storage provider in Europe. This refers to a decentralized blockchain-based protocol that lets anyone rent spare local storage and is a key Web3 component.
To satisfy the compute and storage needs, Eliiovp employs Supermicro’s A+ AS-1124US® and AS-4124GS® servers, running quad-core AMD EPYC™ 7543 and 7313 CPUs and as many as 8 AMD Instinct™ MI100 and MI250 GPUs to further boost performance.
What makes these servers especially potent is that Eliovp rewrote its code to run on this specific AMD Instinct™ GPU family. As a result, Eliovp’s blockchain calculations run up to 35% faster than what it saw on previous generations of its servers.
One of the attractions of the Supermicro servers is the capability to leverage the high-density core count and higher clock speeds as well as the 32 memory slots. And it comes packaged in a relatively small form factor.
“By working with Supermicro, we get new generations of servers with AMD technology earlier in our development cycle, enabling us to bring our products to market faster," said Elio Van Puyvelde, CEO of Eliovp. The company was able to take advantage of new CPU and GPU instructions and memory management to make its code more efficient and effective. Eliovp was also able to reduce overall server power consumption, which is always important in blockchain applications that span dozens of machines.
Featured videos
Events
Find AMD & Supermicro Elsewhere
Related Content
Microsoft Azure’s More Capable Compute Instances Take Advantage of the Latest AMD EPYC™ Processors
Featured content
Microsoft Azure’s More Capable Compute Instances Take Advantage of the Latest AMD EPYC™ Processors
Azure HBv3 series virtual machines (VMs) are optimized for HPC applications, such as fluid dynamics, explicit and implicit finite element analysis, weather modeling, seismic processing, and various simulation tasks. HBv3 VMs feature up to 120 Third-Generation AMD EPYC™ 7v73X-series CPU cores with more than 450 GB of RAM.
Increasing demands for higher-performance computing mean that the cloud-based computing needs to ratchet up its performance too. Microsoft Azure has introduced more capable compute virtual machines (VMs) that take advantage of the latest from AMD EPYC™ processors. This means that developers can easily spin up VMs that normally cost thousands of dollars if they were to purchase their physical equivalents.
This story's focus is on two of Azure's series: HBv3 and NVv4. In most cases, a single virtual machine is used to take advantage of all its resources. High-performance examples of Azure HBv3 series VMs are optimized for HPC applications, such as fluid dynamics, explicit and implicit finite element analysis, weather modeling, seismic processing, and various simulation tasks. HBv3 VMs feature up to 120 Third-Generation AMD EPYC™ 7v73X-series CPU cores with more than 450 GB of RAM. This series of VMs has processor clock frequencies up to 3.5GHz. All HBv3-series VMs feature 200Gb/sec HDR InfiniBand switches to enable supercomputer-scale HPC workloads. The VMs are connected and optimized to deliver the most consistent performance. Get more information about AMD EPYC™ and Microsoft Azure virtual machines.
A Dutch construction company, TBI, is using the Azure NVv4 to run computer-aided design and building modeling tasks on a series of virtual Windows desktops. The NVv4 VMs are only available running Windows powered by from four to 32 AMD EPYC™ vCPUs and offering a partial to full AMD Instinct™ M125 GPU with memory ranging from 2GB to 17GB. Previous generations of NV instances used Intel CPUs and NVIDIA GPUs that offer less performance.
TBI chose this solution because it was cheaper, easier to support and keep its software collection updated. Using virtual desktops meant that no client data was stored on any laptops, making things more secure. Also, these instances delivered equivalent performance, taking advantage of the SR-IOV technology.
Supermicro offers a wide range of servers that incorporate the AMD EPYC™ CPU and a number of servers optimized for applications that use GPUs. These servers range from 1U rackmount servers to high end 4U GPU optimized systems. Whether you’re using it on-prem or you’re building your own cloud, Supermicro’s Aplus servers are optimized for performance and technical computing applications and they run Azure and other systems well. Get more information about Supermicro servers with AMD’s EPYC™ CPUs.
Featured videos
Events
Find AMD & Supermicro Elsewhere
Related Content
Supermicro and Qumulo Deliver High-Performance File Data Management Solution
Featured content
Supermicro and Qumulo Deliver High-Performance File Data Management Solution
One of the issues that’s key to delivering higher-performing computing solutions is something that predates the PC itself: managing distributed file systems. The challenge becomes more acute when the applications involve manipulating large quantities of data. The tricky part is in how they scale to support these data collections, which might consist of video security footage, life sciences data collections and other research projects.
Storage systems from Qumulo integrate well into a variety of existing environments, such as those involving multiple storage protocols and file systems. The company supports a wide variety of use cases that allow for scaling up and out to handle Petabyte data quantities. Qumulo can run at both the network edge, in the data center and on various cloud environments. Their systems run on Supermicro’s all non-volatile memory express (NVMe) platform, the highest performing protocol designed for manipulating data stored on SSD drives. The servers are built on 24-core 2.8 GHz AMD EPYC™ processors.
Qumulo provides built-in near real-time data analytics that let IT administrators predict storage trends and better manage storage capacity so that they can proactively plan and optimize workflows.
The product handles seamless file and object data storage, is hardware agnostic, and supports single data namespace and burstable computing running on the three major cloud providers (AWS, Google and Azure) with nearly instant data replication. Its distributed file system is designed to handle billions of files and works equally well on both small and large file sizes.
Qumulo also works on storage clusters, such as those created with Supermicro AS-1114S servers, which can accommodate up to 150TB per storage node. Qumulo Shift for Amazon S3 is a feature that lets users copy data to the Amazon S3 native format for easy access to AWS services if the required services are not available in an on-prem data center.
For more information, see the white paper on the Supermicro and Qumulo High-Performance File Data Management and Distributed Storage solution, powered by AMD EPYC™ processors.
Featured videos
Events
Find AMD & Supermicro Elsewhere
Related Content
Red Hat’s OpenShift Runs More Efficiently with Supermicro’s SuperBlade® Servers
Featured content
Red Hat’s OpenShift Runs More Efficiently with Supermicro’s SuperBlade® Servers
The Supermicro SuperBlade's advantage for the Red Hat OCP environment is that it supports a higher-density infrastructure and lower-latency network configuration, along with benefits from reduced cabling, power and shared cooling features. SuperBlades feature multiple AMD EPYC™ processors using fast DDR4 3200MHz memory modules.
Red Hat’s OpenShift Container Platform (OCP) provides enterprise Kubernetes-bundled devops pipelines. It automates builds and container deployments and lets developers focus on application logic while leveraging best-of-class enterprise infrastructure.
OpenShift supports a broad range of programming languages, web frameworks, databases, connectors to mobile devices and external back ends. OCP supports cloud-native, stateless applications and traditional applications. Because of its flexibility and utility in running advanced applications, OCP has become one of the go-to places that support high-performance computing.
Red Hat’s OCP comes in several deployment packages, including as a managed service running on the major cloud platforms, as virtual machines, and on “bare metal” servers, meaning a user installs all the software needed for the platform and is the sole tenant of the server.
It’s that last use case in which Supermicro’s SuperBlade servers are especially useful. Their advantage is that they support a higher-density infrastructure and lower-latency network configuration, along with benefits from reduced cabling, power and shared cooling features.
The SuperBlade comes in an 8U chassis with room to accommodate up to 20 hot-pluggable nodes (processor, network and storage) in a variety of more than a dozen models that support serial-attached SCSI, ordinary SATA drives, and GPU processor modules. It sports multiple AMD EPYC™ processors using fast DDR4 3200MHz memory modules.
A chief advantage of the SuperBlade is that it can support a variety of higher-capacity OCP workload configurations and do so within a single server chassis. This is critical because OCP requires a variety of server roles to deliver its overall functionality, and having these roles working inside of a chassis means performance and latency benefits. For example, you could partition a SuperBlade’s 20 nodes into various OCP components such as administrative, management, storage, worker, infrastructure and load balancer nodes, all operating within a single chassis. For deeper detail about running OCP on the SuperBlade, check out this Supermicro white paper.
Featured videos
Events
Find AMD & Supermicro Elsewhere
Related Content
Build an Accelerated Data Center with AMD's Third-Gen EPYC™ CPUs
Featured content
Build an Accelerated Data Center with AMD's Third-Gen EPYC™ CPUs
“AMD EPYC™ processors are now a part of the world’s hyperscale data centers,” said Lisa Su, AMD’s CEO. Meta/Facebook is now building its servers with powerful third-generation AMD EPYC™ CPUs.
If you're making plans to build a high-performance data center, be sure to take a close look at the latest version of AMD's EPYC™ CPU chipsets, which were code-named “Milan X.”
Servers that employ AMD’s third-generation EPYC™ CPUs are so powerful that Meta/Facebook is now building its servers with them, using the new single-socket cloud-scale design, which is a part of their Open Compute Project. “AMD EPYC™ processors are now a part of the world’s hyperscale data centers,” said Lisa Su, AMD’s CEO, in the presentation at which she debuted the processors.
This latest generation of AMD EPYC™ CPUs uses an innovative packaging option of 3D stacking of chiplets for high-performance computing applications. Higher density cached memory is stacked on top of the processor to deliver more than 200 times the interconnected density of prior chiplet packaging designs. “It is the most flexible active-on-active silicon technology available in the world,” Su said. “It consumes much less energy and fits into existing CPU sockets, too.” AMD's latest chipsets satisfy the higher demands of cloud computing and electronic circuit design applications.
Jason Zander, EVP Microsoft Azure, said that Microsoft's partnership with AMD has let the cloud computing company deliver cloud instances that can run up to 12 times the speed of earlier offerings. “That rivals some supercomputers,” he said. Azure has configured some of the most powerful virtual instances, which are running on the latest AMD EPYC™ processors. They are available from 16 cores up to 120 cores and can share 448 GB of memory and 480 MB of L3 cache among the processors. For deeper information, see this Microsoft blog.
Circuit design demands the fastest processors. “The next step for AMD is to deliver more differentiation in value with a focus on performance per core,” said Dan McNamara, general manager of AMD’s Server Business Unit. “In our tests comparing Synopsys VCS chip-design simulation software running on older and newer AMD EPYC™ CPUs, engineers were able to complete 66% more jobs in the same elapsed time, thanks to having a larger L3 cache. This means that more data can be kept closer to the processor for better performance.” These faster product design lifecycles mean faster times to market since designers can save time in the testing process.
Featured videos
Events
Find AMD & Supermicro Elsewhere
Related Content
Gain Business Insights Faster by Building the Right Infrastructure for Performance-Intensive Computing
Featured content
Gain Business Insights Faster by Building the Right Infrastructure for Performance-Intensive Computing
A white paper from IDC projects a new role for IT leaders in preparing the infrastructure required to properly power performance-intensive computing (PIC) for enterprise workloads, such as data-driven insights, AI/machine learning, big data, modeling and simulation and more. Get the full white paper to learn best practices and avoid pitfalls when implementing performance-intensive computing infrastructure.
Organizations use data-driven insights to gain competitive advantage over their rivals. Competitive differentiation is often realized through the delivery of new products and services or enhancements to existing products and services. It can also be achieved by streamlining and optimizing business operations. A data-driven business reduces the time needed to realize business advantage by creating an environment conducive to forming business-differentiating insights.
As a result, IDC projects that a new chapter in the relationship between IT and the business is about to begin. The new phase will push IT further in a strategic direction, increasing its influence on business outcomes.
The trend is expected by IDC to play out over the next four to five years, thrusting IT into a new role implementing a foundational infrastructure designed to foster timely, data-driven insights, at scale. The new infrastructure will be designed to support Performance-Intensive Computing (PIC). Investments in new performance-intensive workloads will be more significant than those used for corporate IT and other business applications.
IDC defines performance-intensive computing as the process of performing large-scale, mathematically intensive computations, commonly used in analytics, machine learning and technical computing — and now increasingly required for artificial intelligence and big data and analytics in the commercial space.
Performance-intensive computing workloads have evolved at an accelerated pace. An overwhelming majority of respondents in IDC's 2021 IT Enterprise Infrastructure Survey agreed that PIC workloads are important or even critically important to their business.
But a general-purpose infrastructure won’t get the job done. Common pitfalls organizations encounter have to do with people, organizational models, business process and access to the technology required to succeed.
- Learn the five areas that organizations need to evaluate when planning to develop their PIC infrastructures.
- What are the common reasons why PIC projects fail?
- Find out about performance-intensive solutions from Supermicro and AMD.
Get the full IDC white paper: Gaining Deep and Timely Insights with Performance-Intensive Computing Infrastructure.
Featured videos
Events
Find AMD & Supermicro Elsewhere
Related Content
Offering Distinct Advantages: The AMD Instinct™ MI210 and MI250 Series GPU Accelerators and Supermicro SuperBlades
Featured content
Offering Distinct Advantages: The AMD Instinct™ MI210 and MI250 Series GPU Accelerators and Supermicro SuperBlades
Using six nanometer processes and the CDNA2 graphics dies, AMD has created the third generation of GPU accelerators, which have more than twice the performance of previous GPU processors and deliver 181 teraflops of mixed precision peak computing power.
AMD and Supermicro have made it easier to exploit the most advanced combination of GPU and CPU technologies.
Derek Bouius, a senior product manager at AMD, said “Using six nanometer processes and the CDNA2 graphics dies, we created the third generation of GPU chipsets that have more than twice the performance of previous GPU processors. They deliver 181 teraflops of mixed precision peak computing power.” Called the AMD Instinct MI210™ and AMD Instinct MI250™, they have twice the memory (64 GB) to work with and deliver data at the rate of 1.6 TB/sec. Both these accelerators are packaged as fourth generation PCIe expansion cards and come with direct connectors to Infinity Fabric bridges for faster I/O throughput between GPU cards -- without having their traffic go through the standard PCIe bus.
The Instinct accelerators have immediate benefit for improving performance in the most complex computational applications, such as molecular dynamics, computer-aided engineering, weather and oil and gas modeling.
"We provided optimized containerized applications that are pre-built to support the accelerator and run them out of the box," Bouius said. “It is a very easy lift to go from existing solutions to the AMD accelerator,” he added. It’s accomplished by bringing together AMD’s ROCm™ support libraries and tools with its HIP programming language and device drivers – all of which are open source. They can unlock the GPU performance enhancements to make it easier for software developers to take advantage of its latest processors. AMD offers a catalog of dozens of currently available applications.
Supermicro’s SuperBlade product line combines the new AMD Instinct™ GPU accelerators and AMD EPYC™ processors to deliver higher performance with lower latency for its enterprise customers.
One packaging option is to combine six chassis with 20 blades each, delivering 120 servers that provide a total of more than 3,000 teraflops of combined processing power. This equipment delivers more power efficiency in less space with fewer cables, providing a lower cost of ownership. The blade servers are all hot-pluggable and come with two onboard front-mounted 25 gigabit and two 10 gigabit Ethernet connectors.
“Everything is faster now for running enterprise workloads,” says Shanthi Adloori, senior director of product management for Supermicro. “This is why our Supermicro servers have won the world record in performance from the Standard Performance Evaluation Corp. three years in row.” Another popular design for the SuperBlade is to provide an entire “private cloud in a box” that combines administration and worker nodes and handles deploying a Red Hat Openshift platform to run Kubernetes-based deployments with minimal provisioning.
Related Resources
Featured videos
Events
Find AMD & Supermicro Elsewhere
Related Content
AMD and Supermicro Work Together to Produce the Latest High-Performance Computers
Featured content
AMD and Supermicro Work Together to Produce the Latest High-Performance Computers
Solving some of business’ bigger computing challenges requires a solid partnership between CPU vendor, system builders and channel partners. That is what AMD and Supermicro have brought to the market with the third generation of AMD's EPYC™ processors with AMD 3D V-Cache™ and AMD Instinct™ MI200 series GPU accelerators wrapped up in SuperBlade servers built by Supermicro.
“This has immediate benefits for particular fields such as crash and digital circuit simulations and electronic design automation,” said David Weber, Senior Manager for AMD. “It means we can create virtual chips and track workflows and performance before we design and build the silicon." The same situation holds for computational fluid dynamics, he added, "in which we can determine the virtual air and water flows across wings and through water pumps and save a lot of time and money, and the AMD 3D V-Cache™ makes this process a lot faster.” Without any software coding changes, these applications are seeing 50% to 80% performance improvement, Weber said.
The chips are not just fast, they come with several built-in security features, including support for Zen 3 and Shadow Stack. Zen 3 is the overall name for a series of improvements to the AMD higher-end CPU line that have shown a 19% improvement in instructions per clock, lower latency for doubled cache delivery when compared to the earlier Zen 2 architecture chips.
These processors also support Microsoft’s Hardware-enforced Stack Protection to help detect and thwart control-flow attacks by checking the normal program stack against a secured hardware-stored copy. This helps to boot securely, protect the computer from firmware vulnerabilities, shield the operating system from attacks, and prevent unauthorized access to devices and data with advanced access controls and authentication systems.
Supermicro offers its SuperBlade servers that take advantage of all these performance and security improvements. For more information, see this webcast.