Skip to main content

Compute

Discuss hardware, software, and computing resources related to your projects.

  • 7 Topics
  • 0 Replies
7 Topics
jrgosalvez
Community Manager
jrgosalvezCommunity Manager
posted in Compute
TOPS as a Metric

Anbu Valluvan, AI Engineer at SQOR.ai, shared his perspective on TOPS as a metric and why HP ZGX Nano AI Station with NVIDIA GB10 Grace Blackwell technology offer a powerful way to transform high TOPS into usable AI performance.  TOPS (Tera Operations Per Second) measures how many trillion operations a processor can execute each second, making it a popular shorthand for AI compute power. Higher TOPS generally means faster training and inference, especially for large models. However, TOPS reflects ideal peak performance, not always what workloads achieve in practice. Real results depend on factors like memory bandwidth, architecture efficiency, and sustained thermal performance. The HP ZGX Nano AI Station will deliver ~1,000 TOPS in a desktop-friendly form factor—enough to run massive language models locally. Unified CPU-GPU memory and NVLink connectivity keep data flowing to the compute units, helping workloads achieve closer-to-peak performance. Energy-efficient Grace CPUs and robust

50
jrgosalvez
Community Manager
Anbu V
Explorer
Anbu VExplorer
published in Compute
TOPS as a Hardware Performance Metric for AI: Evaluating HP ZGX Systems

The past few years have seen an exponential growth of AI applications, from generative models with hundreds of billions of parameters to real-time vision systems. This explosion in AI capability drives an equally intense demand for computing power. In response, hardware vendors have adopted metrics like TOPS (Tera Operations Per Second) to rate AI acceleration.TOPS indicates how many trillions of operations a processor can perform each second under ideal conditions. It has become a key benchmark for comparing AI chips. A higher TOPS suggests the ability to process more data or larger models in the same time frame – critical when training massive neural networks or running complex inference workloads. Yet, as useful as TOPS is, it’s not a simple race for the highest number. Other factors (like memory and efficiency) affect real-world performance, and HP’s ZGX systems illustrate how TOPS is a powerful but nuanced metric.In this article, I’ll break down what TOPS really means, how HP ZGX

120
Anbu V
Explorer
janetteguzman
HP Employee
janetteguzmanHP Employee
published in Compute
AI Edge Computing for Intelligent Traffic Monitoring

A recent paper titled “Edge computing and AI-driven intelligent traffic monitoring and optimization” explores how edge computing can be used to decentralize AI compute by processing data near IoT devices and vehicles to reduce latency and cloud dependency. The paper finds that edge computing using SLAM technology adequately places compute and storage closer to data sources and end users to enable faster data processing and response times of AI workloads for real-time traffic management, autonomous driving, and V2X communication.  Top five take-aways:1. Bring compute close to the data source: Edge computing places processing power closer to data sources (e.g., IoT devices, vehicles), enabling real-time data analysis and response. This reduces latency, enhances reliability, and alleviates bandwidth constraints, particularly crucial for traffic management and autonomous driving during peak periods.2. Decentralize to improve system efficiency and economy of resource you own:By decentralizi

760
janetteguzman
HP Employee
jrgosalvez
Community Manager
jrgosalvezCommunity Manager
posted in Compute
Hybrid Memory to Boost GPU Performance

Stanford University research team explores gain cell memory technology to address the limitations of current SRAM and DRAM memory in GPUs. By combining features of both SRAM and DRAM, the goal is to speed up data access and reduce power consumption.Top Three Takeaways:Gain Cell Memory: hybrid gain cells combine the speed of SRAM with the capacity of DRAM, to overcome the memory wall problem in GPUs by reducing data transfer delays. Innovative Materials: combining different materials for transistors (ALD ITO FET and Si PMOS), the hybrid gain cells provide faster data access, non-destructive reads, and significantly longer data retention compared than traditional DRAM. Potential for SoC Applications: can serve as a drop-in replacement for SRAM, offering higher capacity at lower fabrication costs, making it attractive for SoC manufacturers in datacenter GPUs, CPUs, and embedded systems.Check out Chris Mellor article on Blocks & Files: “Stanford team proposes hybrid gain cell memory to

650
jrgosalvez
Community Manager
TomSadlerHPAI
Explorer
TomSadlerHPAIExplorer
posted in Compute
The Hidden Costs Of Cloud And Where To Find Overspending

The Hidden Costs Of Cloud And Where To Find Overspending (forbes.com) I know this Article is leaning towards FinOps, but that principles and strategies are on similar trajectories for us working in AI and Data Science.  Finding the right mix of computing is a fine art and takes time and understanding to find the perfect mix for you.  Cloud Spending: Gartner predicts that enterprise IT spending on public cloud computing will exceed 51% by 20251. However, 30% of cloud spend is currently wastedOptimization Strategies: Key strategies include right-sizing infrastructure, managing workload placement, leveraging multi-cloud environments, and automating the deletion of idle instances.What are your thoughts on a Hybrid modality? 

920
TomSadlerHPAI
Explorer
jrgosalvez
Community Manager
jrgosalvezCommunity Manager
published in Compute
Data prep 10x faster by using Pandas on GPU

HP and NVIDIA announce AI Workstations that are optimized for pandas dataframes. =========================================================================NVIDIA and HP Supercharge Data Science and Generative AI on Workstations Coming to Z by HP AI Studio, NVIDIA CUDA-X Data Processing Libraries Boost Python Pandas Software for Millions of Data ScientistsMarch 7, 2024HP Amplify — NVIDIA and HP Inc. today announced that NVIDIA CUDA-X™ data processing libraries will be integrated with HP AI workstation solutions to turbocharge the data preparation and processing work that forms the foundation of generative AI development.Built on the NVIDIA CUDA® compute platform, CUDA-X libraries speed data processing for a broad range of data types, including tables, text, images and video. They include the NVIDIA RAPIDS™ cuDF library, which accelerates the work of the nearly 10 million data scientists using pandas software by up to 110x using an NVIDIA RTX™ 6000 Ada Generation GPU instead of a CPU-only

780
jrgosalvez
Community Manager
jrgosalvez
Community Manager
jrgosalvezCommunity Manager
posted in Compute
New Compute Throughput Tip - Increase Batch Sizes, Tune

New computers with powerful GPUs can bottleneck at the CPU if code is not modified to increase batch sizes sent to the GPUs. To improve processing performance of your system, update and tune hyperparameters in your code to deliver larger batch payloads to your GPU.Tuning to balance overfitting and underfitting, we found a 5x performance on model training, giving our team more time to do experiments. Adil Lheureux published a good read on the topic called “How to maximize GPU utilization by finding the right batch size” that you should check out. 

440
jrgosalvez
Community Manager
Terms & ConditionsCookie settings