The past few years have seen an exponential growth of AI applications, from generative models with hundreds of billions of parameters to real-time vision systems. This explosion in AI capability drives an equally intense demand for computing power. In response, hardware vendors have adopted metrics like TOPS (Tera Operations Per Second) to rate AI acceleration.
TOPS indicates how many trillions of operations a processor can perform each second under ideal conditions. It has become a key benchmark for comparing AI chips. A higher TOPS suggests the ability to process more data or larger models in the same time frame – critical when training massive neural networks or running complex inference workloads. Yet, as useful as TOPS is, it’s not a simple race for the highest number. Other factors (like memory and efficiency) affect real-world performance, and HP’s ZGX systems illustrate how TOPS is a powerful but nuanced metric.
In this article, I’ll break down what TOPS really means, how HP ZGX leverages it, and how to interpret TOPS alongside other criteria when selecting AI hardware.
Understanding TOPS: Technical Foundation
TOPS stands for Tera Operations Per Second, a measure of computational throughput. One tera-op equals one trillion operations (for example, additions or multiplications) completed in one second. In practice, TOPS is calculated based on the number of parallel math units and their clock speed. For instance, roughly 2 × (number of MAC units) × (frequency) (in hertz) divided by 10^12 gives the TOPS.
A simple example: an AI chip with 1,000 MAC (multiply-accumulate) units at 1 GHz can achieve about 2 TOPS of theoretical throughput. Higher TOPS generally come from either more processing cores or lower numerical precision (or both). Modern AI accelerators support reduced-precision data types like INT8, FP8, and FP4, which allow more operations per second than full FP32 math.
HP’s ZGX platforms, built on NVIDIA’s Grace-Blackwell architecture, exemplify this. The ZGX Nano AI Station (powered by a single NVIDIA GB10 Grace-Blackwell superchip) delivers around 1,000 TOPS of AI performance by using fifth-generation Tensor Cores with FP4 precision. Likewise, the larger ZGX Fury AI Station, leveraging the GB300 “Ultra” Grace-Blackwell chip, reaches an enormous 20,000 TOPS at FP4. These peak TOPS figures are measured under ideal conditions – essentially the theoretical maximum if every processing unit is busy on low-precision operations.
It’s important to note the difference between peak vs. sustained performance.
Peak TOPS is what the silicon could do on paper, whereas sustained performance is what you actually get on typical workloads. Architectural features in HP’s Grace-Blackwell systems (like unified CPU-GPU memory and NVLink connectivity) aim to help real workloads approach the theoretical peaks. Still, no system runs at 100% of its TOPS at all times – real code has bottlenecks that I will explore later. For now, TOPS provides a technical foundation and common yardstick to compare the raw compute capacity of AI hardware across different devices.
Why TOPS Matters for Hardware Evaluation
In evaluating AI hardware, especially cutting-edge workstations like HP ZGX, TOPS is a crucial metric because it correlates with how well a system can handle demanding AI workloads. A higher TOPS rating often means the system can process more neural network operations in parallel, enabling faster training times and higher inference throughput. For HP’s ZGX line designed for AI developers, this translates directly into better performance on real tasks. For example, the ZGX Nano’s ~1000 TOPS empowers it to run large language models up to 200 billion parameters locally, something previously limited to data centers. That means enterprise teams can prototype or fine-tune generative AI models (think GPT-style chatbots or domain-specific LLMs) on an office workstation without offloading to the cloud.
Similarly, the ZGX Fury’s 20,000 TOPS is so high that it becomes suitable for heavy lifting like training and inferencing giant models with hundreds of billions of parameters. Beyond raw throughput, TOPS/Watt (efficiency) is also a consideration. The Grace-Blackwell architecture in ZGX is designed for power-efficient AI compute – the GB10 chip delivers its 1-petaflop (1000 TOPS) performance within a standard power envelope. HP’s implementation benefits from Arm-based Grace CPU cores (20 cores) known for their energy efficiency, meaning more operations per second per watt of power. This efficiency is crucial in an enterprise setting, where electrical and cooling costs are non-trivial. In real-world use, these TOPS advantages manifest in diverse scenarios. Enterprise generative AI workloads (like building an internal ChatGPT-style assistant or summarizing large documents) are optimized on ZGX systems by using those plentiful tera-ops to chew through tokens faster.
Computer vision models also benefit – high TOPS can drive real-time image or video analysis with complex neural nets. And for ML engineers tuning models or running hyperparameter searches, a high-TOPS machine shortens iteration cycles, enabling quicker experimentation. Importantly, many buyers do look at TOPS ratings as a quick index of capability. It helps them match a system to their needs: a team doing lighter on-prem inference might choose the ZGX Nano (with 1k TOPS) for its balance of performance and cost, whereas a team aiming to train large models or serve heavy AI workloads may justify the ZGX Fury with 20k TOPS. In summary, TOPS matters because it encapsulates an AI system’s horsepower in one number – and for HP’s AI-focused workstations, that number speaks to how well they’ll handle the cutting-edge AI tasks businesses are tackling today.
TOPS in Different Hardware Categories
To put HP’s ZGX systems into perspective, it helps to compare TOPS across different classes of hardware – from mobile AI chips all the way to data-center-grade AI engines.
Mobile NPUs (Neural Processing Units) in smartphones or ultralight laptops typically deliver performance on the order of tens of TOPS. For instance, Apple’s A17 Pro chip includes a neural engine rated around 35 TOPS, and Qualcomm’s latest Snapdragon X Elite platform reaches roughly 45 TOPS on its AI cores. These NPUs specialize in INT8/INT4 operations to hit those figures within tiny power budgets. They enable on-device AI features (like image enhancements or speech recognition) but obviously sit at the low end of the TOPS spectrum.
Moving up, consumer and professional GPUs provide much higher TOPS, thanks to massively parallel architectures and dedicated AI tensor cores. A high-end PC GPU like NVIDIA’s GeForce RTX 4090 can perform on the order of 1,300 TOPS (INT8) using its tensor units with sparsity optimizations. (Without sparsity, it’s roughly half that – still many hundreds of TOPS.) This illustrates that a desktop GPU alone already pushes into four-figure TOPS territory.
Notably, AMD’s flagship GPUs currently advertise lower TOPS (e.g. ~120 TOPS for certain precisions), reflecting differences in architecture and focus on graphics vs. AI. Traditional server CPUs without AI accelerators, on the other hand, usually don’t reach anywhere near those numbers in TOPS – they might only offer a few hundred GFLOPS in general-purpose compute, which would be a fraction of one TOPS. That’s why accelerators (GPUs, NPUs, FPGAs) are essential for modern AI workloads.
Now enter the HP ZGX AI stations, which straddle the line between workstation and supercomputer. The ZGX Nano (HP’s version of NVIDIA DGX Spark) comes with the Grace-Blackwell GB10 SoC; its ~1,000 TOPS of FP4 AI performance puts it in the same league as some data center GPUs or small clusters, but in a compact desktop form factor. Meanwhile, the ZGX Fury (based on the mighty GB300 Ultra chip) boasts an astounding 20,000 TOPS, an order of magnitude above even the fastest single GPUs. In fact, 20,000 TOPS (20 petaops) is data-center class performance – comparable to multi-GPU AI servers – now condensed into what looks like a tower workstation. This massive number is achieved by combining advanced 5th-gen Tensor Cores, FP4 precision, and an architecture that essentially packs the power of multiple GPUs plus a CPU into one package. To appreciate how different hardware categories achieve high TOPS, consider their design trade-offs.
Mobile AI chips rely on highly specialized low-precision units (and sometimes share workload with GPU/CPU), all tuned for power efficiency. They hit respectable TOPS for tasks like image filters, but their memory and thermal constraints keep them far below workstation-class performance.
GPUs achieve high TOPS through sheer parallelism (thousands of cores) and moderate precision (FP16/INT8); they also benefit from substantial memory bandwidth with on-board VRAM. The HP ZGX systems (and similar AI workstations) push this further by integrating CPU and GPU in one coherent memory space – for example, the ZGX Fury’s GB300 has 784 GB of unified memory accessible at high bandwidth via NVLink-C2C between the Grace CPU and Blackwell GPU. This unified architecture means the enormous compute units can be fed data efficiently, which is part of how it sustains those 20k TOPS in practice.
In essence, ZGX Nano and Fury bring data-center TOPS to the workstation category, far beyond what standard PCs or even high-end GPUs deliver. ZGX Nano’s ~1000 TOPS would be considered overkill in a phone and is on par with some AI servers, whereas ZGX Fury’s 20,000 TOPS squarely targets workloads that traditionally required multi-node clusters. This comparison highlights that TOPS is not static across categories – it scales with form factor and use-case, and HP’s ZGX family sits at the cutting edge, where workstation convenience meets supercomputer-level TOPS performance.
Limitations and Considerations
While TOPS is a handy measure of raw compute, it’s not a complete indicator of real-world performance. Relying solely on TOPS can be misleading – there are scenarios where a device with lower TOPS outperforms a higher-TOPS device due to other factors. One major factor is memory bandwidth and capacity. AI accelerators need to move large amounts of data (model weights, activations) in and out of computing units. If a chip can’t feed its cores fast enough, a portion of its TOPS will sit idle. Real applications often hit memory bottlenecks, especially with big models. This is why HP’s ZGX architecture emphasizes unified memory and high-bandwidth links. The ZGX Fury, for instance, provides 784 GB of coherent memory and an NVLink-C2C interconnect to the Grace CPU, so that the Blackwell GPU can access data without the latency and throughput limitations of PCIe. Such integration helps ensure more of those 20,000 trillion operations per second can be used effectively, rather than wasted waiting on data. Cooling and power are another consideration. A system might attain a high TOPS rating in theory, but under sustained load it could throttle if the cooling solution can’t dissipate heat. Enterprise buyers must consider thermal design and power delivery – effectively, can the system run near its peak TOPS for long durations without overheating or drawing beyond available power? HP’s ZGX stations are built as turnkey AI systems, meaning they come with appropriately robust cooling and power supplies to handle these workloads (for example, the ZGX Fury’s chassis is akin to NVIDIA’s DGX Station design, which is a heavy-duty tower built for continuous AI training use). But it’s a reminder that an overclocked gaming GPU advertised with high TOPS might not sustain that level in 24/7 AI use if not adequately cooled.
Precision trade-offs also play a role in interpreting TOPS. A high TOPS number usually assumes lower precision operations (like INT8 or FP4). If your workload requires higher precision (FP16 or FP32) for accuracy, the effective operations per second will be much lower.
For example, that 20,000 TOPS (which is measured at FP4) would correspond to a smaller number of TFLOPS at FP16. So, when evaluating HP ZGX or any AI system, you need to align the TOPS metric with the precision your models need. It’s here that TOPS alone isn’t sufficient – you should also look at metrics like FLOPS for higher precision, or see if the vendor provides benchmarks on specific tasks.
History has shown that not all TOPS are usable. In industry analyses, some AI accelerators achieved, say, 7.5× the TOPS of a GPU but only delivered ~4× the actual neural network throughput, meaning almost half of the theoretical compute went unutilized. Causes include memory limits, suboptimal software, or the mismatch between theoretical ops and the ops a given network actually needs. This phenomenon is sometimes dubbed “dark silicon”, where portions of the chip can’t be fully leveraged in practice.
The takeaway for a prospective buyer is to be cautious: TOPS is a necessary but not sufficient metric.
HP’s ZGX systems mitigate common bottlenecks through their design (fast unified memory, strong cooling, optimized software stack), but one should still consider real benchmarks. Evaluate things like how many images per second can be processed, or how fast a known model (e.g., ResNet-50 or BERT) runs on the system, in addition to the TOPS number. In short, don’t judge an AI system by TOPS alone – view it in context of memory, precision, and system architecture. The good news is that HP engineered the ZGX line to address many of these limitations, marrying high TOPS with the throughput, memory, and stability needed to actually use those tera-ops effectively.
TOPS vs. Other Performance Metrics
How does TOPS compare to other metrics, and should it be the only thing you consider? The answer is balance – TOPS is just one piece of the performance puzzle. Traditionally, FLOPS (Floating Point Operations Per Second) was the headline figure for compute performance, especially for CPUs and GPUs. In fact, TOPS is essentially FLOPS but often counted with lower precision or integer ops (for example, 1 FP16 operation might count as 2 FP8 operations, doubling the “tera-ops” count). While FLOPS (particularly in double or single precision) still matters for scientific compute, in AI the focus has shifted to TOPS because AI algorithms tolerate and even thrive on lower precision arithmetic for speed gains. Still, when comparing systems, it’s useful to know both. For instance, HP ZGX Fury’s 20,000 TOPS at FP4 corresponds to a certain number of petaflops at FP16 – a metric you might compare against a traditional GPU cluster in an HPC context. Another crucial metric is memory bandwidth (measured in GB/s or TB/s). This indicates how fast data can be supplied to those computing units. A system with 1000 TOPS and insufficient bandwidth may underperform a 500 TOPS system with ample bandwidth. For example, NVIDIA’s Blackwell GPU in HP’s ZGX has up to 8 TB/s memory bandwidth with its HBM3e memory, which is extremely high and necessary to keep its tensor cores busy. When assessing performance, look at memory specs – HP ZGX’s unified memory architecture means the CPU and GPU share a large pool, which reduces data copying and can improve effective throughput.
Latency is another consideration. TOPS measures throughput, but latency (the time to execute a single operation or a single inference end-to-end) can be just as important for real-time applications. Sometimes a system might have high TOPS but due to pipeline depth or batching requirements, the response time for one query is slow. If you’re deploying an interactive AI service (say, an AI assistant that needs to respond within 100 milliseconds), you’ll care about latency benchmarks in addition to TOPS.
Lastly, consider standardized benchmark scores (like MLPerf) which combine aspects of TOPS, memory, and software optimization to give a more holistic performance picture. These can often differentiate hardware in ways raw TOPS can’t.
In practice, a balanced evaluation is best. TOPS is a useful first-order metric, but it “does not provide a complete assessment and must be considered in context of specific applications and system architecture”.
In summary, use TOPS as a starting point to gauge a system’s class, but also weigh other metrics like FLOPS for needed precision, memory bandwidth for data-heavy workloads, and latency for responsiveness. HP’s approach with ZGX underscores this balanced philosophy – tremendous TOPS paired with robust memory and infrastructure – and that is exactly how buyers should approach performance metrics: in balance, not in isolation.
Future Trends and Conclusion
Looking ahead, AI’s appetite for compute shows no sign of slowing. Models are growing in size (trillions of parameters are on the horizon), and organizations want faster results and the ability to iterate quickly. This will inevitably drive hardware to offer even more TOPS – such that soon we may talk about hundreds of thousands of TOPS as new chips and multi-chip solutions emerge. HP ZGX systems are poised to evolve with this trend.
HP’s close collaboration with NVIDIA (as seen with the Grace-Blackwell adoption) means future ZGX generations will likely incorporate next-gen superchips with even greater performance. Scalability will also be key. Instead of a single box, we might see HP offer clustered solutions – essentially linking multiple ZGX Fury units or similar via high-speed interconnects. In fact, NVIDIA’s platform already allows linking two Grace-Blackwell systems to handle models with over 400 billion parameters seamlessly.
In conclusion, TOPS remains a powerful metric for understanding AI hardware performance, especially when comparing options like HP’s Z by HP ZGX systems. It encapsulates the remarkable computational leaps enabling today’s AI revolution. But remember that it’s one metric among many. HP ZGX exemplifies this balance: sky-high TOPS backed by the memory, bandwidth, and engineering to make those TOPS count. As AI buyers, appreciating both the power and the nuances of TOPS will help you make informed decisions – choosing systems that not only look good on paper, but deliver effective performance for your AI workloads. With HP ZGX, the takeaway is clear: Tera-ops can drive your AI forward, but it’s how you harness them that truly unlocks AI at scale.
References:
- With Blackwell GPUs, AI Gets Cheaper And Easier, Competing With Nvidia Gets Harder
- Meet the AI PC: what it is, what it does, and how to get started - Edge Up
https://edgeup.asus.com/2024/meet-the-ai-pc-what-it-is-what-it-does-and-how-to-get-started/
- CPU, GPU, and NPU: Understanding Key Differences and Their Roles in Artificial Intelligence | by Antonio Troise | Medium
- NVIDIA DGX Spark - new little AI supercomputer - Rost Glukhov | Personal site and technical blog
https://www.glukhov.org/post/2025/07/nvidia-dgx-spark/
- HP, Dell shed more light on their competitors to DGX Station AI workstation but they won't be cheap | TechRadar
- NVIDIA Puts Grace Blackwell on Every Desk and at Every AI Developer’s Fingertips | NVIDIA Newsroom
- Nvidia GeForce RTX 5090 versus RTX 4090 — How does the new halo GPU compare with its predecessor? | Tom's Hardware
- Are Tera Operations Per Second (TOPS) Just hype? Or Dark AI Silicon in Disguise? - KDnuggets
https://www.kdnuggets.com/2020/05/tops-just-hype-dark-ai-silicon-disguise.html
- Unlocking the Future of Work with Unparalleled Performance for Advanced AI-Workflows | HP
Official Site