What Is GPU Computing (GPGPU)?

Why GPGPU Became Critical

GPGPU means using GPUs for non-graphics workloads.

• CPUs excel at general control logic and low-latency orchestration.
• GPUs excel at applying the same operation to very large data sets in parallel.

When a problem is data-parallel, GPUs can deliver major acceleration.

Short History

• Early phase: repurposing graphics APIs for compute.
• Transition phase: dedicated compute frameworks.
• Modern phase: CUDA, OpenCL, and mature ecosystems.

CPU vs GPU: What Matters

1. Execution model

GPUs run thousands of similar threads efficiently. CPUs remain better for branch-heavy heterogeneous control flows.

2. Memory hierarchy

Performance depends on memory access quality:

• coalesced accesses,
• local data reuse,
• reduced host-device transfers.

3. Compute resources

Architectures differ in FP64, FP32, FP16, integer, and tensor capabilities. Raw FLOPS alone is not enough.

4. Energy efficiency

GPUs can offer strong performance per watt on well-structured workloads.

Core Programming Model

Stream processing dominates:

• data streams,
• massively applied kernels,
• target buffer outputs.

Typical patterns:

• map,
• reduce,
• scan,
• gather/scatter,
• sorting and search at scale.

APIs and Ecosystem

• CUDA,
• OpenCL,
• SYCL,
• DirectCompute,
• Metal.

Selection depends on portability, tooling maturity, production hardware, and team skills.

When GPGPU Works Best

• large datasets,
• weak inter-element dependencies,
• high arithmetic intensity,
• transfer overhead amortized.

Frequent use cases:

• vision and image/video,
• linear algebra and FFT,
• physical simulation,
• bioinformatics,
• AI/ML,
• cryptography and signal processing.

Common Pitfalls

• Excessive branching.
• Offloading problems that are too small.
• Too many CPU-GPU transfers.
• Optimization without profiling.
• Wrong precision strategy for the target architecture.

Practical Evaluation Method

1. Rapid qualification (data parallel? enough volume? transfer ratio acceptable?).
2. Minimal prototype and end-to-end timing.
3. Iterative optimization (memory layout, occupancy, divergence reduction).
4. Industrialization (monitoring, multi-GPU strategy, CPU fallback).

Summary

GPGPU is powerful but not magical. The best results come from memory-aware parallel design and solid CPU-GPU orchestration.

Sources:

• https://en.wikipedia.org/wiki/General-purpose_computing_on_graphics_processing_units