Graphics Processing Units (GPU)

Graphics Processing Units (GPU)

• A GPU (Graphics Processing Unit) is a specialized processor designed to handle parallel processing tasks, primarily for rendering graphics in computers, gaming consoles, and mobile devices.

• Modern GPUs are also widely used for scientific computing, artificial intelligence (AI), machine learning (ML), and cryptocurrency mining due to their ability to perform massive parallel computations efficiently.

1. Key Features of a GPU

1. Specialized architecture for parallel processing capabilities

■ Unlike a CPU (Central Processing Unit), which has a few powerful cores optimized for sequential processing, a GPU consists of thousands of smaller cores that execute multiple tasks simultaneously.

■ Ideal for graphics rendering, deep learning, simulations, and large‒scale data processing.

2. High‒speed computation

■ Designed to process multiple data streams simultaneously, making it significantly faster than a CPU for tasks like image processing, video rendering and neural network training.

3. Memory bandwidth

■ GPUs have high memory bandwidth, allowing them to quickly access and process large amounts of data.

■ Uses GDDR (Graphics Double Data Rate) memory, which has higher bandwidth than traditional RAM to handle large datasets quickly.

4. Optimized for Al and Deep Learning

■ Modern GPUs include Tensor Cores and RT Cores (Ray Tracing Cores) for AI acceleration and realistic graphics rendering.

■ NVIDIA's CUDA and AMD's ROCm allow for general‒purpose GPU computing (GPGPU), enabling non‒graphics applications to leverage GPU power.

• GPU computing offers unprecedented application performance by offloading compute‒ intensive portions of the application to the GPU, while the remainder of the code still runs on the CPU. From a user's perspective, applications simply run significantly faster.

• CPU + GPU is a powerful combination because CPUs consist of a few cores optimized for serial processing, while GPUs consist of thousands of smaller, more efficient cores designed for parallel performance. Serial portions of the code run on the CPU while parallel portions run on the GPU.

2. Connection between CPU and GPU

• Fig. 6.5.2 shows how a GPU is typically connected with a modern processor. A GPU is an accelerator (or a co‒processor) that is connected to a host processor.

• The host processor and GPU communicate to each other via PCI Express (PCIe) that provides 4 Gbit/s (Gen 2) or 8 Gbit/s (Gen 3) interconnection bandwidth.

• This communication bandwidth often becomes one of the biggest bottlenecks; thus it is critical to offload the work to GPUs only if the benefits of using GPUS outweigh the offload cost.

3. GPU Architecture

• Fig. 6.5.3 illustrates the major components of a general‒purpose graphics processor.

• At a high level, the GPU architecture consists of several streaming multiprocessors (SMs), which are connected to the GPU's DRAM.

■ Each SM has a number of single‒instruction multiple data (SIMD) units, also called stream processors (SPs), and supports a multithreading execution mechanism. GPU architectures employ two important execution paradigms :

■ SIMD / SIMT

■ Multithreading

SIMD / SIMT

■ GPU processors supply high Floating Point (FP) execution bandwidth, which is the driving force for designing graphics applications. To make efficient use of the high number of FP units, GPU architectures employ a SIMD or SIMT execution paradigm.

■ In SIMD, one instruction operates on multiple data (i.e., only one instruction is fetched, decoded and scheduled but on multiple data operands). Depending on the word width, anywhere from 32, 64 or 128 FP operations may be performed by a single instruction on current systems. This technique significantly increases system throughput and also improves its energy‒efficiency.

Multithreading

■ The other important execution paradigm that GPU architectures employ is hardware multithreading.

■ GPU processors use fast hardware‒based context switching to tolerate long memory and operation latencies.

■ The effectiveness of multithreading depends on whether an application can provide a high number of concurrent threads.

■ Most graphics applications have this characteristics since they typically need to process many objects (e.g. pixels, vertices, polygons) simultaneously.

■ In conventional CPU systems, thread context switching relatively much more expensive : all program states, such as PC (Program Counter), architectural registers and stack information, need to be stored by the operating system in memory. However, in GPUs, the cost of thread switching is much lower due to native hardware support of such process.

■ The degree of multithreading (the number of simultaneous hardware thread contexts) is much lower in CPUs than in GPUs (e.g. two hyperthreads Vs. hundreds of GPU thread).

4. GPU vs. CPU: Key Differences

Review Questions

1. What is GPU?

2. State key features of GPU.

3. Explain the GPU architecture.

4. Give comparison between CPU and GPU.