Multicore Architectures

Computer Organization and Architecture

Chapter 6.

Next Generation Computer Architecture

Multicore Architectures

• As modern applications demand higher performance and energy efficiency, computer designers increasingly rely on multicore architectures. A multicore processor integrates two or more independent processing units (cores) on a single chip. Each core is capable of executing instructions simultaneously, enabling true parallelism within the processor.

• Multicore systems improve performance not by increasing clock speed, but by executing multiple threads or programs at the same time. This approach allows better utilization of transistors, lower power consumption, and improved throughput compared to single‒core processors.

1. Classification of Multicore Architectures

• Multicore architectures can be classified based on several important design factors:

1. Number of processors (Cores) : This refers to the number of processing units integrated into the system. A multicore processor may consist of:

♦ Dual‒core processors → 2 cores

♦ Quad‒core processors → 4 cores

♦ Hexa‒core processors → 6 cores

♦ Octa‒core processors → 8 cores

♦ Higher counts (12, 16, 32+ cores) in servers and workstations.

More cores allow more parallel tasks to run simultaneously, improving performance in multitasking and multithreaded applications.

2. Processor‒to‒Processor communication : Cores on a multicore chip must communicate and share data efficiently. Common approaches include:

♦ Shared bus communication : All cores communicate over a common bus. Effective for a small number of cores but becomes a bottleneck as cores increase.

♦ Interconnect networks : Advanced multicore chips use mesh, ring or crossbar networks for high‒speed communication.

♦ Shared memory access : Cores communicate by reading/writing to shared memory, making programming easier.

Efficient communication is essential to avoid delays and maintain high throughput.

3. Cache and Memory implementations : Memory design strongly impacts multicore performance. Typical implementations include:

♦ Private L1/L2 Caches : Each core has its own small, fast cache to store frequently used data.

♦ Shared Last‒Level Cache (LLC), usually L3 : All cores can access a shared cache to improve data sharing.

♦ Cache coherence protocols (MESI, MOESI) : Required to ensure all cores see a consistent view of memory.

Multicore systems also use memory models like:

♦ UMA (Uniform Memory Access) : All cores access memory at equal speed.

♦ NUMA (Non‒Uniform Memory Access) : Access time depends on the location of memory relative to the core.

4. I/O bus and Front Side Bus (FSB) implementation

♦ Multicore processors must efficiently communicate with external memory and I/O devices.

♦ Older systems used a Front Side Bus (FSB) to connect the CPU to the memory controller. Modern processors use faster interconnects such as:

o Intel QuickPath Interconnect (QPI)

o AMD infinity fabric

♦ These high‒speed links allow multiple cores to access memory and I/O without bottlenecks.

2. Common Multicore and Multiprocessing Configurations

• Fig. 6.1.1 describes three typical configurations used in modern systems. These configurations differ in how processing units are organized.

Type 1: Hyperthreaded (Logical multicore) system

■ It uses hyperthreading technology. Hyperthreading (also known as Simultaneous Multi‒Threading or SMT) is a technology that allows a single physical processor to act like two or more logical processors to the operating system and application programs.

■ Logical vs. Physical : In hyperthreaded technology, the multiple processors are logical instead of physical. The physical chip still contains only one core.

■ Hardware duplication : The architecture involves some duplication of internal hardware components, such as register files and program counters, but not enough to qualify as a separate, complete physical processor.

■ Function : It allows the single core to execute instructions from multiple threads concurrently by issuing instructions from different threads in the same clock cycle, thereby hiding memory latency and boosting utilization of the execution units.

■ Advantage : Increases instruction throughput without doubling physical hardware.

■ Limitation: Not true multicore ‒ performance gain is smaller than adding real cores.

Type 2: Classic multiprocessor system

■ A classic multiprocessor system consists of two or more independent, Complete Processing Units (CPUs) that are physically separate chips connected on a motherboard.

■ Physical separation : Each processing unit is a separate physical chip with its own dedicated hardware.

■ Tightly coupled : These systems are considered tightly coupled because all processors share a single common main memory and are controlled by a single operating system.

■ Communication : They typically communicate via a shared bus architecture (like the FSB) or a dedicated interconnect. The challenge in these systems is maintaining cache coherence across the separate chips.

■ Advantages :

 ♦ True parallel processing.

 ♦ Suitable for servers and scientific computing.

■ Drawbacks :

 ♦ Higher cost and power usage.

 ♦ Communication overhead increases with more processors.

Type 3 : True multicore processor system

■ A multicore system represents the modern approach to multiprocessing, providing two or more complete processors (cores) integrated onto a single physical chip (die).

■ Integration : The cores share the same silicon substrate and often share higher levels of the cache hierarchy (e.g., L2 or L3 cache) and the memory interface.

■ Efficiency : Integrating multiple cores onto one die offers significant advantages in power consumption, communication latency and manufacturing costs compared to classic multiprocessor systems. The short physical distance between cores allows for very fast, low‒latency communication.

■ Advantages:

♦ Faster communication between cores compared to separate chips.

♦ Lower power consumption.

♦ Highly scalable and efficient.

■ Applications:

 ♦ Used in desktops, laptops, smartphones and modern servers.

3. Advantages of Multicore Architectures

• Multicore designs offer significant improvements in performance, efficiency and design flexibility :

■ Parallelism : The most fundamental advantage is the ability to execute multiple tasks or threads simultaneously across different cores. This is true parallelism, leading to substantial speed‒up for complex workloads.

■ Energy efficiency: By distributing the workload across multiple cores, each operating fos at a lower clock speed, the overall system can complete the task using significantly less power and generating less heat. This improved power efficiency is critical for mobile devices and data centers.

■ Scalability : Processors can be scaled by simply integrating additional cores onto the die without requiring a complete and major redesign of the core microarchitecture itself.

■ Improved performance : Multicore architectures deliver significant speed gains, especially for multithreaded applications that are designed to divide their work, such as media processing, modern gaming, scientific simulations and AI / machine learning tasks.

4. Challenges of Multicore Design

• Implementing and utilizing multicore systems effectively introduces several significant technical and programming challenges :

■ Software compatibility : The full potential of multicore hardware can only be realized if programs are written to use multiple threads, a concept known as parallel programming. Many legacy programs and single‒threaded applications cannot benefit from the additional cores.

■ Cache coherence overhead : Cores typically have their own private caches (L1/L2). The system must employ complex protocols (like MESI) to ensure memory consistency (cache coherence) across all cores. This coherence maintenance introduces overhead as cores must constantly communicate and check the status of shared data blocks.

■ Synchronization and locking : When multiple threads access and modify shared data structures (like queues or tables), they must coordinate their access to prevent data corruption. This coordination involves using mechanisms like locks, semaphores, or mutexes. Implementing synchronization correctly is difficult and can lead to performance bottlenecks (if locks are held too long) or deadlocks (if threads wait indefinitely for each other).

■ Thermal management : Despite running at lower individual clock speeds for efficiency, a large number of densely packed cores on a single die still generates a high amount of total heat. Effective thermal management (cooling) is essential to prevent performance throttling or damage to the chip.

Review Questions

1.What is a multicore processor ?

2. Why do modern systems prefer multicore processors instead of increasing clock speed?

3.Explain the classification of multicore architectures.

4.What is the difference between logical and physical cores?

5.What is a shared bus in multicore processors?

6.What is cache coherence?

7. What do UMA and NUMA stand for?

8. Define hyperthreading.

9. Explain hyperthreading architecture.

10. What is a classic multiprocessor system?

11. What is a true multicore processor?

12. Discuss the architecture and advantages of true multicore processor systems.

13. List any two advantages of multicore processors.

14. What are the challenges of multicore design?