Performance Metrics of Computer

Performance Metrics

• The performance is an important attribute of a computer. It is an important criteria for selection of a computer.

• There are number of ways by which performance of a computer can be measured. In this section we study them and see the best way to measure performance. We also see the limitations of performance measurement.

 

1. Defining Performance

• When we say one computer is faster than another, we compare their speeds and observes that the faster computer runs a program in less time than other computers.

• The computer center manager running a large server system may say a computer is faster when it completes more jobs in an hour.

• The computer user is always interested in reducing the time between the start and the completion of the program or event, i.e. reducing the execution time.

• The execution time is also referred to as response time. Reduction in response time increases the throughput (the total amount of work done in a given time). The performance of the computer is directly related to throughput and hence it is reciprocal of execution time.

Performance_A = 1 / Execution time_A

• This means that for two computers A and B if the performance of A is greater than the performance of B, we have

Performance_A > Performance_A

1 / Execution time_A > 1 /Execution time_B

Execution time_B > Execution time_A

• That is, the execution time on B is longer than that on A, if A is faster than B.

• In discussing a computer design, we often want to relate the performance of two different computers quantitatively. We will use the phrase "A is n times faster than B" or equivalently "A is n times as fast as B" to mean.

Performance _A / Performance_B = n

• If A is n times faster than B then the execution time on B is n times longer than it is on A:

Performance_A / Performance_B = Execution time_B / Execution time_A = n

Example : 1

If computer A runs a program in 10 seconds and computer B runs the same program in 15 seconds, how much faster is A than B ?

Solution:

We know that A is n times faster than B if

Performance_A / Performance_B = Execution time_B / Execution time_A = n

Thus the performance ratio is

15/10 = 1.5

and A is therefore 1.5 times faster than B.

• In the above example, we could also say that computer B is 1.5 times slower than computer A, since

Performance _A / Performance_B = 1.5

means that

Performance_A / 15 = Performance_B

• For simplicity, we will normally use the terminology faster than when we try to compare computers quantitatively. Because performance and execution time are reciprocals, increasing performance requires decreasing execution time. To avoid the potential confusion between the terms increasing and decreasing, we usually say "improve performance" or "improve execution time" when we mean "increase performance" and "decrease execution time".

                    

2. Measuring Performance

• One of the important measure of a computer performance is a time. The computer that performs same task in a least time is the fastest. Program execution time is measured in seconds per program. However, time can be defined in different ways depending on what it measures.

CPU Time

■ CPU execution time or simply CPU time is the time the CPU spends computing for particular task and does not include time spend waiting for I/O or running other programs.

■ CPU time can be divided into the CPU time spent in the program, called user CPU time and the CPU time spent in the operating system performing tasks on behalf of the program, called system CPU time.

■ Differentiating between system and user CPU time is difficult to do accurately because it is often hard to assign responsibility for operating system activities to one user program rather than another and because of the functionality differences among operating systems. We use CPU performance to refer user CPU time.

Performance Metrics

■ Users and designers often examine performance using different metrics. If we could relate these different metrics, we could determine the effect of a design change on the performance as seen by the user. Since we are confining ourselves to CPU performance at this point, the bottom‒line performance measure is CPU execution time.

■ A simple formula relates the most basic metrics (clock cycles and clock cycle time) to CPU time:

CPU execution time for a program = CPU clock cycles for a program × Clock cycle time

■ Alternatively, because clock rate and clock cycle time are inverses,

CPU execution time for a program = CPU clock cycles for a program / Clock rate

• This formula makes it clear that the hardware designer can improve performance by reducing either the length of the clock cycle or the number of clock cycles required for a program.

Example : 2

Computer A runs a program in 12 seconds with a 3 GHz clock. We have to design a computer B such that it can run the same program with in 9 seconds. Determine the clock rate for computer B. Assume that due to increase in clock rate, CPU design of computer B is affected and it requires 1.2 times as many clock cycles as computer A for execution this program.

Solution:

Given: Clock rate_A = 3 x 10⁹ cycles/sec

CPU time A = 12 seconds

 CPU timeB = 9 seconds

We have,

CPU time_A = CPU clcok cycles_A / Clock rate_A

12 seconds = CPU clock cycles_A / (3 x 10⁹ cycles/sec)

CPU clock cycles_A = 12 seconds × 3 × 10⁹ cycle / sec = 36 x 10⁹ cycles

The CPU time for computer B can be given as

CPU time_B = CPU clcok cycles_B / Clock rate_B = [ 1.2× CPU clock cycles_A ] / [ Clock rate_A ]

9 seconds = [ 1.2 × 36 × 10⁹ cycles ] / [ Clock rate_B ]

Clock rate_B = [ 1.2 × 36 × 10⁹ cycles ] / [ 9 seconds ] = 4.8 cycles/sec = 4.8 GHz

Example : 3

A program runs in 10 seconds on computer A, which has a 2 GHz clock. Try to help a computer designer build a computer, B, which will run this program in 6 seconds. The designer has determined that a substantial increase in the clock rate is possible, but this increase will affect the rest of the CPU design, causing computer B to require 1.2 times as many clock cycles as computer A for this program. What clock rate should we tell the designer to target?

Solution: Given: Clock rate_A = 2 × 10⁹ cycles/sec

CPU time_A = 10 seconds

CPU time_B = 6 seconds

We have,

CPU time_A = CPU clock cycles_A / Clock rate_A

10 seconds = CPU clock cycles_A / [2× 10⁹ (cycles/second)]

CPU clock cycles_A = 10 seconds × [2× 10⁹ (cycles/second)] = 20 x 10⁹ cycles

CPU time for B can be found using this equation:

CPU time_B = [1.2 × CPU clock cycles_A ] / Clock rate_B

6 seconds = [1.2 × 20 × 10⁹ cycles_A] / Clock raе_B

Clock rate_B = [ 1.2 × 20 × 10⁹ cycles] / 6 seconds = [ 0.2 × 20 × 10⁹ cycles] / seconds

= [4 × 10⁹ cycles] / seconds = 4 GHz

To run the program in 6 seconds, B must have twice the clock rate of A.

Basic performance equation

The basic performance equation is given by

T = [N× CPI ] / R = (N× CPI) / Clock rate = N × CPI × Clock cycle time

where CPI: The average number of clock cycles each instruction takes to execute.

It is given by

CPI = CPU clock cycles / Instruction count

N: Number of instructions.

R: Clock rate measured in clocks / seconds.

Example: 4

Let us assume that two computers use same instruction set architecture. Computer A has a clock cycle time of 250 ps and a CPI of 2.0 for same program and computer B has a clock cycle time of 500 ps and a CPI of 1.2 for the same program. Which computer is faster for this program and by how much?

Solution: We know that each computer executes the same number of instructions for the program; let's call this number N. First, find the number of processor clock cycles for each computer:

CPU clock cycles_A = N×20

CPU clock cycles_B = N× 1.2

The CPU time for each machine will be

CPU time_A = CPU clock cycles_A × Clock cycle time_A

= N× 2.0 × 250 ps = 500 N ps

CPU time_B = CPU clock cycles_B × Clock cycle time_B

= N × 1.2 × 500 ps = 600 N ps

Thus we can say that computer A is faster. The amount faster is given by the ratio of the execution times.

CPU Performace_A / CPU Performance_B = Execution times_B / Execution time_A =

600 N ps / 500 N ps =1.2

We can conclude that computer A is 1.2 times faster than computer B for this program.

Example: 5

Table 1.11.1 (a) shows the two code sequences with number of instructions of different instruction classes within each code sequence, respectively. The instructions are classified as A, B and C according to the CPI as shown in Table 1.11.1 (b). From the given information:

i) Determine which code sequence executes the most instructions.

ii) Determine which code sequence will execute quickly,

iii) Determine the CPI for each code sequence.

Solution:

 i) Code sequence 1 executes : 4+2 + 4 = 10 instructions.

Code sequence 2 executes : 8+2+2 = 12 instruction.

Therefore, code sequence 2 executes more instructions.

ii) CPU clock cycles required to execute these code sequences is given as

CPU clock cycles 1 = (4 × 1) + (2 × 2) + (4 × 3) = 20 cycles

CPU clock cycles 2 = (8 × 1) + (2 × 2) + (2 × 3) = 18 cycles

Thus, we can say that code sequence 2 is faster than code sequence 1, even though it executes two extra instructions.

iii) We know that

CPI = CPU clock cycles / Instruction count

CPI₁ = 20 / 10 and CPI₂ = 18 / 12 = 1.5

• The above example shows that we can not rely on one factor to access performance. While comparing two computers we must look at all the components of performance listed in Table 1.11.2.

• If some of the factors are identical, performance should be determined by comparing all the nonidentical factors. Since CPI varies by instruction mix (A measure of the dynamic frequency of instructions across one or many programs), both CPI and instruction count must be compared, even if clock rates are identical.

• The performance of a program also depends on the algorithm, the language, the compiler, the architecture and the actual hardware.

 • Table 1.11.2 gives the information of how these components affect the factors in the CPU performance equation.

Example: 6

Consider three different processors P1, P2 and P3 executing the same instruction set. Pl has a 3 GHz clock rate and a CPI of 1.5. P2 has a 2.5 GHz clock rate and a CPI of 1.0. P3 has a 4.0 GHz clock rate and has a CPI of 2.2.

a) Which processor has the highest performance expressed in instructions per second?

b) If the processors each execute a program in 10 seconds, find the number of cycles and the number of instructions in each processor.

Solution:

a) P2 has the highest performance.

Performance of P1 (instructions/sec) = (3 x 10⁹) / 1.5 = 2 × 10⁹

Performance of P2 (instructions/sec) = (2.5 x 10⁹) / 1.0 = 2.5 × 10⁹

Performance of P3 (instructions/sec) = (4 × 10⁹) / 2.2 = 1.82 x 10⁹

b)

 Number of cycles = Time × Clock rate

Cycles (P1) = 10 × 3 × 10⁹ = 30 × 10⁹

Cycles (P2) = 10 × 2.5 × 10⁹ = 25 × 10⁹

Cycles (P3) = 10 × 4.0 × 10⁹ = 40 × 10⁹

Time = (Number of instructions × CPI) / Clock rate

Number of instructions = (Time × Clock rate) / CPI = Number of cycles / CPI

No. of Instructions (P1) = 30×10⁹ /1.5 = 20x 10⁹

No. of Instructions (P2) = 25×10⁹ / 1.0 = 25 × 10⁹

No. of Instructions (P3) = 40×10⁹ / 2.2 = 18.18 × 10⁹

Example: 7

Consider two different implementations of the same instruction set architecture. The instructions can be divided into four classes according to their CPI (class A, B, C and D). P1 with a clock rate of 2.5 GHz and CPIs of 1, 2, 3 and 3 respectively and P2 with a clock rate of 3 GHz and CPIs of 2, 2, 2 and 2 respectively. Given a program with a dynamic instruction count of 1.0 × 10⁶ instructions divided into classes as follows: 10% class A, 20% class B, 50 % class C and 20 % class D, which implementation is faster? What is the global CPI for each implementation? Find the What is the clock cycles required in both cases

Solution:

a) Class A: 10⁵ instruction,

Class B: 2 × 10⁵ instruction,

Class C: 5 × 10⁵ instruction,

Class D: 2 × 10⁵ instruction.

Time = No. instruction × CPI / clock rate

Total time P1 = (10⁵ + 2×10⁵×2 + 5×10⁵ × 3 + 2×10⁵×3) / (2.5 × 10⁹) = 10.4 × 10 ^{‒ 4} S

Total time P2 = (10⁵×2+ 2×10⁵ × 2 + 5×10⁵ × 2+2×10⁵ × 2) / (3 × 10⁹)

      = 6.66 × 10^{‒ 4} S

CPI (P1) = 10.4× 10^{‒ 4} × 2.5 × 10⁹ /10⁶ = 2.6.

CPI (P2) = 6.66 × 10^{‒ 4} × 3 × 10⁹ /10⁶ = 2.0

P2 implementation is faster.

b)

Clock cycles (P1) =10⁵ × 1 + 2 × 10⁵ × 2+5 × 10⁵ × 3+2 × 10⁵ × 3 = 26 × 10⁵

Clock cycles (P2) =10⁵ × 2+2 ×10⁵ ×2+5× 10⁵ ×2 + 2 × 10⁵ ×2+2 × 10⁵ × 2

        = 20 × 10⁵

 

Review Questions

1. What is performance?

2. Define response time and execution time.

3. What is user CPU time and system CPU time?

4. Explain the concept of relative performance with the help of example.

5. State the basic performance equation.

6. State the performance factors,

7. State the components that affects the factors in the CPU performance equation and reason behind it.

8. State the CPU performance equation and discuss the factors that affect performance.

9. Explain the important measures of the performance of a computer and derive the basic performance equation.