1.1.1 - CPU performance and pipelining
A higher advertised clock speed does not guarantee that every program finishes sooner. You will explain how clock frequency, cores and cache affect actual work, then follow overlapping instruction stages in a pipeline. The useful question is which part of a particular workload each change can improve.
Clock frequency and useful work
Clock speed, or clock frequency, is the number of timing cycles per second. A frequency of 3 GHz means three billion cycles per second. These cycles coordinate internal steps; they are not automatically three billion completed instructions per second.
Holding the processor design and the work required constant, a higher frequency provides more cycles each second and can reduce execution time. But an instruction may take several cycles, different processors can do different amounts of work per cycle, and a processor waiting for data cannot turn every extra cycle into useful progress. Comparing GHz alone across different designs is therefore weak evidence.
For a deliberately simple estimate, suppose a fixed task needs 600 million cycles with no additional waiting. At 2 GHz, time is cycles divided by cycles per second: 600 million / 2 billion = 0.30 seconds. At 3 GHz it is 0.20 seconds. The second frequency is 1.5 times the first, so the same work takes two-thirds as long. This is conditional on the cycle count staying the same; it is not a universal speed claim.
Higher frequencies can increase power consumption and heat, requiring appropriate cooling. If thermal limits force the processor to reduce its operating frequency, its sustained performance can differ from a quoted peak. A sensible comparison measures a relevant workload under comparable conditions and then uses the specifications to explain the result.
Cores need work they can share
A core is a processing unit able to execute its own instruction stream. A multicore CPU can run different programs or different parts of a parallel program concurrently. The software must expose work that can be assigned to those cores; the mere presence of four cores does not split one sequential instruction stream into four equal independent tasks.
Suppose a program resizes four independent photos. If each resizing task takes 2 seconds on one core and resources are sufficient, one core processing them in sequence takes 8 seconds. Four cores processing one photo each can ideally finish the resizing in 2 seconds. Starting the tasks, moving data and collecting results add overhead, so the practical time is likely longer than that ideal.
Now suppose a computation must finish step A before it knows the input to step B. These are dependent steps. Running B sooner on another core would not help because its correct input is not ready. Parallel work may also compete for shared memory bandwidth. Both dependencies and resource contention limit scaling.
More cores can still improve responsiveness when several programs run at once: one core can work on a background calculation while another handles a foreground application. That is a different benefit from making a single program four times faster. Always connect the claimed improvement to the software's ability to use the cores and the workload's limits.
Cache reduces memory waiting
Cache is small, fast memory close to or within the processor that keeps copies of frequently or recently needed instructions and data. Accessing a suitable cached value is usually quicker than obtaining it from main memory. Cache is a copy layer; it does not replace the need for main memory or make every program value permanently available at CPU speed.
A cache hit occurs when the required item is present in the cache. A cache miss requires obtaining it from a slower level, creating a delay before the CPU can use it. Program behaviour matters. A loop that repeatedly uses a small group of instructions and values can benefit because those items are reused; a scan over data far larger than cache may produce more misses.
Increasing cache capacity can let more useful items remain available, reducing misses for some workloads. A faster cache reduces hit-access delay. These are distinct factors: capacity describes how much it holds; speed describes how long an access takes. A larger cache does not guarantee a gain if the existing cache already holds the working data or if the program rarely reuses it.
As a simple illustration, a task reads the same small table 100 times. Keeping that table in cache avoids repeatedly fetching the same data from main memory. If a different task streams each value once through a much larger dataset, doubling cache might have much less effect. The explanation must identify the reuse and the avoided wait, rather than only say “more cache is faster”.
Pipelining increases throughput
Pipelining overlaps stages of different instructions so different processor components can work at the same time. A simple model has fetch (F), decode (D) and execute (E), each taking one time slot. While one instruction executes, the next can decode and a later one can fetch. They are different instructions at different stages, not one instruction performing all three stages simultaneously.
Without overlap, four instructions take 12 slots: F-D-E for the first, then F-D-E for the second, and so on. In an ideal three-stage pipeline with no conflicts, the schedule is:
| Slot | Fetch stage | Decode stage | Execute stage |
|---|---|---|---|
| 1 | I1 | — | — |
| 2 | I2 | I1 | — |
| 3 | I3 | I2 | I1 |
| 4 | I4 | I3 | I2 |
| 5 | — | I4 | I3 |
| 6 | — | — | I4 |
The first instruction still takes three slots from its fetch to completion. This is its latency in the model. After the pipeline fills, one instruction completes each slot: the throughput, instructions completed per unit time, is higher. Four instructions finish in six slots rather than twelve.
Pipelining improves the rate of instruction completions by overlapping different instructions. In this model, the one-slot stages and the first instruction's three-slot latency remain unchanged.
For instructions in this particular ideal three-stage model, completion takes slots. The extra two account for filling and draining. This formula depends on one-slot stages, sufficient resources and no stalls; it is not a general description of all processors.
Dependencies and branches interrupt the overlap
A pipeline's ideal schedule can fail when a later instruction needs something that is not ready. For example, I1 calculates a value and I2 immediately uses it. If the model makes that result available only after I1's execute stage, I2 may have to wait. A stall pauses progress until a required value or resource becomes available; the inserted waiting slots reduce throughput.
A branch creates a different problem. Before the branch outcome is known, the processor may already have fetched instructions from the sequential path. If the branch takes a different target, those wrongly fetched instructions must be discarded and the correct ones fetched. This is a pipeline flush. Contemporary designs can predict a branch and use techniques to reduce waiting, but an incorrect prediction still has a cost. No specific prediction implementation is required here.
The speed-up also depends on stage balance and resources. If execution is much slower than fetch, the slower stage limits the rate at which instructions leave the pipeline. If instruction and data access require the same resource at the same time, the accesses may conflict. More overlap therefore improves efficiency under suitable conditions, rather than guaranteeing a fixed multiplier for every instruction sequence.
For a workload that repeatedly branches based on newly read input, raising clock speed, adding cores or enlarging cache might affect different limits. A supported judgement identifies the actual bottleneck: shorter cycles cannot remove a dependency, extra cores need independent work, and cache helps only relevant memory access. This is how the performance factors fit together.