1.1.2 - GPUs, multicore and parallel systems
Some workloads contain many operations that can happen together, while others depend on a strict sequence. You will distinguish CPU cores, GPUs and wider parallel systems, then judge when sharing work can help. Graphics is an important example, but the same reasoning explains non-graphics applications.
Ways to perform work in parallel
Suitable work can be divided among resources that execute at the same time.
Parallel processing and multicore processors
Parallel processing performs parts of a computation simultaneously using multiple processing resources. A multicore processor contains multiple execution cores within one processor package; it is one way to provide those resources.
Each core can execute a separate instruction stream. A parallel system is broader than a multicore processor: it may use CPU cores, GPU processing units, several processors in one computer, or processors distributed across connected computers.
Software divides suitable work into tasks and allocates them to the resources. A computer processing four independent video files might assign one file to each CPU core. A network of computers might each analyse a different batch of records and send results back. In both examples, the operations genuinely overlap if resources are executing at the same time. One core rapidly switching between programs gives concurrency, but that alone does not mean their instructions execute simultaneously.
Multicore systems can improve the responsiveness of several running programs. One core may process a simulation while another handles interaction in a separate application. A single program benefits when it contains parallel work, such as independent chunks, and its software uses those cores. Adding cores cannot automatically remove an order dependency inside the program.
For a small idealised calculation, four independent tasks take 3 seconds each on one core. A serial schedule takes 12 seconds. Four equally fast cores can finish them in 3 seconds if allocation and collection take no time and there is no shared-resource limit. The stated ideal conditions are essential; the total work remains four tasks.
Why GPUs suit graphics work
A graphics processing unit (GPU) is designed to perform large numbers of suitable operations in parallel. Compared with a general-purpose CPU designed for flexible control and complex instruction streams, a GPU typically devotes many processing units to high-throughput work over data. A GPU processing unit is not simply equivalent to a full CPU core; comparing core counts directly can mislead.
Graphics contains many similar operations on large sets of vertices or pixels. To shade an image, a program may apply related arithmetic to many pixels. A GPU can process many of those values together instead of making one general-purpose core handle them all sequentially. Specialist graphics operations and hardware also support rendering tasks. The resulting throughput can support higher frame rates, higher resolution or more detailed scenes when the GPU was the limiting resource.
Single Instruction, Multiple Data (SIMD) describes applying the same operation to multiple data elements. It is a useful model for why uniform arithmetic over many pixels can suit parallel hardware. Actual GPUs can organise instruction groups in related ways; the essential idea here is many similar operations on data, rather than memorising a commercial design's terminology.
Dependencies still exist within graphics, and different pixels may take different paths through a program. The GPU does not remove every wait. A game limited by a sequential CPU operation or data loading may gain little from a faster GPU. State a graphics-linked benefit and its mechanism, rather than assume any processor upgrade improves every aspect of gameplay.
For example, repeatedly multiplying each pixel's brightness by a common factor gives many similar independent operations. Following a single chain of decisions where each decision depends on the preceding result offers much less of that parallelism. This contrasts workloads, not “GPU arithmetic versus no CPU arithmetic”: both can perform arithmetic.
GPU applications beyond graphics
The work pattern that helps graphics also occurs in non-graphics problems. The question is whether a large part of the computation can be expressed as many similar operations over data, not whether its final output is an image.
In scientific modelling, many cells in a grid may calculate their next state from a snapshot of neighbouring values. With that snapshot fixed for a step, suitable cell calculations can run in parallel, then results are combined before the next step. Dependencies between simulation steps remain, so each step must finish correctly before the next uses its results.
In machine learning, training or inference often performs large matrix operations. Many multiplications and additions can be spread across GPU resources, improving throughput for suitable datasets and software. This is a computing workload example, not a claim that the presence of a GPU alone makes a model accurate.
In data mining, a program may compare a query or calculate a score across a large number of records. If the same calculation can be independently applied to each record, GPU parallelism can help. If each record instead requires a long, different series of decisions or constant transfer of small amounts of data, a CPU implementation may be more suitable.
Consider calculating a fixed weighted score for a million records. Each score uses only its own record and the same weights, so the records form independent data-parallel work. However, copying all records to the GPU and returning results takes time. A tiny dataset might finish faster on the CPU because there is little computation to repay that transfer and setup cost.
Why speed-up has limits
Parallelism is useful only when the work and overhead permit a gain. Dependencies constrain order: if task B needs task A's result, B cannot complete correctly before A provides it. A serial portion remains a limit even when the rest is split across many resources.
For example, a job takes 2 seconds of unavoidable sequential preparation followed by four independent 2-second tasks. One core takes 2 + 8 = 10 seconds. Four equal cores ideally take 2 + 2 = 4 seconds, a speed-up of 10 / 4 = 2.5, before any extra overhead. Four cores do not deliver four times the performance because the preparation is still serial. Even indefinitely many cores cannot remove those 2 seconds.
Communication and coordination overhead include distributing data, synchronising stages and combining results. A distributed system also pays network transfer costs. If tasks are too small, the cost of organising them can exceed the time saved. Load imbalance matters too: if one task takes much longer than the others, completing the whole job waits for it while other resources may sit idle.
Resources may be shared. Several cores can compete for memory bandwidth, and moving data between CPU and GPU can limit an acceleration. Parallel algorithms must also coordinate shared updates: two tasks overwriting the same result incorrectly is a correctness problem, not useful speed. Appropriate partitioning or synchronisation is needed.
A justified recommendation therefore identifies independent work, the hardware's suitable operations, data size and transfer costs, and remaining dependencies. Parallel CPU cores suit several flexible tasks; a GPU suits large regular data-parallel workloads; distributed processors can supply more resources but need suitable communication patterns. These are contextual decisions rather than guaranteed rankings.