4.1.3, 4.1.4, 4.1.5 - Operant conditioning and shaping

Behaviour and consequences

Operant conditioning concerns how the consequences of an action change the future likelihood of that kind of action.

An operant is behaviour that acts on the environment, such as pressing a lever, asking a question or completing a task. In operant conditioning, an action is followed by a consequence that makes similar behaviour more or less likely in future. The consequence cannot change the action that has already happened: the claim concerns subsequent behaviour.

Suppose a learner submits work and receives encouraging feedback. If submission becomes more frequent because of that consequence, the feedback has reinforced submission. Calling something a reward because the giver intended it to be pleasant is not enough to show reinforcement. Feedback that one learner values may embarrass another. We need evidence of the effect under the particular conditions.

Skinner’s research used controlled arrangements in which an animal’s response, such as a rat’s lever press, produced a consequence such as food. Response frequency could then be compared under different contingencies. A contingency is the dependence of a consequence on behaviour: food follows pressing the lever rather than being delivered regardless of pressing. Such arrangements make a behaviour–consequence relationship clearer than observing animals that receive food at unrelated times.

This contrasts with classical conditioning. Classical conditioning pairs stimuli so that a cue elicits a response; operant conditioning makes a consequence dependent on an action. Both involve learning from the environment, but identifying what is associated is essential. “Food is involved” does not decide which mechanism is operating.

Adding, removing, increasing and decreasing

Two separate distinctions classify a consequence: whether something is added or removed, and whether the behaviour increases or decreases.

Positive means that a stimulus is added. Negative means that a stimulus is removed. These words do not mean morally good and bad. Reinforcement increases behaviour; punishment decreases it. Combining the distinctions gives four possibilities.

In negative reinforcement, removal is the favourable consequence for the organism. The seat-belt warning is unpleasant, but stopping it reinforces buckling. It is therefore an error to call every unpleasant event punishment. Conversely, removing something is not automatically negative reinforcement: removing a valued privilege can punish the action that preceded it.

A consequence may affect different people’s behaviour in different ways. A teacher might intend a reprimand to reduce joking, but public attention might instead increase joking. Under that observed effect, the attention is functioning as positive reinforcement. Classification requires naming the target behaviour, the change to the environment and the subsequent effect.

These examples explain mechanisms rather than recommend harsh discipline. Ethical behaviour modification should consider wellbeing and teach a useful alternative behaviour, rather than simply try to suppress an unwanted action.

Reinforcers and schedules

Both the reinforcing value of a consequence and the rule governing when it occurs affect learning.

A primary reinforcer has reinforcing value without acquiring that value through association with another reinforcer. Food for a hungry organism and water for a thirsty organism are examples. Their value depends on the organism’s current state: food is generally less effective after satiation. A secondary reinforcer, also called a conditioned reinforcer, acquires reinforcing value through learning. Money and exchangeable tokens matter because of what they can obtain; praise can matter because of its history of association with approval and valued outcomes. A token is not primary merely because it is tangible.

A schedule of reinforcement specifies which occurrences of a behaviour produce reinforcement. Under continuous reinforcement, each occurrence is reinforced. This makes the contingency easy to learn and can help establish a new response. Under partial or intermittent reinforcement, only some occurrences are reinforced. Behaviour can then persist when a particular response brings no reward, because nonreward already formed part of its history. This is resistance to extinction, not permanent immunity to extinction.

The main intermittent schedules differ in whether reinforcement depends on response number or elapsed time, and whether that requirement is predictable.