Observing Practice Performance
One way we can assess learning is to record levels of a performance measure during the period of time a person practices a skill. A common way to do this is to illustrate performance graphically in the form of a performance curve, which is sometimes referred to as a learning curve. This is a plot of the level achieved on the performance measure for each time period, which may be time in seconds or minutes, a trial, a series of trials, a day, etc. For any performance curve, the levels of the performance measure are always on the Y-axis (vertical axis), and the time over which the performance is measured is on the X-axis (horizontal axis).
Performance Curves for Outcome Measures
We can graphically describe performance by developing a performance curve for an outcome measure of performance. An example is shown in figure 11.1, which depicts one person's practice of a complex pursuit tracking task. The task required the person to track, or continuously follow the movement of, a cursor on a computer monitor by moving the mouse on a tabletop. The goal was to track the cursor as closely as possible in both time and space. Each trial lasted about 15 sec. The outcome measure of performance was the root-mean-squared error (RMSE), which was described in chapter 2.
Performance curve for one person learning a pursuit tracking task. The performance measure is the root-mean-square error (RMSE) for each trial. Notice that because the performance measure is error, lower values represent better performance than higher values.
Notice that in this graph you can readily observe two of the four behavioral characteristics associated with learning. First, improvement is evident by the general direction of the curve. From the first to the last trial, the curve follows a general downward trend (note that because the performance measure is error, improvement involves decreasing error). Second, we can also see increased performance consistency in this graph. The indicator of this performance characteristic is performance on adjacent trials. According to figure 11.1, this person showed a high degree of inconsistency early in practice but became slightly more consistent from one trial to the next toward the end of practice. The expectation would be that the person would increase this consistency with additional practice trials.
General types of performance curves. When a person is learning a new skill, the performance curve for an outcome measure typically follows one of four general trends from the beginning to the end of the practice period for a skill. This period of time may be represented as a certain number of trials, hours, days, and so on. The trends are represented by the four different shapes of curves in figure 11.2. Note that in contrast to figure 11.1, the curves in this figure show better performance when they slope upward. Curve a is a linear curve, or a straight line. This indicates that proportional performance increases over time; that is, each unit of increase on the horizontal axis (e.g., one trial) results in a proportional increase on the vertical axis (e.g., one second). Curve b is a negatively accelerated curve, which indicates that a large amount of improvement occurs early in practice, with smaller amounts of improvement later. This curve is the most prominent type of performance curve for motor skill learning. It represents the classic power law of skill learning, which we will discuss in chapter 12. Curve c is the inverse of curve b and is called a positively accelerated curve. This curve indicates slight performance gain early in practice, but a substantial increase later in practice. Curve d is a combination of all three curves and is called an ogive or S-shaped curve.
Four general types of performance curves. Each curve is based on higher performance scores (which would be on the Y-axis or vertical axis) representing better performance than lower scores.
stability the influence on skill performance of perturbations, which are internal or external conditions that can disrupt performance.
performance curve line graph describing performance in which the level of achievement of a performance measure is plotted for a specific sequence of time (e.g., sec, min, days) or trials; the units of the performance measure are on the Y-axis (vertical axis) and the time units or trials are on the X-axis (horizontal axis). This curve is sometimes referred to as a learning curve.
Each curve in figure 11.2 shows better performance as the curve slopes upward. However, as we noted earlier, there are instances in which the slope of the curve is in a downward direction to indicate performance improvement. This occurs when the performance measure is one for which a decrease in the performance level means better performance. For example, measures involving error (as you saw in figure 11.1) or time (such as speed and reaction time) follow this characteristic as performance is improving when the amount of error or time decreases. In such cases, the directions of the performance curves would be opposite to those just described, although the shapes of the curves would be the same.
It is important to note that the four curves presented in figure 11.2 are hypothetically smoothed to illustrate general patterns of performance curves. Typically, performance curves developed for individuals are not smooth but erratic, like the one in figure 11.1. However, there are various statistical procedures that can be used for curve smoothing when the reporting of research results warrants it. Finally, various individual, instructional, and motor skill characteristics can influence the type of curve that will characterize a person's performance as he or she learns a skill. You will learn about several of these characteristics in various chapters of this textbook.
Performance curves for kinematic measures. When we use performance production measures, such as kinematics, we cannot always develop performance curves like the one in figure 11.1. This is the case because a kinematic measure typically does not lend itself to being represented by one number value for each trial. As you learned in chapter 2, kinematic measures involve performance for a period of time within a trial. It is important to include this time component in the graphic representation of a kinematic measure.
To assess improvement and consistency in performance for a series of practice trials, researchers commonly show one performance curve graph for each trial or a group (i.e., block) of trials. To show improvement and consistency changes, they depict a representative sample of trials from different stages of practice.
You can see an example of this approach to kinematic measures in figure 11.3. The task required participants to move a lever on a tabletop to produce the criterion movement displacement pattern shown at the top of this figure. Each participant observed the criterion pattern on a computer monitor before attempting to produce it by moving the lever. The four graphs located below the criterion pattern show one person's performance for 800 trials. The graphs show the practice trials in blocks of ten trials each. To demonstrate performance changes during practice, figure 11.3 shows four of the blocks of trials, each representing a different portion of the 800 trial session. Each graph shows two performance characteristics: the person's average (mean) pattern drawn for the block of ten trials (the solid line); and the variability (SD, or standard deviation) of the patterns drawn for the same block of trials (dashed lines).
Results of an experiment by Marteniuk and Romanow showing changes in performance accuracy (displacement) on a tracking task at different practice trial blocks for one participant. The graph at the top shows the criterion pathway for the tracking task. [From Marteniuk, R.G.,& Romanow, S. K. E.(1983). Human movement organization and learning as revealed by variability of movement, use of kinematic information, and Fourier analysis. In R. A. Magill (Ed.), Memory and control of action.]
A CLOSER LOOK Examples of Motor Skill Performance Adaptability Demands
|Closed Skills |
| || |
from wet sand, dry sand, etc.
from various locations in the sand trap
to various pin locations on the green
when shot has various implications for score
| || |
one- and two-shot free throws at various times of the game
one-and-one shot situations at various times of the game
with various crowd conditions (e.g., quiet, loud, visible behind the basket)
various types of backboards
| || |
on various types of surfaces
in various settings (e.g., home, mall, sidewalk)
while carrying various types of objects
alone or while carrying on a conversation with a friend
|Open Skills |
| || |
various types, speeds, and locations of pitches
various ball-and-strike counts
various people-on-base situations with various numbers of outs
left-handed and right-handed pitchers
| || |
balls that are different shapes, weights, sizes, etc.
various speeds and directions
in the air, on the ground
with one or two hands
| || |
various sizes of cars
various street and highway conditions
with or without passengers
various weather conditions
To determine improvement in performance, compare the early to the later practice trials by examining how the shape of the person's produced pattern corresponds to the shape of the criterion pattern. The graphs in figure 11.3 show that as the person practiced more, the produced pattern became more like the criterion pattern. In fact, in trials 751 through 760, the participant was making a pattern almost identical to the criterion pattern.
To assess changes in consistency, compare how far the standard deviation lines are from the mean pattern for each block of trials. For trials 1 through 10, notice how far the standard deviation lines are from the mean. This shows a large amount of trial-to-trial variability. However, for trials 751 through 760, these lines are much closer to the mean, indicating that the person more consistently produced the same pattern on each trial of that block of trials.
Another means of inferring learning from performance examines the persistence characteristic of improved performance due to practicing a skill. A common means of assessing this characteristic is to administer a retention test. You have been experiencing this approach to assessing learning since you began school. Teachers regularly give tests that cover units of instruction. They use these retention tests to determine how much you know, or have retained from your study. The teacher makes an inference concerning how much you have learned about a particular unit of study on the basis of your test performance.
The typical way to administer a retention test in a motor skill learning situation is to have people perform the skill they have been practicing after a period of time during which they have not actually practiced the skill. (Note that this period of time of no practice is sometimes referred to as a retention interval, which is a term commonly used in the study of memory.) The purpose is to determine the degree of permanence or persistence of the performance level achieved during practice. The actual length of time between the end of practice and the test is arbitrary. But the amount of time should be sufficiently long to allow the influence of any performance variables to dissipate to determine what was learned during practice. The critical assessment is the difference between the person's performance level on the first practice day and on the test. If there is a significant improvement between these two periods of time, then you can be confident that learning has occurred. You will see examples of how researchers have used retention tests to assess learning in the remaining chapters of this book.
One way to assess how well a person learns a serve in tennis is to use a transfer test, such as performing the serve in a tennis match.
© Karl Weatherly/Getty Images RF
The third means of inferring learning examines the adaptability aspect of performance changes related to learning. This assessment method involves using transfer tests, which are tests involving a novel situation to which people must adapt their performance of the skill they have been practicing to the characteristics of this new situation. Researchers have typically used two types of novel situations to assess learning, which practitioners can adapt for their own needs. One is a new context in which the people must perform the skill; the other is a novel variation of the skill itself. Rather than consider specific examples of how researchers have used various types of transfer tests to assess learning, we will consider how each of these types of novel situations has been used in motor learning research. You will see several specific examples of the use of these types of transfer tests in research studies in the remaining chapters of this book.
Novel context characteristics. Practitioners and researchers can use various kinds of context changes in transfer tests. Context refers to the conditions in which a skill is performed. One context characteristic that researchers have commonly used is to change the availability of augmented feedback, which is the performance information a person receives from some external source. For example, in many practice situations, the person receives augmented feedback in the form of verbal information about what he or she is doing correctly or incorrectly. If you were assessing learning to discover how well the person could rely on his or her own resources to perform the skill, then you would like to know how that person would perform without augmented feedback available. For this, an effective context change for the transfer test would be to have no augmented feedback available. It is important to note that some researchers refer to a test that involves this type of context change as a retention rather than a transfer test, because the practiced skill is performed during the test.
Another context characteristic a test administrator can change is the physical environment in which a person performs. This is especially effective for a learning situation in which the goal is to enable a person to perform in locations and situations other than those in which he or she has practiced. For example, if you are working in a clinic with a patient with a gait problem, you want that patient to be able to adapt to the environmental demands of his or her everyday world. Although performing well in the clinic is important, it is less important than performing well in the world in which the patient must function on a daily basis. Because of this need, the transfer test in which the physical environment resembles one in the everyday world is a valuable assessment instrument.
The third aspect of context that can be changed for a transfer test is the personal characteristics of the test taker as they relate to skill performance. Here, the focus is on how well a person can perform the skill while adapting to characteristics of himself or herself that were not present during practice. For example, suppose you know that the person will have to perform the skill in a stressful situation. A test requiring the person to perform the skill while emotionally stressed would provide a useful assessment of his or her capability to adapt to this situation.
Changes in the environmental context and personal characteristics provide not only opportunities to assess a person's capability to adapt what has been learned, but also opportunities to assess the stability of what has been learned. Environmental context changes, such as the presence of other people walking in a hallway or an obstacle in the pathway, can serve as perturbations that might alter the performance of a skill. Requiring a person to perform a skill while emotionally stressed can do the same. The degree to which the person's performance is disrupted by these external and internal perturbations provides evidence of the amount of performance stability a person has acquired as a result of practice.
Novel skill variations. Another aspect of adaptability related to skill learning is a person's capability to successfully perform a novel variation of a skill he or she has learned. This capability is common in our everyday experience. For example, no one has walked at all speeds at which it is possible to walk. Yet, we can speed up or slow down our walking gait with little difficulty. Similarly, we have not grasped and drunk from every type of cup or glass that exists in the world. Yet when we are confronted with some new cup, we adapt our movements quite well to the cup characteristics and successfully drink from it. These examples illustrate the importance to people of producing novel variations of skills. One of the ways to assess how well a person can do this is to use a transfer test that incorporates this movement adaptation characteristic.
Note that one of the methods that can be used to get people to produce a novel skill variation is to alter the performance context in some way so that they must adapt their movements to it. In this way, the transfer test designed to assess capability to produce novel skill variations resembles a transfer test designed to assess capability to adapt to novel performance context features. The difference is the learning assessment focus.
retention test test of a practiced skill that a learner performs following an interval of time after practice has ceased.
transfer test test in which a person performs a skill that is different from the skill he or she practiced or performs the practiced skill in a context or situation different from the practice context or situation.
Another method of assessing learning involves the observation of the stabilities and transitions of the dynamics of movement coordination related to performing a skill. According to this approach, when a person begins to learn a new skill, he or she is not really learning something new, but is evolving a new spatial and temporal coordination pattern from an old one. When viewed from this perspective, learning involves the transition from the initial movement coordination pattern (i.e., the intrinsic dynamics), represented by a preferred coordination pattern the person exhibits when first attempting the new skill, to the establishment of the new coordination pattern. (For detailed discussions see Kostrubiec, Zanone, Fuchs, & Kelso, 2012; Tallet, Kostrubiec, & Zanone, 2008; and Zanone & Kelso, 1994.) We will discuss the concept of intrinsic dynamics again in chapters 12 and 13. Stability and consistency of the coordination pattern are important criteria for determining which coordination state (initial, transition, or new) characterizes the person's performance.
For example, a person who is learning handwriting experiences an initial state represented by the coordination characteristics of the upper arm, forearm, and hand while engaged in handwriting at the beginning of practice. These characteristics make up the preferred spatial and temporal structure the person and the task itself impose on the limb, so the limb can produce movement approximating what is required. This initial stable state must be changed to a new stable state in which the person can produce fluent handwriting. Learning is the process that occurs during the transition between these two states and during the development of the consistency and stability of the new state.
An excellent example of this approach to assessing skill learning is an experiment by Lee, Swinnen, and Verschueren (1995). The task (see figure 11.4) required participants to learn a new asymmetric bimanual coordination pattern. (We briefly considered in chapter 7 the motor control difficulties associated with these types of tasks.) To perform the task, they simultaneously moved two levers on a tabletop toward and away from the body at the same rate (15 times in 15 sec). Their goal was to produce ellipses on the computer monitor. To accomplish this, they had to coordinate the movement of their arms so that the right arm on each cycle was always 90 degrees out-of-phase with the left arm. Recall from the discussion of relative phase in chapter 2 that this means that the position of the right arm's lever at any point in time had to be 90 degrees different from the position of the left arm's lever. For example when the left arm's lever was at 0°, the right arm's lever had to be at 90°. This 90-degree difference had to be maintained throughout the 15 sec of movement.
The task and results from the experiment by Lee, Swinnen, and Verschueren. The top panel shows the task, in which participants moved two levers to draw ellipses on the computer monitor (the dotted lines on each graph represent the goal ellipse pattern). The series of graphs shows the results as the left-arm × right-arm displacements of one person for the pretest and posttest (and some intermediate) trials for each of three practice days. [From Lee, T. D., et al. (1995). Relative phase alterations during bimanual skill acquisition. Journal of Motor Behavior, 27, 263–274.]
Lab 11 in the Online Learning Center Lab Manual provides an opportunity for you to experience the influence of a performance variable during practice as you learn a new motor skill.
The initial coordination pattern for the two arms for one participant is shown in figure 11.4 as the arm-to-arm displacement relationship performed on the pretest. The diagonal lines seen in the day 1 (pretest) graph were the result of the person moving the arms in-phase. The consistency of this coordination pattern is indicated by the amount of overlap of the fifteen diagonal lines produced during the pretest. Notice the person's tendency to produce that same diagonal pattern on the pretest trial on day 2, after having performed sixty practice trials of the ellipse pattern on day 1.
By the end of day 3, this person had learned to produce the ellipse pattern. Evidence for this is the consistent production of fifteen ellipses in both the pretest and the posttest trials on day 3. However, notice the instability of the performance in the many trials between the old and the new stable patterns (exhibited on the day 1 pretest and the day 3 posttest). This instability occurs during the transition between two stable states and characterizes the process of learning a new skill.
You read in chapter 9 that the dual-task procedure is commonly used to assess attention demands of activities or tasks. Because attention demands are important in the learning of motor skills, the dual-task procedure can be a means of determining if changes in attention demands for a skill change as a learner becomes more skillful, which, according to Kahneman's theory, is predicted as a characteristic of learning. Because of this expected change, we would assume that we could assess changes in attention demands of a skill or task as a means of assessing learning. Research evidence supports this expectation. For example, Gabbett, Wake, and Abernethy (2011) demonstrated that, for a fundamental rugby skill (the draw and pass), attention demands are lower for high-skilled than lesser-skilled players. In the experiment, the researchers engaged players in a standard drill involving the skill. The dual-task procedure required players to verbally respond as quickly as possible to the frequency level of a tone (high, mid, or low) they heard while they performed the draw and pass skill.
A CLOSER LOOK An Example of Practice Performance That Misrepresents Learning
An experiment by Winstein et al. (1996) is a good example of how practice performance may not represent the influence of a variable on the learning of a motor skill (see figure 11.5).
Purpose of the experiment: Which of three different knowledge of results (KR) conditions would be best as an aid to help people learn a partial-weight-bearing task? This task is a skill often taught by physical therapists. (KR refers to performance-outcome information a person receives from a source external to himself or herself.)
The task: The participants' goal was to learn to support 30 percent of their body weight while stepping on a floor scale with a preferred leg while on crutches. The target amount of weight was marked on the scale for each person. Participants in one group could see the scale needle move as they were stepping on the scale (concurrent KR). These participants were able to correctly adjust their weight on each trial. Two other groups received augmented feedback after performing the task (terminal KR). Participants in these groups could not see the scale needle during each trial, but saw a red line on the scale after completing one trial or a five-trial set (the five-trial group saw five red lines, each marked with the corresponding trial number of the set).
Practice trials and retention test: All three groups performed eighty practice trials on one day. Two days later, they performed a retention test2 that consisted of twenty trials without any KR about the amount of weight they applied to the scale.
Results: During the practice trials the concurrent KR group performed with very little error. The two terminal KR groups performed with significantly more error than the concurrent group. However, on the retention test the concurrent group performed significantly worse than at the end of the practice trials and worse than both of the terminal groups. The terminal feedback groups performed with about the same amount of error as they produced at the end of the practice trials.
Conclusion: It is important to notice that if the retention test had not been given, the conclusion about the best KR condition for learning this task would have favored the concurrent condition. However, this conclusion would be based on performance when the various types of KR were available to the participants. The more valid way to determine which feedback condition is best for learning is when no KR is available, because it reflects the therapy goal of enabling people to perform the partial-weight-bearing task in daily living conditions, which is with no augmented feedback. When the participants were tested under this condition on the retention test, the conclusion was that the concurrent KR was the worst learning condition. Thus, performance during practice misrepresented the influence of the KR conditions on learning.
Results of the experiment by Winstein et al. (1996) showing that performance during practice can misrepresent learning. The graph shows that during practice the group who received augmented feedback concurrently performed better than the other two groups. But on the retention test, this concurrent feedback group's error increased to a level that was worse than that of the other two groups. [From Winstein, C. J. et al. (1996). Learning a partial-weight-bearing skill: Effectiveness of two forms of feedback. Physical Therapy, 76, 985–993.]