Wade Gilbert, Pierre Trudel, Stéphane Gaumond, and Léon Larocque, University of Ottawa, Canada

The first systematic observation instruments used to analyse the behaviours of coaches were developed over 20 years ago (Piéron, 1994; Trudel & Gilbert, 1995). Numerous instruments have since been developed and modified to fit a diverse range of research questions and contexts (Darst, Zakrajsek, & Mancini, 1989). Although systematic observation of coaches' behaviours has provided information on the frequency and type of coach interventions (e.g., Lacy & Darst, 1985; Tharp & Gallimore, 1976; Trudel, Côté, & Bernard, 1996), the results typically remain isolated from the subject matter (Seaborn, Trudel, & Gilbert, 1998). For the purpose of the present study, the combination of the subject matter (content) of a coach's verbal behaviours and characteristics of the delivery (pedagogical intervention) is labelled pedagogical content interventions.

The formulation of the term 'pedagogical content interventions' is based on Shulman's (1987) description of pedagogical content knowledge. Pedagogical content knowledge is defined as "that special amalgam of content and pedagogy that is uniquely the province of teachers" (Shulman, 1987, p. 8). In his early writings on the topic, Shulman (1986) concluded that researchers' negligence towards this form of knowledge created a 'missing paradigm' in research on teaching. In the sport setting, Vickers (1990) echoed Shulman's comments and attributed this missing paradigm as a primary reason for the absence a standard curriculum in sport.

Researchers in sport pedagogy have only begun to unravel the complexity of pedagogical content knowledge (e.g., Graber, 1995; Rovegno, 1994). Although this body of research has provided descriptions of its structure, how it is developed and how it influences teaching, a profile of pedagogical content knowledge in-action, referred to as pedagogical content interventions, has yet to be provided. The purpose of this study was to develop and apply an instrument to analyse youth sport coaches' pedagogical content interventions. The Systematic Analysis of Pedagogical Content Interventions (SAPCI) instrument is presented, using data from a three-year project with amateur ice hockey coaches.

Youth sport is an important context in which to study the pedagogical content interventions of coaches. Most children spend more time participating in non-school based athletics than in school-based sport programs (Ross, Dotson, Gilbert, & Katz, 1985; Trudel et al., 1996). There is no standard curriculum for coaches to use as a guide, and therefore they are free to decide on the pedagogical content presented to their athletes. This can, and will, result in wide variations in pedagogical content for athletes, even at the same age and competitive level. In addition, youth sport coaches in most countries typically have sparse formal training and may lack the foundational understanding of the sport and pedagogical concepts needed to make informed decisions when selecting pedagogical content (De Knop, Engström, & Skirstad, 1996). Based on the characteristics of the typical youth sport setting, and a gap in the sport science literature on the coaching-learning process, an important question that needs to be answered is: What is the profile of youth sport coaches' pedagogical content interventions? The remainder of the paper is divided into two sections. In the first section the development of the SAPCI instrument is explained. A sample of the type of results that can be obtained through the application of the SAPCI instrument is presented in the second section.

The SAPCI instrument was developed based on the Theoretical Knowledge Structures approach to understanding sport instruction (Vickers, 1990). The Theoretical Knowledge Structures model links the subject matter of a sport or activity with methods of teaching or coaching. The theoretical knowledge structure for ice hockey provided by Vickers was revised based on Trudel's (1987) ice hockey classification system. The SAPCI is an event recording instrument used to record pedagogical content interventions provided by coaches to their athletes during games and practices. Event recording is a form of systematic observation that is used to chronicle the frequency of discrete observable behaviours (van der Mars, 1989a). SAPCI allows a researcher to document frequency counts for the content (What), the form (How and When) and the receiver (Who) of a coach's pedagogical content interventions.

Validity

The SAPCI instrument has undergone numerous revisions over the course of a three-year project. Three researchers, all with extensive practical and research expertise in ice hockey, prepared the instrument, thereby enhancing its face validity. The three researchers had a combined total of 60 years experience in ice hockey as players, coaches and administrators and they were also certified coaches in Canada's National Coaching Certification Program (NCCP) and the Canadian Hockey Association (CHA). One of the researchers has since been hired as the head ice hockey coach of an American collegiate team. In terms of research experience in ice hockey, the group of researchers has published extensively on ice hockey coaching (e.g., Gilbert, Trudel, & Haughian, 1999; Trudel et al., 1996; Wilcox & Trudel, 1998).

Member check interviews were also conducted with eight of the coaches who participated in the three-year project. In the member check interviews, each coach was asked to review and validate coding decisions made by the researchers. Although rarely used in systematic observation studies of coaches' behaviours, this was considered a crucial step in establishing the validity of the results, because coding coaches' pedagogical content interventions can be highly subjective. In general, the coaches agreed with the researchers' interpretations. On the few occasions when a difference of opinion was noted, the researcher clarified the coach's position, modified the coding instrument if necessary, and re-coded any questionable interventions.

For the development of the SAPCI instrument, data were collected with 15 coaches of ice hockey teams in Canada. Most of the coaches (n=13) taught athletes in the youth sport (aged 10-15 years) context, either at the recreational (n=2) or the competitive level (n=11). The other two coaches were employed as university (aged 18-25 years) coaches. Nine of the coaches instructed male athletes and six coaches instructed female athletes. All of the coaches had obtained certification through the joint NCCP/CHA coaching program. A total of 85 ice hockey events were videotaped (55 games and 30 practices) in the three-year project. By gathering data across a wide range of ice hockey contexts it was expected that the full scope of pedagogical content interventions for ice hockey coaching would be observed, thereby enhancing the content validity of the instrument.

Data Collection Procedure

The coaches were videotaped in their natural environments during regularly scheduled ice hockey events. Prior to the first videotaping session with each coach, a semi-structured demographic interview that lasted approximately 60 minutes was conducted. The purpose of this interview was to obtain background information about each coach, including the coach's approach to coaching and coaching objectives, as well as to establish rapport with the coach before videotaping. At each videotaped event the coach was interviewed in a brief (5 minute) pre-event interview to obtain information about the coach's plan or objectives for that event. The event was then videotaped while the coach wore a wireless microphone. The video recorder was set-up across the playing surface from the players' bench where the coach was situated during games. For the practices, the video recorder was set-up in varying locations to obtain an optimal view of the coach and the athletes. The video recorder was focused on the coach at all times. The researchers also wore a headset to listen to the coach's pedagogical content interventions. Immediately following the event, a brief (5 minute) post-event interview was conducted to discuss any modifications made to the coach's original pre-event plan.

Coding Procedure

A joint-coding procedure was used to record the coaches' pedagogical content interventions. Two trained coders jointly coded each videotaped event (game or practice). Due to the sometimes highly subjective nature, and extreme brevity, of coaches' pedagogical content interventions, coding decisions were made through a consensual dialogue between the two coders. For example, coaches occasionally transmitted pedagogical content through one-word statements (e.g., "Shoot" or "Deep"). The coders familiarised themselves with the coaches in the study by reviewing the interviews conducted with the coaches (demographic interview and pre- and post-event interviews). The purpose of reviewing the transcripts was to gain an awareness of each coach's idiosyncrasies and coaching styles.

Reliability

In an effort to establish acceptable levels of reliability, reliability tests were conducted at periodic intervals throughout the development and application of the instrument. Reliability tests were conducted after each series of games or practices was coded for each coach (typically four games or three practices). The objective of the reliability tests was to ensure that the coders could obtain consistent coding results using the SAPCI. The reliability scores were calculated using two methods. The first method was based on scored-interval reliability testing (van der Mars, 1989b). For the purpose of the present paper, in which an event recording instrument was developed, this method will be referred to as the scored-event method. Each pedagogical content intervention that was recorded was compared across two coding sessions, separated by at least one week. A reliability score was then calculated for each component of the SAPCI instrument, using the number of agreements / (number of agreements + number of disagreements) x 100. This method provides a more accurate portrait of reliability than simply calculating the total number of agreements, as typically suggested for event recording instruments (van der Mars 1989b).

The second method used to calculate reliability was the kappa test (Cohen, 1960). The kappa formula provides an agreement coefficient that takes account of factors that can influence reliability scores such as chance agreements and the number of codes used (Suen & Ary, 1989). The kappa test has been shown to be an established method for calculating reliability in sport pedagogy research with systematic observation instruments (e.g., Sharpe, Spies, Newman, & Spickelmier-Vallin, 1996; Trudel, et al., 1996).

In the early stages of development it was not possible to achieve acceptable reliability scores for all of the components of the SAPCI. However, with the refinement of the coding instrument and coder training, reliability scores above the acceptable standards of 80% for the scored-event (van der Mars, 1989b) and .60 for the kappa (Hartman, 1977) were achieved. For example, 16 reliability tests were conducted while coding the data that is presented in the second part of this paper. The reliability scores ranged from 83% - 100% (scored-event) and .71 - 1.00 (kappa) for all components of the instrument.

The SAPCI coding system allows researchers to record data on four components of a coach's pedagogical content interventions: 'What' (is the content of the intervention), 'When' (is it transmitted), 'How' (is it transmitted) and 'Who' (is the receiver). In Table 1, the categories and codes for each coding component are presented.

The first component of the SAPCI is used to record the subject matter of the coaches' pedagogical content interventions. The subject matter is divided into 4 categories, 6 sub-categories and 28 codes. The four categories are Individual Techniques, Individual Tactics, Team Tactics, and Rules.

Individual techniques. Individual techniques are defined as the fundamental skills required to play the game (Perron & Chouinard, 1991). There are five codes in the Individual techniques category: Skating (ITSKT), Puck control (ITPUC), Passing (ITPAS), Shooting (ITSHT), and Goalie Technique (ITGOT).

Individual tactics. Individual tactics are defined as individual actions used "by one player to gain advantage over one or more opponents" (Perron & Chouinard, 1991, p. 264). The individual tactics category is divided into three sub-categories: Offence, Defence, and Neutral.

Offence is defined as when a player, on the team of the coach who is being coded, has possession of the puck. There are three codes in the Offence sub-category.

Deke another player (ITODP): pedagogical content related to how a player can manoeuvre with the puck around one or more opponents.

Deke the goalie (ITODG): pedagogical content related to how a player can manoeuvre with the puck around the opposing goalie.

Other (ITOO): any pedagogical content related to a player with possession of the puck that does not fall into one of the preceding categories.

Defence is defined as when an opposing player has possession of the puck. There are three codes in the Defence sub-category.

Against the puck-carrier (ITDPC): pedagogical content related to how a player can attempt to take puck possession away from an opponent.

Against the non-puck carrier (ITDNP): pedagogical content related to how a player can attempt to cover or impede the progress of an opponent without the puck.

Other (ITDO): any pedagogical content related to a player when an opponent has possession of the puck that does not fall into one of the preceding categories.

Neutral is defined as when neither the player nor the opponent has possession of the puck. There are two codes in the Neutral sub-category.

Work along the boards (ITNWB): pedagogical content related to how one player can overcome his or her opponent when attempting to gain possession of the puck while competing with one or more opponents along the boards of the ice surface.

Face-offs (ITNFA): pedagogical content related to how one player can overcome his or her opponent when positioned in a face-off position, unrelated to his or her position in relation to the rest of his or her team-mates, before the puck is dropped to commence play.

Team tactics. Team tactics are defined as all individual and collective actions by the players on a team, organised, co-ordinated and unified rationally with the objective of gaining an advantage over the opponent (Perron & Chouinard, 1991). Similar to the Individual tactics category, there are three sub-categories in the Team tactics category: offence, defence and neutral.

Offence - there are six codes in the Offence sub-category.

Offensive principles (TTOPR): pedagogical content related to general offensive tactics that are not bound by any zone or area of the ice surface, and when the team has possession of the puck. (Example: When a coach instructs the athletes to spread out when they are in possession of the puck).

In defensive zone (TTODZ): pedagogical content related to team strategy when the team has possession of the puck in the third of the ice surface from the end boards behind their own net to their own blue-line.

In neutral zone (TTONZ): pedagogical content related to team strategy when the team has possession of the puck in the middle third of the ice surface from one blue-line to the other blue-line.

In offensive zone (TTOOZ): pedagogical content related to team strategy when the team has possession of the puck in the third of the ice surface from the opponent's blue-line to the end boards behind the opponent's net.

Powerplay (TTOPP): pedagogical content related to when the team has a player-advantage over their opponent and the team has possession of the puck.

Short-handed (TTOSH): pedagogical content related to when the team is at a player-disadvantage against their opponent and the team has possession of the puck.

Defence - there are six codes in the Defence sub-category.

Defensive principles (TTDPR): pedagogical content related to general defensive tactics that are not bound by any zone or area of the ice surface when the opponent has possession of the puck. (Example: When a coach instructs the defencemen to never let an opponent get behind them).

In defensive zone (TTDDZ): pedagogical content related to team strategy when the opponent has possession of the puck in the third of the ice surface from the end boards behind the team's own net to their own blue-line.

In neutral zone (TTDNZ): pedagogical content related to team strategy when the opponent has possession of the puck in the middle third of the ice surface from one blue-line to the other blue-line.

In offensive zone (TTDOZ): pedagogical content related to team strategy when the opponent has possession of the puck in the third of the ice surface from the opponent's blue-line to the end boards behind the opponent's net.

Powerplay (TTDPP): pedagogical content related to when the team and has a player-advantage over their opponent and the opponent has possession of the puck.

Short-handed (TTDSH): pedagogical content related to when the team is at a player-disadvantage against their opponent and the opponent has possession of the puck.

Neutral - there is one code in the Neutral category.

Face-offs (TTNFA): pedagogical content related to how one or more players can overcome their opponent when positioned in a face-off position, related to their position in relation to the rest of their team-mates, before the puck is dropped to commence play.

Rules. Rules are defined as the official playing guidelines for the sport of ice hockey as described in the Canadian Hockey Association rule book (CHA, 1993).

Positional rules (RUPO): pedagogical content related to guidelines regarding the ice surface and player location.

Penalty rules (RUPE): pedagogical content related to guidelines for infractions that impedes an opponent's progression and that result in a penalty for the violator of the infraction.

The second component of the SAPCI documents the playing status of the athletes when they receive pedagogical content interventions from the coach.

In action (IA): Games - when the receiver of the pedagogical content intervention is on the playing surface and is an active participant while the game is in progress. Practices - when the receiver of the pedagogical content intervention is on the playing surface and is an active participant in the practice drill.

Not in action (NA): Games - when the receiver of the pedagogical content intervention is either on the bench or on the playing surface while the game play is stopped. Practices - when the receiver of the pedagogical content intervention is either on the bench or on the playing surface and not an active participant in the practice drill.

The third component of the SAPCI documents the form of the coaches' pedagogical interventions. This component is only recorded if the pedagogical content is transmitted to the athletes when they are Not in action (NA). It is believed that due to the dimensions of the playing surface and the speed of the game that pedagogical content transmitted to athletes when they are In action (IA) is not likely to be received by the athletes, or at least not as clearly. The form of the pedagogical content is divided into three categories: Instruction, Feedback, or a combination of Instruction and Feedback.

Instruction - pedagogical content transmitted to the athletes that includes an explanation on how to execute a skill or strategy, or to play the game within the rules.

General Instruction (GI): pedagogical content transmitted to the athletes that does not include specific cues on how to execute a skill, tactic, or to play the game within the rules.

Specific Instruction (SI): pedagogical content transmitted to the athletes that does include specific cues on how to execute a skill, tactic, or to play the game within the rules.

Feedback - pedagogical content transmitted to the athletes based on their performance on the ice.

Descriptive correct (DC): pedagogical content transmitted to the athletes based on the athletes' correct performance of a skill, tactic, or rule of the game.

Descriptive error (DE): pedagogical content transmitted to the athletes based on the athletes' incorrect performance of a skill, tactic, or rule of the game.

Instruction and Feedback - pedagogical content transmitted to the athletes that includes an explanation on how to execute a skill or strategy, or to play the game within the rules, and is based on the athletes' performance on the ice.

The fourth component of the SAPCI documents the receiver of the coaches' pedagogical interventions. This component is only recorded if the pedagogical content is transmitted to the athletes while they are Not in action (NA). Furthermore, this component is only recorded during practices. In games, the playing surface is large and the coach is stationary (unlike practices where the coach is typically on the ice with the players and roams around the ice). In games, many players are typically clustered around the play, and the coaches' transmissions must compete with noise from cheering fans, music, opponents, and officials. After attempting to record the receiver of the coaches' pedagogical content interventions during games, it was determined that it was too difficult to reliably record the receiver during games.

Individual player (IP): pedagogical content transmitted to one player only.

Sub-group of players (SG): pedagogical content transmitted to more than one player but less than the entire team.

Entire team (ET): pedagogical content transmitted to all players on the team at the same time.

Results of SAPCI Application

In the remainder of the paper, an example of the results obtained using the latest version of the SAPCI to code the pedagogical content interventions of ice hockey coaches is presented. The results are taken from a sample of eight coaches of youth (age 10-15 years) competitive male ice hockey teams, videotaped in 32 games and 24 practices. The results for the games are presented first, followed by the results for the practices.

'What'. On average (see Table 2), the coaches provided 101 pedagogical content interventions per game (3223 pedagogical content interventions / 32 games). However, there were wide variations among the coaches (Range = 56 - 151). All of the coaches except Coach 6 transmitted pedagogical content mostly related to team tactics (Range = 55.6% - 88.0%). When they transmitted pedagogical content related to team tactics, offensive team tactics were favoured at nearly a 2:1 ratio. With the exception of Coaches 6 and 7, individual tactics were found to comprise less than 20% of the pedagogical content interventions. However, when the coaches did discuss individual tactics with their players during games it was overwhelmingly related to defensive individual tactics. Pedagogical content related to individual techniques was seldom transmitted to the athletes during the games, constituting an average of only 12.1% of the pedagogical content interventions (Range = 7.3% - 16.4%). Lastly, the coaches rarely transmitted pedagogical content related to the rules of ice hockey during the games (Range = 0.5% - 5.3%).

'When' and 'How'. In general (see Table 3), the coaches preferred to transmit pedagogical content to the athletes when they were in action (60.2%) as opposed to when they were not in action (39.8%). This is true for all but two of the coaches (Coaches 4 and 5).

When the coaches transmitted pedagogical content in the form of instruction, it was typically specific instruction (Mean = 36.2%, Range = 19.8% - 57.0%). When the coaches provided pedagogical content in the form of feedback, they generally preferred to comment on athletes' correct execution (DC) of skills, tactics or rules. This trend was found for all but two of the coaches (4 and 8). Lastly, when the coaches transmitted pedagogical content that combined feedback and instruction, they preferred to provide feedback on an incorrect execution of a skill, tactic or rule combined with instruction to help the athlete improve his or her performance (DE + SI / GI).