Printable version (PDF) of this article.

Executive Summary: 100 Year Review of Leadership Intervention Research -- Briefings Report 2004-01, Gallup Leadership Institute

By Bruce Avolio, et al., The Gallup Leadership Institute, College of Business Administration at the University of Nebraska - Lincoln

Academic Citation: Bruce Avolio, et al., "100 Year Review of Leadership Intervention Research: Briefings Report 2004-01, Gallup Leadership Institute," Kravis Leadership Institute, Leadership Review, Vol. 5, Winter 2005, pp 7-13

Keywords: Leadership interventions, meta-analysis, laboratory versus field

About the Authors: Contributing researchers at the Gallup Leadership Institute included Bruce Avolio, Adrian Chan, Nicole Chan, Holtjona Galanxhi-Janaqi, Josh Gitlitz, Sean Hannah, Katie Hassler, Linda Hietbrink, Tim Hodges, Nate Hood, Larry Hughes, Christine Foster, Ed Lankas, Cuifang Li, Ketan Mhatre, Steve Norman, Rebecca Reichard, Fred Walumbwa, Lisa Wilson, and Weichun Zhu E-mail: bavolio2@unl.edu

This two-year project began with a simple leadership question: Do leadership interventions make a difference? Many claim that training programs develop leaders, but how many have actually proven this to be the case?

During the past two years the Gallup Leadership Institute began investigating all of the literature on leadership intervention studies conducted over the last 100 years using a well-established quantitative technique called "meta-analysis."

At the outset of our review of the literature, we classified all leadership models into two broad categories: traditional leadership and new genre. By traditional, we refer to theories that dominated leadership research up to the late 1970s, including trait, behavior, and contingency approaches to leadership. By new genre, we refer to theories that have dominated leadership research in the 1980s to date, including charismatic, inspirational, transformational, and visionary leadership.

Using standard meta-analysis procedures, we calculated differences across leadership intervention conditions in each study (e.g., trained versus untrained) and aggregated those results across all studies within each class. Overall, the results of the meta-analysis indicated that experimental interventions did make a positive difference, and the difference was about the same for new genre versus traditional models of leadership.

Purpose of the Study
Our primary purpose was to investigate and analyze all leadership intervention studies that have been conducted over the last 100 years, utilizing meta-analytic techniques, to assess their results. Meta-analysis is a quantitative procedure used to review and evaluate prior research studies. This procedure is especially useful in situations where the number of studies to review becomes large and evaluating trends based on summary judgment may be unreliable. For example, an author reviews 200 studies and concludes the results are mixed when in fact the trend is positive after quantitatively calculating the effects.

In addition to investigating the overall effects of leadership interventions, we also examined the following: Does the impact of leadership differ (a) as a function of whether the study is conducted in a laboratory or field setting?; (b) as a function of nationality (i.e., U.S. vs. non-U.S.)?; or (c) as a function of industry/organization (i.e., military, industry, educational, etc.) settings?

Procedures
Inclusion Criteria. The inclusion criteria set for this study spanned all leadership intervention studies conducted over the last 100 years. This definition required that 1) the phenomenon under investigation was a leadership phenomenon, 2) that the researcher manipulated this phenomenon, and 3) that the effects of that leadership intervention were quantitatively measured. We accepted both true and quasi-experiments, although quasi-experiments (those without random assignment) comprised less than 25% of the intervention studies found over the last 100 years. Because our goal was to achieve a full population sample of all interventions, our search spanned all sources of studies, whether they were articles, books, conference papers, dissertations, or unpublished studies.

Given our goal of a full population of leadership interventions, we conducted an exhaustive, four-phase search process. In phase I, a team of twelve associates conducted a comprehensive literature review and developed a list of 18 leadership research streams found in the literature. Through group brainstorming sessions, a list of 124 search words and phrases to identify studies in each of these streams was developed for use in electronic searches. The team then conducted an initial search of 24 electronic databases.

Phase II comprised a validation search conducted approximately one year later. A second team of eight associates replicated the phase I search to determine if any articles were missed in phase I or published within the intervening year. Additionally, with the assumption that authors of intervention studies would largely draw from other intervention studies for theoretical support, the reference lists of all articles found in phase I were themselves reviewed to see if any referenced articles met our inclusion criteria. Lastly, the reference lists of 30 other previously-published leadership meta-analyses were screened to ensure we included all cited interventions.

In phase III, letters were sent to 670 scholars, asking them to review and validate the proposed listing of studies for inclusiveness, with all returned recommendations investigated. After search phases I-III failed to produce any sizeable groups of studies prior to 1970, we conducted a final phase IV search, manually searching leadership handbooks and other reference sources.

The four search phases combined netted 465 studies that were determined as possibilities for inclusion. Each study was then subjected to review by a two-person inter-rater team to ensure it met all inclusion criteria. A total of 201 studies passed this review, based on the above detailed criteria, and were moved forward into the coding phase.

Description of Variables Coded During the Analysis. A project-specific database was developed to capture 86 different data fields from each of the 201 studies. This provided a potential database of 17,286 data points, limited only by data not reported by the authors, and otherwise not able to be calculated.

As meta-analysis requires numerous decision rules during coding and analysis which will affect the quality of the methodology and the interpretability of the findings, the criteria and rules used for this project are discussed below.

Coding Accuracy and Interrater Agreement. A coding team of twelve associates was formed and trained to conduct data extraction and coding from the studies. This group was broken down into six two-person teams each assigned an equivalent group of articles. All 86 data fields for each study were coded separately by both a primary coder and a secondary coder. Data were then shared between team members for inter-rater reliability coding. Initial interrater reliability was 90%. All discrepancies in coding were then reinvestigated and consensus reached by team members or referred to a third coder, ultimately resulting in 100% agreement.

Studies by Category
Laboratory versus field. Sixty-three percent of the studies were conducted in laboratory settings, with a majority of these studies conducted in educational settings, using student samples. A total of 30% of studies were conducted in field settings. Figure 2 presents this breakdown.

National Setting (U.S. versus non-U.S.). Sixty-eight percent of studies in the sample were conducted within the United States, with 28% conducted in other countries, and 4% not reported. Less than 10% of studies reported the nationality of leaders, and less than 11% reported follower nationality (see Figure 3 for a breakdown).

Organization Type. Studies were coded into four categories of organization type: Military, industry, educational, and an 'other' category. The breakdown was as follows: 15% military, 9% industry, 64% educational and 12% other (see Figure 4).

Length of Intervention. Figure 5 presents the 142 studies that reported intervention length.

Key Findings

• Attempts at changing the impact of leadership on others through interventions worked. If the effects were random, there would be a 50/50 chance of an effect being observed. On average, we found a 67% chance of an effect.
• We discovered that interventions based on new genre and traditional leadership models had similar levels of impact, 66% versus 65%.
• Leadership effects were found to be higher overall in laboratory settings versus field settings.
• We found that 64% of the studies were conducted on college campuses, and the majority of participants were undergraduates.
• Researchers have proposed that leadership is both a universal and a culture specific phenomenon. Our analysis, did not show any difference in leadership intervention effects between U.S. and non-U.S. samples.

Implications for Research

• There are still relatively few studies that have examined the impact of leadership intervention over an extended period of time.
• Future research needs to explore what aspects of these interventions contribute to the greatest positive effects e.g., providing feedback to enhance awareness, offering people strategies to practice, etc.
• Leadership interventions matter, yet not all leadership theories/training will have the same impact on followers. We must test how to link different models of leadership to the specific types of outcomes we expect to be most impacted.
• Over 100 years, only 11 interventions in total lasted over one month. Hence, we have yet to fully test the impact of leadership interventions on follower performance.
• Individual factors such as leader/follower age, gender, and ethnicity have not been adequately explored.

Implications for Practice

• Raising people's beliefs about working with more capable, motivated followers can itself have a positive impact on performance.
• The inclusion of an evaluation strategy for leadership interventions, can itself positively impact the effects of leadership interventions on performance.
• Preliminary evidence indicates that interventions that work in one culture may also have a similar positive impact in other cultures.
• Evidence is pretty clear that attempts at changing leadership styles and behaviors did work.

Contributing Researchers