Printable version (PDF) of this article.

Developing the Leader's Strategic Mindset: Establishing the Measures

By John Pisapia, Daniel Reyes-Guerra, and Eleni Coukos-Semmel, Florida Atlantic University

Academic Citation: John Pisapia, Daniel Reyes-Guerra, and Eleni Coukos-Semmel, "Developing the Leader's Strategic Mindset: Establishing the Measures," Kravis Leadership Institute, Leadership Review, Spring 2005, Vol. 5, pp. 41-68

Keywords: Successful leadership, reflection, systems thinking, reframing, PSLQ

About the Authors: John Pisapia is a Professor of Leadership and Policy Studies at Florida Atlantic University. Daniel Reyes-Guerra is the Principal in Residence at Florida Atlantic University. Eleni Coukos-Semmel is the Clinical Teaching Assistant at Florida Atlantic University. John Pisapia's e-mail address: bpisapia@bellsouth.net

This paper assumes that successful leaders working in complex, ambiguous or chaotic environments think more strategically than less successful leaders in those same environments. Finding a lack of appropriate instrumentation to test this assumption, a valid and reliable instrument was created for that purpose. Three cognitive processes associated with strategic thinking were identified as potential distinguishers between successful leaders and less successful leaders: reflection, systems thinking and reframing. A literature review of these processes was conducted, definitions for each of the processes were created and then potential items to describe them were developed. These definitions and items were submitted to an expert panel for content validation and refinement. The instrument was then administered to 113 practitioners to test item reliability. This test yielded a Cronbach's alpha for the total instrument (.91). The reliability coefficients for the subscales were Systems Thinking (.83) Reflecting (.85) and Reframing (.72). This paper describes the rationale supporting its assumption, reviews the cognitive processes and the procedures used to develop and validate the instrument. The 38-item instrument, referred to as Pisapia's Strategic Leadership Questionnaire (PSLQ), will be further tested to insure its ability to discriminate between systems thinking, reflection and reframing and successful leadership. Paper contains 64 references, 4 tables, and 1 figure.

In every country, leaders struggle with meeting the demands of a globalized society and local constraints. In a sense the world is becoming one place. During the latter half of the last century, leadership challenges were largely framed in the cold war context that created a stabilizing and linear notion of change. Today, the stability of all institutions is challenged by the rapid speed of change driven by globalization, shifting populations and integration of advanced information and communications technologies which diminishes the impact of time, space and distance. This transition from the modern industrial age to the postmodern information age with its accompanying complexity is creating a profound challenge for all organizational leaders.

In a concurrent research effort, Pisapia (in process) notes that leaders who find themselves in such messy, chaotic, complex environments fail for a number of reasons:

• They are trained in and rely upon a linear thinking mindset that does not work in situations characterized by ambiguity and complexity.
• They are unable to identify critical societal and institutional forces impacting their environment and thus do not connect their organizations to the current major themes associated with success.
• Their concept of change is also linear. Therefore, they overuse quantifiable parameters in the change process and seek to rationally plan their way to success.
• They do not see their organizations as dependent upon the actions and views of other organizations and individuals. Therefore, they do not connect with significant forces on their critical path of success.
• They do not connect the vital concepts of necessary organizational changes to the minds and spirit of their followers. Therefore, they are unable to benefit from empowerment and self-management.
• They use a limited set of leadership actions to influence followers to join in a common cause. Therefore, they are effective only when conditions match their one-dimensional set of leader actions.

He further asserted that the traditional heroic model of leadership seems no longer sufficient to create major and lasting change. Leaders today are surrounded by paradoxes: change and stability; linear and nonlinear thinking; localism and globalism. It is natural that some leaders are better than other leaders in understanding, interpreting and leading in these multi-polar, "messy" environments. Pisapia argues that they are better because they have adopted a strategic mindset to make sense of complexities facing the organization and applying identifying non-linear change opportunities.

Leaders today, he concluded, are faced with developing a new leadership framework. The postmodern reality requires leaders who can understand their strategic context and remain confident, competent and flexible in order to adapt their organizations. In these environments the leader's (a) ability to influence is as important as his or her position, (b) conceptual ability is indispensable in recognizing interdependencies, interrelationships and patterns, (c) ability to make consequential decisions requires both powers of analysis and intuition, and (d) communication and mediating skills are crucial. The new leader framework requires leaders to shift the balance from C² (command and control) toward C² (coordination and collaboration) in practicing their craft. Furthermore, he notes, these features of the strategic mindset should reside not only in executive leaders but also managerial leaders.

We are not alone in our assessments. Chilcoat (1995), for example, suggests that effective leaders demonstrate more complex mental processes than ineffective leaders. Moreover, Leithwood and Steinbach (1992) believe that efforts to improve the effectiveness of educational leaders may be more productive if more consideration were given to improve the quality of thinking and problem solving rather than simply focusing on actions or behaviors. It has long been commonly accepted that today's leaders lead in environments replete with ambiguities. March (1994) suggests:

"Ambiguity refers to a lack of clarity or consistency in reality, causality, or intentionally. Ambiguous situations are situations that cannot be coded precisely into mutually exhaustive and exclusive categories. Ambiguous purposes are intentions that cannot be specified clearly. Ambiguous identities are identities whose rules or occasions for application are imprecise or contradictory. Ambiguous outcomes are outcomes whose characteristics or implications are fuzzy." (p. 178)

In this environment, the key ingredients to success are leaders that recognize organizational effectiveness, continuity, and underlying change patterns (Bolman and Deal, 1994; Drucker, 1995). Schreyogg and Noss (2000) and Weick and Quinn (1999) all support the claim that there is an over reliance on linearity which does not fit with today's realities of "fast and furious change." Most researchers point to the importance of development of conceptual skills in order that leaders can: (a) apply information and concepts to practice, (b) see the organization as a whole and to understand how various parts of the organization relate to and effect each other, and (c) discern meaning in and to establish relationships between events and bits of information that, at first glance, would appear to be discrete and unrelated. The ability to think in this manner seems critical to the leaders' ability to manage complexity in the 21st century. Interpretation and meaning-making are hallmarks of transformational leadership working in complex environments. (Bass and Avolio, 1994; Leithwood, Jantzi, and Steinbach, 1999)

The research is less clear on how this is actually accomplished. Most interpretations begin with the leader's personal mastery (Covey, 1989; Senge, 1990) and his or her ability to think flexibility and strategically (Bonn and Christodoulou, 1996). The mental processes necessary to work in ambiguous, complex and conflictual situations have been described as analytical-logical or emotional-intuitive (Cohen et. al.,1993; Kets de Vries, 2001), conceptual (Chilcoat, 1995; Magee and Somervell, 1998), strategic thinking (Bonn, 2001; Pisapia, in process), holistic thinking (Senge, 1990), sensemaking (Weick, 1995), ecological thinking (Capra, 2002), practical thinking (Fallesen,1995), contextual thinking (Capra, 2002), process thinking (Capra, 2002), creative thinking (Kendall, 1990, de Bono 1996), and opportunity or obstacle thinking (Neck and Manz,1992; Godwin, Neck and Houghton, 1999).

Cognition is the way thinking is done. Mental or cognitive processes enable us to acquire knowledge by manipulating ideas and processing new information and beliefs in our minds. Information, memory, reasoning, application of schemas and biases, making attributions and thinking-through a problem are examples of cognitive processes. Many important processes have been identified in the literature such as: chunking (Simon 1947, 1999; Newell and Rosenbloom, 1981; Agor, 1988), cognitive reduction (Simon, 1947), cognitive heuristics (Stanwick, 1996), cognitive maps/schemas (March and Simon, 1958; Simon 1947; Stanwick, 1996), mental imagery (Anthony et. al., 1993; Stanwick, 1996), creativity (De Pree, 1989), mental models and schemas (Senge, 1990; Weick, 1995; Riedel, Morasth, & McGonigle, 2000), critical thinking (Baron, 1994; Halpren, 1996; Cohen et. al. 2000), pattern recognition (Simon, 1947, 1999; Cohen et al., 2000), reframing (Morgan, 1986; Bolman and Deal, 1994), reflection (Dewey 1933; Argyris and Schön, 1978; Schön, 1983), and systems thinking (Senge, 1990), Moreover, some have argued (e.g., Perkins, 1995) that there may be too many strategies for the decision maker to remember, consider, select, and apply.

The assumption guiding this work is that modern leaders must possess a strategic mindset which can be developed through the use of advanced cognitive processes. This paper defines three cognitive processes (systems thinking, reframing and reflection) thought to enable leaders to be successful in times of complexity, and then specifies how these processes can be measured. When applied, these processes should help the leader create a mindset that makes sense of complexities facing the organization and enables the leader to identify, predict, respond and adapt to non-linear change opportunities and challenges. The purpose of this study was to create a valid and reliable instrument that can be used in future studies to test the assumption that in complex environments, successful leaders think differently than less successful leaders.

Regardless of the cognitive architecture that is presumed to underlie human cognition, knowledge must be retrieved, activated, and/or recreated to influence actions and perceptions. Three cognitive processes: systems thinking, reframing and reflection were isolated as potential distinguishers between successful and less successful leaders. It is apparent from the descriptions found in the paragraphs that follow that the three processes complement each other. For example the role of context, mental models, and framing is evident in each process. However, there is enough dissimilarity to warrant individual descriptions. Leaders use information gathered through systems thinking and reframing during the process of reflection to make sense out of situations. These processes assist leaders in (a) reframing situations so they become clearer and understandable; (b) reflecting and developing theories of practice which guide actions; and, (c) thinking in more holistic ways. They also aid leaders in seeing events and problems in terms of concepts which are useful ways of thinking effectively about problems. The relationship of these cognitive processes to the strategies used by strategic leaders is illustrated in Figure 1.

Figure 1
The Cognitive Processes Needed to Practice Strategic Leadership
© Pisapia, 2005.

Systems Thinking

Systems thinking rests on the notion that the "whole is greater than the parts." In systems thinking the whole is primary and the parts are secondary. This makes systems thinking useful in understanding organic organizations. In traditional systems thinking (analytic/linear/reductionist thinking), the parts are primary and the whole is secondary. This shift in mindset from parts to wholes was the breakthrough that modern theorists needed to understand living organisms (Parsons,1960; Senge, 1990; Capra, 2002). This analysis first develops the notion of a genetic code (intrinsic properties of systems) common to all open systems and its value to system thinkers. Then, system thinking processes are explicated.

Consider the notion that system dynamics are controlled by inherent properties (the DNA) which tend to produce activity (change or stasis) in the system. If a leader understands the properties he or she can become aware of current behavior and anticipate future behavior. Therefore, there is a need to have this understanding of systems properties to fully engage in systems thinking. Simply, leaders can become better systems thinkers if, as a precondition, they develop a deep understanding of basic system properties. With knowledge of the DNA in place, effective leaders should be able to move into the skill of systems thinking that allows them to use multiple perspectives and think critically in conceptual terms about the environment in which their organizations are embedded.

Bertalanffy (1968), the father of general systems theory, identified interaction, interrelationships, and interdependence as the main characteristics of systems. Unless there is interactivity among the elements, it is not a system. As previously indicated, the major contribution of his work is its focus on wholes and not just the parts of systems. He also proposed that systems are also hierarchical in nature. All systems except the largest have supra-systems, and all systems except the smallest have subsystems. These hierarchies are unlike traditional hierarchies in the Weberian sense. They are more web-like, nested hierarchies. As Capra (2002) points out that there are no pyramids in nature only networks nesting within other networks. This focus on connectedness in terms of context and relations created a mindshift from analysis to context, from structure to process.

Capra (2002) says that in order to understand something, one begins by putting it into a larger context rather than take it apart. In this view, systems thinking is contextual. A key characteristic of systems thinking is the ability to shift one's attention back and forth between systems levels. Therefore, because of this interrelatedness, a leader could anticipate that a change in one part of the system will produce changes in other system components. The more interdependent your organization is with another the more interaction you can expect to occur.

Modern systems thinkers recognize that it is impractical to isolate the organization from its environment. They recognize that systems have permeable or impermeable boundaries that distinguish them from supra or sub systems with which they are interrelated. Therefore, leaders can anticipate that if the boundary of their organization is too permeable (i.e., receives too much environmental stress) the organization will lose its focus, explode, and disintegrate. On the other hand, if it is too impermeable, it will implode and collapse. Therefore, leaders should actively practice boundary maintenance by establishing and loosening boundary barriers that regulate stress on the system. Additionally, the greater the complexity and turbulence found in a system's environment, the more leaders should anticipate the need for establishing boundary-spanning roles to influence intersystem relations.

Another systems property describes systems as in a constant state of dynamic equilibrium. Dynamic equilibrium refers to a continual state of adaptation that allows the system to grow and change in a stable way. It has been noted that left unattended, systems tend toward entropy, inertia, randomness and disorder. Systems decrease this tendency by exchanging energy and information with their environments and transforming the inputs to valued outputs to remain in balance with its environment. Capra (2002) describes this phenomenon: a dynamic balance is created that combines consistency of pattern with continual structural change. In this new thinking, there are cyclical changes and evolutionary changes which are not just a reaction to the environment. Therefore, if leaders do not perceive changes in their organization's environment, the system is thrown into a state of imbalance. On the other hand, leaders who do anticipate environmental changes and make efforts to adapt to the changes stay in a state of dynamic equilibrium. As Katz and Kahn (1978) and other systems researchers have noted, leaders who do not move to adapt their organizations, or do it too late, cannot survive.

The capacity of a system to self-correct by adjusting to feedback is another key element. Positive feedback enables a system to maintain its behavior while negative feedback prompts the system to adapt. Feedback is received from the external environment in the form of consumer concerns or needs, profit margins and internal and external audits. Leaders who understand that it is normal for systems to adjust to feedback will monitor the "noise" in the system to make adjustments before it becomes overwhelming.

Finally, the property of equifinality maintains that systems can achieve different results from similar inputs and similar results from different inputs. Leaders who follow the equifinality concept can be confident that: (a) initial inputs do not decide the degree to which a goal will be achieved; (b) results may vary even though inputs are "equal," and; (c) there is no one "best way" to achieve an organizational goal. Therefore, they will adjust strategies to influence results their organizations seek.

Armed with these understandings of the intrinsic properties of systems to guide leader action, the analysis now moves to development of the notion of the cognitive processes necessary to engage in systems thinking. Senge (1990) describes a series of mindshifts that need to occur, so leaders: (a) perceive interrelationships rather than linear cause-effect chains, and (b) understand change as a process rather than an event. He asserts that this shift from linear thinking to holistic thinking must first take place in the mind of the leader and the work group.

There are several abilities necessary to properly utilize the process of systems thinking. Systems thinking requires that the leader understands that he or she is part of the feedback process, not standing apart from it. This understanding represents "a profound shift in awareness" that there is connectivity between members of organizations that influences the way a system works. The perspective gained from looking at feedback in this way "suggests that everyone shares responsibility for problems generated by a system." (Senge, 1990, p. 78) This feedback perspective becomes especially significant when leading organizations. Organizations are always involved in processes that determine the output and direction of the organization. Senge recommends that in order to understand a balancing feedback process the systems thinker must "start at the gap - the discrepancy between what is desired and what exists… then look at the actions being taken to correct the gap." (p. 87) The leader must then translate the understanding into action. Senge emphasizes that the "bottom line of systems thinking is leverage - seeing where actions and changes in structures can lead to significant, enduring improvements." (p. 114)

The value of systems thinking in leadership for organizations is even greater in the post modern period where it is thought organizations must be learning organizations to be successful. Gailbraith (2001) makes an important emphasis concerning "utilizing the importance of structural relationships for planned and sustained change." (p. 2) For Gailbraith, the focus is on two feedback loops. "All dynamics in systems arise from the interaction of just two types of feedback loops; positive (self-reinforcing) and negative (self-correcting or balancing) loops. (p. 4) Senge also considers these loops, and focuses his analytical lens on ten systems archetypes drawn from those loops that systems thinkers have to understand and know how to act upon. The test of a systems thinker would be to be able to recognize the archetype and then act accordingly. Senge (1990) has developed and explained the archetypes in both his seminal work of The Fifth Discipline and The Fifth Discipline Fieldbook, and they will not be described further here.

Finally, a major component of systems thinking is looking for connections in the system (Beckhard & Pritchard, 1992). What is important is for the systems thinker to see and understand the ways that changes to one part of a system might affect, and be affected by, other parts of the system. This understanding and action can only be achieved through recognition of systems properties and dynamics, as well as the specific systems archetypes.

Drawing on all of the preceding properties and processes, a definition of systems thinking was extracted, tempered by the understanding of systems properties. The definition that resulted views systems thinking to be the ability to see systems holistically by understanding the properties, forces, patterns and interrelationships that shape the behaviors of the systems which provide options for action. This definition was submitted to a panel of experts who teach organizational behavior, and revisions were made based on their input. (See Table 1). This definition was used in creating test items to measure systems thinking which is further explained in the methodology section of this paper.

Table 1.
The cognitive processes necessary to practice systems thinking.

Reframing

Leaders frame and place all situations in context. Framing is a cognitive process that helps us gather and organize information and create knowledge. It involves sorting and interpreting the meaning of new information, events, and experiences. Framing imposes assumptions and provides a language for analysis of behavior in which certain aspects of situations are chosen to be considered and interpreted. Typically, we reach for frames when trying to understand new, complicated events and to have control over how communications, goals and initiatives will be perceived. However, the manner in which a leader frames a situation is crucial to his or her understanding and public reasoning. Framing a problem is a subjective process. As Cuban (1990) explains,

"It depends upon one or more facts that show a discrepancy between what is and what ought to be done. It depends upon the perceptions of the person or group that interpret the data and do the defining. What shapes [these perceptions] are [one's] previous personal and work experiences, [one's] beliefs and values, the position [one holds] within an organization, and the expected role [one] is to play within that organization. "(p. 2)

While each frame (also referred to as lenses, metaphors, mental models, paradigms, and frameworks) provides information, it can also create blind spots by not allowing some information to surface. The process of reframing a situation attempts to negate this weakness. Therefore, reframing is the process of examining the same situation from multiple perspectives. Typically, we reframe when trying to understand new, complicated events, to promote our own preferred outcome or change how actions are perceived. Each perspective (frame) provides information to understand the situation in its entirety. Properly conducted, reframing enables leaders to diagnose their context, define critical issues, interpret them and communicate in ways that empowers their followers. As Kouzes and Posner (1993) assert, the "more you know about the world, the easier it is to approach it with assurance." (p. 299)

Reframing has grown in importance since the mid-1980s when Gareth Morgan's (1986) influential book, Images of Organizations, was published. The main thesis was that "situations and/or problems could be framed and reframed in different ways allowing new kinds of solutions to emerge." (p. 337) This work was soon followed by Bolman and Deal (1991) in which they noted that the inability of managers to "consider multiple perspectives continually undermines efforts…to change organizations." (p. 309) These important books described the value of the reframing approach and its ability to analyze and respond to organizational situations through the use of multiple frames.

Fluency in multiple frames is considered a basic skill for post modern leaders. It is critical for them to make their dominant frames explicit and to widen their frame repertoire. Reframing is a conscious effort by leaders to switch attention across multiple perspectives, frames, mental models and paradigms in order to generate new insights and options for actions. The goal is to produce usable knowledge. The reframing process is accomplished by rotating through appropriate conceptual models for the activities and events observed. This process can overcome Bolman and Deal's (1991) assertion that frames filter out some things and allow others things to pass through quickly.

The ability to reframe complex problems is an important precondition to the effective exercise of leadership. Bolman and Deal (1991) suggest that, "the ability to reframe experiences enriches and broadens a leader's repertoire and serves as a powerful antidote to self entrapment." (p. 4) For them, the leader's ability to reframe a problem involves a conscious effort to size up a situation using multiple lenses. Managers who master the ability to reframe report a liberating sense of choice and power. They are able to develop unique alternatives and novel ideas about what their organization needs. They are attuned to the people and events around them. They are less often startled by organizational perversity, and they learn to anticipate the turbulent twists and turns of organizational life. The result is managerial freedom - and more productive, humane organizations. (p. 17)

There are several other abilities necessary to properly utilize the process of reframing. Foremost is that multiple frames must be used. The use of multiple frames offers leaders the ability break out of traditional perspectives and structures and to identify and to introduce changes. As Morgan says, "situations and problems can be framed and reframed in different ways allowing new kinds of solutions to emerge." (p. 337) Similarly, for Bolman and Deal (1991) the leader's "inability to consider multiple perspectives continually undermines efforts…to change organizations." (p. 309) Furthermore, mastering the ability to diagnose a situation from multiple perspectives is the first step in helping others to do so. It requires that the leader hold multiple images simultaneously in his or her mind. Morgan (1986) contends that leaders need to develop the: "capacity to remain open and flexible, suspending immediate judgments whenever possible, until a more comprehensive view of the situation emerges. They are aware of the fact that new insights often arise as one reads a situation from "new angles," and that a wide and varied reading can create a wide and varied range of action possibilities. Less effective managers and problem solvers, on the other hand, seem to "interpret everything from a fixed standpoint. As a result, they frequently hit blocks that they can't get around." (pp. 11-12)

Another crucial ability within reframing is that judgment must be suspended until the process is completed. Misuse of reframing can result in leaders often making the mistake of taking a stance on a preconceived solution before a problem has been clearly defined and understood. The importance of a leader's ability to recognize a problem that has been presented with a predetermined solution, and then to reframe the problem in solution-free terms cannot be overstated. While different and equally sound solutions to a problem may exist, the way that a problem is framed essentially determines whether a predetermined solution will be conceived or whether other solutions will be considered. (Copland, 2000)

Another reframing ability leaders must develop is to identify and understand the metaphors, mental models, paradigms and frameworks that are being used to frame a problem, situation or issue. Metaphors are linguistic expressions which allow leaders to understand new information in the context of what they already know and reject information that does not fit the existing metaphors. Metaphors connect random information to myths, ideologies and stereotypes. These metaphors can reinforce current worldviews or they can guide future actions. Mental models can be defined as a prototype for thinking and acting, and/or a filter through which individuals observe their world. Senge (1990) describes them as deeply ingrained assumptions, generalizations, and images that influence how we understand the world and how we behave within it. This identification of mental models enables leaders to:

• Maximize their ability to grasp motives, beliefs, concerns, interests, unconscious behaviors and customs.
• See the world as others see it, to live in their time frames, to capture the events in and on their own terms and grasp the culture.
• Provide the administrator with access to the emotional reactions of the group, and
• Build on their own, and the group's, implicit knowledge.

Paradigms, on the other hand, are the most powerful frames because of their power of persuasiveness. They are well researched and have a large following. Paradigms illustrate big ideas and how they are connected. As approximations of reality, they have an internal logic and a set of assumptions which enable practitioners to make inferences about some event. The paradigms that one ascribes to in common practice are known as one's "worldview."

Spillane et. al. (2002) call mental models, paradigms, and frameworks "schemas" which they describe as knowledge structures that link together related concepts used to make sense of the world and to make predictions. They are composed of an individual's prior knowledge and experience, including tacitly held expectations and beliefs about how the world works. These schemas

"serve as a lens influencing what the individual notices in the environment and how the stimuli that are noticed are processed, encoded, organized, and subsequently interpreted. . . .schemas are not just collections of associated features; rather, they seem to encode causal explanations or theories about how the world operates." (p. 394)

Spillane and his colleagues further point out that research on schemas also stresses the presence of mental models. They describe the value of mental models in the following way: "people construct intuitive models from their experience, apart from formal instruction, and use those models to envision a situation, essentially 'running' the model to make predictions about its causes and outcomes." (p. 395)

Finally, reframing may be used to consider and offer new ideas. It is central to discovering both strategies and goals. Schön (1983) clarifies this point, "As inquirers frame the problem of the situation, they determine the features to which they will attend, the order they will attempt to impose on the situation, the directions in which they will try to change it. In this process, they identify both the ends to be sought and the means to be employed." (p. 165) The importance of reframing is found in its ability to plant and nurture new values in an organization while jettisoning old meanings and understandings. (Snow et al., 1986) When the stories and understandings are no longer relevant and do not provide guidance it's time for the leader to reframe the situation.

Drawing on the discussion provided in this section, a definition of reframing was extracted from this analysis. The resulting definition is that reframing is the ability to switch attention across multiple perspectives, frames, mental models and paradigms in order to generate new insights and options for actions. This definition was submitted to a panel of experts who teach organizational behavior and revisions were made. Table 2 displays the definition and the abilities used in reframing.

Table 2.
The cognitive processes necessary to practice reframing.

Reflection

Reflection is a cognitive skill that involves careful consideration of any belief or practice that promotes understanding of situations and then applying newly gained knowledge to these situations. It relies on subjecting evidence, perceptions and experience to critical scrutiny in order to make sense and meaning of situations prior to weaving the thinking into a theory of practice. By reflecting on both successes and failures we begin to unpack the assumptions and values that lie beneath rules, regulation and processes in our work and everyday life. This constant effort of reevaluation and interpretation is an integral part of how leaders make sense of situations. Even though the leader is without all the information needed, the use of reflection will offer the best possible options for action and prediction.

For Dewey (1933) reflection is a disciplined, active meaning making process and not simply equated with "mulling" something over. It creates a deeper understanding of relationships and connections among experiences and ideas. He claimed that the reflective thinker

"...must evaluate the potential solutions to the problem in light of existing information, information that may be incomplete and unverifiable…. Reflective thinking requires the continual evaluation of beliefs, assumptions, and hypotheses against existing data and against other plausible interpretations of the data. The resulting judgments are offered as reasonable integrations or syntheses of opposing points of view. Because they involve ongoing verification and evaluation, judgments based on reflective thinking are more likely to be valid and insightful than are beliefs derived from authority, emotional commitment, or narrow reasoning."(Dewey, 1933, p. 6-7)

The ability to combine reflection and action is a central theme found in the literature. For example, Dewey (1933) referred to reflection as a systematic process of thinking and reflective actions and dispositions. Korthagen (1988) defined reflection as a cycle of thinking and acting which includes action, looking back on the action, awareness of essential aspects, creating alternative methods of actions, and trial, which is the start of a new cycle of reflection. Argyris and Schön (1978) made a significant contribution to this discussion when they split theory and action. They argued that people have implicit mental maps (theories in use) that tell them how to act in certain situations and other theories (espoused theory) which are used to explain our actions to others. Espoused theory is the way we answer when someone asks how we would behave in a certain situation. However, it is the maps (which people may be unaware of) rather than the theory which actually guide actions. They argue that more effective leaders would benefit from a greater correspondence between theory in use and espoused theory. They offer reflection as the process to use to bring about this.

Schön (1983) further explained that reflection can be used in-action and/or on-action. Reflection on-action is sometimes characterized by "stepping back" and considering multiple perspectives and solutions. In this type of reflective activity, effective leaders reflect on their practice in terms of personal experience, previous successes, hopes, preferences, strengths and weaknesses, and desired outcomes. On the other hand, reflecting in-action is sometimes thought of as the ability to "think on one's feet" by drawing on a repertoire of experiences, feelings, mental maps, metaphors and theories of practice from prior reflections on-action. By including reflection in-action as part of his concept, Schön recognized that practitioners faced with short time lines and decisions have to be rapid. He encouraged them to think as artists. Reflection is not something leaders just do "on their time off" or when they have time to think about things. This point has sometimes been lost on practitioners with a limiting view of reflection who see primarily as making sense of an experience after it has happened.

Reflection requires the ability to not be hindered by one's own personal intervening constructs. There is an element of reflection that requires a disengagement of the self while being constantly engaged in rethinking and reexamining. For Kegan (1994) reflection is as not just a skill, but as "an active demonstration of a mind that can stand enough apart from its own opinions, values, rules, and definitions to avoid being completely identified with them. It is able to keep from feeling that the whole self has been violated when its opinions, values, rules, or definitions are challenged." (p. 231) This ability of personal value suspension is facilitated by creating an internal conversation. Reflection, according to Schön (1983), involves "spot surfacing, criticizing, restructuring, and testing of intuitive understandings of experienced phenomena; often it takes the form of a reflective conversation with the situation. This form of reflection could either involve 'Internal Conversations' (What is this telling me? How could I have handled it better? What will I do next time) or 'Reflective Dialogues' (Drop-in conversations with a respected colleague to share both questions and insights, to bounce ideas around, often generating other insights and confirming the sense of direction)." (p. 3)

Several taxonomies of reflection are found in the literature: technical reflection (Senge, 1990) practical reflection (Senge 1990) and critical reflection (Mezirow 1990; Senge 1990; King & Kitchener, 1994). Technical reflection often comes from academic knowledge and involves following routines and a plan. Practical reflection (sometimes referred to as tacit knowledge) is based on reflection on experience and how things work or don't work in practice. It is practice oriented and very useful in specific contexts where its focus is on what is "possible" and is more creative and spontaneous than technical thought. Critical reflection asks what is "right" and is more questioning of assumptions of policies and processes. It requires, as King and Kitchener (1994) say, "the continual evaluation of beliefs, assumptions, and hypotheses against existing data and against other plausible interpretations of the data." (p. 7)

Senge (1990) uses the three types of reflection when he describes professional practice based on reflective thinking in terms of levels. He says, "The first level is technical reflection which is concerned with examining the efficiency and the effectiveness of means to achieve certain ends. The second level, practical reflection, involves examining not only the means but also the ends, questioning the assumptions and the actual outcomes. The third level is critical reflection, which considers the moral and ethical issues of the social compassion and justice along with the means and the ends, encompassing the first two levels." (p. 2) Of the three types, critical reflection is the most necessary for transforming oneself and organizations. As Mezirow (1990, p. 12-13) points out, "We become critically reflective by challenging the established definition of a problem being addressed, perhaps by finding a new metaphor that reorients problem-solving efforts in a more effective way."

Argyris and Schön (1978) have a similar way of describing reflective thought. They differentiate between single and double loop learning. Argyris and Schön describe single-loop learning as reaction to circumstances based on taken for granted values, goals, and frameworks. They point out that, in single loop learning, reflection is commonly focused on making the organization more efficient and the detection and correction of error. The emphasis is on techniques and making techniques more efficient (Usher and Bryant, 1989). In single loop learning the governing variables of a situation are not questioned. If reflection occurs in this situation it is simply to make the organization more efficient. They just look for another strategy to achieve its present objectives. Single-loop learning seems to be present when goals, values, frameworks and, to a significant extent, strategies are taken for granted.

In contrast, the metacognitive process of double loop learning requires the questioning of frames and values since the mental models of the leader may result in modifying the organization's priorities, policies, culture and practices. White (2002) points out that "the core of double loop learning is to discern the 'big picture' and not simply let the dynamics of the issue at hand dominate the thinking process." (p. 3) As he describes it, in double loop learning, the person tries to stand outside the situation and observe what is happening, analyzing how the problem has been framed, observing his or her own behavior as well as that of the other people in the situation, looking at the structural properties and the underlying dynamics of the situation. This quality of going below the surface and looking at the foundations is recurrent in the literature. It is through the use of double loop learning that the individual engaged in reflection can adopt or reinforce the strategies that emerge from the new analysis of frameworks and values. In a sense, it is a way of evaluating one's intellectual performance through an internal dialectic and enables us to learn our way out of dilemmas. The ability to engage in critical reflection and double loop learning, therefore, is essential to reflection and reframing and systems thinking. For example, Schön (1983) believes reframing is necessary for double loop learning and vice versa. However, double-loop learning "involves questioning the role of framing . . . which underlie actual goals and strategies." (p. 165)

Reflection also requires the ability to use current perceptions, experience, and knowledge and that of others from past experience to create an understanding of the present and the future. King and Kitchener (1994) outlined assumptions concerning the development of this aspect of reflective judgment. They indicate that individuals function within a "developmental range" of stages. For example, one of these stages, that of reasoning (which they consider to be stage six of development), they discuss how a reflective judgment must carry with it the understanding that:

• Knowledge is uncertain and must be understood in relationship to context and evidence.
• Some points of view may be tentatively judged as better than others.
• Evidence on different points of view can be compared and evaluated as a basis for justification. (p. 254)

The use of others' experiences and point of view as a part of one's own reflective practice is also found in Rodgers (2002) analysis of Dewey's work. She points out, "an experience, then, is not an experience unless it involves interaction between the self and another person, the material world, the natural world, an idea, or whatever constitutes the environment at hand." (p. 846) Argyris and Schön (1996) as well see the use of others perspectives as necessary for reflection. They indicate that reflective practice involves thoughtfully considering one's own experiences in applying knowledge to practice while being coached by professionals in the discipline.

The relationship between individual and organizational learning, leadership and reflection is another strong theme found in the literature. For example, Dewey, recognizing its reflective nature, gave significant consideration to the relationship of reflection and learning. Rodgers (2002), interpreting Dewey, summed up this relationship with the following points:

• Reflection is a meaning-making process that moves a learner from one experience into the next with deeper understanding of its relationships and connections to other experiences and ideas. It is the thread that makes continuity of learning possible, and ensures the progress of the individual and, ultimately, society. It is a means to essentially moral ends.
• Reflection is a systematic, rigorous, disciplined way of thinking, with its roots in scientific inquiry.
• Reflection needs to happen in community, in interaction with others.
• Reflection requires attitudes that value the personal and intellectual growth of oneself and of others. (p. 845)

Drawing on the preceding section a definition of reflecting was extracted. The resulting definition portrays reflection as the ability to weave logical and rational thinking through the use of perceptions, experience and information to make judgments as to what has happened and then create intuitive principles that guide future actions. This definition was submitted to a panel of experts who teach organizational behavior and revisions were made. This definition found in Table 3 was used in creating test items to measure reflecting.

Table 3
The cognitive processes necessary to practice reflecting

In summary, systems thinking, reframing and reflection in all of their forms are important cognitive skills for leaders to possess. The perspective one takes away from this literature is that there is no one best way to create corrective or unique solutions. Each situation presents different motives, problems and preferred outcomes that result in different choice of strategies. The job of the leader is to select the process that fits the circumstances. Systems thinking gives the leader the ability to see patterns and interrelationships. Reframing provides the advantage of multiple perspectives. Reflection gives the leader the ability to see why certain choices work and others do not, which is tied directly to the uniqueness of each choice situation.

The analysis also led to the belief that leaders who possess the ability to engage in these cognitive processes will be more effective than those who possess these abilities in lesser quantities. The use of these processes enables the leader to build a reservoir of insights and intuitions which can be called upon when confronted with ambiguity, complexity and dilemmas. To test this assumption, the procedures to construct a valid and reliable instrument are detailed in the next section of this paper.

Although the primacy of reflecting, reframing, and systems thinking are supported by the literature, no assessment tools were readily found which measure a leader's ability to perform these processes. The literature is replete with (a) knowledge and indicators of leadership capability, (b) past performance appraisal ratings and/or assessment center evaluations, and (c) instruments regarding leaders' attributes, processes, and functions, there is a paucity of instrumentation measuring the leader's mindset. As Bonn (2001) noted there is a need to develop tools for testing the strategic thinking ability of leaders. She asserted that although there are a number of tests available that look at aspects of strategic thinking, e.g. creativity (Hocevar and Bachelor, 1989) or divergent thinking (Torrance, 1974), more comprehensive testing measures need to be developed. Such research should lead to the identification of criteria which best predict strategic thinking ability. Following Bonn's advice and our own failure to identify a test for our purposes, our attention turned to the next needed step of the creation of a tool that evaluates the mental processes of leadership.

The general design of this study was nonexperimental in nature. The project was undertaken in three basic phases. In the first phase, a literature review was conducted to support or refute the constructs as promoted by Pisapia (in process) as well as create the knowledge base to draw on for the eventual creation of an instrument. This process resulted in creation of the conceptual framework guiding the study. Drawing on the analysis, clear definitions of systems thinking, reframing and reflecting and the abilities necessary to utilize these processes were extracted. The definitions and necessary abilities to be measured were submitted to a panel of three experts who teach organizational behavior. The panel members were instructed to analyze the definitions and abilities to determine content validity. They were asked to indicate (yes/no) whether the definition and description of each of the cognitive processes adequately described the concepts of reflection, reframing, and systems thinking. Revisions were made based on their responses. The resulting definitions and abilities are found in Tables 1, 2, and 3 at the end of this paper. Once the definitions and abilities were secured the instrument development was begun. This phase is fully described under instrument development in following paragraphs. In the final phase of the study, two pilot studies were designed and conducted to establish the reliability of the instrument. These pilot studies are described below in the instrument reliability section.

Following recommendations of Crocker and Algina (1986), a pool of items representative of the three cognitive processes was developed and submitted to the panel of experts who clarified and eliminated items. The remaining items were pilot tested; items were clarified and eliminated until reliability could be established.

Instrument Validity

Content validity was achieved in the following manner. An initial 189 item pool was created using direct phases or sentences found within the literature describing specific behaviors, actions or assumptions involved in the cognitive processes. Following Schwartz's (1987) criteria of simplicity, familiarity and unambiguousness, the item pool was narrowed to 64 items. The 64 items were placed in a random order in an instrument format. The items were not presorted by cognitive skill construct. The instrument was given to the panel of experts for an item analysis. The panel members were asked to use the given definitions to sort and rank the 64 items by cognitive process area. They judged whether each of the given statements was a valid measure of one or all of the three cognitive skills. The protocol required that all three panel members agreed on placement to retain an item. Moreover, they were asked to add any items they felt had been omitted in each of the cognitive process areas and add any comments or suggestions to the instrument. The panel also provided feedback on clarity of each of the items. When an item was deemed to be relevant, but not clear, it was rewritten. Through this iterative process, additional items were deleted and 52 items were retained for the first version of the instrument. Additionally, the panel was also asked to judge the importance level of 14 system property items on a 5-point Likert scale (anchored by 5 = critically important; 1 = not important) of each of the items. Those items with a mean score of 4.0 or higher were retained for the pilot instrument. Seven of the 14 items were retained and added to the instrument. These procedures resulted in at total of 59 items found on version one of the instrument now called Pisapia's Strategic Leadership Questionnaire (PSLQ). A response mode asking the frequency that the leader uses the 52 measures of systems thinking, reframing and reflection on a Likert Scale was incorporated into the instrument. Respondents were asked to indicate their level of agreement with the 7 statement regarding system properties on a Likert Scale was also incorporated into the instrument.

Instrument Reliability

Reliability of the PSLQ instrument was estimated using correlations between cognitive skills subscales in two pilot studies. Version one of the PSLQ was administered to a group of masters level students (n=34) enrolled at Florida Atlantic University preparing for leadership positions. All subjects held full time supervisory, management or leadership positions in addition to pursing their degrees. Coefficient alphas were estimated for the items of each sub-scale, the sub-scales, and the total scale. SPSS version 11.5 was used to evaluate item responses. Results showed (a) how each item correlated with the total score; (b) how the items correlated with one another; and, (c) how subscales related to one another. Twelve (12) items having a negative or low correlation with the total score of the sub-scale were deleted at this point of instrument development. Reliability tests were run again on the remaining 47 items and the sample yielded a high Cronbach alpha (.89) for the total instrument. The Cronbach alphas for the subscales were moderately high for Reflecting (.84) and Systems Thinking (.82) and only moderate for Reframing (.72).

The revised PSLQ with 47 items (Version 2) was administered to a second sample of like students at the same university (n = 79). Responses from these individuals were used to determine an additional set of reliability estimates for the instrument. Again, items having a negative or low correlation with the total score of the sub-scale were deleted. Two items were deleted during this phase of instrument development. This second pilot test group yielded a slightly higher Cronbach's alpha for the total instrument (.91). The reliability coefficients for the subscales were somewhat higher than the first group for Systems Thinking (.83) and Reflecting (.85) and stayed the same for Reframing (.72). (see Table 4) These results were encouraging as they were above the .70 threshold as indicated by Nunnally and Bernstein (1994). The pilot tests resulted in creation of PSLQ version 3 composed of 45 items which can be administered in twenty minutes. The PSLQ can be retrieved from the author of this paper.

Table 4
Means, Standard Deviations, and Item Correlations for Pilot Study Version 2 (n= 79)

The analysis revealed that crucial cognitive processes needed by strategic leaders in the post modern organizational world were not being evaluated. This circumstance created a vacuum in an area of leadership identification that has both been recognized over 70 years ago in seminal works regarding reflection (Dewey, 1933, Argyris & Schön, 1978) and brought to the forefront over 10 years ago concerning reframing (Morgan, 1986; Bolman & Deal, 1991) and systems thinking (Bertalanffy, 1968; Senge, 1990). This study sought to fill this vacuum.

In this paper the process used for designing, testing, and validating an instrument to measure the cognitive processes of leaders was described. The final product, the PSLQ, is an instrument consisting of 38 items organized into the following three scales: reflection, reframing and systems thinking. The preliminary item analysis-examining the inter-correlations of individual survey items - as well as the means told us to retain the remaining 38 items for substantive analysis. Therefore, we combined the item means and standard deviations from both groups participating in the reliability testing procedure (n = 113). The three scale mean ratings range from 3.46 - 3.85 on a 5-point scale. Standard deviations of individual items are less than 1.00, indicating a strong agreement among most reviewers on the ratings for a given domain of items. An additional piece, a survey on systems properties (7 items) was included for further study in terms of the relevance of understanding systems properties for the ability to engage in systems thinking, reframing and reflection. The PSLQ received high scores on validity and reliability tests suggesting that the instrument will yield the information it claims. Validity was established by subjecting the definitions and instrument items to a panel of experts in an iterative process until there was agreement on both definitions and items. Sample size for the first reliability runs of the both the first and second versions of the instrument have been adequate for reliability testing but inadequate to conduct either exploratory or confirmatory factor analysis.

At this point in the instrument's development there are some lingering issues. The overall scale reliability of .91 and the high reliabilities for systems thinking, reflecting and reframing lead to the possibility that the scale is measuring one construct instead of three. This concern cannot be settled with the current subject sample used for scale development. To overcome this concern a third administration to at least 300 subjects is necessary to perform satisfactory factor analyses.

Second, the subjects in the first and second administration were graduate students all of whom were working in supervisory or management positions in addition to pursuing their studies. These samples were acceptable for item scaling and reliability testing but were insufficient to conclude that the use of strategic thinking constructs can discriminate between successful and less successful leaders. Therefore, the third administration of the instrument should be to subjects holding leadership positions in the profit and a non-profit sector. The point of this third administration should be to demonstrate that the three subscales are independent measures of cognitive processes and that the instrument discriminates between successful and less successful leaders. This protocol will insure not only the validity and reliability of the instrument but also its ability to predict leader success.

Finally, the data exhibits high means with little variance. This condition is often found in self reported behaviors. However, it is an issue that should be examined in the next iteration of the instrument. Until then a cautionary stance should be taken during interpretation of scores stemming from the PSLQ.

If these issues can be overcome through future research, it is possible that the PSLQ will be able to predict potential leader effectiveness and therefore can be a useful tool for leadership development and/or screening individuals for leadership positions. At this point, it can reasonably be concluded that the analysis has provided a clear description of the cognitive processes necessary to promote organizational change and adaptation and that the instrument shows promise of filling a gap in our ability to understand the relationship of these processes and leader success in complex and chaotic environments.

Agor, W. (1988). Finding and developing intuitive managers. Training and Development Journal, 42 (3), 68-70.

Anthony, W.; Bennett, R.; Maddox, N; & Wheatley, W. (1993). Picturing the future using mental imagery to enrich strategic environmental assessment. Academy of Management Executive, 7 (43-56).

Argyris, C. & Schön, D. (1978). Organizational learning: A theory of action perspective. Reading, Massachusetts: Addison Wesley.

Argyris, C. & Schön, D. (1996). Organizational learning II: Theory, method and practice. Reading, Mass: Addison Wesley.

Baron, J. (1994). Nonconsequentialist decisions. Behavior and Brain Sciences, 17 (1), 1-10.

Bass, B.M. & Aviolo, (1994).Improving Organizational Effectiveness through Transformational Leadership. Thousand Oaks, CA: Sage.

Beckhard, R. & Pritchard, W. (1992). Changing the essence: The Art of creating and leading fundamental change in organizations. San Francisco, CA: Jossey-Bass Publishers.

Bertalanffy, L.V. (1968). General system theory: Foundations, development, applications. New York: George Braziller.

Bolman, L. G. & Deal, T. E. (1991). Leadership and management effectiveness: A multi-Frame, multi-Sector analysis. Human Resource Management, 30, 509-534.

Bolman, L. G. & Deal, T. E. (1994). Looking for leadership: Another search party's report. Educational Administration Quarterly, 30 (1), 77-96.

Bonn, I. (2001). Developing strategic thinking as a core competency. Management Decision, 39 (1), 63-70.

Bonn, I. & Christodoulou, C. (1996). From strategic planning to strategic management. Long Range Planning, 4 (29), 543-551.

Capra, F. (2002). The Hidden Connections, Doubleday, New York, NY.

Chilcoat, Richard (1995). Strategic Art: The new discipline for the 21st century. U.S. War College.

Cohen, M., Adelman, L., Tolcott, Martin, Bresnick, F. & Freeman, M (1993). A cognitive framework for battlefield commanders' situation assessment. Ft. Leavenworth, KS: US Army Research Institute.

Cohen, M, Thompson, B, Adelman, L. Bresnick, T., Shastri, L. & Riedel, S. (2000). Training critical thinking for the battlefield: Basis in cognitive theory and research. Ft. Leavenworth, KS: U.S. Army Research Institute.

Copland, M.A. (2000). Developing the problem-framing skills of prospective principals. Paper presented at the Annual Meeting of the American Educational Research Association (New Orleans, Louisiana, April 24-28, 2000). ERIC :ED 442 192

Covey, S. (1989). The seven habits of highly effective people. New York: Dell Publishing.

Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.

Cuban, L. (1990). Problem-finding: Problem-based learning project. Palo Alto, CA: Stanford University School of Education.

de Bono, E. (1996). Serious Creativity, HarperCollins Business, London.

DePree, M. (1989). Leadership is an art. New York: Simon & Schuster.

Dewey, J., (1933). How We Think. Chicago: Henry Regnery.

Drucker, P.F. (1995). Managing in a time of great change. Oxford: Butterworth.

Fallesen, J. (1995). Overview of practical thinking skills for battlefield command. Research report 1685. Alexandria, VA: U.S. Army Research Institute.

Galbraith, P. (2001). Systems thinking: A lens and scalpel for organizational learning. ERIC Document Reproduction Services ED 453 186.

Godwin, J., Neck, C. & Houghton, D. (1999). The impact of thought self-leadership on individual goal performance: A cognitive perspective. Journal of Management Development, 18 (2) 153:169.

Halpren, D. (1996). Thought & knowledge: An introduction to critical thinking. Mahwah, NJ: Lawrence Erlbaum Associates.

Hovecar, D. and Bachelor, P. (1989). A taxonomy and critique of measurements used in the study of creativity. In Glover, J.A. et al. (Eds), Handbook of Creativity. New York, NY: Plenum Press.

Katz, D. & Kahn, R. (1978). The social psychology of organizations (2nd Ed.). New York: John Wiley & Sons.

Kegan, R. (1994). In over our heads: The mental demands of modern life. Cambridge, MA: Harvard University Press.

Kendall, A. (1990). The creative leader: Concepts for Air Force leadership. Maxwell AL.: Air University.

Kets de Vries, M. (2001). The leadership mystique: A user's manual for the human enterprise. New York: Prentice Hall

King, P.M. & Kitchener, K.S. (1994). Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey Bass.

Korthagen, F. A. (1988). The influence of learning orientations on the development of reflective teaching. In J. Calderhead (Ed.), Teachers' Professional Learning (pp.35-50), Philadelphia, PA: Falmer Press.

Kouzes, J., & Posner, B. (1993). Credibility: How leaders gain and lose it, why people demand it. San Francisco: Jossey-Bass.

Leithwood, K. A. & Steinbach, R. (1992). Improving the problem-solving expertise of school administrators. Education and Urban Society, 24 (3), 317-345.

Leithwood, K., Jantzi, D., & Steinbach, R., (1999). Changing leadership for changing times, Buckingham: Open University Press.

Magee, R., & Somervell, B. (1998). Strategic leadership primer. Carlisle Barracks, PA: U.S. Army War College.

March, J.G. & Simon, H.A. (1958). Organizations. New York New York, Wiley.

March, J.G. (1994). A primer on decision-making: How decisions happen. New York : Free Press

Mezirow, J. (1990). Fostering critical reflection in adulthood : a guide to transformative and emancipatory learn. San Francisco : Jossey-Bass Publishers

Morgan, G. (1986). Images of Organization, Sage, Beverly Hills, CA.

Neck C.P. & Manz, C.C., (1992) Thought self leadership: The Influence of self talk and mental imagery on performance. Journal of Organizational Behavior, 13, 681-699.

Newell, A. & Rosenbloom, P. (1981). Mechanisms of skill acquisition and the law of practice. In J. R. Anderson (Ed.) Cognitive skills and their acquisition. Hillsdale, NJ: Lawrence Erlbaum.

Nunnally, J.C, & Bernstein, I.H. (1994). Psychometric theory (3rd. ed.). New York: McGraw-Hill, Inc.

Parsons, T. (1960). Structure and process in modern societies. New York: Macmillan Company.

Perkins, D. N. (1995). Outsmarting IQ: The emerging science of learnable intelligence. New York: Free Press.

Pisapia, J.R. (in process). Strategic leadership: A primer.

Riedel, S., Morath, R. & McGonigle, T. (2000). ARI Workshop Proceedings. Ft. Leavenworth, KS. US Army Research Institute

Rodgers, C. (2002). Defining reflection: Another look at John Dewey and reflective thinking. Teachers College Record, 104 (4), 842-866.

Schreyogg, G., & Noss, C., (2000). Reframing change in organization: The equilibrium logic and beyond. Academy of Management Proceedings. ODC 2000 Best Paper Award.

Schön, D.A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.

Schwarz, N. (1997). Questionnaire design: The rocky road from concepts to answers. In L. Lyberg, P. Biemer, M. Collins, E. DeLeew, C. Dippo, N. Schwarz, & D. Trewin (Eds.), Survey measurement and process quality (pp. 2945). New York: John Wiley & sons.

Senge, P. (1990). The fifth discipline: The art and practice of the learning organization. New York: Doubleday.

Simon, H.A. (1947). Administrative behavior (1st ed.). New York: Harper and Row.

Simon, H. (1999), Making managing decisions, in Agor, W. (Ed.), Intuition in Organizations, Sage, Newbury Park, CA, pp. 23-39.

Spillane, J.P; Reiser, B.J.; Reimer, T. (2002). Policy implementation and cognition: Reframing and refocusing implementation research. Review of Educational Research, 72 (3) 387-431.

Stanwick, P. (1996). Mental imagery: An alternative to top management team replacement for declining organizations. Journal of Organizational Change Management, 9 (2), 47-65.

Torrance, E.P. (1974), The Torrance Tests of Creative Thinking: Technical-norms Manual, Scholastic Testing Services, Bensenville, IL.

Usher, R. & Bryant, I. (1989). Adult education as theory, practice and research. London: Routledge.

Weick, K., & Quinn, R. (1999). Organization change and development. Annual Review of Psychology, 50, 361-386.

Weick, K. (1995). Sensemaking in Organizations. Thousand Oaks, Ca: Sage Publications.

White, D. (2002). Reflective practice: Wishful thinking or a practical leadership tool. Paper presented at the International Conference to study the Issues and Challenge facing Catholic Educational Leadership. Sydney, Australia.