The Evolution of IS Research Since 1980: From Frameworks to Impact:

Peter G.W. Keen, a prominent figure in Information Systems (IS) research, delivered a pivotal keynote speech at the inaugural International Conference on Information Systems in 1980. His address was both a critique and a call to action for the nascent field of IS research.

Keen’s central argument was incisive and provocative. He observed that IS researchers had become overly focused on generating conceptual frameworks, to the detriment of the field’s practical relevance and scientific rigor. His critique can be summarized in three key points:

Let’s expand on the key points from Peter Keen’s influential keynote speech at the first International Conference on Information Systems (ICIS) in 1980:

1. Proliferation of Frameworks:

Abundance of Conceptual Models: IS researchers were producing numerous theoretical frameworks attempting to explain various aspects of information systems in organizations.

• Lack of Consolidation: These frameworks often overlapped or competed, leading to a fragmented theoretical landscape.
• Emphasis on Novelty: There was a tendency to create new frameworks rather than build upon or refine existing ones.
• Complexity: Many frameworks were intricate and multi-faceted, making them difficult to comprehend or apply holistically.

2. Difficulty in Testing:

Lack of Operationalization: Many frameworks were not clearly operationalized, making empirical testing challenging.

• Methodological Limitations: The field lacked established, rigorous methodologies for testing complex, multi-variable frameworks.
• Data Collection Challenges: Gathering comprehensive data to test holistic frameworks was often impractical or prohibitively expensive.
• Construct Validity Issues: The abstract nature of many constructs in these frameworks made it difficult to ensure they were being measured accurately.

3. Limited Practical Applicability:

Gap Between Theory and Practice: Frameworks often failed to translate into actionable insights for IS professionals.

• Oversimplification of Complex Realities: Theoretical models sometimes failed to capture the nuanced, context-dependent nature of real-world IS implementations.
• Lack of Prescriptive Power: While frameworks might describe IS phenomena, they often fell short in guiding practical decision-making.
• Rapid Technological Change: The fast-paced evolution of technology often outpaced the development and validation of theoretical frameworks.

4. Appearance of “Fraudulence”:

Credibility Crisis: The combination of untested theories and limited practical relevance risked undermining the field’s credibility.

• Academic-Practitioner Divide: This approach widened the gap between academic research and industry needs.
• Lack of Cumulative Tradition: The constant generation of new frameworks hindered the development of a solid, cumulative body of knowledge.
• Questionable Value Proposition: It became difficult to justify the value of IS research to both academic institutions and industry partners.

5. Need for Reference Disciplines:

Interdisciplinary Approach: Keen advocated for integrating theories and methods from established fields like management, psychology, and computer science.

• Methodological Rigor: Adopting proven research methodologies from reference disciplines could enhance the empirical testing of IS theories.
• Theoretical Grounding: Anchoring IS concepts in established theories could provide a stronger foundation for the field.
• Practical Relevance: Drawing from applied disciplines could help bridge the gap between academic research and practical application.

6. Implications for the Field:

Identity Crisis: Keen’s critique sparked ongoing debates about the nature and boundaries of IS as a discipline.

• Methodological Diversity: It encouraged the adoption of a wider range of research methods, from positivist to interpretive approaches.
• Focus on Relevance: Subsequent research placed greater emphasis on practical implications and real-world applicability.
• Theory-Practice Balance: Keen’s speech influenced efforts to balance theoretical development with empirical validation and practical application.

7. Long-term Impact:

Rigor vs. Relevance Debate: Keen’s critique initiated a long-standing discussion about balancing academic rigor with practical relevance in IS research.

• Evolving Research Standards: It led to more stringent requirements for empirical testing and practical implications in top IS journals.
• Interdisciplinary Collaboration: Encouraged more cross-disciplinary research and collaboration in the IS field.
• Reflexivity in Research: Prompted ongoing self-examination and critique within the IS research community.

Keen’s assessment was that this situation made the IS field appear “fraudulent” – a harsh but thought-provoking characterization. He suggested that this approach to research risked isolating IS from both the scientific community (due to lack of rigorous testing) and practitioners (due to limited applicability).

To address these issues, Keen advocated for IS researchers to look beyond their immediate domain to “reference disciplines.” This suggestion encouraged IS scholars to draw upon established theories, methodologies, and practices from related fields such as computer science, management, psychology, and sociology. By doing so, IS research could:

1. Ground its frameworks in more established scientific traditions

2. Adopt rigorous testing methods from other disciplines

3. Enhance the practical relevance of its findings by connecting to real-world problems and solutions

Keen’s speech was a watershed moment in IS research. It challenged the community to elevate its standards of scientific inquiry and to bridge the gap between theory and practice. His critique has continued to resonate in the decades since, influencing the evolution of IS as a discipline and encouraging a more balanced approach between theoretical development, empirical testing, and practical application.

This keynote set the stage for ongoing debates about the identity, rigor, and relevance of IS research – themes that continue to be central to the field’s development to this day.

Since Peter Keen’s seminal critique in 1980, the field of Information Systems has undergone significant evolution. However, the question remains: Have we made substantial, visible progress? To answer this, we need to examine multiple dimensions of advancement in IS research:

1. Methodological Rigor:

Have our research methods improved in rigor?

Positive Developments:

• Adoption of more sophisticated statistical techniques
• Increased use of mixed-method approaches
• Greater emphasis on construct validity and measurement
• Development of IS-specific methodologies (e.g., design science research)

Ongoing Challenges:

• Balancing rigor with relevance
• Keeping pace with rapidly evolving technologies
• Addressing the complexities of socio-technical systems

Assessment: While methodological rigor has undoubtedly improved, there’s an ongoing need to refine and adapt our methods to the unique challenges of IS phenomena.

2. Impact and Cross-Disciplinary Influence:

Are we disseminating ideas beyond our own bubble, and are other disciplines listening?

Signs of Progress:

• Increased citations of IS research in related fields (e.g., management, computer science)
• Growing number of IS scholars publishing in non-IS journals
• Adoption of IS concepts in other disciplines (e.g., digital transformation in strategic management)

Persistent Issues:

• Risk of insularity in some IS research streams
• Challenges in communicating IS insights to broader audiences
• Limited recognition of IS as a reference discipline for others
• Assessment: While there’s evidence of growing influence, IS research still struggles to consistently impact broader academic and practitioner communities.

3. Field Identity and Consistency:

Have we developed a recognizable identity with consistency between our stated focus and actual research?

Positive Indicators:

• Establishment of core IS journals and conferences
• Development of IS-specific theories and frameworks
• Growing recognition of IS departments in academic institutions

Ongoing Debates:

• Continued discussions about the core identity of IS
• Tensions between techno-centric and socio-technical approaches
• Balancing the field’s breadth with the need for a coherent core

Assessment: While IS has made strides in establishing its identity, there’s still a lack of consensus on the field’s core, leading to ongoing identity debates.

4. Adaptability to Changing Contexts:

How well do we adapt to evolving technologies, institutional demands, and societal changes?

Strengths:

• Rapid incorporation of emerging technologies into research agendas (e.g., AI, blockchain)
• Increased focus on societal impacts of IS (e.g., privacy, digital divide)
• Adaptation to new research paradigms (e.g., big data analytics)

Challenges:

• Keeping pace with the accelerating rate of technological change
• Balancing long-term theoretical development with the need to address current issues
• Adapting to changing institutional pressures (e.g., emphasis on research impact)

Assessment: The field has shown adaptability, but sometimes struggles to lead rather than follow technological and societal changes.

The progress of IS research since 1980 presents a mixed picture. On one hand, there have been significant advancements in methodological rigor, growing cross-disciplinary influence, and the establishment of a more defined field identity. The adaptability of IS research to new technologies and societal issues is also commendable.

However, challenges persist. The field continues to grapple with balancing rigor and relevance, fully establishing itself as a reference discipline, achieving consensus on its core identity, and consistently leading rather than following technological trends.

Moving forward, the IS community needs to:

• Continue refining methodologies while maintaining practical relevance
• Actively engage with other disciplines and practitioners to expand influence
• Work towards a more cohesive field identity while embracing diversity
• Strive to anticipate and shape technological and societal changes rather than merely reacting to them

By addressing these challenges, IS research can build on its progress and enhance its impact on both academic knowledge and practical applications in the ever-evolving digital landscape.