INTRODUCTION
The field of distance education remains in an early stage of development. As recently as fourteen years ago, Shale (1988) suggested that distance education had asserted itself but could not define itself. Existing as a sub-field in education, distance education informs its practice from social science theory (education, psychology, sociology, anthropology) and the more recent professional field of education. The creation of a knowledge foundation all its own, based on sound empirical research leading to theory development, is now an imperative (Miller and Husman,1996; Garrison, 2000).
Fostering the generation of original and innovative research in distance education requires redirection on several fronts, including research methods instruction of students in graduate distance education programs (Ramakrishna et al., 1999). The majority of graduate students in distance education come from a practitioner background (Monahan, 1994) and their focus has likely been on the critique and consumption of research findings rather than on the underpinnings and procedures for knowledge construction through research. In this short monograph, we argue that the best preparation for critiquing and assimilating the results of reported research is developed through an understanding of sound research design and methods, that an emphasis on basic research in graduate programs in distance education will encourage theory development in this maturing field and that a major emphasis during the period of graduate instruction in research methods must be placed on developing the intangible components of curiosity and intellectual excitement
The identification of the emphases (topics, focus, foundational perspectives) and requirements in any graduate level program rests on preconceived notions about what is to be fostered in graduate study, as well as what is required by way of subject specific knowledge and skill. The following assumptions about the tenor of a typical graduate program in distance education are made:
Assumption 1. Graduate students must be engaged in advanced, systematic examination of knowledge in a specific discipline or field of study. A conceptual and methodological mastery of the social science research process is required for the analysis, evaluation and implementation of the research concepts, designs and processes to which they will be exposed. For students who wish to be discerning consumers of knowledge, a basic course may be adequate. For students who wish to be researchers themselves, more advanced courses in specific research methods and research tools are required.
Assumption 2. Knowledge attainment in research methods occurs at a level beyond fact acquisition and comprehension to in-depth levels of application, thoughtful analysis, concept integration in original ways and information evaluation. Learning about and informing research activity facilitates the development of well-reasoned argument. Learning the process of identifying a sound research or project question requires exposure to and understanding of rational, cautious and thorough thought processes. In addition, identification of a problem or question must be made in the context of and with reference to an already well-reasoned body of existing literature. Analyzing, synthesizing and evaluating current knowledge on an issue or question is a central part of the research process. These activities contribute to the development of higher-order thinking skills desirable in graduate students who aspire to professional or academic positions in distance education.
Assumption 3. Expansion of current personal knowledge of a field relative to collective knowledge available and opportunities for individual creative endeavors must be included in study opportunities. The research focus must be applied to activities that are undertaken to develop and to inform individual knowledge and decision-making. Such work taps the collective knowledge base in order to develop individual understanding. Graduate study research moves research activity to the process of creating knowledge in a field or discipline. Research, in this sense, is designed to replicate, verify or augment that which is already known. This research may be descriptive, exploratory (in reference to possible relationships between concepts) or explanatory (testing the plausibility of cause and effect relationships between concepts).
Assumption 4. Methodologies available for basic
and applied research, for assessing practice, guiding policy, and solving
practical problems and for contributing to collective knowledge through
well-grounded and substantive research questions must be integral components
of the program.
Assumption 5. Graduates will eventually hold credentials that both
allow and require them to act as developers and gatekeepers of knowledge
in distance education. To this end, they will develop a high level of
content knowledge with an understanding of the scope and nature of research
and publication enterprise that creates and shapes that knowledge.
Of these five, assumptions 3 through 5 speak directly
to the call for an expanding cohort of distance education researchers.
Because of the on-going merging of instruction and technology, distance
education has now found itself at the centre of attention and of scrutiny
and the need for a further enhancement of its theoretical underpinnings
is needed. This evolution will require the education and credentialling
of competent and enthusiastic researchers whose task it will be to strengthen
and broaden through substantive and focused research initiatives the field
of distance education.
Research Competencies
A traditional and effective way to address the content and structure of the research component of a graduate-level distance education program is to categorize both the research competencies that are held by the typical incoming student (entry-level competencies) and the competencies that will be the focus of overall program (graduate program competencies including those outside of formal courses) and also the weight (courses, hours of instruction) that this component is given within the larger context of full program of studies.
Entry-level competencies
Admission criteria to graduate-level distance education master’s
programs differ but, for many applicants, an understanding of and expertise
in the steps of the systematic and rigorous research process appears to
be very limited and, in some cases, seriously lacking. To a large degree,
the level of the research competencies the new students bring to a graduate
program is primarily a function of the undergraduate discipline from which
they graduated. Students who hold a baccalaureate degree in the social
sciences (educational psychology, psychology, sociology, anthropology)
are typically better versed in researched methodologies and often have
carried out systematic research. Deficiencies reflective of the typical
case are little or no undergraduate preparation in research design, little
or no undergraduate preparation in data analysis (quantitative or qualitative),
and lack of understanding of epistemological foundations for social science
research. In many cases, this results in the distance education student
calling on what might colloquially be described as “street smarts”
or personal experience when focusing on the generation of specific research
problem or question and the concomitant rationale and proposed methodology.
Graduate program competencies
The desired learning outcomes that occur during and that result from the
successful completion of a master’s level distance education degree
program and that are directly tied to research methodologies and procedures
are many and varied. The following list of general outcomes should be
viewed as a partial but also typical cataloguing of the program goals.
1. To prepare all students for the informed interpretation of the literature;
2. To prepare some students for program-related distance education research;
3. To prepare some students for the role of practitioner-researcher (Jarvis, 1999)
4. To prepare some students for future graduate studies;
5. To examine the epistemological foundations for social science research;
6. To foster a set of laudable graduate student attributes: development of reasoned argument (Mauch & Birch, 1998), speculative audacity , curiosity, rigour and a sense of wonderment.
The first 3 of these outcomes can be addressed by the offering of graduate-level courses in research methods, the fourth and fifth outcomes by inclusion in one of a set of advanced courses while the sixth is somewhat more elusive. A framework as to how this outcome might be addressed and nurtured in a distance education program is addressed later in this paper. Of these six outcomes, this final one could rightfully be considered the most important of the lot. Indeed, one could argue that, if this sine qua non of the research process is underplayed or even omitted, the efforts to master the concepts and skills that are encompassed by outcomes 1 through 5 will not result in an enriched and comprehensive perspective.
Proportion of total course load
An important program design consideration is the proportion of instructional
weight or credits to be allotted to instruction in this subject matter.
For those students who plan to carry out original research in their program
(basic research course or courses) and for those who also plan to pursue
further graduate studies (Ph.D., Ed.D.) (advanced research course or courses),
the following percentages are suggested weightings within a program:
These values are intended to take into account the state of the typical applicant’s research competencies as described in the above section on entry-level competencies. That is, of the total number of credits that must be earned to be accredited with a master’s degree, one of five should focus on the research methods. It should be noted that the advanced course weighting includes the proportion of instructional time allocated to the basic course(s). Of course, if a particular distance education program’s pool of new students is stronger in research knowledge and experience than the demographic that has been set out in this paper, then these weightings would have to be adjusted.
BASIC COURSE CONTENT
To address the competencies listed above, adequate and effective preparation of graduate students in distance education for the informed consumption and critical analysis of reported research and for the carrying out of degree-related and future research requires a well-integrated suite of courses. One obvious strategy is to prepare a set of basic research methods courses that are primarily designed (1) to bring those students whose entry-level competencies are lacking to some reasonable and commonly-accepted criteria and (2) to provide a strong grounding for program-related tasks (e.g., literature reviewing for a course, secondary analyses of data) including proposal writing for a thesis or for a project, both instances of original analysis and critical thinking. With this in mind, the focus of these particular courses should be on general principles of the research process, both quantitative and qualitative, with a view to developing initially consumption/interpretation of results competencies that are necessary for all graduate students (researchers or practitioners) rather than on the underlying epistemological underpinnings to both quantitative and qualitative methodologies (Huck, 2000; Onwuegbuzie, 2000). This subject matter would be tackled in the advanced courses.
For outcomes 1 through 3, the following outline (table 2) identifies the topics to be addressed, the specific content and the sequence in order to ensure a comprehensive understanding of the research enterprise and the conceptual and skill development necessary for research design, data collection and data analysis skills with due regard for the two main research approaches:
| Topic | Content |
| Theory and the construction of knowledge | 1. History and benefits of social science research 2. Relationship between theory development, research and professional practice 3. Relationship between types of research methods and the construction of knowledge 4. Theory construction from practice |
|
1. Perspectives and approaches for basic and applied research 2. Research approaches that reflect quantitative and qualitative perspectives 3. Practitioner research as applied research 4. Computational versus textual modes for acquiring data |
| Quantitative methodologies | 1. Critical elements of experiments and quasi-experiments 2. Types of experimental design 3. Threats to validity in experimental and survey design 4. Survey design and its place in social science research 5. Features of questionnaire and survey design 6. Advantages and disadvantages of types of surveys 7. Descriptive and inferential statistics 8. Sampling procedures 9. Power and effect size 10. Use of computer programs for data analysis 11. Interpretation of data analysis |
| Qualitative methodologies | 1. Scope and nature of field research 2. Steps in field research studies 3. Multiple social and ethical issues in field research 4. Scope and nature of historical-comparative research 5. Reliability and validity as they relate to historical-comparative and field research 6. Contrasting and comparison of historical-comparative and field research 7. Sources of evidence and the quality of types of secondary data 8. Types of manual coding 9. Use of computer programs for data analysis 10. Value of negative evidence in qualitative data. |
| Dissemination and consumption of research findings | 1. Process, requirements and benefits of rigorous literature review 2. Types and quality of literature sources 3. Procedures for evaluating the quality of literature sources 4. Publication standards |
Table 2. Topics and content of basic research methods course(s)
From the perspective of instructional systems design, it is highly improbable that the topics listed above could be accommodated in a single course. Not unexpectedly, those graduate students whose previous academic work was in disciplines that do not lend themselves to systematic, empirical research (e.g., humanities, theology, arts) would be hard pressed to master this content in the context of a one-term course or single course. An obvious strategy would be to design two courses with sufficient time allocated for practical tasks associated with the use of data analysis software (e.g., SPSS, AtlasTi). A strength of this suggested course structure is that, upon successful completion, the graduate student would be in good standing with respect to outcomes 1 through 3.
In addition to the design and production of methodology courses that are both systematic and comprehensive as delineated above, graduate students also require access to pertinent and easily accessible databases for both coursework and for research proposal generation. For distance education students, many if not all of the large and pertinent databases (e.g., E.R.I.C., PsychLit) are available via telecommunication links and, as a result, do not put these distance education students at a disadvantage. Second, the institution that is awarding the graduate degree must make available to their students software for both numerical and textual analysis. This can be accomplished by means of site licenses or, in some cases, by allowing for data analyses to be carried out on-line. Finally, the home institution must put into place personnel and procedures to support their student researchers.
ADVANCED STUDY IN RESEARCH METHODS
For students who wish address outcomes 4 through 6 (see above), a basic course or set courses that covers the topics and subject matter delineated in table 2 will not be sufficient. A deeper understanding of not only particular elements or components of more complex and sophisticated procedures and techniques but also of the fundamental issues of the history and evolution of attempts to systematize the framework for the deciding on the validity of a standard model and its associated propositions (i.e., epistemology) would have to be the focus of advanced course offerings. Table 3 contains a suggested list of topics and appropriate content:
| Topic | Content |
The evolution of science, social science and types of inquiry
|
1. Sources of knowledge (empirical, rational) and the roles of
inductive and deductive science 2. Application of the scientific method to the social world 3. Distance education research as an interdisciplinary field of inquiry 4. Dimensions of social science research and distinguish among basic research , applied research and evaluation |
Epistemological foundations of research
|
1. Positivism, interpretivism and perspectives on the margins:
critical, postmodern and feminist views 2. Links between epistemology and research methods 3. Integration of research methods and reconciling epistemological assumptions. |
|
1. Mathematical foundations for probability distributions 2. Theoretical distributions for statistical inference 3. Hypothesis testing and confidence intervals 4. Multi-factor ANOVA – fixed, random and mixed effects 5. Structural equation modeling 6. Meta-analysis |
Advanced qualitative foci |
1. Phenomenological underpinnings 2. Ethnomethodology 3. Field research, grounded theory and case studies 4. Historical-comparative methods 5. Coding, concept formation and analytic memo writing 6. Authenticity and verifiability in qualitative data analysis |
Table 3. Topics for advanced research methods course(s)
It should be noted that approximately half of the instructional emphasis for these advanced courses is centered on the epistemological issues that underlie the quest for a powerful and comprehensive standard model. The issue of causality and how research may be designed to uncover systematic patterns of factors or interrelated factors is at the heart of such endeavour. If distance education is to move beyond the generation of models that are primarily descriptive in nature, then new cohorts of researchers must be attuned to the underlying philosophical nature of any line of inquiry that attempts to lay bare these components of the standard model.
ARGUMENTS FOR IMPLEMENTATION
It is a given that the primary goal for the instruction of research methods in distance education should be to prepare students to make contributions to the field of distance education as a result of professional growth and through research, basic and/or applied. With respect to the former, a first step is to instruct to a high level of mastery the content in table 2 above so that graduate students will indeed possess a sufficiently sophisticated set of competencies with which to assess and to interpret the huge volume of published reports that are in the professional literature and in the public domain. The ability to discriminate between solid and substantive reports of research and opinion pieces, between generalizable and non-generalizable results and results and between causal and correlational conclusions is necessary to avoid accepting erroneous or unfounded promulgations of purported significant findings.
A second argument focuses on just exactly what is needed for the continuing and effective development of a standard model for distance education. Most, if not all disciplines and fields of study, have generated over years, decades and even centuries a commonly-accepted framework that reflects the immediate state of knowledge and that acts as a vehicle to make sense of the results of the on-going attempts to understand and to explain (and, in some cases, predict) the phenomena that are at the focus of inquiry. To date, attempts to produce such a standard model for distance education have not been particularly powerful even if one makes use of standard models from the subsuming disciplines of education, sociology and psychology for purposes of incorporating or integrating those models with the goal of generating a model for distance education. The only remedy for this is to unleash a strong cohort of well-prepared students who will address the central problems or questions in the existing model of distance education and who will, as a result of their efforts, further develop theory in the field.
Third, the establishment and implementation of a set of effective procedures for both basic and applied research will serve to enhance the quality of instruction of the graduate program itself and, perhaps of even greater importance, will increase the probability that the graduates of that program will produce high-quality research during their tenure in the program and in their subsequent professional lives. To increase the probability of this eventuality, this set of procedures might include research consultancy support, limited funding for data collection purposes and exposure within the graduate program and with external agencies (e.g., professional organizations) about the results of these research initiatives.
Traditionally, the most effective way to attend to these arguments has been to implement an apprenticeship model of instruction and learning with the goal of graduating students who are capable of executing well-designed and substantive independent research. Arguably, there is no one curriculum or instructional approach that can guarantee that this type of individual will unerringly emerge from any program of study but, as an overarching goal, the long-standing apprenticeship model has been both ubiquitous and effective. This professional development model can be replicated in distance education programs, with content appropriately designed as described above and with state-of-the art information and communication technologies to facilitate student/supervisory relations throughout.
CURIOSITY AND SENSE OF WONDERMENT
As described in outcome six, a very important learner outcome in a graduate-level degree program is the experience of being engaged in what can unabashedly be described as the “sine qua non” of the graduate experience. In fact, the authors would be quite comfortable in going quite a bit further and citing this program outcome as not “one of the most important” but as the most important outcome for graduate-level students. Here we are alluding to the intrinsic rewards that follow on what is often a long, arduous and, in some instances, a lonely process. What can get lost in the very complex and multi-factored experience of working through the requirements for a graduate program is the excitement and intellectual sense of accomplishment that follow from the acquisition of the concepts and skills necessary for the generation and implementation of an innovative and substantive research endeavour. In addition, a contingent result is often the self-perception that a powerful and well-grounded set of research tools has been internalized and mastered and that an entirely new world-view has been acquired. This new “world-view” is the profound understanding that there are systematic and rigorous means to answer questions about phenomena in distance education.
With respect to how one might foster this sense of curiosity and wonderment, two considerations might prove to be effective. The first is the awareness, either as a result of formal instruction (see table 3) or informal discussions, that, when one enters into graduate school, one is poised to engage in a long-standing tradition in Western culture in which a mix of reason, experience, creativity and intuition converges in the generation of powerful explanatory models. Historically, this has meant grappling with what has been described as Hume’s problem of induction and how, in the development and validation of a standard model, the major and pivotal constructs in the standard model can be tested by attempts to falsify hypotheses deduced or drawn from the relationship between and among those constructs. A close study of the contribution of those luminaries who have grounded their analyses in the framework of the correspondence theory of truth (Russell, Tarski, Wittengstein, Popper) would both be rewarding and edifying.
A second consideration -and one would it would be incumbent on the instructional designers of the courses in research methodologies to provide- is opportunities for the graduate students to maximize the meaningfulness of the instructional tasks by structuring them in such a way that the individual student is encouraged to scrutinize data, either textual or numerical, that he or she has observed or recorded. This type of individualization is not novel in instructional design and it is argued that the inclusion of this strategy would do much to promote the learner outcome of curiosity and wonderment. A particularly apt quotation that captures the essence of this goal is the following:
“Nevertheless, science is certainly capable of instilling great emotional satisfaction; indeed, many physicists stand in reverence before the grandeur of Nature and the beauty of its structure as revealed by their theories and discoveries. Listen to Poincare: ‘The scientist does not study nature because it is useful; he studies it because he delights in it, and he delights in it because it is beautiful. If nature were not beautiful, it would not be worth knowing, and if nature were not worth knowing, life would not be worth living.’ ” (Newton, 1997, p. 34)
The Nobel prize-winning physicist, Richard Feynman, has voiced a similar perspective or sentiment:
I think it is very important - at least it was to me – that if you are going to teach people to make observations, you should show that something wonderful can come from them. I learned then what science was about. It was patience. If you looked, and you watched, and you paid attention, you got a great reward from it (although possibly not every time). As a result, when I became a more mature man, I would painstakingly, hour after hour, for years, work on problems – sometimes many years, sometimes shorter times – many of them failing, lots of stuff going into the wastebasket, but every once in a while there was the gold of a new understanding that I had learned to expect when I was a kid, the result of observation.” (Feynman, 1999 , p. 82)
For students who are enrolled in a graduate-level degree program, including distance education, instruction in and the internalization and mastery of the central concepts and requisite skills in social science research and the concomitant methodologies are essential. The acquisition of discipline-specific concepts and skills are, of course, extremely important but unless these are complemented by both an appreciation for and a profound understanding of what constitutes knowledge (the epistemological underpinnings of inquiry) and of the tools that are used to advance knowledge (various research methodologies) when social science research is being designed and implemented, the graduate student’s experience and education must be deemed to be incomplete. Both underpinning and complementing these desired learner outcomes is the recommendation that much attention in the design of the materials for the research—focused courses be allotted to attending to the instilling and nurturing of how exhilarating and rewarding the research process can be.
Foot Notes:
Webb and Glesne (1998) report on the claims of research methods instructors
in this regard, stating that no instructors interviewed were willing
to claim that speculative audacity could be taught, many agreed that
students’ imaginations can be nurtured and that many students
can become competent researchers.
REFERENCES
Feynman, R. (1999). The pleasure of finding things out. Perseus Publishing.
Garrison, D. R. (2000). Theoretical challenges for distance education
in the 21st Century: A shift from structural to transactional issues.
International Review of Research in Open and Distance Learning, 1(1),
pp. 1-17.
Huck, S. (2000). Reading statistics and research (3rd edition). Addison-Wesley.
Jarvis, P. (1999). The practitioner-researcher: Developing theory from
practice. San Francisco: Jossey-Bass.
Mauch, J.E. & Birch, J.W. (1998). Guide to the successful thesis
and dissertation. New York: Marcel, Dekker, Inc.
Miller, M.T., & Husman, D.E. (1996). A holistic model for primary
factors in the ecology of distance education course offerings. Journal
of Distance Education, 11 (1), pp. 101-110.
Monahan, T. C. (1994). The usefulness and motivational value of research
methods courses for education professionals. Paper presented at the
Annual Meeting of the Eastern Educational Research Association, Sarasota,
FL, February 11, 1994.
Newton, R. (1997). The truth of science. Harvard University Press: Cambridge,
MA.
Onwuegbuzie, A. J. (2000). On becoming a bi-researcher: the importance
of combining quantitative and qualitative research methodologies. Paper
presented at the Annual Meeting of the Association for the Advancement
of Educational Research, Ponte Vedra, FL, November, 2000.
Ramakrishna, C. P. & Prasad, V. S. (1999). Action research projects
in distance education: a manual. Indira Gandhi National Open University,
Distance Education Council.
Shale, D. (1988). Toward a reconceptualization of distance education.
American Journal of Distance Education, 2(3), 25-35.
Webb, R.B. & Glesne, C. (1998). Teaching qualitative research. In The handbook of qualitative research in education. Toronto: Harcourt Brace Jovanovich.
Biodata of authors:
Dr. Tom Jones:
Dr. Tom Jones is an Associate Professor in the Centre for Distance Education
at Athabasca University. His areas of expertise are distance education
course development and delivery, instructional technology, human-computer
interaction and epistemology. Dr. Jones has also worked in public schools,
in government and in the corporate sector. He is presently a core faculty
member in the Master of Distance Education degree program.
Conatc address:
Tom Jones, Ph.D.
Associate Professor
Centre for Distance Education Athabasca University
1 University Drive Athabasca, AB T9S 3A3
(780) 675-6180 (780) 675-6170 (fax)
tomj@athabascau.ca
Dr. Cleveland-Innes has studied and practiced in the field of adult, distance and higher education for the last eighteen years. Her research focus is social factors in education, having studied in both sociology and education as a graduate student. Current projects include student role adjustment in on-line learning, affective learning outcomes in online learning and factors in deep and surface learning. She is Assistant Professor, the Center for Distance Education at Athabasca University.
Conatc address:
M. Cleveland-Innes, Ph.D.
Assistant Professor
Centre for Distance Education Athabasca University
1 University Drive Athabasca, AB T9S 3A3
(780) 675-6180 (780) 675-6170 (fax)
martic@athabascau.ca
 |
 |