Turkish Online Journal of Distance Education-TOJDE January  2011 ISSN 1302-6488 Volume: 12 Number: 1 Article 2


 

 

CONTENT ANALYSIS OF RESEARCH PROJECTS

 SUBMITTED BY UNDERGRADUATE STUDENTS

 (2000-2009) AT THE ZIMBABWE OPEN UNIVERSITY:

Implications for Quality Assessment

 

Caleb KANGAI

Richard BUKALIYA

Farirai MUSIKA

Mapuranga BABRA

Zimbabwe Open University, 

Mashonaland East Region,

Marondera, ZIMBABWE

 

INTRODUCTION

 

The major concern about Distance Education continues to be its quality (Dellana, Collins, and West, 2000). Most research studies focusing on quality have concentrated on comparing distance education with conventional education. These studies have found no significant difference between distance education and conventional education in terms of quality (Phipps and Merisotis, 1999; DeSantis, 2002; Russell, 2000). However, an issue that has continued to attract attention of open and distance learning (ODL) educators, scholars and researchers is how ODL institutions, whatever their structure, context or circumstances, can assess their own quality (Myrdal, 1994). According to Myrdal (1994), some of the critical questions that continue to fuel the “distance education” quality debate are; What makes distance education an experience that would be described as one of quality? How can distance education improve the quality of the ODL it offers?  How can an institution providing ODL assess its own quality effectively? These are large and difficult questions.

 

Agreeably, all institutions providing ODL will have some existing systems and procedures for assessing the quality of what they do. But not all have addressed the assessment of quality within their organizations in a systematic way as much as they need to (Mertens, 2005). Challenges facing ODL institutions are that procedures for assessing quality can be ad hoc, piecemeal, unsystematic, too reliant on individual discretion, and standards of practice can be unnecessarily inconsistent and variable. These are some of the challenges experienced by lecturers at the Zimbabwe Open University particularly in the assessment of students’ research projects. Inspired by the need to have a systematic approach that could be used in assessing the quality of students’ research projects, a team of four lecturers at the ZOU conducted a content analysis of 400 research projects submitted by undergraduate students between the years2000-2009.

 

The study led to the development of a content analysis model the four researchers found to be very useful to both research students and their supervisors in the quality assessment of students’ research projects.

This article discusses how this model was developed and its application in the assessment of students’ research projects.

 

BACKGROUND OF THE STUDY

 

The Zimbabwe Open University (ZOU) is an Open and Distance Learning (ODL) institution in Zimbabwe, established to cater for a substantial component of people who, by design or unintentionally, could not be accommodated in conventional universities, by offering them the opportunity to study in their homes and in their workplaces through distance education.  The ZOU was established on 1st March 1999 through an Act of parliament (Chapter 25:20), with an initial enrolment of 624 students registered for the Bachelor of Education degree programme. By 2004 ZOU had become the largest university in the country and second largest in Southern Africa compared to University of South Africa (UNISA), with a student enrolment of approximately 13 000. However, the student population, in the year, 2010, dropped to approximately 10 000. During the time of this study, in 2010, ZOU had four faculties;

 

Ø  the faculty of Arts and Education,

Ø  the faculty of physical Sciences,

Ø  the faculty of Commerce and Law and

Ø  the faculty of Social Sciences, offering 30 undergraduate degree programmes, 3 diploma courses, 5 masters degree programmes and 2 doctoral degrees.

 

The ZOU teaches its students using multi media methods that include the module (print text), face-to-face tutorials and ICTs. Students are assessed through the use of assignments, tests, examinations and research projects.

 

The present study is part of a longitudinal study to investigate issues of quality in the assessment of students through the research projects. Researchers at the ZOU have identified a number of challenges facing research students and their supervisors (Nyawaranda (2005; Kangai and Mapolisa (2008).

 

According to Nyawaranda, (2005), challenges include the following:

 

Ø  Students failing to complete research projects in time

Ø  Low submission rate of 40%

Ø  Students working on non-researchable topics

Ø  Plagiarism

Ø  Poor quality of research projects 

Ø  Lack of competence (by supervisors) in approving students’ research topics

Ø  Inconsistence in marking students’ research projects

Ø  Lack of uniformity in the supervision and marking of students’ research projects

 

These challenges have impacted negatively on the quality assessment of students’ research projects. It is against this background that the present content analysis study was undertaken.

 

Purpose of the Study

The purpose of the study was to conduct a content analysis of research projects submitted by ZOU undergraduate research students in order to establish their present status, gaps, and areas of saturation and to develop a model that could be used in the quality assessment of these projects.  

 

Statement of the Problem

One of the topical issues that have provoked heated academic debate in programme management meetings at the ZOU is the question of quality in the assessment of students’ research projects (Izuagie, 2001).

 

The central issue confronting research students and their supervisors at the ZOU is how to achieve quality, effectiveness and productivity in the new changing environment. The research problem is that not all projects’ supervisors and markers have addressed the assessment of students’ projects in a systematic way.  For example double marking of projects have produced extreme differences in marks awarded to the same project. The present study sought to assess, through a content analysis, the quality of research projects submitted by ZOU undergraduate students and to develop a content analysis model that could be used for quality assessment of these projects.

 

Research Questions

The present study was guided by the following research questions:

 

  1. Do ZOU undergraduate students have a bias in the choice of research topics, research paradigm, and research design and data collection methods?
  2. What is the status (methodological appropriateness and methodological quality) of students’ research projects?

 

Importance of the Study

Research is the process by which new knowledge is created. As such it influences the direction in which an academic field or a community moves. The findings of the current study are of interest to many groups in distance education; however, we wished to provide this information primarily for the benefit of research students and their supervisors who should be aware of how quality can be enhanced during supervision and assessment of research projects. The information presented in this article is also of great benefit to lecturers who are involved in the teaching of research theory and methodology. The findings are also of great value to developers of distance education materials.

 

LITERATURE REVIEW

 

The research Project: Definition and structure

The present study is a content analysis of research projects submitted by ZOU undergraduate students (2000 – 2009). Reckase (1995) defined a research project as a purposeful collection of student work that exhibits to the student and others, the student's efforts, progress, or achievement in a given area. The major goal of project work is to assess the ability to apply knowledge to solve real-life problems. The student must take considerable control over the assessment through planning and applying knowledge in perhaps new and different ways.  Projects, thus attempt not only to assess the end products, but to some extent, the process that went into creating them as well.

 

THEORETICAL FRAMEWORK

 

Although there is no universally accepted format for the research process most studies adhere to the sequence of scientific enquiry. Scientific enquiry is the search for knowledge by using recognized methods in data collection, analysis and interpretation. The scientific method is usually a sequential research process. The typical steps in the scientific method are:

 

 

Ø  Define a problem

Ø  state the hypotheses to be tested

Ø  collect and analyse data

Ø  interpret the results and draw conclusions about the problem (McMillan and Schumacher, 1993).

 

Research has found that different universities and even faculties in the same university may use the same process with minor modifications. Students’ research projects at the ZOU are structured following seven stages;

 

Ø  The research topic/title

Ø  Research questions/Hypotheses

Ø  Literature review

Ø  Research methodology

Ø  Data collection

Ø  Data analysis, presentation and discussion of findings

Ø  References

 

The Research Topic/Title

Most ZOU degree programmes require students to undertake research and submit a research project report within a period of 12 months. The research projects are unstructured. This means students choose their own topics, research paradigm, research design and data collection instruments with minimum guidance from their supervisors.   The research process begins with the student identifying a research problem and formulating a research topic which he or she then submits to the programme coordinator for approval. The role of the coordinator is to assess whether the topic is researchable, relevant to the programme for which it is undertaken, feasible, concise, and unambiguous, of public interest and clearly delimits the area of study. The coordinator then allocates the student to a supervisor on the basis of the supervisor’s proximity and expertise in the student’s research topic. Bogdan and Biklen (1992) has noted that right at the initial stage, quite often, students fail to define a research problem from either a conventional or technical meaning. Kratwohl (1985) identifies lack of training and experience in research methods as an obstacle to students’ progress in research. Thus the quality of a research project can be affected by the choice of a research topic, lack of training and the use of inappropriate methods. The present study sought to establish the methodological appropriateness and methodological quality of projects submitted by ZOU undergraduate students from 2000-2009.

 

Research Questions/Hypotheses

Research studies may state the research problem in question form or as a hypothesis. According to Hedrick,et. al., (1993) the research topic operationalises objectives of the proposed research which then lead to research questions or hypotheses. Research questions clarify and focus the researcher towards what information (data) needs to be collected, from what sources and under what conditions. The researcher’s worldview (Paradigm) influences the nature of the research questions and proposed methodology (Mertens, 2005). The question format is often preferred because it is simple and direct. Psychologically, it orients the researcher to the immediate task: to develop a design to answer the question (Mertens (2005).  A research hypothesis is a tentative statement of the expected relationship between two or more variables. A hypothesis directs the study. Actually the study is undertaken in order to either accept or reject the hypothesis. Research questions are mainly associated with qualitative research and hypotheses are associated with quantitative studies.

 

The present study sought to establish whether students’ research projects were guided by research questions or hypotheses. Borg and Gall (1989) put research questions into two categories: descriptive questions and relational questions. The former are based on the question “What is…..?, while the relational studies are predicted by questions such as “What factors determine, cause or lead to….?”. This current study is based on Hedrick et. al., (1993) taxonomy of research questions. Hedrick et. al., (1993) posit that research questions can be classified into four types: descriptive, normative, co relational and impact.

 

Descriptive research questions are designed to produce information about what is or has been happening in relation to the target of the research. Normative research questions go beyond description and require that information generated in response to descriptive research questions be compared with some standard or expected observation. Correlative research questions are used to identify relationships to enable the explanation of phenomenon. Hedrick, et. al., (1993) point out that data derived in response to such questions indicates the strengths and direction of a relationship between two or more variables, not causality. Impact research questions aim at identifying effects, to establish causal links between an independent variable (the intervention) and a dependent variable (the anticipated change).

 

Thus the type of research question determines the type of data to be collected for a particular study. Research data can be either in numbers or words. According to Leedy (1993) the nature of the data and the research question dictate the methodology that must be employed in a particular study.

 

Research that collects, analyze and present data in the form of numbers is said to be quantitative. Research that uses words is qualitative. Research that collects both quantitative and qualitative data is said to use mixed methods.

 

Research paradigms 

A paradigm is a way of looking at the world.  It is composed of certain philosophical assumptions that guide and direct thinking and action of the researcher. The labels quantitative, qualitative and mixed methods are linked to particular research paradigms. Lather (1992) and Lincoln and Guba (2000), have identified four major paradigms that have influenced educational research:  Postpositivism, Constructivist, Transformative and Pragmatic. The present study sought to establish the research paradigm most preferred by undergraduate students and the extent to which the paradigms were related to the other concepts in the research projects (research design, data collection instruments, form of data collected and procedures for data analysis and presentation.

 

Postpositivism

This was the dominant paradigm that guided early educational research. It is commonly referred to as the quantitative paradigm. Positivism is based on the rationalistic, empiricist philosophy that originated with Aristotle, Francis Bacon, John Lock, August Compte, and Emmanuel Kant (Mertens, 2005).  Positivism assumes that the social world can be studied in the same way as the natural world and there is a method for studying the social world that is value free and that explanations of a causal nature can be provided and events or observations are influenced by a single theory or paradigm. 

 

Constructivist

The proponents of constructivist paradigm were Campbell and Stanley (1963, 1966), Cook and Campbell (1979). 

 

According to Cook and Campbell (1979), constructivist paradigm rejects the position that events or observations are influenced by a single theory or paradigm. The basic assumptions guiding the constructivist paradigm are that knowledge is socially constructed by people active in the research process and that researchers should attempt to understand the complex world of lived experiences from the point of view of those who live it (Schwandt 2000). The constructivist paradigm, also known as qualitative paradigm, emphasizes that research is the product of the researchers and cannot be independent of them (Mertens, 2005).

 

Transformative

The transformative paradigm assumes that there are multiple realities shaped by social interaction. There is an interactive link between researcher and participants.  Knowledge is socially and historically situated. Research studies adopting the transformative paradigm include action research, critical theory, participatory research and feminist theory. Researchers following the transformative paradigm fix themselves self- consciously as participatory activists and believe that when one immerses oneself in the activity, there is likelihood that one elicits more data (Lincoln & Guba, 2000). The main focus of transformative researchers is to seek to unearth, disrupt, and transform existing ideological and or institutional arrangements. This is the research paradigm usually adopted by political activists and gender lobbyists when they seek to change gender and political ideologies.

 

Pragmatism

Pragmatism is a paradigm that provides the underlying philosophical framework for mixed methods research (Maxcy, 2003; Patton, 2002; Tashakkori and Teddlie, 2003). These philosophers rejected the scientific notion that social science enquiry was able to access the truth about the real world solely by virtue of a single scientific method. To this end, the most appropriate paradigm is the pragmatic paradigm in which the researcher is not confined to one research paradigm (Patton 2002).  When analyzing methodological appropriateness and methodological quality of research projects one must be aware of the paradigmatic debates that currently exist between quantitative and qualitative researchers.

 

The Quantitative and Qualitative Paradigm Debate

Most research work is guided by either the quantitative (logical positivism) or qualitative (constructivist) research paradigm or a mixture of the two (pragmatism) (Guba, 1978). Researchers have long debated the relative value of qualitative and quantitative inquiry (Patton, 1990). Each approach represents a fundamentally different inquiry, and researcher actions are based on the underlying assumptions of each paradigm.

 

Quantitative research reflects the traditional scientific approach to problem solving. It uses experimental methods and numerical data to test hypothetical generalizations.  It assumes that there is a single reality that can be broken down into variables. The purpose of this type of research is to test hypotheses that have been developed before the research project started and to form conclusions that can be generalized to other situations. The emphasis in this approach is upon measurement, comparison, and objectivity.  Qualitative research, broadly defined, means any kind of research that produces findings not arrived at by means of statistical procedures or other means of quantification. Qualitative research uses a naturalistic approach that seeks to understand phenomena in context-specific settings.

 

Where quantitative researchers seek causal determination, prediction, and generalization of findings, qualitative researchers seek instead illumination, understanding, and extrapolation to similar situations. The most obvious distinction between quantitative and qualitative research is the form of data collection and presentation.

 

The quantitative approach has dominated the field of education since the 1950's and is considered by some as the major approach to educational research. Proponents of the quantitative paradigm include Dodds, Lawrence, & Guiton, 1984); Kember & Dekers, 1987); Osman & Wagner, 1987). These researchers maintain that quantitative methods are inherently preferable to non-quantitative methods. Contrary to this view, Smith and Manning (1982) suggest that qualitative research is better for initial exploratory research into unknown or unfamiliar phenomena.

 

It enables the researcher to develop concepts, conjectures, interpretations and theories empirically grounded in the investigated systems. In the real world, events cannot be teased out from the context in which they are inextricably embedded, and understanding involves the relationships among all of the many parts of the whole.

 

Thus qualitative methods- which emphasize both inner and outer knowledge of man in his world- are preferable (Filstead, 1970). This argument has led to the present confusion between the quantitative and qualitative paradigms (Owens, 1982). The prevailing theoretical understanding is that a combination of the two paradigms can generate more powerful analyses than either approach could generate alone (Miles, 1983). Researchers such as Strauss & Corbin (1990); Patton, (1990), believe that qualitative and quantitative research methods can be effectively combined in the same research project.

 

In support of this view, Russek and Weinberg (1993) claim that by using both quantitative and qualitative data, their study of technology-based materials for the elementary classroom gave insights that neither type of analysis could provide alone. However, the aim of the present study was not to enter into the ‘paradigm war’ but to contribute knowledge towards quality assessment of students’ research work. This article in no way suggests that one paradigm is inherently superior to the other but that both have an appropriate context.

 

A research paradigm (quantitative, qualitative or mixed methods) has a direct influence in the choice of research design to be adopted for a particular study. Methodological appropriateness is established when there is a perfect link between the research paradigm and the research design.

 

Research Design

According to McMillan and Schumacher (1993), research design refers to the plan and structure of the investigation used to obtain evidence to answer research questions.  The design describes the procedures for conducting the study, including when, for whom, and what type of data will be obtained. In actual fact, it is the nature of data to be collected that dictates the research design that must be employed in a particular study. The design indicates how the research is set up; what happens to the subjects and what methods of data collection are used. 

 

The purpose of the design is to provide the most valid, accurate answers possible to the research question.  Since there are many types of research questions and many types of designs, it is important to match the design with the research question.  For the purposes of the present study, research designs were classified according to four categories; true experimental, quasi-experimental, non-experimental and mixed methods.

The current study sought to establish the most preferred research design by ZOU students and the appropriateness of the designs used.

 

Data Collection Techniques  

Another way of classifying research besides research designs eg,.experimental, non- experimental, is to examine the technique used in the study to collect the data. There are basically six ways to collect data: observations questionnaires, interviews, documents, tests, and unobtrusive measures (Mcmillan and Schumachar, 1993).

 

These techniques can be classified as either quantitative or qualitative. Quantitative techniques use numbers to describe phenomenon while qualitative techniques use narrative descriptions. Most of the techniques can be used with any of the research designs. Quantitative techniques are used with experimental, descriptive and correlational designs (Mcmillan and Schumachar, 1993).

 

Examples of quantitative techniques include structured observations, standardised interviews, tests, questionnaires, unobtrusive measures.

 

Qualitative techniques are used with descriptive surveys, ethnographic, historical, content analysis and case study designs. Examples of qualitative techniques include ethnographic observations, ethnographic interviews, and documents. The present study sought to establish whether the data collection techniques used in the projects were appropriate to the research designs. 

 

METHODOLOGY

 

The present study employed both quantitative and qualitative research techniques and was guided by grounded theory. Grounded theory relates abstract concepts to propose a theory as an explanation of the phenomena.

 

The theory is “grounded” in that it is developed from the data in contrast to testing a theory from the literature.  Data are presented to describe each concept and its relationships to the other concepts.

 

Research Design

The present study involved content analysis of research projects submitted by ZOU undergraduate students. There are two general categories of content analysis: conceptual analysis and relational analysis. Both types of content analysis were applied. Conceptual analysis focused on establishing the existence and frequency of certain concepts in the projects.

 

The concepts were chosen for examination and the number of their occurrences (quantity) within the projects recorded. In this study conceptual analysis was undertaken by using open coding. Open coding is the part of analysis that pertains specifically to naming and categorizing phenomena through close examination of data (Mertens, 2005:424). In order to come up with a checklist of concepts and codes, we initially analyzed 50 research projects.

 

Our analysis produced a checklist of six subcomponents of the research project: research questions, research paradigm, research design, data collection instruments, data form, procedures for data analysis and presentation. These concepts were categorized and coded as presented in Table: 1.

 

 

Table: 1Checklist of Concepts, Categories and Codes

 

Concept

Category

Code

Research Questions

1.Descriptive

2.Normative

3.Correlational

4.Impact

D

N

C

I

Data Form

1.Numbers

2.Words

N

W

Research Paradigm

1.Post Positivist (Quantitative)

2.Constructivist (Qualitative)

3.Transformative

4.Pragmatic (Mixed Methods)

PP

C

TP

MMP

Research Design

  1. True Experimental

       (Quantitative)

 

 

3.    Quasi-Experimental

(Quantitative)

4.    Non-Experimental

(Qualitative)

 

 

     5.  Mixed Methods

(Quantitative- (Qualitative)

 

 

·   Post-test equivalent group

·   Pretest-posttest

·   Solomon four

·   Analytical survey

-

 

·   Descriptive survey

·   Ex post facto

·   Case study

·   Content analysis

·   Historical

       -

 

Pt

PtPt

SF

AS

 

Q

 

 

DS

EPF

CS

CA

H

MMD

 

Data collection instruments 

·   Questionnaires

·   Interviews

·   Observations

·   Tests

·   Documents & records

Q

I

O

T

D & R

 

 

Reliability of the Methods

We checked the reliability of the coding a (95% agreement is suggested; .8 for Cohen's kappa).

 

To establish the dependability of the categorization decisions, one researcher coded all the items and a second researcher independently coded a 10% random sample of those items. The proportion of simple agreement between the coders was 0.96.

 

Sample

Data for the current study was collected from a sample of 400 research projects, which were submitted by ZOU undergraduate students between te years 2000-2009. The sampling techniques involved the following steps.

 

First, we identified the four faculties in the university: Arts & Education, Social Sciences, Commerce & Law and Physical Sciences. From each faculty two undergraduate degree programmes were randomly selected using the hat method. Each Faculty has a minimum of four undergraduate degree programmes. From each degree programme, 50 research projects were randomly selected. (See Table: 2)

 

 

Table: 2

Random sample of research projects content

analyzed for the current study

 

FACULTY

DEGREE PROGRAMME

PROJECTS

Arts & Education

1. Bachelor of Educational Management

2. Bachelor of Arts, English & Communication

50

50

Social Sciences

1.Bachelor Science (Honours) in Counselling

2.Bachelor of Science (Honours) in Special

   Education 

50

 

50

Commerce & Law

1.Bachelor of Commerce in Human Resource

   Management

2. Bachelor of Commerce in Marketing

50

 

50

Physical Sciences

1.Bachelor of Science Degree in Agriculture

2.Bachelor of Science Degree in Geography &

   Environmental Sciences

50

 

50

 

TOTAL

400

 

Using the checklist of concepts and codes, the four researchers conducted a conceptual and relational analysis of a sample of the 400 research projects submitted by ZOU undergraduate students from 2000–2009. The first objective of the present study was to establish the status of students’ research work, its existing gaps and areas of saturation.In order to achieve this objective, we analyzed each project and identified the existence of the concepts and their frequency.

 

The analysis focused on the following concepts: research questions, research paradigms, research designs, and data collection methods.

 

The second objective of the study was to undertake a relational analysis of the research projects in order to establish the quality and appropriateness of the methods applied.

 

Findings of the present study are discussed below with reference to the research questions that guided the study, the prevailing research theory and findings of previous research studies.  

 

DISCUSSION OF FINDINGS

 

Dominant Research Questions and Hypotheses

The first concept we analysed was the research question or hypotheses that guided each study.

 

Findings of the current study revealed that, across faculties, the majority of research projects one hundred and ninety-five (48.75%) used descriptive research questions, followed by ninety-four (23.5%) impact research questions, Sixty-five (16.25%) correlational questions and forty-six (11.5%) used normative research questions (See Table. 3 below).

 

Most studies in the department of Agriculture were guided by hypotheses and studies in the department of geography used both hypotheses and research questions.

 

Table: 3

Research questions most preferred by ZOU undergraduate students

 

FACULTIES

RESEARCH QUESTIONS

Descriptive

Normative

Correlation

Impact

N

%

N

%

N

%

N

%

Arts & Education

53

13.25

8

2

25

6.25

14

3.5

Social Sciences

80

20

14

3.5

0

0

6

1.5

Commerce & Law

46

11.5

11

2.75

14

3.5

29

7.25

Physical Sciences

16

4

13

3.25

26

6.25

45

11.25

Totals

195

48.75

46

11.5

65

16.25

94

23.5

 

Studies in the faculties of Arts & Education, Commerce & Law and Social Sciences were mainly descriptive hence they were guided by descriptive research questions. Descriptive studies are mainly used to describe peoples’ characteristics such as their attitudes, beliefs, habits, values, demographics, behaviours, opinions, desires, ideas, and other types of information.

 

Our findings agree with Mcmillan & Schumacher (1993) who have noted that descriptive research is popular in social research because it does not involve manipulation of independent variables.

 

Descriptive research questions typically ask “what is “and imply a survey research design (Mcmillan & Schumacher, 1993:86).  Mertens, (2005:107) says descriptive researches are designed to produce information about what is or has been happening in relation to the target of the research. Borg and Gall (1996 p. 373) characterize descriptive research as “the most basic of methods, which involves simply describing the characteristics of individuals or other phenomena.”

 

The use of descriptive research questions by ZOU students, therefore, may be an indicator or a measure of students’ level of research skills and the quality of training and project supervision availed to the students. Impact research questions were second most preferred and were popular amongst research students from the faculty of Physical Sciences. Research projects from Agriculture were mainly experimental, hence they used impact questions and hypotheses (11.25%) and correlational questions (6.25%).

 

The Dominant Research Paradigm

The second concept we analysed was the research paradigm. The present study revealed that quantitative research dominated in the physical sciences, particularly in Agriculture (72%), and the qualitative paradigm influenced a significant number of studies in the faculties of Arts & Education (B.Ed, 10% and English, 18%), Commerce & Law (Human Resource, 22% and Marketing, 16%), and Social Sciences (Counseling, 20% and Special Education, 12%).

 

Another finding was that there was a growing trend towards the use of mixed methods (B.Ed, 54%; English, 46%; Human Resources, 52%; Marketing, 40%; Counselling, 44% and Special Education, 58%).

 

The pragmatic paradigm was more dominant in the department of Geography with sixty-six percent of the projects applying both quantitative and qualitative techniques (See table 4 below).

 

 

Table: 4

The Dominant Research Paradigm

 

 

Paradigms

Faculties

Arts& Educ.

Com & Law

Social Science

Physical Science

B.Ed

Eng.

HR

Market

Coun

Sped

Agric

Geo

N

%

N

%

N

%

N

%

N

%

N

%

N

%

N

%

 

Positivism

(Quantitative)

3

6

4

8

2

4

9

18

8

16

6

12

36

72

1

2

 

Constructivist

(Qualitative)

5

10

9

18

11

22

8

16

10

20

6

12

0

0

2

4

 

Pragmatism

(Mixed Method)

27

54

23

46

26

52

20

40

22

44

29

58

2

4

33

66

 

No Paradigm

15

30

14

28

11

22

13

26

10

20

9

18

12

24

4

8

 

Total

50

100

50

100

50

50

50

100

50

100

50

100

50

100

50

100

 

 

Another interesting finding was that 22% of the projects did not identify a paradigm. Leaders in the field of research do not all do not agree as to the need to acknowledge an underlying paradigm, nor do they agree on the role that such paradigms serve in the research process.

 

Whilst Patton (2002) says research paradigms are unnecessary and possibly handicapping, Schwandt, (2000), argues that research paradigms are inescapable.

 

The need for a paradigm is also supported by Mertens (2005), who maintains that a researcher’s theoretical orientation (paradigm) has implications for every decision made in the research process, including choice of methods. Skrtic (1991), argues that the absence of a research paradigm, renders the research atheoretical since it rests on an unexamined and unrecognized theory. Thus, the fact that 22% of ZOU research students from the faculties of Arts & Education, Commerce & Law and Social Sciences proceeded to undertake their studies without acknowledging their theoretical paradigm or philosophical assumptions (whether by design or otherwise) raises questions pertaining to the quality of training and supervision research students receive from their tutors.

 

The present study shows that gaps of knowledge and skills with regards to the role of research paradigms still exist amongst ZOU research students and their project supervisors. The question that comes to mind is “Do ZOU research students fully understand the role played by paradigms in research?” We resolved to address this question in a follow up study.

 

The Dominant Research Design

The third concept we analysed was the research design. We based our analysis on Borg and Gall’s classification of research questions. According to Borg and Gall (1989) there are two categories of research questions- descriptive and relational. Descriptive questions lead to non- experimental designs and relational questions lead to experimental designs and the use of both leads to mixed methods. We, therefore, identified and classified research designs as experimental, non-experimental or mixed methods. Our findings revealed that 72% of research projects from the faculty of Physical Sciences, Agriculture in particular, adopted experimental research designs.

 

Research projects from Geography (66%) were mostly case studies and applied mixed methods. Research projects from the faculties of Arts & Education, Commerce & Law, Social Sciences employed non-experimental research designs confined to descriptive surveys (89%), case studies and ex post facto studies.

 

This study revealed a clear distinction between research methodologies used in social sciences and those used in physical sciences.

 

Whilst the descriptive survey research design dominated in almost 89% of studies in the faculties of Arts & Education, Commerce, and Social Sciences, experimental designs were mostly applied in Agriculture.

 

Our findings agree with Mertens (2005); Borg and Gall (1996); and McMillan & Schumacher (1993) who have noted that descriptive surveys are the most basic of methods, popular and are used pervasively in social research.

 

Descriptive surveys are mainly used to describe peoples’ characteristics such as their attitudes, beliefs, habits, values, demographics, behaviors, opinions, desires, ideas, and other types of information. 

 

Certainly, there is a place for descriptive designs in educational research, but whether this should occupy almost 90% of students’ research at the ZOU is a topic for discussion and further research.

 

Perraton (1988) argues that the confinement of students’ research projects to one or two research designs is a function of the lack of theoretical foundations in research methodology. The present findings, therefore, fuel academic debate about the level of students’ research skills and the quality of training and supervision they receive from their project supervisors.

 

Data Collection Instruments

The fourth concept we analysed involved data collection instruments. Our analysis revealed that the questionnaire was the most preferred data collection instrument in the faculties of Arts & Education (35%), Social Sciences (36%), and Commerce & Law (35%). (See Table: 4).

 

A combination of the questionnaire and interviews was also equally preferred, Arts & Education (35%), Social Sciences (21%), and Commerce & Law (38%).

 

Studies in the faculty of Physical Sciences, particularly in the department of Agriculture applied tests and observations (42%), tests and documents (16%) and tests and questionnaires (10%).

 

This study revealed a clear distinction between data collection instruments applied in social research-qualitative studies in (Arts & Education, Commerce & Law and Social Sciences) and quantitative studies in natural sciences.

 

This is because whilst quantitative research emphasizes objective measurements and collection of numerical data through tests, observations, questionnaires or surveys (Saunders et.al., 1997), qualitative research focuses on understanding social phenomena through interviews and personal comments (Strauss & Corbin, 1990).

 

 

Table: 5

Data collection instruments mostly preferred

by ZOU undergraduate research students

 

 

 

DATA COLLECTION

TECHNIQUES

FACULTIES

Arts &

Education

Social

Sciences

Commerce & Law

Physical Sciences

N

%

 

%

N

%

N

%

Questionnaires only

35

35

36

36

35

35

5

5

Test only

0

0

0

0

0

0

0

0

Interviews only

13

13

23

23

15

15

15

15

Observation only

0

0

2

2

0

0

2

2

Documents

2

2

0

0

5

5

0

0

Group Discussion

0

0

5

5

2

2

0

0

Questionnaire & Interviews

35

35

21

21

38

38

10

10

Questionnaire, Interview &

Observations

10

10

1 2

1 2

4

4

0

0

Questionnaires & Documents

5

5

1

1

1

1

0

0

Test & Interviews

 

 

 

 

 

 

1

1

Tests & Questionnaire

 

 

 

 

 

 

10

10

Tests & Documents

 

 

 

 

 

 

16

16

Tests & Observations

 

 

 

 

 

 

42

42

 

Our findings show that most of the qualitative studies in the faculties of Arts & Education, Commerce and Law used a combination of the questionnaire and the interview because they collected both quantitative and qualitative data.

 

RELATIONAL ANALYSIS

 

The second objective of the present study was to conduct relational analysis of the research projects submitted by ZOU undergraduate students between 2000-2009 in order to assess methodological appropriateness and methodological quality. Relational analysis of the research projects was conducted through the use of axial coding and selective coding (Mertens, 2005: 424). Axial coding is the part of the analytic process in which the researcher puts the parts of the data identified and separated in open coding back together to make connections between categories (Mertens, 2005: 424). Selective coding involves the process of selecting one core category (the story line) and relating the other categories to it (Mertens, 2005). Strauss and Corbin, (1990) use the term paradigm to refer to the storyline. For the present study we first identified the research question as the core category (storyline) and the rest of the concepts as sub-concepts. We then developed a model that linked these sub-concepts to the storyline (research question). (See Figure: 1 below) By using the relational analysis model that we had developed, we classified (for each project) the research question as descriptive or relational (differences, impact, co relational) and the type of data collected as qualitative, quantitative or both.

 

We then classified and analyzed the research paradigm, research design, and data collection instruments and data analysis procedures under these three paradigms. To establish methodological appropriateness and methodological quality, we analyzed the links between the research question (core concept) and the following sub-concepts: data collected, research paradigm, research design, data collection instruments and techniques employed in the analysis and presentation of research findings.

 

 

Research Question

 

 
 

 

 

 


                                                         

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Figure: 1

Relational Analysis Model for research projects

 

Our analysis led to the discovery of four categories into which research studies by ZOU undergraduates could be classified:

 

Ø  Truly qualitative research,

Ø  Truly quantitative research,

Ø  Truly Mixed methods research and 4) Partially Mixed Methods (Table 6)

 

 

 

Table: 6

Types of Research studies at the Zimbabwe Open University

 

 

Paradigm

Faculties

Arts & Education

Commerce & Law

Social Sciences

Physical Sciences

Total

 

N

 

N

 

N

 

N

 

N

%

Truly Quantitative

7

 

11

 

13

 

44

 

75

18.75

Truly Qualitative

14

 

18

 

15

 

2

 

49

12.25

Truly Mixed Methods

3

 

1

 

2

 

34

 

40

10

Partially Mixed Methods

47

 

46

 

51

 

4

 

148

37

No Paradigm

29

 

24

 

19

 

16

 

88

22

Total

100

 

100

 

100

 

100

 

400

100

 

Truly qualitative research

Our study revealed that 49 (12.2 5%) of the studies were purely qualitative. These studies came from the faculties of Arts & Education, Commerce & Law and Social Sciences. These studies were guided by descriptive research questions, employed descriptive surveys or case study research designs, collected data in words, analysed data in context and presented findings in words. Only two (0. 5%) research projects from the faculty of Physical Sciences were qualitative. Patton (2002), notes that the choice of qualitative methods might be appropriate in many educational research studies because they are based on humanistic values.

 

Truly quantitative research

Quantitative research accounted for 75 (18.75%) studies of which 36 (9%) came from Agriculture and the rest from faculties of Arts & Education, Commerce & Law and Social Sciences. Most of the projects from the faculty of Physical Sciences, Agriculture in particular were guided by hypotheses or relational (differences, impact, correlational) research questions.

 

These studies were mainly quantitative, employed experimental research designs, collected numerical data through the used tests and observations, analysed data using statistical packages such as SPSS and presented research findings using frequency tables, statistics, graphs and percentages.

 

These projects demonstrated a high degree of methodological appropriateness and methodological quality in that quantitative data was collected, analysed and presented through the use of quantitative methods.

 

Fully integrated mixed methods

Our analysis revealed that 40 (10%) research projects used mixed methods throughout the research process. These studies were mainly from the department of Geography 33 (8.25%) and the rest from the faculties of Arts & Education, Commerce & Law and Social Sciences.

 

Almost all studies from Geography, indicated that they were influenced by both quantitative and qualitative research paradigms, employed case study research design, collected both numerical and textual data, analysed and presented data using both words and numbers. Teddlie and Tashakkori (2002) describe the following characteristics as those of a truly mixed approach methodology.

 

Ø  It would incorporate multiple approaches in all stages of the study (i.e problem identification, data collection, data analysis and final inference)

Ø  It would include a transformation of the data and their analysis through another approach eg. content analysis of data followed by a quantitative analysis of the same data. 

 

In this study these characteristics were revealed in the projects from Geography.

 

Partial Mixed methods

Our analysis revealed that 148 (37%) research projects from mainly the faculties of Arts & Education, Commerce & Law and Social Sciences employed mixed methods. These studies initially indicated that they were qualitative, employed the descriptive survey research design but used questionnaires and interviews to collect data in numbers and words. Lincoln and Guba (2000) identify qualitative methods as the preferred methods for researchers working in the constructivist paradigm. However, they also recognize that quantitative methods can be used within this paradigm when it is appropriate to do so. 

 

According to Mertens (2005) qualitative and quantitative data collection can occur in parallel form or sequential form. Mixed methods can also involve the conversion of qualitative data to a quantitative form or vice versa.  According to Mertens (2005) mixed methods can be applied at four different levels:  problem identification, data collection, data analysis, and discussion of research findings.

 

Greene and Caracelli (2002) argue that social researchers mix methods to a varying degree at various points in their research and still call their work mixed method research. However ZOU undergraduate researchers called their research work qualitative.

 

These findings have a bearing on the current ‘quantitative – qualitative’ paradigmatic debate. Proponents of mixed methods (Patton, 1990; Strauss & Corbin 1990) believe that qualitative and quantitative research methods can be effectively combined in the same research project.

 

In fact, Patton (1990) advocates for a "paradigm of choices" that seeks "methodological appropriateness as the primary criterion for judging methodological quality. Methodological appropriateness, Patton (1990) argues, will allow for a "situational responsiveness" that strict adherence to one paradigm or another will not.

 

Patton is also supported by Lincloln and Guba (2000) who note that many changes have occurred in the status of paradigms and choice of methods over the last decade such that various paradigms are beginning to “interbreed”. Russek and Weinberg (1993) have also argued in support of mixed methods.

 

They claim that by using both quantitative and qualitative data, their study of technology-based materials for the elementary classroom gave insights that neither type of analysis could provide alone.

 

However, Mertens (2005) argues that the field of research has not yet reached the point of full integration of paradigms. Our findings support Mertens (2005) in that all the studies that made use of mixed methods in the faculties of Arts & Education, Social Sciences and Commerce and Law) except for Geography, initially indicated that the studies were qualitative but went on to apply both quantitative and qualitative techniques and procedures during data collection, analysis and presentation.

 

CONCLUSIONS AND DIRECTIONS FOR FUTURE RESEARCH

 

This study was a content analysis of research projects submitted by ZOU undergraduate students between 2000-2009. The study aimed to achieve two objectives. The first objective was to conduct a conceptual analysis of the projects in order to establish the status of students’ research work, its gaps and areas of saturation. The present study found out that for the past decade, undergraduate research work at the ZOU has been characterized by four features.

 

Ø  A clear distinction existed, between research methodologies applied by students in the faculties of Arts & Education, Commerce & Law and Social Sciences on one hand and those applied by students in the faculty of Physical Sciences on the other hand. And in the faculty of Physical Sciences another distinction existed between the departments of Agriculture and Geography. Whilst (%) of studies in the department of Agriculture were quantitative and used simple experiments, (%) of studies in the department of Geography were case studies and employed mixed methods.

Ø  The study also found out that although most research studies from the faculties of Arts and Education, Commerce and Law and Social Sciences were qualitative, basic descriptive surveys, they also employed quantitative techniques in data collection, analysis and presentation.

Ø  A significant number (22%) of projects across faculties did not identify the research paradigm.

Ø  Most research projects in the faculties of arts and education, Social Sciences, Commerce and Law were guided by descriptive research questions, whereas projects in the faculty of Physical Sciences were guided by relational research questions and hypotheses.

Ø  Questionnaires and interviews were the most preferred data collection instruments for studies in the faculties of Arts and Education, Commerce and Law and Social Sciences. Studies in the faculty of Physical Sciences used tests and observations.

 

The second objective was to conduct a relational analysis of the projects in order to assess the methodological appropriateness and methodological quality of the projects. The present study revealed that students’ projects demonstrated a high degree of methodological appropriateness and methodological quality. Appropriate linkages were established between research questions, data collected and the methods used in data collection, analysis and presentation. The use of mixed methods also enhanced the quality and appropriateness of the research methodology.

 

However, the confinement of students’ projects to basic descriptive surveys (for qualitative studies) and to simple experiments (for quantitative studies) reflects limited knowledge and low- level research skills in both research students and their supervisors. Thus the present study raises a number of critical questions: What is the level and quality of knowledge and research skills ZOU students have? What is the quality of the preparatory courses ZOU students undertake before embarking on their research projects? Do these courses adequately equip students with the knowledge and research skills students require for them to undertake and produce quality research projects? What is the quality of supervision research ZOU students get during the conduct of their research projects? How can ZOU enhance the quality of project supervision, quality of students’ research work and quality assessment of students through research projects? These questions call for further research studies.

 

Implications

The current study has a number of implications in the assessment of students through research projects. Primarily, it serves to inform the Zimbabwe Open University about the status of students’ research work for the past ten years. This study contributes knowledge on how the current developments taking place in research theory have influenced research activities at the ZOU.

 

The study also points in the direction educational students’ research activities at the ZOU are taking. The study contributes useful knowledge and skills that can be used in the assessment of ODL students through research work. Research students and their supervisors can use the relational analysis model developed in this study to assess quality of their own research work.

 

Developers of teaching and learning materials can also use findings of this study to develop courses and materials that adequately equip students with the right knowledge and skills required for research.

 

There is, at the ZOU, an urgent need to evaluate the current courses that prepare students for research work. These courses are:

 

Ø  Introduction To Research Methods

Ø  Statistics For Educational Research

Ø  Computer Application In Educational Research

 

BIODATA and CONTACT ADDRESSES of AUTHORS

 

C KANGAICaleb KANGAI (Mr.) is a lecturer teaching at the Zimbabwe Open University in the Faculty of Education. He holds two Masters Degrees-the first in Educational Management and the second in Business Administration.  His research areas include issues of quality and effectiveness in the management of Open and Distance Learning (ODL). He has published three modules for undergraduate and post -graduate students at the Zimbabwe Open University; Dissertation Guidelines for the Master of Education in Educational Administration, Planning and Policy Studies. (2002); Strategic Management in Education (2009) and The Politics of Education (2008)

 

Caleb KANGAI

Zimbabwe Open University 

Mashonaland East Region,

P.O. Box 758, Marondera, ZIMBABWE

Tel:  +26327921922, Cell : +263913547204

Email:calebkangai@gmail.com

 

BUKALIYARichard BUKALIYA (Mr.) is a lecturer teaching at the Zimbabwe Open University in the Faculty of Education. He holds a Masters Degree in Educational Administration, Planning and Policy Studies. His research interests include issues in Distance Education. To date the researcher has not published any article but it is envisaged that this article is the beginning in a series of other articles to come.

 

Richard BUKALIYA

Zimbabwe Open University 

Mashonaland East Region,

P.O Box 758, Marondera, ZIMBABWE

Tel: +26327921922

Cell: +263733686817

Email:bukaliar@yahoo.com

 

FARIRAI

 

Farirai MUSIKA (Ms) is an Assistant lecturer teaching at the Zimbabwe Open University in the Faculty of Social Sciences. She holds a Bachelor of Science Honours degree in Psychology. Her research interests include issues in Distance Education.

 

 

Farirai MUSIKA

Zimbabwe Open University 

Mashonaland East Region, P.O Box 758,

Marondera, ZIMBABWE

Tell: +26327921922

Cell: +263734353277 

Email: farimusika@gmail.com

 

MAPURANGAMapuranga BABRA (Mrs.) is a lecturer teaching at the Zimbabwe Open University in the Faculty of Social Sciences.  She holds a Masters degree in Special Education. She has co-authored the following three modules published by Zimbabwe Open University She has A Focus to Intellectual Disabilities, Assessment and Rehabilitation and Interventions: Facilitating Life Skills

 

Mapuranga BABRA

Zimbabwe Open University, 

Mashonaland East Region, P.O Box 758,

Marondera, ZIMBABWE

Tel: +26327922267

Cell: +263912917026

Email: mapubee@gmail.com

 

REFERENCES

 

Bogdan R. C. & Biklen S. K. (1992) Qualitative Research for Education. Boston Allyn and Bacon.

 

Borg, W. R. & Gall, M. D. (1989). Educational Research. White Plains. N Y. Longman.

 

Campbell, D. T. & Stanley, J C. (1963). Experimental and Quasi-Experimental Designs for Research on Teaching. Chicago, Rand McNally.

 

Campbell, D. T. & Stanley, J. C. (1966). Experimental and Quasi-Experimental Designs for Research. Skokie, I L. Rand McNally.

 

Cook, T. D. & Campbell, D. T. (1979). Quasi-Experimentation: Design and analysis issues for field settings. Chicago, Rand McNally.

 

Dellana, S., Collins, W. & West, D. (2000). Online education in a management science course-effectiveness and performance factors. Journal of Education for Business, 76,43-48.

 

DeSantis, C. (2002). ELearners.com. Retrieved November 2, 2002, http:// elearners. Com

 

Dodds, A. E., Lawrence, J. A. & Guiton, P. (1994). University students’ perceptions of influences on external study. Distance Education, 5(2), 174-185.

 

Filstead, W. J. (1970). Qualitative Methodology. Chicago: Markham. Gage, N. L. (1989). The paradigm Wars and their Aftermath. Educational Researcher, 18(7), 4-10.

 

Gall, M., Borg, W. & Gall, J. (1996). Educational research: An introduction (6th ed.). New York: Longman.

 

Guba, E. G. (1978). Toward a methodology of naturalistic inquiry in educational evaluation. Monograph 8. Los Angeles: UCLA Center for the Study of Evaluation.

 

Greene, J., and Caracelli, V. J. (2002). Making paradimatic sense of mixed methods practice. Thousand Oaks, C A: Sage.

 

Hedrick, T. E., Bickman, L. & Rog D. (1993). Applied Research design: A practical Guide: Newbury Park. C A: Sage.

 

Kangai, C. V. & Mapolisa, T. (2008). Citation analysis of research projects submitted by Bachelor of Education (EAPPS) students’ (2000-2004) to the department of education at the Zimbabwe Open University: Implications for educators and librarians.

 

Kember, D. & Dekkers J. (1987). The role of study centres for academic support, Distance Education, 8  (1), 4-17.

 

Krathwohl, D. R. (1985). Methods of Educational and Social Science Research. White Plains. N Y. Longmans.

 

Lather, P. (1992). Critical frames in Educational Research: Feminist and post structural perspectives: Theory and Practice.

 

Lincoln, Y. & Guba, E. (1985). Naturalistic inquiry. New York: Sage. Moon, S., Dillon, D., & Sprenkle, D. (1990). Family therapy and qualitative research.

 

Lincoln, Y. S. & Guba, E. G. (2000). Paradigmatic controversies, contradictions and emerging confluences. Thousand Oaks C A: Sage.

 

Maxcy, S. J. (2003). Pragmatic threads in mixed methods research in the Social Sciences: The search for multiple modes of inquiry and the end of philosophy of formalism. Thousand Oaks, C A: Sage.

 

McMillan, J. H. & Schumacher, S. (1993). Research in Education: A Conceptual introduction, 3 th Edition. Happer, Collins College Publishers.

 

Mertens, D. M. (2005). Research and Evaluation in Education and Psychology: Integrative diversity with quantitative, qualitative and mixed methods. 2nd Edition London. Sage Publications.

 

Miles, M. B. (1983). Qualitative dataas an attracive nuisance: The problem of analysis. In J. Van Maanen (Ed.), Qualitative methodology. Beverly Hills, CA: Sage.

 

Myrdal, S. (1994). Teacher education on line: What gets lost in electronic communication; Educational measurement: Issues and practice.

 

Nyawaranda, V. A. (2005). Supervising Research Projects/ Dissertations. A Paper delivered at the Zou Workshop in Mashonaland Central Region. Bindura.

 

Osman, R. & Wagner G. A. (1987). New Zealand management students’ perceptions of communicationtechnologies in correspondents education. Distance Education, 8(1), 47-63.

 

Owens, R. (1982). Methodological rigor in naturalistic inquiry: Some issues and answers. Educational Administration Quarterly, 18(2), 1-21.

 

Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park , CA: Sage.

 

Patton, M. Q. (2002). Qualitative Evaluation and Research Methods. Thousand Oaks,  C.A: California, Sage.

 

Perraton, H. (1988). A theory for distance education. In D. sewart, D. Keegan, & B. Holmberg (Ed.), Distance education: Interenationalperspective (pp.34-35) New York: Routledge.

 

Phipps, R. & Merisotis, J. (1999). What’s the Difference? A review of Contemporary Research on the Effectiveness of Distance Education in Higher Education. The institute for Higher Education policy.

 

Reckase, M. D. (1995). Portifolio assessment: A theoretical estimate of score reliability.

 

Russek, B. E. & Weinberg, S. L. (1993). Mixed methods in a study of implementation of technology-based materials in the elementary classroom. Evaluation and Program Planning, 16 (2), 131-142.

 

Russell,, T. L. (1999). The no significant difference phenomenon. Raleigh, NC: North Carolina State University.

 

Schwandt, T. (2000). Three epistemological stances for qualitative enquiry: Interpretivism, Hermeneutics and Social Constructionism. Beverly Hills, C A: Sage.

 

Smith, R. B. & Manning, p. K. (1982). A Handbook of social science methods, volume 2: Qualitative methods. Cambridge, MA: Ballinger.

 

Straus, A. & Corbin, J. (1990). Basics of qualitative research. Newbury Park , CA: Sage.

 

Teddlie, C. & Tashakkori, A. (2003). Major issues and controversies in the use of mixed methods in Social and Behavioural Sciences. Thousand Oaks, C A: Sage.