INTRODUCTION

The success of an online learning project depends, to a great extent, on the continuous study and improvement of the major components of the online learning paradigm. One of the major thrusts of online learning is focused on providing activities that shift the control of learning activities from the teacher to the student. Students are the most important stakeholders in any learning environment. Thus, we must know our students well in order to engage them effectively online. We should know their skills in the use of computers, competencies, attitude towards online learning etc. This will help us evaluate how good online learners our students are (Great Basing College, 2003., Extension Online, 2003) and also help us identify and address possible causes of attrition (Frankola) in our courses.

Course content must also be subjected to periodic evaluation for effective learning. The best evaluators for an instructional material are its learners. The kind of study reported in this paper is necessary to find out, amongst other things, whether our contents contain, in the right measures, Elsenheimer’s five key points for keeping learners engaged namely, entertainment, interaction, control, usability and customization (Elsenhiemer, 2003).

This paper is structured as follows. Sections 2 and 3 outline the development and delivery processes of our courses, respectively. Section 4 presents results of our case study. Section 5 is a summary of the paper and Section 6 is acknowledgement.

Course development
Our development team consists of six faculty members who jointly acted as subject matter experts, instructional designers, graphic designers and programmers. Thus, the team can be viewed as multi-disciplinary as advocated by the British Open University, as quoted in (Care, and Scanlan, 2001). The development tool was mainly Macromedia’s suite of packages.

Our development process identifies the following seven-point steps:

• Preparation: This phase involves development team formation and project proposal writing.
• Analysis and scheduling: Outlining details of the course coverage or course blueprint and project’s action plan.
• Design of course architecture: Design course’s top-level architecture, presentation template, units format and structure etc.
• Content development: Flesh the skeleton produced in the design phase. Animation sketches and corresponding narration transcripts are written at this phase.
• Integration & animation: The final content developed and vetted is now transferred into the final presentation.
• Recording and synchronization: Record the presentation and synchronize the voice narration with the text and animation motions. Use an acoustic expert or text-to-speech software for recording the presentation.
• Deployment: Compile the course for the Web and create students’ course packs. Deploy course onto the available learning management systems (WebCT, Blackboard etc).

The course content is organized into forty lectures for easy mapping to traditional face-to-face lectures. Each lecture takes, on average, 35 minutes to complete when viewed with the full audio explanations and animations. Further details can be found in our earlier publications (Junaidu and J. Al-Ghamdi , 2002)

Course delivery
The course is delivered almost completely online. The traditional three weekly meetings are combined into a single weekly meeting. Part of the weekly face-to-face meeting is used as a question-and-answer session as well as taking a quiz. The laboratory component is, however, conducted in the traditional manner, with an instructor in charge. This integrated method of online delivery has proved successful in keeping our online students involved, reducing their anxiety and increasing their motivation. This is supported by NYUonline’s research which shows that providing a one-hour live session for every four hours of self-paced study was a highly effective mix (Frankola).

Students are provided with hard copies of the screen dumps of the Author ware presentation for off-line study. A CD is also provided containing the compiled course material as posted on WebCT. For online study of the material in the college laboratories, students are provided with walkman-style headphones so as to reduce disruption in the laboratories caused by audio over speakers.

Students’ survey
Our study covers three computer science courses, ICS 102: Introduction to Computing, ICS 201: Introduction to computer science and ICS 202: Data structures. These courses correspond to the three-course sequence, for covering practical computing fundamentals, advocated by the IEEE-CS/ACM joint task force on Computing Curricula (Chang, C et al. 2001), At the time of this study the first two courses were being offered online for the first time and the third course for the fifth time. Students were required to take the courses online without choice. A total of 54 ICS 102 students, 56 ICS 201 students and 80 ICS 202 students were surveyed.

The survey consists of twenty six questions divided into four categories. The first category asks questions about students’ time management skills, study habits and motivation. The second category is about the contents of the online course and about how well organized and user-friendly it is. The third and fourth categories of the questions, respectively, deal with the laboratory component of the course and students’ assessment measures.

Each question has five options numbered 1 to 5 with some reverse semantic differentials as shown in Table 1. Questions for which there were no responses have a tag of zero. Other responses have the tags indicated and these tags will be used, for brevity, in the subsequent tables.

The surveys were conducted towards the end of the semester two major examinations in the courses. We now report the results of the questionnaire according to the four categories.

Students’ skills, study habits and motivation
There are eight questions in this category as follows:

Q1: I manage my time well in this course
Q2: I do not like online study, so I’m not motivated in this course
Q3: I study the course material only using the printable slides
Q4: I spend more time studying this course than other non-online courses
Q5: I always work with other students in this course
Q6: I normally work alone in all other courses
Q7: Any other comments regarding your time management, study habits and motivation?

Responses to Questions 1 to 6 are shown in Table 2. Sample comments on Question 7 are shown at the end of the section.

Regarding time management, the results in this table show that 37% percent of ICS102 students agree or strongly agree that they manage their time well, compared to 28% who disagree or strongly disagree and 35% who are neutral on this question. Majority of ICS 201 students, 38%, are neutral on the time management questions, compared to 33% who disagree or strongly disagree, while majority of ICS 202 students, 41% disagree or strongly disagree, compared to 36% who are neutral.

With respect to motivation, 52% of ICS 202 students, 45% of ICS 102 students and 42% of ICS 201 students agree or strongly agree that they dislike online learning and hence they are not motivated. 16%, 15% and 24% respectively are neutral on this question.

Based on the responses on the first two questions, ICS 102 students appear to be more open-minded probably because this is their first course in the department and that they have not taken any other course in the department with which to compare. On the other hand, ICS 202 students appear to be more negative probably because they were influenced by their predecessors.

More ICS 202 students, 67%, disagree that they study the course material only from the printable slides compared to 51% in ICS 102 and 48% in ICS 201. Only a small percentage, 9% in ICS 102, 5% in ICS 201 and 14% in ICS 202, of the respondents are neutral on this question. This high percentage (67%) in ICS 202 is because of the fact that there is much more animations in this course than in the others. Consequently, it is impossible to study most parts of the course from the printed static material.

A higher percentage of students in all the three courses, 36% in ICS 102, 65% in ICS 201 and 50% in ICS 202, agree that they spend more time in their study of the online course compared to the time they spent in other courses taught in the traditional way. 32%, 16% and 22%, respectively, of the students are neutral and the remainders disagree or strongly disagree on this question. This result is not unexpected, because the students are no longer passive receivers of information in the learning process but take control of and undertake a self-directed independent learning activity.

Regarding teamwork in the online course, 34% of ICS 102, 43% of ICS 201 and 50% of ICS 202 students agree or strongly agree that they partake in teamwork with others. On teamwork in other non-online courses, however, 75% of ICS 102 students, 67% of ICS 201 students and 61% of ICS 202 students agree or strongly agree that they work alone in other non-online courses.

Results of these two questions (Questions 5 and 6) suggest that the online experience promotes interaction and teamwork among students in all the three courses. This is a potential advantage that online learning offers and should be exploited to good effect.

Q7: Any other comments regarding your time management, study habits and motivation?

Here are sample students’ responses to this question (edited to remove typos and correct grammar):

• I divide my time to all courses according to their importance. I could not give this course the time it deserves
• This course took most of my time (at the expense of other courses)
• I just messed up. I accept that I did not manage my time well

Course organization and usability
There are five questions in this category, which are:

Q8: The course content is well organized
Q9: The course covers too much material
Q10: The animations in the presentations are useful
Q11: The navigation tools in the presentation are adequate
Q12: Any other comments regarding the course contents and organization?

The results in Table 3 show that 52% of ICS 102 students, 35% of ICS 201 students and 51% of ICS 202 students agree that the courses are well organized. On the other hand, about 10% the students surveyed in each course, strongly disagree. Regarding the coverage in the three courses, 49% of ICS 102 students, 51% of ICS 201 students and 60% of ICS 202 students agree or strongly agree that the courses covered too much material. On the other hand, 36%, 15% and 17%, respectively agree that the courses coverage was not too much. While ICS 202 results on this question are not unexpected, because about 25% more material is covered compared to the traditional face-to-face course, results on the other courses may be affected by students’ apprehension of the online paradigm.

On the question of the usefulness of the animations in the course, 54% of ICS 102 students’ respondents strongly agree the animations are useful and 30% of the students abstained. ICS 201 students seem unsure about this fact as 45% of them abstained and 32% of remainder agree or strongly agree that the animations are useful. An overwhelming majority of ICS 202 students, however, agree that the animations are useful. These results are not unexpected because there are more animations in ICS 202 compared to the other two courses.

On the adequacy of the control buttons that help users navigate the course material, 41% of ICS 102, 49% of ICS 201 and 43% of ICS 202 students are neutral. 29%, 38% and 23% of the respondents, respectively, agree or strongly agree that the navigational controls are adequate. We are aware that the controls need improvement to provide lower-levels of control like using a slider to ‘jump around’ within the presentation.

Q12: Any other comments regarding the course contents and organization?

Here are sample students’ responses to this question:

• It started nicely but became unbalanced later: more must be taught on methods before starting other topics
• Some examples not easy to understand (e.g. on GridBagLayout)
• Navigation is poor, provide a glossary and search options

Course laboratory component
There are six questions in this category as follows:

Q13: The lab material is useful
Q14: The lab should be made online also.
Q15: I understand the course material only after taking the lab
Q16: It is good to have a short pre-lab quiz to help us prepare in advance
Q17: It will be better to combine the single weekly meeting with the lab
Q18: Any other comments regarding the course contents and organization?

We note here that the laboratory part of the course is offered in the traditional way, with an instructor lead. Each lab session consists of a set of examples followed by a set of lab tasks. Students earn no points for solving the tasks. The lab grade constitutes 20% of the total course grade. The lab grade is distributed among four lab quizzes and a final lab test.

Table 4 shows that 81% of ICS 102, 69% of ICS 201 and 61% of ICS 202 students agree or strongly agree that the lab material is useful. On the other hand, only 15%, 28% and 19% of the respondents, respectively, disagree or strongly disagree on the usefulness of the lab material. On the question of making the lab component online, all three students’ groups indicate strong disapproval of the idea. The results show 85% of ICS 102, 76% of ICS 201 and 88% of ICS 202 students disagree or strongly disagree with the proposal of making the lab online. This result shows that the university has to do more to address the problem of students’ apprehension to online learning.

Response to Question 15 indicates students’ heavy reliance on the lab material: 63% of ICS 102, 48% of ICS 201 and 51% of ICS 202 students agree or strongly agree that they understand the course only after taking the lab. On the other hand, 17%, 36% and 29% of the students, respectively, disagree or strongly disagree. ICS 201 respondents have the highest percentage (52%) of agreement on having a pre-lab quiz for each session followed by ICS 102 students with 48% agreeing or strongly agreeing. ICS 202 students, however, is almost evenly divided with 45% in agreeing and 46% disagreeing. The author’s personal experience is that pre-lab quizzes are quite useful as they encourage students to come to the lab more prepared and therefore deriving more benefits from the labs.

Thirty two percent of ICS 102 students agree or strongly agree that weekly one-hour session and the three-hour lab session should be combined into a single four-hour weekly meeting. Thirty seven percent of the students disagree or strongly disagree and the remaining students are neutral. 53% of ICS 201 and 29% of ICS 202 students agree or strongly agree on this question. On the other hand 26% and 50% of the students, respectively, disagree or strongly disagree on combining the two meetings into one.

Q18: Any other comments regarding the course contents and organization?

Some responses are:

• No comments, it is very useful and helpful
• It is just a repetition of the lecture material; the exercises can be done at home!
• Lab tasks too long, must provide solution for each lab’s work

Course assessment
There are eight questions in this category, thus:

Q19: The regular quizzes are helpful in studying the course
Q20: I study only to prepare for quizzes and exams
Q21: The home works help me to study and understand the course material
Q22: The number of quizzes and home works is right
Q23: It is better to have more quizzes but fewer home works
Q24: It is better to have more home works but fewer quizzes
Q25: The home works should be replaced with a course project
Q26: Any other comments regarding the course assessment and evaluation?

We note that we give quizzes every other week in the course to ensure that students do not fall behind. There are also homework exercises every other week such that homework is submitted covering the quiz material on or before the quiz date. Table 5 shows an overwhelming agreement among the students on the usefulness of the bi-weekly quizzes with 86% of ICS 102, 91% of ICS 201 and 84% of ICS 202 students agreeing or strongly agreeing. The results of this question with that of Question 20 (which indicates that 40% of students from each of the three courses agreeing that they study only to prepare for a quiz or exam) show that a good number of the students are deadline-oriented and that they need to be pushed to study the course material.

ICS 102 students overwhelmingly agree on the usefulness of the course homework exercises with 85% of the respondents agreeing or strongly agreeing. 69% ICS 202 students agree or strongly agree that the home works are valuable and drive students study of the course. 47% of ICS 201 students agree or strongly agree that the home works help them in studying the course but 30% of the students are neutral. About 70% of the students in the three groups, on average, agree or strongly agree that the number of quizzes and home works are just right.

When asked whether they prefer more quizzes and less home works, the students groups are almost evenly divided. While 56% of ICS 102 students disagree or strongly disagree, 53% of ICS 201 students agree or strongly agree to this question. ICS 202 students almost break even on this question with 40% agreeing and 43% disagreeing. On the question of having more home works and less quizzes, however, all students groups disapprove of the idea. 71% of ICS 102 respondents and 66% of those in ICS 202 disapprove of the idea of replacing the home works with a course project. 48% of ICS 201 students also disapprove while 22% of the respondents are neutral.

Q26: Any other comments regarding the course assessment and evaluation?

Sample responses to this question are:

• Homework exercises make willing CS students good
• Course project should be given on the first day of the semester! Online course should not have more than one instructor. Assign grade for students’ participation in the discussion forum
• The course is great, the instructors are greater … but the online thing really killed us…

SUMMARY AND CONCLUSIONS

This paper presented results of a case study carried out at KFUPM on the development and delivery of three undergraduate courses online. The study used a questionnaire designed to get students’ inputs on various essential elements for successful online learning. These include students’ motivation, time management and study habits, the course content and coverage extent and courses’ evaluation criteria.

Figure 1: Students Responses to Question 2

The results presented in Section 4 show that our students are generally deadline-oriented in that they have to be pushed to study. This means that they are not as self-paced independent learners as online study requires and that they lack time management skills. The fact that our students lack motivation (Figure 1) and are somewhat apprehensive to online learning may not be unconnected to the fact that all students in our pilot offerings are required to take the course online, without option. A general lesson, therefore, is that online study should be optional; only learners who believe that quality education can be achieved through online study may be advised to take online courses. The same should be the case for teachers who may facilitate online courses. Nevertheless, the university should organize awareness programs to address the concerns of student and instructor skeptics on the prospects of online education.

Figure 2: Role of Animations in Online Courses

Another important outcome from our study is that course contents should be rich in multimedia animations. These animations constitute the major difference between online presentations and textbook presentations. Without animations that add value to your course, students will study only from the printed course material. On the other hand, the more creative animations there are in your presentations the more useful they are likely to rate your course. This is exactly what Figure 2 depicts for our three courses.

Figure 3: Should Labs be Online?

The results shown in Figure 3 buttress our earlier point that our students are not as instructor-independent as should online learners. These results show an overwhelming disapproval of making the lab components of the course online. The students’ also show high approval of the usefulness of the instructor-led lab component and a strong approval also of the frequent quizzes and homework assignments given on alternate weeks.

ACKNOWLEDGEMENT

I commend the university (KFUPM) for initiating the online project and the college of computer sciences and engineering for pioneering this effort in the university. Without this initiative, this work would not have been possible and, without the college Dean’s concerted effort and motivation this study would not have come to fruition. I acknowledge the support of my colleagues with whom the online courses were developed and delivered. I also acknowledge the understanding of the project team leader, Dr. Jarallah Al-Ghamdi, for creating an environment conducive for debating ideas for the benefit of all.

REFERENCES

Care, W. D., & Scanlan, J. M., April 2001. Planning and managing the development of courses for distance delivery: Results from a qualitative study. The Online Journal of Distance Learning Administration, 4(2). Retrieved from http://www.westga.edu/~distance/ojdla/summer42/care42.html

Chang, C et al. (2001), Computing Curricula 2001 (Computer Science), Final Report, Dec., 15, 2001. from: http://www.computer.org/education/cc2001/final/index.htm

Great Basing College, 2003. Self Evaluation for Potential Online Students. from: http://faculty.gbcnv.edu/onlinestudentsurvey.htm
J. Al-Ghamdi and S. Junaidu, Emulating F2F for Distance Learning Students, International Journal of Applied Computing & Informatics (IJACs), Submitted.

J. Elsenhiemer, 2003. Learning Cicuits. Terms of Engagement: Keeping Learners Online. from: http://www.learningcircuits.org/2003/feb2003/elearn.html

Frankola, K. Tips for Increasing E-learning Completion Rates. Workforce Magazine http://www.workforce.com/archive/feature/22/26/22/index_printer.php

S. Junaidu and J. Al-Ghamdi , Developing an Online Data Structures Course using Authorware, Journal of the United States Distance Learning Association (USDLA), Volume 16, No. 10, October 2002

S. Junaidu and J. Al-Ghamdi, Tips for Developing Media-rich Online Courses, Journal of the United States Distance Learning Association (USDLA), Volume 16, No. 12, December 2002.

University College, Berkeley, Extension Online, 2003. What Makes a Good Online Student? from: http://learn.berkeley.edu/sampleclass/u01/u1note03.html

University College, Berkeley, Extension Online, 2003. What Makes a Successful Online Student?from:http://www.ion.illinois.edu/IONresources/onlineLearning/StudentProfile.asp

Biodata o Author:

Sahalu JUNAIDU
Dr. Junaidu is an Assistant Professor of Computer Science. He received his Ph.D. from St. Andrews University, Scotland in 1998. His areas of interest include parallel computing, programming languages and e-Learning. He has been actively involved in the development and delivery of online course material for the past four years. Contact Sahalu at
sahalu@ccse.kfupm.edu.sa
Information & Computer Science Department
King Fahd University of Petroleum & Minerals
Dhahran-SAUDI ARABIA