Drew Hurley
.
Recent discussion at Santa Fe Community College about "Distance Learning" and the
development of appropriate academic Internet courses prompted me to conduct an informal
experiment in my Introductory Sociology classes during the Fall semester of 1997. I selected
this course for two primary reasons. First, this is a "Gordon Rule" course with a 3000 word
writing requirement, and the basis for meeting this requirement was available through Prentice-Hall's
"Online Study Guide." Second, I am already in the midst of converting the "Astound"
based video lectures I use in this course into HTML format so that all of these visual
enhancements can be available to students at any time over the Internet. Therefore, I believed
that this course provided us with the best opportunity to measure the effectiveness of a highly
content specific course presented on the Internet (process specific Internet courses have
previously been demonstrated to be very successful).
I began this semester, as I have for the previous 15 years. At the end of the first week of classes,
after every student had the opportunity to acquire their textbook, Research Manual and Video
Lecture Guides from the College Bookstore, I spent a class period going over the details of the
"Gordon Rule" requirement for the course. Essentially, there are three required formal papers:
an 1800 word research analysis, a 600 word Quantitative Study and a 600 word Qualitative
Study. After giving everyone a chance to consider these requirements, I began the next class
session by asking for any questions about the written requirements for the course. After fielding
a few questions from each of the three classes, I then proposed the alternative requirement,
which all students could volunteer to select. This assignment would require each student to go
to the "Online Study Guide" and answer all of the essay questions for each of the first 16
chapters that they were required to read during the semester (20 chapters were assigned). They
were given the URL and one week to decide if they wished to volunteer for the Online essays.
[The Internet address is = http://www.prenhall.com/macionis/.]
I believed that answering these essay questions would provide a suitable alternative to the strong
research emphasis that I place on my course requirements and provide an appropriate optional
mechanism for the traditional Extra Credit (Secondary Analysis) Study that I make available to
students after they have completed their "Gordon Rule" course requirements. There were
between 8 and 14 questions for each chapter, with an average of 11 questions for the 16 chapters
required. Answering these essay questions would easily produce the required 3000 words for
each student.
The students remaining with the Traditional assignments have the opportunity to earn a total of
500 points on their written papers: 400 points for the three required papers (at 200 for the major
paper and 100 each for the two smaller papers) and and 100 points on the Extra Credit exercise.
Similarly, those students electing to write the Online essays would earn up to 25 points for each
chapters essay questions for each of the first 16 (out of 20) chapters that were assigned. They
could also obtain extra credit for answering the questions for the final 4 chapters that were
assignment.
The comparable nature of this alternative methods of meeting the "Gordon Rule" requirement
for Introductory Sociology persuaded me to initiate this experiment despite my misgivings about
forgoing the strong research emphasis that I place on these assignments. On the other hand,
since I was the one who wrote the "Online Study Guide," I was aware of the fact that every
chapter had one or two questions which were highly provocative or analytical. Since all other
aspects of the course requirements were identical for both sets of students, this initial
investigation focused primarily upon the performance differences of these two sets of students
(the dependent variable). The control group would be the Traditional students and the
experimental group would be known as the Online students.
The actual conduct of the study went very smoothly, despite the fact that it necessitated a few
more grading sessions on my part (I always return graded papers the class period after I receive
them). One minor glitch occurred early in the study when the Prentice-Hall Website went off-line
for three days (without informing anyone; they shut down to upgrade the webpage
templates, so that it responded better to student input). So I hurriedly emailed a duplicate set of
essay questions to each of the Online students. From that point on, each of these students had
two ways to submit their essays to me: by direct email reply or by submitting them through the
"Online Study Guide."
The preliminary results of this study seem to strong favor the students who selected the Online
essays. In fact, that is correct, however, a close examination of the data reveals that it is so for
reasons quite different from what we initially expected. Please allow me to explain.
Among the students who finished the term, these students clearly performed better (i.e., had
consistently higher grade averages) than those who elected to meet the traditional written
requirements for the semester. It should also be noted that these traditional students performed
at about the same level that students have usually performed in this course during the past 15
years. Table One shows a direct comparison for the Final course grades for these two groups of
students. Please understand that all of the best students did not suddenly volunteer for the
Online essays. There were several clearly outstanding students left in the ranks of those electing
to perform the traditional assignments. Moreover, at least half of those in the Online group were
assuredly not the most talented of students. At the beginning of the study, the ability levels of
both groups appeared to be genuinely equal. None the less, the final grade average among the
Online students was 84.33% and the final average for the Traditional students was 74.32%.
While the results from Table one significantly favor the students in the Online group, there are a
number of other factors which bear upon these performances. Attendance has been consistently
correlated with course grades in this sociology course. Typically speaking, the better one's
attendance, in this course, the better one's grade. In this semester when many students were
afflicted with the flu, Table two specifically compares the attendance ratios for these two
groups. Not surprisingly, the Online essay group had substantially fewer average absences (a
full one third less, 6.25 compared to 9.14) during the semester.
Because of the flu and a variety of other reasons, overall attrition (student dropout) was
particularly high in my Introductory Sociology classes this semester. Of the 73 students who
began the semester only 56 completed it. Table three compares the attrition rates for both
groups. Those in the Online group also seemed to suffer less from attrition (20%) than the
traditional students (25%).
The results of these comparisons suggest that the success of the online students may well be due
more to traditional classroom variables than the initial proposition suggested. Instead of the
Internet providing an alternative to the classroom setting, it may be better thought of as an
addition to it. Students in the Online group dropped out less frequently and had one third fewer
absences than did those in the Traditional group. One clear implication of this finding is the
suggestion that these students used this performance vehicle to supplement their class efforts by
doing more work out of class; thus providing indirect confirmation of the First Rule of
Academic Success: "the more you study and prepare the better you will do."
Therefore it is important that we examine the other major variables involved in this study. First,
we shall consider the fundamental premise of the study. Are the grades of the Online students
better than the Traditional students on their required written work? Table Four compares these
groups on the basis of their written requirements. The results are surprising.
.
Grades | Online Students (20) | Traditional Students (36) |
A | 9 | 8 |
B+ | 0 | 2 |
B | 4 | 11 |
C+ | 2 | 1 |
C | 4 | 8 |
D | 0 | 2 |
F | 1 | 4 |
Total Points | 16866 | 28241 |
Average | 84.33% | 74.32% |
.
Absences | Online Students (20) | Traditional Students (36) |
Number Absences | 125 | 329 |
Average | 6.25 | 9.14 |
.
Attrition | Online Students | Traditional Students |
Began Semester | 25 | 48 |
Withdrew by Oct. 16 | 5 | 8 |
Dropped Out | 0 | 4 |
Completed Semester | 20 | 36 |
.
The data from Table Four is almost exactly the opposite of what we might have expected. The Online students average 72.84% while the Traditional students earned 83.98% on their required written assignments. This is a rather significant difference in the data that favors the Traditional students at the expense of the Online group. This confounding data thus supposes the question: how did these groups compare on their respective exams? Table Five compares both groups of students on their exams (both groups took the same exams).
.
Compariosn Period | Online Students | Traditional Students |
First Assignment | 2700 (155) | 6489 |
Second Assignment | 2293 (126) | 2953 |
Third Assignment | 1271 (63) | 3239 |
Total Points | 6264 | 12681 |
Total Average | 72.84% | 83.98% |
.
Exam Points and Average | Online Students | Traditional Students |
Points .... Average | Points .... Average | |
Exam One (100 pts) | 1465 (20) 73.25 | 2522 (39) 64.67 |
Exam Two (100 pts) | 1670 (20) 83.50 | 2676 (36) 74.33 |
Exam Three (100 pts) | 1456 (20) 72.80 | 2399 (33) 72.70 |
Final Exam (200 pts) | 2544 (19) 67.20 | 4243 (32) 66.80 |
Exams Total | 7145 (98) 72.91 | 11840 (172) 68.84 |
.
The only significant differences are found on the first two exams. These outcomes dramatically favor the Online group whose respective score averages were 73.25 and 83.50 while the Traditional group averaged scores of 64.67 and 74.33 respectively. Why did students in the Online group score so much better on first and second exams than students in the traditional group? That initial advantage may be an unintended consequence of exposure to the "Online Study Guide." I told all of the students that I was the author of the "Online Study Guide" and that the questions found there would be similar to the questions they would see on their exams. I specifically suggested to them that the better they did on these questions, the better they would likely perform on their exams. The superior performance of the Online group on the first two exams may well reflect the early advantage they had of responding to my questions on the study guide. It should be noted that performance on the third exam and final exam were not significantly different between the two groups because the Traditional students finally caught up to them on becoming familiar with my style of writing questions and the fact that both groups were skipping over sections on their reading assignments and cutting corners on their study and preparation for these end of the semester exams (hence their declining scores).
If my supposition that early familiarization with my style of questions in the key to understanding the exam score differences between the two groups, this should be born out in the extra credit work performed by both groups. Both groups had the same opportunities for extra credit. Students in the Traditional group had the opportunity to complete an additional assignment with a maximum value of 100 points while students in the Online group could answer the essay questions for the last four chapters that were assigned (also 100 points). All students were given the opportunity to take advantage of the multiple choice and true/false quizzes on the "Online Study Guide" (at 1 point per quiz, a total of 40 points), and there were five extra credit class activities I used during the semester (10 points each). Table Six presents the Extra Credit earnings of both groups.
While performing slightly better on the computer quizzes and class activities, the students in the Online group showed their real mettle by out-performing the Traditional students on the extra assignment option that was available to them. Make no mistake about this fact, either. Only nine of the Traditional students opted to submit an extra credit assignment (the average grade was 87.11). There is a good deal of truth in the old saying, "you can lead a horse to water but you can't make him drink." The opportunity to earn these extra points was there for all students, but only one quarter of the traditional students took advantage of their opportunity, while three quarters of the Online students did. The difference in these average extra credit earnings is profound. The Online group averaged extra earnings of 91.65 points while the Traditional group averaged only 53.03 extra points.
.
Extra Credit Component | Online Students (20) | Traditional Students (36) |
Computer Quizzes (40 pts) | 272 | 414 |
Class Activities (50 pts) | 499 | 711 |
Extra Assignment (100 pts) | 1062 | 784 |
Total Extra Credit | 1833 | 1909 |
Average Per Student | 91.65 | 53.03 |
.
This brings us back to the original premise upon which this study was based: does this data support the notion that the Online program may prove to be a reasonable substitute for the normal classroom experience? It does not. The data created by this comparison strongly suggests that the Online students used this device as a supplement, not as a substitute, to the class room presentation of this course. The Online essay experience provided these students with an additional opportunity to invest their time, energy and talent into insuring that their class performances were successful.
The next step in the research process would be to exempt the Online Students from class attendance requirements and repeat the experimental comparisons, once again. It would be feasible to preform this subsequent follow-up study this coming Fall (1998) semester. By that time, I will have completed the HTML conversions necessary to allow all of the students who miss a class to view these video lectures on their own. Of course, they will be deprived of the sterling classroom discussions that my classes are known for, but they will have the opportunity to see all of the concepts and data that are the focus of these discussions. This is extremely important because I make it a point to include additional material, not covered in the text, in each of these class Video Lectures. Online students will need the opportunity to learn about these particular features and concepts.
Meanwhile, any suggestions or recommendations from faculty colleagues would be greatly appreciated, particularly on how to facilitate the touchy point of exempting Online students from class while continuing to require the attendance of the Traditional students. Your comments, suggestions and recommendations are welcome, and encouraged.
.
Go To Introductory Sociology Page