Engagement and Performance for Female and Male Students in an Online Course
Methods
For this descriptive and historical research, content analysis techniques were used by the researchers to analyze students' engagement and achievement in a graduate course delivered asynchronously to both female and male students. "Content analysis is a technique that enables researchers to study human behavior in an indirect way, through an analysis of their communications" (Fraenkel & Wallen, 1999, p. 405). The content analysis for this study consisted of both qualitative and descriptive techniques as described by Fraenkel and Wallen.
As with any study, it is important for the researcher to establish internal validity, external validity, reliability, and objectivity. However, in the qualitative paradigm these terms are referred to as credibility, transferability, dependability, and confirmability. Credibility and dependability were established using the technique of triangulation. "Perhaps the best way to elicit the various and divergent constructions of reality that exist within the context of a study is to collect information about different events and relationships from different points of view. Different questions, different sources, and different methods should be used to focus on equivalent sets of data" (Erlandson, Harris, Skipper, & Allen, 1993, p. 31). For this study, both content analysis and semi-structured interviews provided triangulation. Generalizability attempts to apply the findings of a study to other contexts. The naturalistic researcher "does not maintain that knowledge gained from one context will have no relevance for other contexts or for the same context in another time frame. 'Transferability' across contexts may occur because of shared characteristics" (Erlandson, et al., 1993, p. 32). In this study, the researchers collected engagement patterns and interview transcripts which provided sufficient detail to allow the reader to decide transferability. The fourth requirement, confirmability, was established by conducting an audit trail.
The naturalistic setting for this study was a graduate course entitled Principles of Adult Education during the Spring 2002 semester. This course was a departure from our usual design for graduate-level distance education courses. Unlike our other distance-delivered graduate course offerings that have included and even emphasized the use of synchronous delivery strategies (face to face meetings, audio and videoconferencing), this course employed only the asynchronous technologies and delivery strategies available through WebCTTM. WebCTTM is a commercial software set of Web course-development tools for creating instructional environments at a distance (WebCTTM, 2001). No synchronous interaction was planned or conducted. The course had a variety of structured learning activities that were designed to enhance student/technology, student/student, student/content, and student/instructor interactions. Although there no synchronous interactions planned or conducted these interactions provided multiple opportunities for rich and continuous communications between and among students and the instructor. The structure of the course also allowed the instructor opportunities to modify instruction (individualizing instructional sequences) along the way, based on students' demonstrated competence and needs.
There were 24 students enrolled in the course (16 male students and 8 female students). Students had 114 days to complete and submit all materials. January 14, 2002, was the first day students could submit assignments and May 7, 2002, was the last day. No time constraints were placed on students as to when assignments should be submitted during this time frame. The instructor worked under the assumption that as graduate students they would take responsibility for their learning.
Students were provided the following written instructions:
Welcome to Agricultural Education 610 "Principles of Adult Education." This course is designed to be asynchronously delivered...meaning you can work on meeting the course objectives at any time or location. You can also work on most assignments out of sequence. For example, you may wish to work on Module 1 and 4 before working on Module 2 and 3.
There are 14 course modules that you will work through over the semester. You will complete 12 assignments along the way (ALL ASSIGNMENTS MUST BE SUBMITTED THROUGH WEBCT'S ASSIGNMENT FEATURE): Four reaction papers; four argument papers; twenty online discussion postings; one student lead instruction; one learning contract; and one application project.
ALL ASSIGNMENTS ARE DUE MAY 7, 2002.
The data collection instrument was based on the research questions. Four categories were used initially to classify the data: initial engagement, continuous engagement, completion of course, and performance in course. Initial engagement was defined operationally as the first day students submitted an assignment. Continuous engagement was defined operationally as the number of days between the submission of the first and last assignment, or the length of time from initial engagement to course completion. Completion of course was defined operationally as the last day students submitted an assignment. Any students who started early and finished late were still classified as continuous engagers. This was verified by our triangulation using follow-up interviews. We discovered that students were often thinking about the course even when otherwise occupied. Performance in the course was defined operationally as the percentage of points earned on each assignment and overall. Students also were grouped by one of three patterns of engagement: starts early and finishes early; starts early and finishes late; starts late and finishes late. A fourth possible pattern of engagement, starts late and finishes early, was not detected.
Additionally, nine students were interviewed by telephone or face-to-face to help the researchers gain a more thorough description of why the students engaged and performed as they did. Interviewees were selected to include at least one of each gender by level of engagement. Students were coded by gender (Male or Female), location (Distance or Campus), and a number based upon when they were interviewed (1 through 9) to ensure confidentiality.
The researchers recognize the design limitations of using intact classes. Caution is warranted against transferring these findings beyond this class. Additional research is needed to support and prove the transferability of findings and recommendations to other naturalistic settings. This study is a part of a larger study examining how students engage and perform in asynchronously delivered courses.