[Queenie]

To learn more about the Echo360 Blended Learning Grants Program, visit http://echo360.com/c...grants-program/

Title

Analysis of Lecture Capture Engagement as a Retention Tool

Institution

Riverland Community College

Project Team

Jan Waller, Dean of Academic Affairs, Liberal Arts and Sciences, Riverland Community College
Alan Erdahl, Ph.D., Anatomy & Physiology Instructor, Riverland Community College
Bill Dowden, Director, Office of Instruction Technology, Academic Affairs, Riverland Community College
Danielle Heiny, Director of Retention & Student Success, Student Affairs, Riverland Community College
Mary Ann Bush, Counselor/ Online Specialist, Career Exploration, Student Affairs, Riverland Community College

Abstract

Riverland will identify indicators within the Echo360 lecture capture/blended learning environment that can be used to predict and then improve retention and student success. The project will use instructional technology tools to analyze indicators, beyond course content, that may indicate at risk students. It is expected that early indicators from Echo360 usage will provide insight into student learning, and predict academic success, even prior to the first test being given to evaluate content understanding. Because testing often does not begin until the third or fourth week of the semester, it is difficult for the online instructor to evaluate students until then and for an online class, the student may already be lost by the fourth week. While initial analysis will be directed at Anatomy & Physiology classes for which we have 3 years of data, results will also be directed toward other classrooms using the Echo360. Successful completion of this grant-funded project will provide instructors using Echo360 with another tool help retain students and then to students to attain their education goals. Success will be measured by statistical analysis of student satisfaction surveys, instructor satisfaction surveys and graduation/retention rates with classes incorporating the Echo360 platform.

Description

Riverland Community College is a rural community college located in Southeast Minnesota. Enrollment has grown steadily since 2000 with a large percentage of growth coming from online and hybrid/blended courses. Riverland full-year equivalents (FYE) is 2563, with online comprising 521 FYE. Riverland has approximately 100 full-time and 90 adjunct faculty. Riverland’s online program began in 2001, and Biology classes were first offered online in the blended learning environment in 2005. Riverland piloted Echo360 as our lecture capture platform to support our hybrid and completely online Anatomy and Physiology classes during Spring Semester 2009. Echo360 lecture capture was also used to support in-class sections of Anatomy and Physiology. Currently Echo360 is used in the online courses for numerous disciplines. Echo360 is also being used in Riverland’s disability services to provide additional support for students online. Since Riverland’s adoption of the Echo360 Lecture Capture Platform, Riverland has received outstanding survey results as to student engagement and student satisfaction with the lecture captures. Survey results are included below for 19 Anatomy and Physiology classes (combined results for online, hybrid and in class sections with access to Echo360):

End of semester ECHO survey: 2009 & 2010 & Spring 2011 combined - 409 Students (19 classes)
Did you try Echo360 recordings? 394 Yes; 15 No; 96.3 %
Did you use Echo360 recordings? 381 Yes; 25 No; 93.8 %
If you used Echo360 approximately what % of the recordings did you listen to? 69.1 %
On average, approximately how many times did you listen to each echo lecture with Echo360? 1.5
Did you find the Echo’s to be a useful learning resource? 394 Yes; 8 No; 98.0 %
Would you like to see other classes at Riverland using Echo360? 395 Yes; 9 No; 97.8 %
Did you notice you were less likely to attend the regular class because of the Echo360? 62 Yes; 196 No; 24.0 %
Did you notice that you were more relaxed in class knowing you had Echo360 to review? 228 Yes; 34 No; 87.0 %
If you used Echo360, please rank the value of Echos as a learning resource from one (low) to ten (high) 8.6

Beginning Fall Semester 2009, Echo360 recordings have been incorporated by the Chemistry, Art, Business, Speech, Psychology, Math and English Departments and have become a regular fixture as part of Riverland’s Allied Health Nursing Program and highly rated Nursing Simulation Lab. Faculty use the Echo360 for regular, hybrid and online classes at Riverland Community College as grown exponentially just in 2 years (see data below):

Instructor Use:
Beginning 2009 - 1 instructor
Beginning 2010 - 8 instructors
Beginning 2011 - 39 instructors
As of 9/12/11 - 52 instructors

Student Use:
2009 - 22,300 hits (1st year)
2010 - 53,700 hits (2nd year)
2011 (thru 9/12/11)- 62,000 hits (2/3 of third year)
Total - 138,000 hits

Echos Recorded:
During 2008 - 11
During 2009 - 356
During 2010 - 1040
During 2011 - 1080 (from 1/1/11 – 9/12/11)
Total as of 9/12/11 - 2487

Biology faculty noticed an increased challenge of student retention in the online environment. Many experts agree that lack of student engagement leads to lower retention rates (Research from Community College Survey of Student Engagement (CCSSE). While Echo360 recordings certainly promote or facilitate student engagement (as documented in the above survey), the question became, “Using early student engagement as a critical gauge in student retention, what level of early engagement (or lack of engagement) should trigger intervention or assistance by the college to ensure student academic success?” Riverland Student Survey data indicates that overall students are engaging the Echo360 and are very satisfied with the technology, the question became whether student engagement patterns with the Echo360 could be used to indicate the need for some type of active college intervention leading to improved student retention.

An analysis of struggling online students led faculty to conclude that non-academic issues were negatively impacting students, often times more adversely than academic issues. The non-academic issues were not related to course content. Faculty member, Dr. Alan Erdahl reports, “As the instructor of the class I feel that content issues are my responsibility, but many other issues present themselves where the instructor is not the most qualified person to address the issues. Demographics, financial concerns, part-time employment, family issues or lack of family support, computer challenges, documented learning disabilities or attention deficit concerns are all factors that may impede student success. There is a growing need to address these life issues and connect students to appropriate services.” Dr. Erdahl proposes using the Echo360 grant funds to create a “retention specialist” to address these needs. The retention specialist would either be qualified to deal with these issues directly or make appropriate referrals to college and community resources that students could access.

Preliminary research using Online Anatomy & Physiology I as well as Anatomy & Physiology II began in January of 2010 to try and determine if there are specific indicators within the students’ usage of lecture capture and the D2L Course Management System that can be used to facilitate online retention efforts both at the faculty level and at the institution level. Results of this analysis showed indicators that may be used very early in the semester to “red flag” students at risk of academic failure. Possible indicators include: time watching Echos; number of Echos sites visited; total number of classroom topics visited; and completion of first week assignments. This type of analysis requires skills beyond the instructor’s content expertise and are very time-intensive. Riverland’s goal is to identify indicators related to early student engagement and deploy interventions . These indicators will be used to identify at risk students and target additional college resources to increase academic success. According to Dr. Erdahl, “The lack of engagement is often the first indication that non-academic issues may be impeding the student’s academic performance.”
This preliminary research using Anatomy & Physiology classes along with other college initiatives, has led Riverland to create a Director of Retention & Student Success position. It is proposed that this grant will be used to pilot a retention initiative. The pilot will incorporate a retention specialist who will determine viable indicators of successful online course completion. These indicators will trigger intervention very early in the semester to target at risk students based on engagement data and computer interactions and skills. The interventions will target students who struggle even before classroom content issues become apparent later in the semester.

The retention specialist will complete the statistical analysis of various parameters related to student usage of lecture capture as an indicator of future success. Those students whose indicators flag them as “at risk” would then receive assistance from the retention specialist in a manner appropriate to their needs. This may include instructor assistance, consultation with a counselor, Office of Instruction Technology guidance, Student Success referrals, course management software or computer skill training, an Echo360 skills tutorial, financial aid information and planning or any other possible issue that is interfering with the students ability to reach their full potential. Technology has reached its original goal of improving access of material to distant students, now the challenge is to use this technology to improve engagement and retention of those new students who are learning in a different environment than ten years ago. Just making the content accessible is not the complete answer, technology must go beyond access and actually help solve some of the issues it has created so it does not actually become a learning barrier in the future.

This grant money would be used to pay for a position that would allow analysis of student usage data of lecture captures within our course management system. This data would then be cross referenced to student success (content mastery) within the class. Data from the 2009, 2010 and 2011, Anatomy and Physiology courses will be used to investigate and determine statistical correlation between lecture capture use and student success. Those indicators that show significant correlation will then be used in Spring 2012 and Fall 2012 classes as early warning indicators. Intervention by the retention specialist will then occur on those flagged students. Content mastery and classroom graduation rates will then be statistically analyzed to determine the success of this project. It is hoped some of these indicators can also be used for other classes using Lecture Capture as part of their courses or other courses can be analyzed to determine possible indicators. This retention specialist would also try to find common indicators between courses that can be used as indicators throughout the college and thereby eliminate the need for course by course evaluation.

At-risk students may be directed to, and supported in, completing on-demand workshops covering topics leading to increased academic success. Workshops will be developed and delivered using Echo360 platform. Topics include active listening; note-taking; test-taking tips; overcoming test anxiety.

Staff and faculty will create recruitment messages and materials that promote the benefits of an education environment that uses Echo360. These messages will use examples of student success based on actual student data and personal testimonies. Messages will inform students of the many benefits of the Echo360 learning environment, including enhanced learning and student engagement which has been shown to increase academic success.

Objective

The primary objective is to determine early indicators with the lecture capture/blended learning environment that can be used to improve retention and student success. The project will be using technology to look at those parameters beyond content mastery that indicate at risk students. It is expected that early indicators from Echo360 usage will provide insight into student learning even prior to the first test is given to evaluate content understanding. Since testing often does not begin until the 3rd or 4th week of the semester, it can be difficult for the online instructor to evaluate a student until then and for an online class, the student may already be lost by the fourth week. While initial analysis will be directed at Anatomy & Physiology classes for which we have 3 years of data, results will also directed toward other classrooms using the Echo360. It is hoped to provide instructors using Echo360 with another tool help retain students and then to help those students attain their education goals. Success will be measure by statistical analysis of student satisfaction surveys, instructor satisfaction surveys and graduation/retention rates with classes incorporating the Echo360 platform.

Dissemination

Primary instructors using Echo360 generally give one or two talks per year within Riverland to promote Echo360. Project results will be presented to at least one of these “Technology in the Classroom” sessions. Riverland presented “Learning Discoveries: Student Effort Relating to Student Success, Using Technology to Evaluate” that presented our findings on using Echo 360 at the 2011 Center for Technology and Learning ITeach Conference in Minnesota and we would propose to present these findings at a Future Center for Technology and Learning Conference. Riverland did a presentation “The Evolution of a Biology Class” at the first annual STEM conference in Orlando, 2010 on Technology in the Biology classroom including the Echo360 platform and it is proposed to present our findings at an upcoming STEM conference. Recently some of the area high schools that are involved in our Concurrent Enrollment (college in the high school) Program have indicated interest in lecture capture and it is anticipated results would be disseminated to this individuals as well.