[Nicole Jouvelakas]

Title

An investigation of study behaviours in learning environments supported by Echo360 lecture recording tools

Institutions

Murdoch University
Perth, Western Australia

Macquarie University
Sydney, New South Wales, Australia

The University of Newcastle
New South Wales, Australia

Chief investigator

Associate Professor Rob Phillips, Educational Development Unit, Murdoch University

Abstract

This project will examine how students study in courses/units of study¹ that make thoughtful use of technologies (including Echo360 lecture recording technologies) to provide flexibility in teaching and learning and cater for an increasingly diverse student cohort. A case study approach will be adopted which investigates the curriculum and the learning process as a whole, rather than investigating Echo360 tools in isolation. Each case study will explore its particular context and develop a detailed understanding of how students study and learn in technology-enhanced environments.

The research builds on a previous, larger study investigating the use and impact of Lectopia and similar technologies across four Australian universities from the perspectives of both students and staff. While students had strong positive views about Lectopia, staff were concerned about student attendance and engagement, and a blurring of the traditional boundaries between internal (on-campus) and external (distance education) students. The previous project provided a statistical overview of student behaviour, but it was not designed to describe how students used Echo360 or other learning technologies to study successfully.

This is the focus of the current proposal, with the following expected outcomes:

• Produce an analytic tool to observe student behaviour as they use Echo360
• Provide evidence of how students study in technology-rich environments; and
• Enable better understanding of what is an effective environment for blended learning, and its implications for Echo360.

Case studies will be units of study across the participating universities which integrate educational technologies in innovative ways and are rated highly by students and peers. A mixed methods approach will be enhanced by data collected from Echo360 and Blackboard usage logs. Targetted students will be interviewed about their study behaviours and preferences. In addition, assessment results will be analysed to explore student engagement and behaviour in order to identify patterns of study which lead to effective learning.

Of specific interest to the study will be the nature and patterns of interaction of students with identifiable curriculum resources, elearning structures and peers and staff and their relationship to academic achievement. This research will enable a richer understanding to be developed of how particular students behave in particular units across the participating universities. It is intended that this will lead to general design principles for educators interested in applying Echo360 technology to maximise student interaction and learning outcomes.

¹The term ‘unit of study’ or ‘unit’ will be used to describe a semester long programme of study on a single subject. In other countries, course, module or paper is used equivalently.

Detailed Overview

Objectives and Outcomes

Case studies in this project will examine how students study in courses/units of study that make thoughtful use of technologies (including Echo360 lecture recording technologies) to provide flexibility in teaching and learning and cater for an increasingly diverse student cohort. The three partner institutions all currently use Lectopia.

The researchers aim to develop a detailed understanding of how particular students study and learn in technology-enhanced environments. At the same time, the project will explore the use of Echo360 (and BlackBoard) usage data as an analytic tool to observe student behaviour, with the aim of developing and trialing a new mixed-methods approach for e-learning research.

The expected outcomes will be to:

• Produce an analytic tool to observe student behaviour as they use Echo360
• Provide evidence of how students study in technology-rich environments; and
• Enable better understanding of what is an effective environment for blended learning and its implications for Echo360.

Significance

This research addresses the following criteria of the Echo 360 research grants program:

• Demonstrate a significant, original contribution to the field of lecture capture research;
• Engage with, and build on, previous research conducted into lecture capture and learning technologies;
• Contribute to research on understanding lecture capture within the broader learning technology field and provide design principles about how best to use ECHO360 technologies; and
• Promote collaboration within Murdoch University’s School of Education and Educational Development Unit and with Macquarie University and the University of Newcastle.

This study seeks to provide insight into the pedagogic benefits of lecture capture best practices through several small scale case studies into blended learning environments.

Background

The last decade has seen a fundamental shift in the nature of teaching and learning in higher education, brought about, in part by increasing student diversity and lifestyle pressures (Anderson, 2006; Australian Vice-Chancellors' Committee, 2007; Phillips et al., 2007), and the widespread uptake of educational technologies. The primacy of the traditional lecture/tutorial approach to university teaching is being challenged (Jones, 2007; Phillips, 2005) and the boundaries between face-to-face and distance education are being blurred (Woo et al., 2008).

Students are demanding flexibility in time and space in order to be able to study effectively. They want to access their learning activities in ways which fit in with their work and family commitments (Maor & Volet, 2007; McInnis & Hartley, 2002 Ch 6). Meeting these student needs has placed some pressure on universities as evidenced in a recent study by Anderson (2006) which reported that 78% of students found that work impacted on their study, and 40% felt that their university did not cater well for students in paid employment.

The increasing diversity of the student cohort adds another element of complexity to the situation. Only 22.6% were aged under 20 in 2003, while 27.3% were aged over 30, placing approximately 50% of students between 20 and 30 (Universities Australia, 2008). A much broader range of language backgrounds (AEI, 2008) and cultures also needs to be accommodated.

Educational technology use is widespread at universities. It provides a level of flexibility sought by students, as a supplement to traditional face-to-face teaching (Harris, Yanosky, & Zastrocky, 2003), in what is called variously mixed-mode, blended or flexible learning environments (Lefoe & Albury, 2004). Tools in widespread use are learning management systems (predominantly BlackBoard) and lecture recording systems (predominantly Echo360, including Lectopia).

The introduction of Echo360 and similar technologies has been one response to the need for flexibility of access. Not surprisingly, Echo360 is gaining in popularity, particularly with students finding that their needs for flexibility have not been met by ‘traditional on-campus teaching paradigms’ (Lefoe & Albury, 2004). With increased demands posed by work and family commitments (Anderson, 2006; McInnis & Hartley, 2002), recent studies have confirmed students’ appreciation of the convenience and flexibility offered by anytime, anywhere access to lectures (Fardon & Neville, 2003; McNeill et al., 2007).

Since Carroll’s (1963) model of factors contributing towards academic achievement, educators have recognised the importance of time on task as an indicator of success in learning. Recent evidence shows that many higher education students fail to use their learning time wisely (e.g., Kuh (2001) and competing demands on students’ online time (such as social networking sites) continue to undermine academic endeavour in e-learning.

One source of data that has been underutilised in research of this type is the usage logs from e-learning applications - automatically recorded student interactions with the online system. This type of data mining has recently been called “academic analytics” (Dawson, McWilliam, & Tan, 2008). The project leader has done previous work in this area (Phillips, 2006; Phillips & Baudains, 2002; Phillips, Baudains, & van Keulen, 2002) and is currently involved in a project mining usage logs to analyse the social networks that students form in online discussion forums (Dawson, 2006a, 2006b).

Previous Work

This project builds directly on a previous, large project funded by the Australian Learning and Teaching Council: The Impact of Web-based Lecture Technologies on Current and Future Practice in Learning and Teaching. This project² investigated the use and impact of iLecture/Lectopia and similar technologies across four Australian universities from the perspectives of both students and staff (Gosper et al., 2008). The research program comprised of two stages; in the first stage, surveys of students and staff were used to capture the diversity of experiences in the use of Lectopia in order to identify and categorise the issues and usage patterns. The second stage involved a more detailed exploration of the educational issues arising from the surveys through a series of vignettes and case studies.

²http://www.cpd.mq.ed...lt/overview.htm

The project found that both staff and students appreciated the flexibility these technologies provided for students as a whole, and specifically distance students, those with disabilities and from non-English speaking backgrounds. This is consistent with other findings (Albon, 2004; Buxton, Jackson, deZwart, Webster, & Lindsay, 2006; Knight, 2006; Signor, 2003; Williams & Fardon, 2005, 2007a). The majority of students think Lectopia makes it easier for them to learn and helps them achieve better results. They found it valuable because of the convenience (they don’t need to attend), the flexibility (it fits with their busy lifestyle), and because it acted as a safety net when they couldn’t attend, allowing them to catch up on missed lectures. As well as providing flexibility of access, students also used the technology as a tool for learning – to pick up on things missed in the lecture, revise for exams and revisit complex ideas. In line with other work (Massingham & Herrington, 2006; Williams & Fardon, 2007b), students reported strong positive perceptions of the benefits Lectopia provided for supporting their learning and achieving better results.

Of the students using Lectopia, over 75% indicated that this was because they could not attend classes, for reasons ranging from time-table clashes, work commitments, family commitments, and caring responsibilities (Gosper et al., 2008). Students made informed choices about whether to attend based on convenience/flexibility, social factors and the educational value they thought they would gain from face-to-face attendance. However, the choice to use Lectopia instead of lecture attendance is not an either-or decision. Some students use Lectopia to replace lecture attendance, some don’t use it at all, preferring to go to lectures, while the more common use is to complement the lecture

Staff views, while still positive, were significantly less positive than students'. Many staff were concerned about falling attendance at lectures and a perceived lack of engagement by students, findings also made by (Massingham & Herrington, 2006). A particular concern was that students would delay listening to lectures until they needed to prepare for examinations, thereby missing an opportunity for ongoing learning and correction of misconceptions (Goldberg, Haase, Shoukas, & Schramm, 2006). On the other hand, some staff had much more positive perceptions of Echo360, particularly those who had taken a whole of curriculum perspective and effectively integrated Echo360 and other technologies into their units of study (Gosper et al., 2008).

A major finding of our previous work is that there is a blurring of the traditional boundaries between internal (on-campus) and external (distance education) students. Internal students are behaving increasingly as distance students, particularly in terms of attendance patterns, while distance students increasingly use technology to develop a social learning environment more common to on-campus students.

While our previous project, consistent with the literature, identified a mismatch between staff expectations and student behaviour with Echo360 and similar technologies, the research was not designed to explain these behaviours. The project provided a statistical overview of student behaviour but it was not able to describe how students used Echo360 or other learning technologies to study successfully.

The methodology proposed in this proposal (combining usage log data with other methods) will be trialed in a small number of case studies with the intention of developing a rich description of how particular students study and learn in technology-enhanced environments.

Project Plan

A mixed methods research approach will be used, with data collected from technology usage logs and reports, targeted interviews and assessment results. The data will be analysed to explore student engagement and behaviour in order to identify patterns of study which may lead to effective learning.

Stage 1: Usage log tool - October - December 2009

Existing Lectopia reporting tools will be explored and extended to report how often Lectopia is used, and its pattern of use by students, for example, those who use Echo360 regularly, those who use it rarely, and those who use it in bursts. The usage log tool will enable a quantitative measure of use of Lectopia which extends beyond its use in this project. We would request technical assistance from Echo360 about how to extract data for analysis.

The tool will be a PHP script with a simple web-based interface which interrogates the existing Lectopia database and displays and exports the data in a form suitable for analysis. The analysis will initially be through Excel functionality such as pivot tables and graphs.

Open source software will be used to develop and deliver the tool.

Stage 2 Case studies - February - June 2010

In stage 2, the usage log tool and other methods will be combined to study student behaviour in at least one unit of study at each institution.

Students will be selected for interview based on their position along two dimensions:

• attendance and non-attendance at lectures
• use or non-use of Echo360

A further selection criterion will be developed, based on their pattern of use of Echo360 (e.g. regularly, only in study periods).

The targeted students will be invited to an interview at the end of the semester. Those that consent will be presented with their usage data from Echo360, and with the reports currently available as part of BlackBoard, and semi-structured interviews will probe their study behaviour, their interest in the subject and their expected performance in this unit.

After the interview, student perceptions of their performance will be correlated against assessment results in the subject, and their degree-level Grade Point Average, to enable judgments to be made about particular study behaviours.

This data will enable a richer understanding to be developed of how particular students behaved in particular units across the participating universities. It is intended that this will lead to general design principles for educators interested in applying Echo360 technology to maximise student interaction and learning outcomes. At the same time, it will inform a larger-scale study seeking results which are more readily applicable across the sector.

Dissemination

Our earlier Echo360 work (Gosper et al., 2008) and work within our universities, recognizes that many staff are concerned about the impact of Echo360 and similar technologies on their teaching practice. The research will explore student behaviours in order to inform decision-making by teaching staff and administrators about how best to take advantage of Echo360.

As such, dissemination is a core part of this project, to staff and administrators. We will establish a dissemination and communications plan to promote the project within our universities and the sector more broadly. The project is closely linked to strategic initiatives at all of the partner universities, and project team members are well placed to use the project outcomes to influence policy and practice decisions at these universities.

Dissemination will occur through various means, to various stakeholders:

• Internal newsletters and a website
• Publication at conferences and in journals (e.g. 2010 Echo360 Community conference, 2010 ascilite conference. ED-MEDIA conference)
• Presentations to peak professional bodies, such as the Australasian Council on Open, Distance and E-learning (ACODE)
• Presentations to policy-making bodies in our institutions
• Press releases
• A video interview of findings for use by Echo360

References

AEI. (2008). Australian Education International, International student numbers [Electronic Version]. Research Snapshot Number 37.

Retrieved 5 November 2008 from http://aei.gov.au/AE.../37SS08_pdf.pdf

Albon, R. J. (2004). iLectures: Their current role in learning. In HERDSA 2004 conference proceedings. Miri, Malaysia. [Online]

Available at http://www.herdsa.or...rs/P070-jt.pdf.

Anderson, M. J. (2006). Degree of fit: University students in paid employment, service delivery and technology. Australasian Journal of

Educational Technology, 22(1), 88-103. [Online] Available at http://www.ascilite..../anderson.html.

Australian Vice-Chancellors' Committee. (2007). Australian University Student Finances 2006. [Online] Available at

http://www.cshe.unim...nances2006.pdf.

Buxton, K., Jackson, K., deZwart, M., Webster, L. a., & Lindsay, D. (2006). Recorded lectures: Looking to the future. Proceedings of

the 23rd annual ascilite conference: Who's learning? Whose technology? [Online] Available at http://www.ascilite....apers/p118.pdf.

Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64(723-733).

Dawson, S. (2006a). Online forum discussion interactions as an indicator of student community. Australasian Journal of Educational

Technology, 22(4), 495-510. [Online] Available at http://www.ascilite....22/dawson.html.

Dawson, S. (2006b). Relationship between student communication interaction and sense of community in higher education. Internet and

Higher Education, 9(3), 153-162.

Dawson, S., McWilliam, E., & Tan, J. P. L. (2008). Teaching smarter: How mining ICT data can inform and improve learning and teaching

practice. In Hello! Where are you in the landscape of educational technology? Proceedings ascilite. Melbourne. [Online] Available at http://www.ascilite....ocs/dawson.pdf.

Fardon, M., & Neville, M. (2003). Case Study: Stages in establishing a large scale streaming media implementation - an institutional

perspective. In S. D. Snyder (Ed.), Proceedings of the EDUCAUSE in Australasia 2003 Conference. Adelaide: University of Adelaide. [Online] Available at http://ilecture.uwa....ucause_N48.pdf.

Goldberg, H. R., Haase, E., Shoukas, A., & Schramm, L. (2006). Redefining classroom instruction. Advance Physiological Education, 30,

124-127.

Gosper, M., Green, D., McNeill, M., Phillips, R. A., Preston, G., & Woo, K. (2008). Final Report: The Impact of Web-Based Lecture

Technologies on Current and Future Practices in Learning and Teaching. Sydney: Australian Learning and Teaching Council. [Online] Available at http://www.altc.edu....port_aug08.pdf.

Harris, M., Yanosky, R., & Zastrocky, M. (2003). Supplemental beats remote in higher education e-learning. 2003

Jones, S. E. (2007). Reflections on the lecture: outmoded medium or instrument of inspiration? Journal of Further and Higher Education, 31(4), 397 - 406.

Knight, R. (2006). Podcast pedagogy divides opinion at US universities. [Electronic Version]. Financial Times online. Retrieved 27 July

2009 from http://www.ft.com/cm...0779e2340.html.

Kuh, G. D. (2001). Assessing what really matters to student learning: Inside the National Survey of Student Engagement (NSSE). Change,

33(3), 10-17.

Lefoe, G., & Albury, R. (2004). Introduction to special issue on distributed learning. Educational Media International, 41(3), 181-182.

Maor, D., & Volet, S. (2007). Interactivity in professional online learning: A review of research based studies. Australasian Journal

of Educational Technology, 23(2), 269-290.

Massingham, P., & Herrington, T. (2006). Does Attendance Matter? An Examination of Student Attitudes, Participation, Performane and

Attendance? In Journal of University Teaching and Learning Practice (Vol. 3). [Online] Available at http://jutlp.uow.edu...ingham_008.pdf.

McInnis, C., & Hartley, R. (2002). Managing Study and Work: The impact of full-time study and paid work on the undergraduate experience

in Australian universities: Department of Education, Science and Training, Commonwealth of Australia. [Online] Available at http://www.dest.gov...._6/eip02_6.pdf.

McNeill, M., Woo, K., Gosper, M., Phillips, R. A., Preston, G., & Green, D. (2007). Using web-based lecture technologies – advice from

students. In Enhancing Higher Education, Theory and Scholarship, Proceedings of the 30th HERDSA Annual Conference [CD-ROM] (pp. np). Adelaide: Higher Education Research and Development Society of Australasia.

Phillips, R. A. (1997). The Developer's Handbook to Interactive Multimedia - A Practical Guide for Educational Applications. London:

Kogan Page.

Phillips, R. A. (2001). A Case Study of the Development and Project Management of a Web/CD Hybrid Application. Journal of Interactive

Learning Research, 12(2/3), 225-243.

Phillips, R. A. (2005). Challenging the Primacy of Lectures: the Dissonance between Theory and Practice in University Teaching. Journal

of University Teaching and Learning Practice, 2(1), 1. [Online] Available at http://jutlp.uow.edu...illips003.html.

Phillips, R. A. (2006). Tools used in Learning Management Systems: analysis of WebCT usage logs. In L. Markauskaite, P. Goodyear & P.

Reimann (Eds.), The 23rd Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education: Who's Learning? Whose Technology? (pp. 663-673). Sydney: Sydney University Press. [Online] Available at http://www.ascilite....apers/p208.pdf.

Phillips, R. A., & Baudains, C. (2002). Learning Botany: Evaluation of a web-supported unit on plant diversity. Perth, Western

Australia: Murdoch University. [Online] Available at http://www.tlc.murdo...t/N265Eval.pdf.

Phillips, R. A., Baudains, C., & van Keulen, M. (2002). An Evaluation of Student Learning in a Web-Supported Unit on Plant Diversity.

In A. Williamson, C. Gunn, A. Young & T. Clear (Eds.), 19th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (pp. 525-534). Auckland, New Zealand. [Online] Available at http://www.ascilite....papers/161.pdf.

Phillips, R. A., Gosper, M., McNeill, M., Woo, K., Preston, G., & Green, D. (2007). Staff and Student perspectives on Web-based Lecture

Technologies: insights into the great divide. In ICT: Providing choices for learners and learning. Proceedings of the 2007 ascilite conference (pp. 854-864). Singapore. [Online] Available at http://www.ascilite....s/phillips.pdf.

Signor, L. (2003). An exploration into the reactions of undergraduate students to virtual lectures. Paper presented at the OLT 2003,

Queensland, Australia. [Online] Available at https://olt.qut.edu....ngs/Signor.doc.

Universities Australia. (2008, 6 October 2008). Student characteristics. Universities Australia Publications Statistics, from

www.universitiesaustralia.edu.au/documents/publications/stats/Students.pdf

Williams, J., & Fardon, M. (2005). On-demand internet-transmitted lecture recordings: attempting to enhance and support the student

experience. Proceedings of Alt-C 2005 (Manchester, September 2005) Association for Learning Technology, 153-161.

Williams, J., & Fardon, M. (2007a). Lecture recordings: extending access for students with disabilities. Paper presented at the ALT-C

conference, Nottingham, UK, Sept 4-6.

Williams, J., & Fardon, M. (2007b). Recording lectures and the impact on student attendance. Paper presented at the ALT-C.

Woo, K., Gosper, M., McNeill, M., Preston, G., Green, D., & Phillips, R. A. (2008). Web-based Lecture Technologies: Blurring the boundaries between face-to-face and distance learning. ALT-J, 16(2), 81-93. [Online] Available at http://www.informawo...text=713240930.