[Robert Minch]

Has anyone done detailed logging and analysis of lecture capture usage? For a project I'm involved with, it would be very helpful to have data on usage behaviors, including who views what, when, for how long, in what mode, etc. Presently our options would seem to include very basic data collected through Blackboard pages that users pass through, standard web server logs, basic iTunesU usage logs, and perhaps a custom web portal we could design to collect data from users before and after viewing materials. A search of this site finds the successful grant application "An investigation of study behaviours in learning environments supported by Echo360 lecture recording tools" that seems to indicate there might be an Echo360 system database that might be queried for more information as well. I'd be interested in knowing about any and all options that might exist. Thanks--Robert.

PS I recognize the privacy implications and am prepared to deal with them appropriately through IRB procedures and/or data anonymizing as needed.

[Brett Dalton]

 Robert Minch, on 19 September 2009 - 02:02 AM, said:

Has anyone done detailed logging and analysis of lecture capture usage? For a project I'm involved with, it would be very helpful to have data on usage behaviors, including who views what, when, for how long, in what mode, etc. Presently our options would seem to include very basic data collected through Blackboard pages that users pass through, standard web server logs, basic iTunesU usage logs, and perhaps a custom web portal we could design to collect data from users before and after viewing materials. A search of this site finds the successful grant application "An investigation of study behaviours in learning environments supported by Echo360 lecture recording tools" that seems to indicate there might be an Echo360 system database that might be queried for more information as well. I'd be interested in knowing about any and all options that might exist. Thanks--Robert.

PS I recognize the privacy implications and am prepared to deal with them appropriately through IRB procedures and/or data anonymizing as needed.

Robert

Assuming you are a seperate Wowza server you should be able to capture the console output of the process to a text file and analyse this.I don't know if Wowza already has some of this information in logs elsewhere but it may be worth a poke.

It wont tell you about the views etc that users are using in ESS as i believe that this is all client side with no interaction with the server (hence the ability to download the entire pres as a single zip file).

However Wowza will show seeks, pause, play etc of the streams, it might involve some correlation of session id's etc but it may give you more info. If it does not includes time stamp info it should be trivial to code a pipe program that takes standard in and prepends a time stamp to the console output. This would allow you to track exact usage patterns etc. The attached picture is a sample of the wowza console output to give you an idea of what it supplies. Again you may be able to change the reporting level of the Wowza server to give more info but you would need to check the docs to see if this is possible.

 Wowza console output.jpg (157.47K)
Number of downloads: 32

Brett Dalton
La Trobe University

[Brett Dalton]

Robert

Thanks to our highly knowedgable Unix admins i got the following info.

Assuming your Wowza server is on Unix it can be done by piping the output of the console to a file, to add the date stamp run a second command

"for i in `seq 1 255` ; do echo `date` >> outputfilename.txt ; sleep 1s; done"

Where outputfile name is the file that the console text is piped to. The sleep command is how often to add a time stamp, in this case every second.

This will produce a file with the Wowza console output with a time stamp inserted every second for 255 seconds, you can obviously increase the duration as needed. I'm sure your unix admins would be able to help.

If your on windows somthing simmilar may also be possible.


BRETT

[paulb]

It may not be exactly what you are looking for but ....

You can monitor Wowza with Nagios (currently looking at doing that here at Griffith) using this fledgling plugin setup.

http://www.linuxiso....wzamediaserver/ (google translate is your friend)

This should enable a live snapshot of more than just the number of current connections. Basically anything you can see in jconsole should be available over jmx.

Sawmill will import raw wowza logs, but I haven't tried it myself so can't comment on its worth.

Hope this helps

Paulb

[Brett Dalton]

 paulb, on 23 September 2009 - 11:23 PM, said:

You can monitor Wowza with Nagios (currently looking at doing that here at Griffith) using this fledgling plugin setup.

Paulb

We do server monitoring using Cacti which gives simmilar info minus the number of connection. Network I/O, memory usage, disk space etc.

Did a bit of reading on JMX interface, there's a link in the Wowza forums with some code examples if your interested.

http://www.wowzamedi...hread.php?t=353

Given ESS and the ESS processors are Java do they have a JMX interface? It could be really useful in managing installs where it can provide more specific info than the more generic metrics I currently get from Cacti (SNMP).

BRETT

[paulb]

Thanks for the link on the Wowza forums, I hadn't spotted that.

As far as I can tell is should be possible to open up any mbeans via jmx. But you might be better off monitoring the trancoder itself (I'm using Lectopia so thats ffmpeg) for memory usage and load average. Maybe via a script...

Using Lectopia you can probably get all this stuff out via Lasso, but I have no experience with Lasso.

Still onward ...

Cheers

Paulb

 Brett Dalton, on 24 September 2009 - 07:48 PM, said:

We do server monitoring using Cacti which gives simmilar info minus the number of connection. Network I/O, memory usage, disk space etc.

Did a bit of reading on JMX interface, there's a link in the Wowza forums with some code examples if your interested.

http://www.wowzamedi...hread.php?t=353

Given ESS and the ESS processors are Java do they have a JMX interface? It could be really useful in managing installs where it can provide more specific info than the more generic metrics I currently get from Cacti (SNMP).

BRETT

[Brett Dalton]

 paulb, on 29 September 2009 - 10:23 PM, said:

Thanks for the link on the Wowza forums, I hadn't spotted that.

As far as I can tell is should be possible to open up any mbeans via jmx. But you might be better off monitoring the trancoder itself (I'm using Lectopia so thats ffmpeg) for memory usage and load average. Maybe via a script...

Cacti give load averages, memory usage and CPU as well. It's a SNMP monitoring tool so all those metrics are available. It's interesting to be able to see exactly how much utilisation of our processors have. Since running multiple instances per box we're clearing our recordings is 20 mins or so for 4 formats. FFMPEG is really a speedy beast when it comes to compression times.

BRETT

[Robert Minch]

Thanks everyone--I will be investigating the options you suggest. I've used Cacti and similar tools before, and if there are any particular custom/proprietary SNMP MIBs that I should know about, I'd appreciate a reference to those. Thanks again--Robert.

[EchoKaitlyn]

 Robert Minch, on 19 September 2009 - 02:02 AM, said:

Has anyone done detailed logging and analysis of lecture capture usage? For a project I'm involved with, it would be very helpful to have data on usage behaviors, including who views what, when, for how long, in what mode, etc. Presently our options would seem to include very basic data collected through Blackboard pages that users pass through, standard web server logs, basic iTunesU usage logs, and perhaps a custom web portal we could design to collect data from users before and after viewing materials. A search of this site finds the successful grant application "An investigation of study behaviours in learning environments supported by Echo360 lecture recording tools" that seems to indicate there might be an Echo360 system database that might be queried for more information as well. I'd be interested in knowing about any and all options that might exist. Thanks--Robert.

PS I recognize the privacy implications and am prepared to deal with them appropriately through IRB procedures and/or data anonymizing as needed.

Robert,

I've done some investigation on how this type of analysis might be done. Wowza does provide a pretty accurate foundation for monitoring viewing characteristics, because it's session-based and logs are written even when a "sudden stop" happens in the playback (such as a user closing the browser, or losing network connectivity). I'd suggest that you look at Sawmill (www.sawmill.net) to analyze the Wowza logs...it already knows the Wowza log format and you can create custom reports from the analyzed data.

By default, the Wowza server will log the stream name (x-sname), the client IP address (c-ip), total file size (x-file-size), and played duration as either time (x-duration) or bytes (sc-bytes). What is not present in the default Wowza installation is the location within a stream that a user seeks to, which allows us to determine whether they watched 20MB of a 25MB file versus watching the same 5MB segment 4 times.

To enable that field:
- edit /usr/local/echo360/server/wowza/conf/log4j.properties
- find the method “log4j.appender.serverAccess.layout.Fields” in the “# Access appender”
- add the value “x-spos” to the (rather long) list of fields that will be logged
- save the file, exit, and restart Wowza to make log4j re-read the configuration file

The position (x-spos) will be populated in any “seek” event, and shows the new location in the stream (as time in milliseconds).
On the next log entry for this stream, you can look at the current value of [x-duration] versus the previous value when the seek occurred to calculate how long this particular segment was played.
By connecting the segments together (again, another calculation) it is possible to determine that the student viewed X% of the presentation material.

Hope that helps!

Kaitlyn

[Brett Dalton]

 EchoKaitlyn, on 13 October 2009 - 12:59 PM, said:

Hope that helps!

Kaitlyn

Kaitlyn

This is absolutely awesome information, I wasn't aware of this package and the extended reporting from Wowza. I know that there is interest in doing more detailed analysis of usage at La Trobe so this will help massivly.

Thanks BRETT