Over the course of 2015, the Media and Instructional design teams at University of Wisconsin-Extension, Division of Continuing Education, Outreach & E-Learning (CEOEL) collaborated to produce Sherlock as a prototype tool for evaluating a learner’s competency in active listening. The initial aim of Sherlock was to provide a method for assessment that was automated, but more closely matched the actual skills exercised in active listening than existing assessments like multiple choice tests. Following initial development and testing, Sherlock was extended so that it could be used to support both assessment and instruction. While Sherlock currently only offers interactive video tagging, there are plans to extend the platform further to support similar activities with other types of media.
Sherlock is a PHP web application built in house at University of Wisconsin-Extension, Division of Continuing Education, Outreach & E-Learning (CEOEL). It is also a Learning Tools Interoperability (LTI) tool, which can integrate with any LTI conformant online learning management systems (LMS). The main functionalities and capabilities of Sherlock are to assess a user’s ability to recognize audio cues or visual cues based on events that occur in a piece of streaming video.
The user recognizes the cues by clicking on a button from a set of five or fewer pre-defined buttons next to the video. Buttons are appropriately named in response to the types of events in the video. Each button has a limited number of times that it can be clicked. The limit is displayed at the top right of each button. In addition, after a button is clicked, it enters a cool down period where it is grayed out for a number of seconds specified by the designer. Buttons turn blue again to indicate that they are active after the preset time passes. The cool down time may be different for each button, and a progress bar is displayed on the button during the cool down period to give the user an indication of how soon they will be able to click it again.
Depending on the exercise, a special button may be presented. The special button is a little bigger than the other buttons, and it is located right next to the timeline. While this special button could be put to other purposes, at current it is implemented to rewind the video by a number of seconds set by the designer. It can also invoke a penalty. After the rewind button is clicked, the video is paused for a couple of seconds and then resumes. When the rewind button is pressed, any button on cool down will reset or restart over again with additional seconds added.
A tag corresponding to the user’s button clicks automatically appears over the timeline below the video. The letter or icon on the tag corresponds with the button letter or icon. The tag is placed at the time when the user believed a cue was displayed or expressed.
At the end of video, the end of the assessment is automatically triggered and Sherlock transitions to a score view with a detailed explanation of the user’s performance for each button. Depending on the exercise type, a review of the assessment may also be available where the user can compare the positions of their tags with the correct positions with justifications.
Sherlock is database driven and can be used inside or outside of an LMS. Each exercise in Sherlock is created with the use of a JSON file and a video hosted on YouTube. The JSON file stores all of the data in regard to the exercise such as the title of the exercise, the number positions, the number of buttons, button limits for each button, the cool down time for each button, etc. At the end of an exercise, another JSON file is created and stored. This JSON file stores all of the positions that the user tagged. Personal user information is not stored in this JSON file.
When Sherlock is used as a stand alone tool (i.e., outside of an LMS), a login system is in place that leverages the Google Sign-In API. Thus, the login system requires the user to have a Google Account. Basic user information will be requested from the Google Account and stored in the database for exercise taking and grading purposes.
As an LTI tool inside of an LMS, Sherlock will not ask for the user to sign in. The LTI API will take care the authentication process. No user information will be stored in the database. Any file generated will not contain any identifying information to point back to the user.