Tag: competencies

CLAMP’s Moodle 3.0 and Moodle 3.1 recommendations

The Collaborative Liberal Arts Moodle Project (CLAMP) recommends the use of Moodle 3.0.x and Moodle 3.1.x for use during the 2016-17 academic year. Our recommendation regarding 3.1 includes a caveat to wait for the Moodle 3.1.1 release.

While we have been keeping an eye on Moodle 3.0.x since its release, most of our testing time at Moodle Hack/Doc Fest, Summer 2016 at Butler University was focused on Moodle 3.1. The newest version of Moodle introduces a number of enhancements including a greatly improved metacourse creation interface, a recycling bin plugin, an overhaul of the submission viewer for assignments, and a new “Competencies” plugin.

While we ran into no major issues during our testing of Moodle 3.1, there were two notable display issues that were scheduled to be resolved in Moodle 3.1.1:

Informal polling at Moodle Hack/Doc Fest, Summer 2016 and the CLAMP Moodle Exchange indicates that a majority of schools plan to go with the 3.0.x release because it has been out longer and had more bug fix releases. This follows the typical pattern with new Moodle version adoption within CLAMP schools.

This document is not meant to be a comprehensive review of all of the changes in Moodle 3.0 and 3.1; the full list of updates can be found in the release notes for Moodle 3.0 and Moodle 3.1.

CLAMP has already released Liberal Arts Editions (LAE) of Moodle 3.0.4 and Moodle 3.1.0. The Liberal Arts Edition version of 3.1.1 will be released approximately a week after Moodle HQ releases its version of 3.1.1.

Update 7/28/2016: MDL-54165’s resolution still has user experience issues. Read the update in the “Assignment grading interface and document annotation” section of this report for details and an updated version of the  ad hoc query for identifying courses that use the annotations feature.

Moodle 3.1 Improvements

Meta Course Link Improvements

Moodle HQ has major significant improvements to the Course Meta Link enrollment interface (see MDL-27628). As a result, CLAMP has removed its own version from the Moodle Liberal Arts Edition for Moodle 3.1 and is pleased to recommend the core solution.

Download selected assign submissions as a zip file

Moodle now allows users to select which assignment submissions they want to download, rather than having to download them all at once (see MDL-52490). This is a step forward in functionality but Moodle 3.1 also changes how the zip file itself is generated thanks to the implementation of MDL-52489 (“Download all submissions as a zip” should maintain the folder structure in students submissions”)

The new-style assignments zip file nests the assignment files in folders for each student, rather than using a single folder and prefixing each file with the student’s username and assignment info. This makes the assignment folder less browsable, and will cause teachers to drill down an extra level to view each students’ work. The plus side is that the files themselves are no longer renamed, but this may be offset by the need to browse an additional layer of folders.

Topic Blocks

Topic blocks can now be deleted, which sends their attached resources (but not the topic block itself) to the recycling bin. Topic blocks also can be renamed more quickly, just like activities and resources.

Pinned discussion topics

The ability to “pin” a discussion topic to the top of a forum was added in Moodle 3.1 and works as expected. By default, the teacher role and higher can pin topics, but that capability can be assigned to other roles in the system.

News Forum changed to Announcements

The “News forum” is now called “Announcements”, which is more in keeping with how that forum is used. The forum itself functions the same as before but colleges may want to update their documentation to reflect the change.

Grade Improvements

As of Moodle 3.0, the grades interface has been improved; there are now separate links for “Grades” and “Grade setup” in the Course administration block (see MDL-51051). The latter is the same as “Categories and items”, but makes more intuitive sense since that page is really about setting up and configuring the grade book.

Search Improvements

Search was improved in Moodle 3.1 (see MDL-31989) but it has additional dependencies that may be beyond the reach of many schools: it requires the installation of both the Solr server and the Solr extension prior to Moodle configuration and setup. Based on the documentation, the improved search respects user access and only returns items you have access to. It also appears to search just about all of the standard objects (e.g., book, assignment, forums, etc.). We did not have Solr setup at Hack/Doc so we were not able to test this further.

Moodle 3.1 New Features

Recycling Bin

The recycling bin (see MDL-48012) works as expected and allows a user to restore deleted content such as resources. It is tied to the course, not the instructor so if one instructor in the course deletes something, another instructor in the same course can restore it. By default, items are deleted from the recycling bin after seven days, but admins can change the time limit.

Assignment grading interface and document annotation

Moodle overhauled the assignment grading interface in 3.1, providing a streamlined interface that bakes in PDF annotation. By default the new interface displays the submitted assignment file as a PDF with the typical grading fields (grade, comments, etc.) to the right.

There are two problems with the feature.

  • Converting to PDF: If the submitted document wasn’t a PDF, Moodle attempts to convert it to a PDF. This relies on having an obscure helper utility called “unoconv” installed on the server. If that utility isn’t there (and it’s likely it won’t be) then the conversion fails and the user is left with a blank PDF. This is an issue with Moodle’s requirement checking regime (if you don’t have unoconv, Moodle shouldn’t try to use this feature) that is actively being worked on in MDL-54165 “New grading interface should hide “editpdf” if unoconv is not enabled”. It is set to be fixed in Moodle 3.1.1, so if this isn’t installed, it won’t try and render the PDF.
  • The annotation tool is clunky: Even if everything is working as intended, the annotation tool remains clunky, being akin to adding a simple paint program to Moodle. Previously this was less of an issue because it was not front and center in Moodle’s workflow, but that’s no longer the case with the new assignment grading interface. Comment on MDL-54818 “Improve assignment PDF annotation” if this issue is important to you.

A number of colleges are planning on turning off the PDF annotation component of Moodle until the issues with it are resolved. However, before turning off the feature we wanted to know how many people were using that particular feature in their courses. As a result, we created an ad hoc query that looks for the use the annotation feature in Moodle assignments by identifying occurrences of “comments” and “annotations” in the database.

Update 7/28/2016: Moodle core’s resolution of MDL-54165 adds a new option to switch between three assignment views: PDF only, PDF and grading interface, grading interface only.

However, this does not resolve the issue that we identified at Moodle Hack/Doc Fest, which is that if the unoconv utility is not installed, a blank PDF appears instead of the assignment. This is potentially confusing to end users and we believe it would be better to default to only the grading interface, with no PDF, when the utility isn’t available.

A second design ticket, MDL-55145 “New grading interface should indicate why the document isn’t displayed in the review panel” was opened to deal with this issue. We strongly recommend that people weigh in on this issue with their opinions regarding the interface.

Annotations Ad Hoc Query: Further testing of our annotation query to determine whether annotation was used in a course revealed an error in the logic that caused it to return the wrong results. That issue has been fixed. Get the revised query. Note that it requires the ad-hoc database queries plugin to run (this plugin is included in the 3.0.x and 3.1.x versions of the Moodle: Liberal Arts Edition).

Competencies

Moodle’s documentation defines competencies as:

“Competencies describe the level of understanding or proficiency of a learner in certain subject-related skills. Competency-based education (CBE), also known as Competency-based learning or Skills-based learning, refers to systems of assessment and grading where students demonstrate these competencies. In Moodle 3.1 it is possible to create and apply frameworks for evaluating students against competencies in Moodle.”

We spent considerable time at Moodle Hack/Doc Fest, Summer 2016 reviewing Competencies and came to the conclusion that it’s a complicated new system with an inherent workflow that isn’t well documented.

In brainstorming use cases for competencies at our colleges the scenarios that stood out the most were those where some sort of interdisciplinary or cross-institutional outcomes needed to be tracked. For example, a writing program that establishes a set of skills that students should develop during their time at the college. Evidence of acquiring competency in those skills could be provided in a variety of ways:

  • Evidence submitted by students
  • Evidence submitted as part of a course activity
  • Evidence submitted by faculty for students.

Moodle Competencies handle this through three tools:

  • Competencies: The building blocks of the system, competencies establish a specific goal, and provide a mechanism for giving evidence that the goal has been met.
  • Competency Frameworks: A collection of competencies.
  • Learning Plans: A method for pushing out a particular set of competencies (possibly taken from multiple frameworks) to students.

The single biggest challenge with Competencies is that it assumes you have an existing offline workflow and framework and want to implement it in Moodle. If you have those things, we expect that the tool makes a lot more sense. If you don’t, then the published documentation isn’t going to help you understand the usefulness of the tool or how to implement it at your college.

In addition, Competencies itself has an implied workflow that isn’t obvious to laypeople. For example, there is a process for students or faculty to request review of competency evidence. The per-student requests for such review appear on faculty’s “My Moodle” page, but it’s not clear if there is a place where faculty or learning plan managers could go to see the progress of an entire student cohort (e.g., not just an individual student’s progress toward meeting competency goals, but the entire cohort’s progress).

The development documentation does a much better job of explaining the purpose of Competencies and does a better job of explaining how the various pieces fit together.

Moodle 3.0/3.1 Plugin Review

We compiled a list of popular 3rd party plugins for Moodle and then determined if they were compatible with 3.0 and 3.1. Of particular note were the following plugins, which list compatibility with 3.0 but not 3.1:

  • Course Overview
  • TurnItIn
  • McGraw HIll

Course Overview was tested under 3.1 and seemed to work just fine. That said, the way it is configured changed in 3.1; it is now difficult to get to all of the settings on one unified page.

View the full list of third-party plugins.

About this report

This report was compiled based on work done at Moodle Hack/Doc Fest, Summer 2016 at Butler University. The following individuals contributed to it: Jason Alley (Lafayette College), Joe Bacal (Smith College), Deryl Botta (Butler University), Kristi Burch (Centre College), Adam Dinnes (Beloit College), Charles Fulton (Lafayette College), Ken Newquist (Lafayette College), Ruth Schwer (Butler University), Jedidiah Rex (Beloit College), and Matt Wright (Butler University).

Questions about the report can be sent to info@clamp-it.org.

Hack/Doc at Butler: Day 2

Moodle spelled out using variable sized artistic block letters
Moodle becomes art thanks to the “Word Art” exhibit in Butler University’s Irwin Library. Photo credit: Ken Newquist

During Day 2 of Hack/Doc Fest at Butler the team tackled a review of third-party plugin compatibility with Moodle 3.0 and 3.1, continued our accessibility work by testing new tools and documenting best practices, reacquainted ourselves with the Workshop plugin, and delved deeply into Moodle 3.1’s new Competencies feature.

Plugin Review

We compiled a list of popular 3rd party plugins for Moodle and then determined if they are compatible with 3.0 and 3.1. Of particular note were the following plugins, which list compatibility with 3.0 but not 3.1:

  • Course Overview
  • TurnItIn
  • McGraw HIll

Course Overview was tested under 3.1 and seemed to work just fine. That said, the way it is configured changed in 3.1; it is now difficult to get to all of the settings on one unified page.

View the full list of third-party plugins.

Accessibility and Course Design

Continuing our work from Monday, we created a “scroll of death course” — aka an exceedingly long course — with a variety of accessibility issues. The course was created using the TinyMCE WYSIWYG editor, but edited using Atto.

As expected, Atto couldn’t fix much of what had been done incorrectly using TinyMCE, if for no other reason than it didn’t have the same editing options. Many of the problems in the course had to be fixed by going into the source HTML and editing it there.

Once the new course is completed, we’re going to create the following example courses that address the “scroll of death” issue while improving accessibility and usability:

  • an accessible course using stock Moodle
  • an accessible course with Collapsed Topics course format
  • an accessible course with the Grid course format

We also intend to look at an option that allows students to switch switch course formats (e.g. grid to topic) to help with accessibility. This requires the user to set a custom profile field called “accessibility” which — when toggled — lets them switch to a designated “accessible” course format.

Screen Readers

We spent some time experimenting with screen readers on Moodle and other websites. We started with ChromeVox, a plugin for Chrome that allows you to tab through a page and reads the elements to you. The lack of mouse support made it more difficult to use and there isn’t an obvious way to toggle the plugin on and off.

We experimented with TalkBack on Android. It works great, but it radically transforms the way you use your phone and introduces a number of double and triple click command for common tasks. It requires you to re-learn how to how to navigate your phone.

We also researched optical character recognition techniques and found ConvenientOCR is a plugin for JAWS. This plugin lets you mouse or tab over an image and then OCR it. We did not get to try this plugin, but it sounds promising. JAWS itself is an expensive piece of software that only runs on Windows, but there is a free trial version for those who want to check it out.

Annotations Workflow

We continued our review of the new Assignment Submission view and the problems associated with it (see MDL-54165 New grading interface should hide “editpdf” if unoconv is not enabled and MDL-54818 Improve assignment PDF annotation).

A number of schools are planning on turning off the PDF annotation component of Moodle until the issues with it are resolved. However, before turning off the feature we wanted to know how many people were using that particular feature at our institutions. We created an ad hoc query that looks for the use the annotation feature in Moodle assignments by identifying occurrences of “comments” and “annotations” in the database.

The annotation query is available through CLAMP’s Github; it requires the ad-hoc database queries plugin to run.

CLAMP House Cleaning

We fixed two issues with the CLAMP website. We’ve resolved the SSL cert error that occurred when you tried to browse www.clamp-it.org under https. We also fixed a bug that would redirect you to an account sign-up page when accessing http://clamp-it.org.

Printing Quizzes in Moodle

We continued looking at ways to print quizzes and documented a technique involving embedding a quiz within a “book” within Moodle. View the documentation.

Workshop

We revisited Moodle’s Workshop module and found it to be a good peer assessment tool and fairly easy to set up. The tool has different student grading options including

  • comments, no grades
  • comments and grades
  • rubrics (which are different than assignment rubrics)

Competencies

We spent considerable time at Hack/Doc reviewing Competencies and have come to the conclusion that it’s a complicated new system with an inherent workflow that isn’t well documented.

Here’s how the documentation describes Competencies:

“Competencies describe the level of understanding or proficiency of a learner in certain subject-related skills:”

In brainstorming use cases for competencies at our colleges the scenarios that stood out the most were those where some sort of interdisciplinary or cross-institutional outcomes needed to be tracked. For example, a writing program that establishes a set of skills that students should develop during their time at the college. Evidence of acquiring competency in those skills could be provided in a variety of ways:

  • Evidence submitted by students
  • Evidence submitted as part of a course activity
  • Evidence submitted by faculty for students.

Moodle Competencies handle this through three tools:

  • Competencies: The building blocks of the system, competencies establish a specific goal, and provide a mechanism for giving evidence that the goal has been met.
  • Competency Frameworks: A collection of competencies.
  • Learning Plans: A method for pushing out a particular set of competencies (possibly taken from multiple frameworks) to students.

The single biggest challenge with Competencies is that it assumes you have an existing offline workflow and framework and want to implement it in Moodle. If you have those things, we expect that the tool makes a lot more sense. If you don’t, then the published documentation isn’t going to help understand the usefulness of the tool or how to implement it at your college.

In addition, Competencies itself has an implied workflow that isn’t obvious to laypeople. For example, there is a process for students or faculty to request review of competency evidence. The per-student requests for such review appear on the “My Moodle” page, but it’s not clear if there is a place where faculty or learning plan managers could go to see the progress of an entire student cohort (e.g. not just an individual student’s progress toward meeting competency goals, but the entire cohort’s progress).

The development documentation does a much better job of explaining the purpose of Competencies and does a better job of explaining how the various pieces fit together.

* Dev: Competencies
* Dev: Competency-based Education


Posts from Moodle Hack/Doc Fest at Butler University: HomepageSprint | Day 1 | Day 2 | Day 3

Hack/Doc at Butler: The Sprint Day

The white, aquaduct-like facade of Irwin Library at Bulter University
The exterior of Irwin Library at Butler University. Photo credit: Ken Newquist

CLAMP kicked off Moodle Hack/Doc Fest on Monday, June 21 with our traditional sprint day. The sprint day exists for everyone to get a jump on the week’s work — a few people (in this case, most of those attending Hack/Doc) arrive early, resolve any logistical challenges (getting to campus, connecting to the local network, making sure sources of caffeine are available throughout the day), and start organizing the week’s worth.

During this sprint we came up with our usual tasks list in Google Drive and people signed up for the things they were interested. Then, in a fit of spontaneous documentation, everyone started using that tasks list to provide updates on their progress. It’s not a thing we’ve done before, but it’s working out very well — it provides a running log of what we were working on, and it’s more efficient because we don’t need to spend as much time reporting out each day.

Topics identified include:

  • Accessibility – Making Accommodations for Users with Disabilities
  • Managing the “scroll of death” on course pages within Moodle
  • The practical implementations of layering quizzes over video
  • Evaluate the Moodle Mobile app
  • Printing from Moodle
  • The Recycle Bin plugin in Moodle (new feature in Moodle 3.1)
  • Evaluate the Mass Action block (3rd party plugin)
  • Evaluate Competencies (new feature in Moodle 3.1)
  • Test out the Global Search
  • Document, test, and name the new umbrella course plugin (3rd party plugin)
  • Look at other major changes in Moodle 3.1
  • Review anonymous forums in Moodle 3.1 (Moodle Liberal Arts Edition)
  • Review Ad-hoc database queries (customsql) (Moodle Liberal Arts Edition)
  • Best practices for using Moodle with other schools

Hack/Doc attendees can view the task list in Google Docs; the initial draft of the list is available in the CLAMP Moodle Exchange.

In the course of our work we came across a gnarly problem involving the new assignment review interface. This interface allows the user to see a student’s submitted assignment alongside all the relevant grading fields (e.g. add a grade, comments, etc.). The student’s work is rendered as a PDF and the teacher can use Moodle’s annotation capabilities to mark up the paper with their responses. The end result is saved as a PDF and sent to the student as feedback.

The problem is with the PDF. If students submitted a PDF, then the process works reasonably well. If they submitted a Word or Open Office document instead, Moodle converts it to PDF for display … or tries to. If you do not have a specific helper application known as unoconv installed on the server (and in our experience, it’s unlikely you will have it), then the PDF conversion fails. Instead you get a blank page. That blank page can then be edited with the annotation tools and submitted as feedback, but that’s not particularly helpful since the student’s original assignment isn’t included.

This is documented in MDL-54165 New grading interface should hide editpdf if unoconv is not enabled and is flagged as a critical bug in Moodle 3.1. The proposed resolution to the bug is to revise the interface to allow it to fail gracefully when this tool isn’t available. We encourage the CLAMP community to watch and vote for this issue.


Posts from Moodle Hack/Doc Fest at Butler University: HomepageSprint | Day 1 | Day 2 | Day 3