TALL blog

In the last few months JISC have been doing some great work looking at EMA, electronic management of assessment.  This has produced a really useful landscape review and there is more to come. For TALL this has brought back lots of memories of the Cascade project, and in particular our focus area: Online assignment handling. This was a few years ago and it may seem that many of our outputs might be outdated, but it is amazing how much of what we were dealing with remains the same.   In particular the recognition that so many issues in this space are about business processes rather than technology, really resonates with our Cascade experience. Certainly we are still working with colleagues to identify where processes can be improved to make things better for everyone and make using technology to support them even more effective.  A good example of this has been the  time it has taken to agree a universal submission time of day (2pm to ensure there is always support available to students immediately before and afterwards) and identify submission free periods (Mondays, Fridays and 2 weeks a year over Christmas and in the summer). Simple in principle, lots of benefits, but a big change from previous norms.

Our work on EMA in the department has come a long way since the Cascade project. Online submission and feedback is now our default mode for assignment handling (with a very small number of exceptions) and the understanding of staff and students of how EMA works, the real benefits and issues, makes discussion of future options and opportunities a much more valuable process.  It is this last point that ties us back around to the EMA work that JISC is undertaking.  While we have achieved a huge amount in this area, there is no doubt in our minds (and those of our students, tutors and administrators) that we could be doing more – and we have made sure we have participated in consultations to share what we are learning and what we want to know. What it is great to see is that we are not alone! The areas we are grappling with are ones that everyone else is dealing with too.  We will be closely watching this space to see what comes next.

Image found at https://flic.kr/p/ce7bZy  /CC BY 2.0

As the IT director for Sainsburys pointed out at BETT a couple of weeks ago ‘self-service’ caused a revolution in retail during the 20th century. It allowed for greater choice, efficiency and of course scale. It put the ‘super’ in supermarket in the same way that the web has put the (potential) ‘massive’ into MOOC.

At first glance the current wave of publicity-garnering MOOCs appear to be the equivalent of self-service education. Big out-of-town locations for education with an increasing range of products that you are free to browse at leisure.

Pick a product and pay for accreditation as you pass through the tills…

CC-NC-ND http://www.flickr.com/photos/sputnik57/3583618864/

This perhaps is a little disingenuous though as there is more effort required than simply putting a course in your basket to gain validation. Automated testing and peer assessment are legitimate ways of assessing levels of knowledge and, if properly designed, increasing understanding. This is the real challenge for MOOCs, as it is for any course; how can we encourage students to think? How do we best mix the ingredients we have available to increase the chances that those engaging with our courses will finish them with *both* increased knowledge and increased understanding? – I hope we can all agree that teaching with a view to increasing understanding is a large part of what higher education institutions are for(?)

I have heard teaching described as ‘what you have to do because there are more of them than there are of you’, it’s inherently about dealing with scale. In this sense many of the pedagogical challenges faced by the designers of MOOCs are the same as those to be found in face-to-face or non-massive courses. The danger though is that xMOOC style self-service education favours those who already equipped with the intellectual and academic techniques required to interrogate a subject. How do we encourage those who don’t have the necessary higher-education ‘literacies’ to wade through swathes of video lectures and online resources? One answer is already hiding in the MOOC format: the ‘event’.

MOOCs generally have a start and finish date which makes them a form of slow-burn event. Even though the web has an always-on, always-connected, constant-flow paradigm it is still largely event driven. We are drawn to specific moments in time which act as way-points in the ceaseless river of information and social noise. MOOCs are useful island in this river with a beginning, middle and end, a simple narrative we can organise around and hopefully contribute to even if we don’t choose to listen to the whole story. The principle of the event can be taken further though as I believe it is highly compelling, especially in an online context. This is what I’m focusing on with the new Oxford Connect format.

Educators and technologists have become adept at putting-the-curriculum-online but we have yet to master the nuances of the live event outside of the lecture theatre. Pi Day Live, the pilot event for Oxford Connect, is designed to be a moment in time where hundreds of participants gather online to take part in collective activity. It will be highly ‘Evented’ (an idea originally attached to ‘virtual worlds’ but which is broadly applicable), encouraging participants to be as Resident as possible for a short period. My hope is that in time this live format will become a valuable way of communicating ideas, concepts and research from Oxford. I envisage this format being used as part of large-scale online courses, incorporating the fellowship of live events to support communities of learners and to act as milestones in a larger pedagogical structure.

Perhaps the live event is what is missing from xMOOCs and the expertise of the connectivists is what’s needed to counter a self-service mentality which disenfranchises those without with the literacies required to go-it-alone in online learning?

As part of Cascade we are updating our online assignment handling system.  Currently the vast majority of assignments in the department are handled in hard copy, so that making this service available more widely will be a big change for many of our students, administrators and academics if they use it.

In our relatively informal consultations up to this point we have encountered widely differing reactions to the prospect of moving this online, both negative and positive and we are now moving forward with a more systematic survey in this area. As part of this we are trying to come up with a list of  attitudes which stakeholders can indicate agreement with or not – we started out with about 5 statements each for academics and students but since consulting a bit more are now up to well over 10 for each.

 Academic statements

1. Many of my students hand write assignments
2. I would be technically confident handling assignments online
3. I am confident that online assignment handling is secure
4. I do not want to spend more time at a screen
5. I think that online assignment submission would speed up the marking process
6. I would welcome being able to use plagiarism detection software
7. I am worried about having to remember more passwords
8. I think typing feedback will take longer than writing it by hand
9. I am worried about students submitting assignments in file types I cannot read
10. I do not want to have to print out assignments
11. I am worried about having a good enough computer to deal with marking assignments online
12. In my subject it is difficult produce electronic assignments e.g. maths notation or Cyrillic script
13. I am worried that online submission will make it easier to plagiarise
14. I think online assignment submission will be more stressful for student

Student statements

1. I currently hand write my assignments
2. I would be confident about the security of submitting my assignments online
3. I have the technical skills to submit an assignment online
4. If I submitted an assignment online, I would like to receive confirmation of receipt by email
5. I would welcome the additional time the option of online submission would offer in meeting a deadline
6. I would be happy to receive my work back electronically
7. I think that online assignment submission would speed up turnaround of my assignments
8. I prefer handwritten comments on my assignment
9. I find typed feedback easier to read
10. I currently submit my assignments by hand
11. I currently submit my assignments by post

We are hopeful we have captured the most common attitudes, but it is hard to balance the positive and negative and there is always the worry that this will draw attention to points of view that would have never have occurred to our stakeholders if we had not brought it up.  If anyone has any experience in this area and can suggest anything they found useful it would be great to hear from you.

TheStar.com reports on a student facing disciplinary action for running a Facebook study group. Sounds like the institution took a dislike to normal study behavior just because it was happening online.

Although I am very closely involved in the development of all our courses, we have so many now that I can no longer follow them as closely as I used to when they finally go live. The course stops being the thing that the author, project manager, web developers, and I (as learning technologist) have laboured over and becomes something that is owned by the tutor (often, but not always, the author) and the students.

As an external observer you can dip in and get a sense of how things are progressing but it is not the same as visiting a course every day, getting to know everyone, and really being part of the course experience.

Unsurprisingly it sometimes feels odd guiding people in creating effective online learning when my “hands on” involvement is so much less than it has been in the past, so I am reduced to getting my affirmation that we are on the right track in other ways. Evaluation forms are always interesting reading (and we do monitor these very closely) but recently we got some very gratifying hard statistics that I had not personally seen before.

In continuing education funding is often predicated on the mysterious completion rates – i.e. it is not just the students who start your course but those who finish who are important to HEFCE….especially hard for us in our fully online courses (an area where completion rates can be VERY low) .

So to the meat – we got the latest stats and we are averaging a 91% completion rate, even more amazing is that this is higher than the face to face rate of about 85%.

Designing courses to encourage completion is something we have given a lot of thought to over the years and we have devoted a lot of our learning design energy in creating learning and assessment in such a way to keep students on-board and motivated. It seems that in 91% of cases it is paying off….

I have just got back from JISCs latest pedagogy experts meeting. Always a good place to find out more about things that are going on in eLearning (from a pedagogy perspective) either JISC funded or otherwise.

Had a very interesting discussion about the future of (e)assessment – I am looking forward to seeing some of the stuff coming out of this area, as whatever about massed mcq question banks I think that social software is going to completely change this space, but this presents a lot of challenges for the institution.

Also an interesting update on the SIMPLE project which looks like a really interesting example of simulations for learning.