Category Archives: assessment

Electronic Management of Assessment (EMA)

Posted on by

7368246546_531d0211df_z

In the last few months JISC have been doing some great work looking at EMA, electronic management of assessment.  This has produced a really useful landscape review and there is more to come. For TALL this has brought back lots of memories of the Cascade project, and in particular our focus area: Online assignment handling. This was a few years ago and it may seem that many of our outputs might be outdated, but it is amazing how much of what we were dealing with remains the same.   In particular the recognition that so many issues in this space are about business processes rather than technology, really resonates with our Cascade experience. Certainly we are still working with colleagues to identify where processes can be improved to make things better for everyone and make using technology to support them even more effective.  A good example of this has been the  time it has taken to agree a universal submission time of day (2pm to ensure there is always support available to students immediately before and afterwards) and identify submission free periods (Mondays, Fridays and 2 weeks a year over Christmas and in the summer). Simple in principle, lots of benefits, but a big change from previous norms.

Our work on EMA in the department has come a long way since the Cascade project. Online submission and feedback is now our default mode for assignment handling (with a very small number of exceptions) and the understanding of staff and students of how EMA works, the real benefits and issues, makes discussion of future options and opportunities a much more valuable process.  It is this last point that ties us back around to the EMA work that JISC is undertaking.  While we have achieved a huge amount in this area, there is no doubt in our minds (and those of our students, tutors and administrators) that we could be doing more – and we have made sure we have participated in consultations to share what we are learning and what we want to know. What it is great to see is that we are not alone! The areas we are grappling with are ones that everyone else is dealing with too.  We will be closely watching this space to see what comes next.

Image found at https://flic.kr/p/ce7bZy  /CC BY 2.0

Attitudes to online assignment handling

Posted on by

As part of Cascade we are updating our online assignment handling system.  Currently the vast majority of assignments in the department are handled in hard copy, so that making this service available more widely will be a big change for many of our students, administrators and academics if they use it.

In our relatively informal consultations up to this point we have encountered widely differing reactions to the prospect of moving this online, both negative and positive and we are now moving forward with a more systematic survey in this area. As part of this we are trying to come up with a list of  attitudes which stakeholders can indicate agreement with or not – we started out with about 5 statements each for academics and students but since consulting a bit more are now up to well over 10 for each.

 Academic statements

  1. Many of my students hand write assignments
  2. I would be technically confident handling assignments online
  3. I am confident that online assignment handling is secure
  4. I do not want to spend more time at a screen
  5. I think that online assignment submission would speed up the marking process
  6. I would welcome being able to use plagiarism detection software
  7. I am worried about having to remember more passwords
  8. I think typing feedback will take longer than writing it by hand
  9. I am worried about students submitting assignments in file types I cannot read
  10. I do not want to have to print out assignments
  11. I am worried about having a good enough computer to deal with marking assignments online
  12. In my subject it is difficult produce electronic assignments e.g. maths notation or Cyrillic script
  13. I am worried that online submission will make it easier to plagiarise
  14. I think online assignment submission will be more stressful for student

Student statements

  1. I currently hand write my assignments
  2. I would be confident about the security of submitting my assignments online
  3. I have the technical skills to submit an assignment online
  4. If I submitted an assignment online, I would like to receive confirmation of receipt by email
  5. I would welcome the additional time the option of online submission would offer in meeting a deadline
  6. I would be happy to receive my work back electronically
  7. I think that online assignment submission would speed up turnaround of my assignments
  8. I prefer handwritten comments on my assignment
  9. I find typed feedback easier to read
  10. I currently submit my assignments by hand
  11. I currently submit my assignments by post

We are hopeful we have captured the most common attitudes, but it is hard to balance the positive and negative and there is always the worry that this will draw attention to points of view that would have never have occurred to our stakeholders if we had not brought it up.  If anyone has any experience in this area and can suggest anything they found useful it would be great to hear from you.

Keeping students on course

Posted on by

Although I am very closely involved in the development of all our courses, we have so many now that I can no longer follow them as closely as I used to when they finally go live. The course stops being the thing that the author, project manager, web developers, and I (as learning technologist) have laboured over and becomes something that is owned by the tutor (often, but not always, the author) and the students.

As an external observer you can dip in and get a sense of how things are progressing but it is not the same as visiting a course every day, getting to know everyone, and really being part of the course experience.

Unsurprisingly it sometimes feels odd guiding people in creating effective online learning when my “hands on” involvement is so much less than it has been in the past, so I am reduced to getting my affirmation that we are on the right track in other ways. Evaluation forms are always interesting reading (and we do monitor these very closely) but recently we got some very gratifying hard statistics that I had not personally seen before.

In continuing education funding is often predicated on the mysterious completion rates – i.e. it is not just the students who start your course but those who finish who are important to HEFCE….especially hard for us in our fully online courses (an area where completion rates can be VERY low) .

So to the meat – we got the latest stats and we are averaging a 91% completion rate, even more amazing is that this is higher than the face to face rate of about 85%.

Designing courses to encourage completion is something we have given a lot of thought to over the years and we have devoted a lot of our learning design energy in creating learning and assessment in such a way to keep students on-board and motivated. It seems that in 91% of cases it is paying off….

Pedagogy Experts Group

Posted on by

I have just got back from JISCs latest pedagogy experts meeting. Always a good place to find out more about things that are going on in eLearning (from a pedagogy perspective) either JISC funded or otherwise.

Had a very interesting discussion about the future of (e)assessment – I am looking forward to seeing some of the stuff coming out of this area, as whatever about massed mcq question banks I think that social software is going to completely change this space, but this presents a lot of challenges for the institution.

Also an interesting update on the SIMPLE project which looks like a really interesting example of simulations for learning.