• SakaiToolsSurvey2007

This pages describes the final version of the Michaelmas 2007 Tools survey. Previous iterations have been archived.

Here is the email message that was sent to stakeholders

Executive Summary

We have attempted to summarise the essence of the findings of the survey below. The highlights are presented along with the percentage of users who felt this way.

A more comprehensive summary of the survey is also available in PDF format.

Survey Size

48 people responded to the survey request; 61% of these are floor managers, ie, they administer part of the weblearn tree. If they were IT support staff then they were instructed to respond on behalf of the people that actually use the system. About three-quarters were quite experienced with WebLearn, 10% do not use the system. We did not canvass any 'students'.

Uses of WebLearn

Over half of the respondents use WebLearn for administration (54%), approximately one third use it for teaching (34%).

Functionality

Choice statements:

• use it for (protected) filesharing (35 say this)
• staff intranet
• providing course information
• setting tests / quizzes

'Essential' Tools

A question was posed as to which tools were essential, useful, not useful or unknown to the respondent. The most 'essential' tools or services were

1. Controlling access (71%)
2. Questionnaire (41%)
3. Timed release (41%)

4. Structured document / EasyWriter (33% / 27%)*

5. Notification of activity (33%)
6. Quicklinks (links to internal / external URLs) (27%)
7. Tracking facilities (30%)

* These tools are listed together as they perfrom similar functions: web page authoring. EasyWriter is a new tool and despite our best efforts, one quarter of respondents hadn't heard of it.

'Useful' Tools

The 'useful' tools (not including those above) were:

• MyWeblearn (53%)

• Discussion Forum (51%)*
• RSS (newsfeed) display (50%)
• Pigeon hole (44%)
• Logbook (42%)
• Short answer paper (40%)
• Multiple choice quiz (37%)

* This tool is a dreadful implentation of a forum; in the authors mind this means that its importance can probably be increased.

'Not Useful' Tools

The tools that were thought to be 'not 'useful were:

• Short answer paper (36%)
• Logbook (33%)
• Peer marking tool (33%)*
• Discussion forum (probably due to interface) (31%)
• Multiple choice quiz (30%)

* this is probably only useful in certain disciplines, eg, I would imagine that the Medics would probably like this tool.

Poorly Implemented Tools

Choice statements:

• Logbook - poor interface: 3 say this,
• Discussion forum - poor interface: 5 say this,
• Want timed release of individual files,
• MCQ / questionnaire, need more flexibility / question types: 3 say this,
• ACL too complex,
• Structured document too complex,

People also complain that they haven't heard of some of the tools.

New Tools

For this section we identified Sakai tools that it would be possible to deploy and that we thought may be of interest to users:

• Add items to a shared calendar ('Schedule') (63%)
• Allow students to upload documents to share with their tutor ('Drop Box') (55%)
• Make course related announcements ('Announcements') (57%)
• Mailing list with archive ('Mail Archive' and 'Mailtool') (55%)
• Create and share reading lists ('Resources' in v2.5) (46%)

• Personal calendar ('Schedule' in MyWorkspace) (43%)

• Detect plagiarism ('Assignments') (39%)
• Switch to student view (39%)*
• Vote ('Poll') (36%)

* Not currently possible but Indiana Uni have nearly finished the code for this.

It was interesting to note that there's very little interest in SCORM (although in the past OUP have asked about it). Also 'Gradebook', 'Chat' and 'Diary' (Blogger) generated little interest.

Comments About Servies / Tools

Statements:

• What happened to Portole (reading lists)?

• Why are certain facilities only avaliable in WebLearn when a Groupware solution would provide a better service?

• Email is often more efficient
• More question types (quizzes), Exam mode (on-line assessment)
• Want sign-up / room booking tool
• Poor interfaces (too many clicks) / too hard to use
• Want site map
• Want shared Groups Store

WebLearn Usability

We asked how effective a range of usability features were, nothing was rated as very good and a small proportion were rated as very poor. The biggest problems were:

• Visual presentation (30%)
• Search (30%)

On the flip side features ranked as good were:

• Consistancy of appearance / terms / actions (65%)
• Speed (quick loading of material) (57%)
• Simplicity of use (53%)
• Readability (lack of clutter) (50%)
• Navigation (44%)

Nobody whatsoever thought that Search was 'Very Good' and only one person thought that navigation was 'Very Good'!

Campus Metaphor

Should we carry the Campus Metaphor forward into the new system?

• Yes 31%
• No 42%
• No opinion 28%

Demographic

Origins:

• 7 from colleges
• 8 from service departments (OUCS and OLI only)
• 15 from academic departments (5 from engineering the rest spread evenly over the 3 divisions)
• 2 from libraries

Role:

• 9 management centred (head of 'body' or project manager)
• 8 lecturer / tutor
• 6 researchers
• 12 administrators
• 19 ITSS / technicians
• 8 information centered (librarians etc)

12 people said they were willing to talk to us face to face.