Summary of Discussions

1. Presentation of results of phase I evaluation

Dr. Ken Waterston of IMRI gave an enlightening presentation on the findings from Evaluation period I. Broadly, the response was positive, with only 14.3% of users indicating they would not use Xgrain again. Extra Help features were requested along with possible search examples. A number of those using the service found the processes unclear, which will need to be addressed if the product is to be successfully embedded into the curriculum.

Discussion:

Other issues that emerged during the discussion were:

There are problems with evaluating a service that might not contain databases relevant to each user, as users will judge the usefulness of a service by relevant hits. Information about databases was also lacking. Roddy MacLeod made a suggestion that a link to the JISC resources guide would be helpful.

In the phase II evaluation the questionnaire should have details about where to send it and include a stamped-addressed envelope.

More help screens will be needed for the phase II evaluation.

The labels of some buttons were confusing to users. EDINA should look at other services such as CSA, ZPORTAL and MetaLib to see how the screen directions are labelled.

There were some detailed discussions on each of the Xgrain screens and it was agreed that the phase II evaluation would elicit more detailed feedback on some of the issues.

2. Criteria for selection of sites for phase II evaluation

Ken Waterston presented a briefing paper on suggestions for the trial period II evaluation. Ken will draw up a model for phase II and circulate it to the L&T associates for comment.

A discussion took place on the suitability of using LTSNs as a contact point for phase II. Some seem to have good links with Academic staff whereas others do not have these contacts. The group agreed that LTSNs should at least have a contact person for the evaluation. Ken Waterston stressed the pre-planning of the evaluation, including advance recruitment of subjects. Valuable time can sometimes be wasted if subjects need to be rounded up at the last minute. The group agreed that Academic staff are more difficult to recruit than students for such studies. Ideas such as incorporating an evaluation in to a conference or event were suggested. All agreed that persuading Academic staff to incorporate Xgrain into teaching will be difficult. Publicity needs to stress ease of use and shortened instruction time due to the use of one interface, not many.

3. Learning materials discussion

The group agreed that case studies in the use of A&I services in learning and teaching are difficult to track down. As part of the Xgrain project, case studies will no longer be actively sought but any received will be accepted. Alan Brine's team at Loughborough have agreed to author some more materials.