The Importance of Evaluation (and Being Earnest)

This week’s readings focused on different ways of designing online learning modules (screencasts and online tutorials), as well as a more broadly applicable method for designing one-shot instructional sessions (the ADDIE model, pictured below).

Image from Pace University's Cornerstone Newsletter (http://www.pace.edu/sites/pace.edu.ctlt/files/ctlt/newsletter/Volume%202%20Issue%201/articles/idm.htm)

The ADDIE Model

One issue I kept coming back to while reading these materials were the “evaluation” and “analysis” steps. It seems that the analysis and evaluation steps are two of the most important steps in the model for designing effective instruction, but they are also two that are the most often overlooked, and the most difficult to accomplish, especially in library instruction situations. In their discussion of money, the author of Creating the One-shot Library Workshop notes that one important thing to consider allocating funds for is to “[e]ncourage a client to help design and administer an evaluation of the learners’ paper, presentation, or other such product” (p. 16, Veldof, 2006). It does seem very important to have the client involved in the design of the evaluation process. As librarians, we can evaluate how well our actual teaching went, but to be effective, the skills we hope to impart through our instruction need to be long lasting and profound. We don’t always have the access to our learners to be able to see whether or not this is the case, though, so we need to keep in contact with those who will have access to our learners in order to judge the ultimate success of our instruction. However, in a lot of the library instruction situations I’ve read about or experienced, this kind of connection can be quite lacking.

As an example, in her 2010 article “Is an Online Learning Module an Effective Way to Develop Information Literacy Skills,” Nicole Johnston that “[a]necdotal evidence from the lecturer suggested that this transfer of skills had indeed taken place, but further research on assessing the quality and extent of this transfer needs to be undertaken”(p. 216). This is the problem I’m pointing out – often what passes for “evaluation” of library instruction sessions tends to be either (a) questionnaires that seem particularly prone to bias (as shown elsewhere in Johnston’s article) or (b) anecdotal evidence from instructors. I don’t mean to seem like I’m saying this is the librarians’ fault, however. It most certainly is not – the blame for this lack of collaboration between instructors and instructional librarians seems to rest, at least in large part, on the instructors themselves, or at least on the culture and structure of many higher-education institutions.

As librarians, we often seem like we want to remain neutral on some topics. The ALA Code of Ethics, after all, calls for remaining “unbiased” and “courteous.” However, it seems that this is a situation in which, in order to best help our patrons, we need to get a little more pushy. It can’t be acceptable for instructors to think that all students need to do to unlock their “Information Literacy Badge” is to attend a few (or sometimes only one!) workshop or class with a librarian. This kind of instruction and training is going to require a much closer collaboration between librarians and other instructors. All the steps of the ADDIE process can involve this collaboration, but it seems that, at the very least, librarian/instructor collaboration MUST occur during the evaluation phase. How can we know if students have truly learned anything if we only see them during a couple one-hour workshops during their 4-year college career?

Advertisements

About rclement643

I am currently a graduate student at University of Michigan's School of Information. This blog is being kept as a place for reflections on the readings for the class
This entry was posted in Uncategorized. Bookmark the permalink.

3 Responses to The Importance of Evaluation (and Being Earnest)

  1. Kelly says:

    I have also been thinking a lot lately about the idea of librarianship and neutrality. One of the questions that came up during the Quasi-con unconference was “What’s the line between education and advocacy?” Of course, there is no one “line” between these two functions, as it depends on context and so many other factors, but I tend to side with the push for greater advocacy as it relates to educational goals like increasing information literacy among students. I can’t help but think that traditional institutions are in some ways hindered by disciplinary borders — say, between writing/research classes and “library skills” efforts — because they reinforce divisions among educators, rather than incentivizing team work.

  2. Meggan says:

    I think the point you make about learning needing to be useful beyond an end-of-worshop survey is really valid. It gets back into the problems of librarian/educator collaboration and communication that Kelly talks about. I wonder what kind of incentive might be effective in creating a culture of collaboration.

  3. northernwood says:

    Agree with you about the often overlooked “evaluation” and “analysis” elements in the instruction design process. And the difficulty or lack of effort in that area partly contributes to our dilemma of evaluating the quality of library service. As you said, “we can evaluate how well our actual teaching went,” but we don’t “have the access to our learners” to be able to tell whether our teaching is really effective.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s