Article

 

Independent Searching During One-Shot Information Literacy Instruction Sessions: Is It an Effective Use of Time?

 

Rebekah Willson
Assistant Professor and Librarian, Mount Royal University
PhD Student, School of Information Studies, Charles Sturt University
Wagga Wagga, New South Wales, Australia
Email:
rwillson@csu.edu.au

 

Received: 2 July 2012 Accepted: 18 Nov. 2012

 

 

cc-ca_logo_xl2012 Willson. This is an Open Access article distributed under the terms of the Creative CommonsAttributionNoncommercialShare Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

 

 


Abstract

 

Objective To test the assumption that giving students time to research independently during a one-shot information literacy instruction (ILI) session, combined with scaffolding, is an effective pedagogical practice and a good use of class time.

 

Methods The study was conducted at a student-focused, four-year undergraduate institution with 8,500 full load equivalent students. Following brief, focused instruction in 10 different ILI sessions, first-, second-, and third-year students in 80-minute one-shot ILI sessions were given time to research independently. The librarian and instructor were present to scaffold the instruction students received. Students were asked to track the research they did during class using a research log and to fill out a short Web survey about their preparedness to do research and the usefulness of the ILI session.

 

Results Students agreed to have 83 research logs and 73 Web surveys included in the study. Students indicated that they felt more prepared to do research for their assignment after the ILI session and rated individual help from the librarian as the most useful aspect of the instruction session. Students did not rate independent time to do research as valuable as anticipated. Examining the research logs indicated that several things are taking place during the ILI session, including that students are demonstrating what was taught in the session in their searches, that their searches are progressing in complexity, and that students are using feedback from previous searches to inform the formulation of search queries. While students appear to be putting independent search time to good use, many students articulation of their thesis statement remains poor and searches continue to be fairly simplistic.

 

Conclusions This study gives evidence that giving independent research time in ILI sessions, with scaffolding, is an effective use of class time. The study also demonstrates that the majority of students are able to use what is taught during classes and that they are using class time effectively, though searching remains fairly simple. The focus of ILI sessions is on skill development, and future research should be on integrating IL into the curriculum to develop more complex skills and thinking needed in the research process.

 

 


Introduction

 

For all practitioners, evidence based practice is challenging. For those who teach information literacy instruction (ILI), evidence based librarianship becomes particularly difficult when teaching sessions are one shot. Many librarians have a single session with a group of students that will last between 50 and 80 minutes, and a significant amount of content to cover. As student learning is the ultimate goal for ILI, the choice of what content to cover (e.g., teaching concepts or skills) and how to teach that content (e.g., lectures or hands-on practice) is of the utmost importance.

 

Collecting evidence for any research is challenging; however, when attempting to practice evidence based librarianship in this restricted context, data collection must be quick and unobtrusive, not taking up precious class time. One way to incorporate data collection into one-shot sessions is to use what already takes place in the class. Many librarians do this by collecting assignments, either research assignments that have been assigned by the course lecturer (Webster & Reilly, 2003) or worksheets that have been assigned by the librarian in the ILI session (Fain, 2011), or a combination of these two methods. For other librarians, course assignments are not available and they may decide worksheets take up more class time than they are willing to give.

 

The author was interested in questions of pedagogy how best to use class time in the ILI session. Typical classes were short lectures followed by lengthy periods of time to search independently, combined with one-on-one help termed scaffolding. Scaffolding is a technique in which a teacher works with students individually to give them support, gradually removing that support as the student is able to work more independently. (Larkin, 2008). The evidence based librarianship project was designed to determine if current practices were helpful to students. The research question was whether giving students time to research independently during a one-shot information literacy instruction session, combined with scaffolding, is an effective use of class time.

 

Literature Review

 

Information literacy instruction is a large part of many librarians work. As such it has been the subject of much study. To better understand what takes place in the classroom, it is important to examine both student behaviour and classroom pedagogy.

 

Information literacy instruction is a complex research topic, with a multitude of factors impacting the learner, the learning environment, and the instruction. For those who teach searching, understanding how students search for online information is an important aspect. In their Information Behavior Model, Urquhart and Rowley (2007) identified many micro and macro factors that impact student information behaviours, including information literacy, search strategies, discipline and curriculum, pedagogy, support and training, information resource design, and access. Many students typically begin searching with Google (Griffiths & Brophy, 2005; Urquhart & Rowley, 2007), relying less on academic resources (Griffiths & Brophy, 2005). Starting searches with Google and limiting use of resources to those that are well known is typical, as ease of use and familiarity are important factors in their choice of resource (Dervin & Reinhard, 2007; Griffiths & Brophy, 2005; Urquhart & Rowley, 2007). Searchers in online environments typically include few terms in their search queries, infrequently use advanced search techniques such as Boolean operators, and infrequently use advanced search features such as limiters (Lau & Goh, 2006; Markey, 2007a, 2007b; Wang, Berry, & Yang, 2003; Willson & Given, 2010). In addition to simple searching, students often expect online public access catalogues and databases to work like a search engine (Griffiths & Brophy, 2005; Novotny, 2004). Searchers with less search experience spend less time thinking about and planning searches, in addition to using fewer self-aware, metacognitive strategies than do searchers with expertise (Tabatabai & Shore, 2005). Overall, undergraduate students use familiar sources in fairly rudimentary ways to satisfy their information needs.

 

Many researchers have examined the effectiveness of ILI. Some research has focused on the general effectiveness of ILI. Portmann & Roush (2004) found that ILI increased library usage, though not library skills. One of Koufogiannakis and Wiebes (2006) findings from their meta-analysis of ILI studies was that, overall, instruction of any variety was better than no instruction. Other research has focused on the effectiveness of particular interventions. Buhay, Best, and McGuire (2010) found that student scores on post-tests were statistically significantly higher when they used clickers in ILI. Marcus and Beck (2003) found that students who took part in a treasure-hunt style self-orientation to the library scored higher on questionnaires and rated the tours more positively than those who were in librarian-led orientation groups. Bren, Hillemann, and Topp (1998) found that using a guided, hands-on instructional method increased undergraduate students retention of information provided during an ILI session.

 

These studies indicate that ILI is effective and that particular interventions can be used. What is missing from studies of ILI effectiveness is an examination of scaffolding and independent search time. One article mentioned scaffolding as part of the ILI instruction (Johnson et al., 2011). No articles were found that addressed independent search time. It is difficult to determine whether this apparent lack of literature is due to these pedagogical practices not being researched, or due to differences in terminology that make the literature difficult to find.

 

Context and Aims

 

Context

 

Mount Royal University is a four-year undergraduate university. The student body is 10,551 full-time students (Mount Royal University, n.d.a). The institution has four categories within its Aims of an Undergraduate Undergraduate Education, with information literacy listed under Intellectual and Practical Skills (Mount Royal University, n.d.b). In 2009/2010, the library taught 710 ILI sessions to over 12,000 students. Typically ILI sessions are one-shot sessions that are either 50- or 80-minutes long and tend to be focused on a specific research project. The vast majority of ILI sessions are hands-on and take place in computer labs.

 

Aims

 

As part of evidence based practice, the author was interested in examining the information literacy instruction she provides, with a view to improving understanding of whether the independent time to search in the ILI session is used effectively and how teaching could be altered to improve learning. Feeling that she was trying to include too much content into one-shot sessions, she cut down on the amount of content presented in class in the lecture format, focusing on the specific research assignment and providing as much time as possible for hands-on work. The focus of these ILI sessions became the students research assignments and the individual help (scaffolding) provided to students by the librarian and the class instructor.

 

The researcher felt that giving students time to work on their research assignment is a more active learning technique and that students would have the opportunity to try what had been discussed in class, along with scaffolding. Part of scaffolding is to work within students zone of proximal development, the gap between what a student can achieve on their own and what they can achieve with help. By focusing on individual time with students to scaffold their work, the researcher believed that instruction could be better tailored to students specific needs (e.g., working within their zone of proximal development), focus on those needing more help and support by providing additional time and attention, while more confident students could get research done during class time.

 

While this was the reasoning behind the original changes made to the ILI sessions, the assumption being made was that independent time to work would be beneficial for students. The author received challenges to this assumption by colleagues in the scholars program who wondered if students were prepared for time to work independently and whether giving students an assignment that introduced concepts might be a more beneficial use of class time. While much research has been done into which method of delivering information literacy instruction (e.g., computer-assisted instruction vs. traditional instruction), fewer librarians have researched what specific aspects of a method of instruction make it beneficial.

 

Methods

 

Participants

 

Students from 10 ILI sessions were the participants in this study. The classes were at the first-, second-, and third-year level from the religious studies, psychology, and general education disciplines. Class sizes ranged from 20-30 students, meaning the participants were drawn from an overall sample of approximately 200-300 students. The classes focused on searching for sources for their assignment, basic search strategies (Boolean operators, truncation, and phrase searching) and database searching. In two sections of a third-year psychology course, students were taught to use MeSH. The inclusion criteria for the study were that students were attending an ILI session that was 80 minutes or longer, as 50-minute sessions were too short to include a Web survey. All students in ILI sessions were asked to perform the same tasks, fill out a research log during independent searching, and to complete a Web-based survey at the end of class. All students responses were examined to inform pedagogy and student learning. Only the data of those students who agreed to participate were included in the study. In total, 73 students agreed to include their Web survey in the study and 83 students agreed to include their research log in the study. This study received approval from the Human Research Ethics Board at Mount Royal University.

 

Web Survey

 

This quantitative study included both a Web survey and research logs. To examine the research question, the researcher designed a Web survey and research log. Section 1 of the Web survey asked whether students had attended a previous ILI session (and if so, how many), trying to gauge students prior experience (see Appendix A). Section 2 of the Web survey was designed to determine the specific aspects of the ILI session the different pedagogical tools used in the classroom that students perceived to be most useful by comparing one aspect to another. Students were asked to rate the activities that took place during the class from most useful (1) to least useful (9). Finally, in Section 3, students were also asked to rate their preparedness before and after the ILI session on a four-point Likert scale. The research referred students to the Web survey URL via the online subject guide for the class. The Web survey was administered at the end of the ILI session, taking approximately five minutes to complete.

 

Research Logs

 

The research logs attempted to uncover students information behaviours which resources they use, how they search, how they modify their searches, what they think about their searching (see Appendix B). The research logs were examined to determine if students used the skills taught in class during independent searching. Other research has described research journals as part of an ongoing research process throughout a class (Smith, 2001; Warner, 2003). The research log used in this study is intended to capture students searching at a particular time and to aid students in recording their search process, similar to Kuhlthaus search logs (2004, pp. 32-33). Bates Berrypicking model (1989) and Kuhlthaus Information Search Process model (2005) were used in the creation of the research log to help students track their search progression and the change in their thinking that leads to search modification.

 

During the independent search time, students were asked to record their work. The research log was on carbonless paper; students kept the top copy while the author kept the bottom. It is important for students to keep track of their searches to understand where they have searched, to understand what they have searched, and to examine how their research might progress. In addition to being a form for data collection, the researcher employed the research log as a pedagogical tool to try to increase students awareness of their searching. The research log was used in preference to computer logs, which do not involve student thought. Students were asked to record their topic/thesis statement and ideas/concepts related to their topic. Students were also asked to record their searches: the date, the resource searched, the search query, what was found, and notes to self. If students asked questions during the ILI session, they were also asked to record their questions: what the question related to, if their question was answered, and what questions they felt might come up later.


Analysis

 

The researcher analyzed the Web surveys using descriptive statistics (frequencies and percentages), which were chosen to summarize the responses from the students in the sample and to provide basic information about the responses. The researcher examined research logs to determine if there was evidence of what took place during the independent search time, while recorded searches were examined to determine where students searched, the search queries students created, the search strategies students used, the ways in which searches changed, and what students wrote about their searching. From looking at how students modified their searches and what they wrote about their searches, the researcher developed categories to describe the commonalities seen. After developing operational definitions for the categories, the author categorized the searches. The categories were: search complexity (searches with two or more terms or use of specific search strategies), search progression (series of searches in which students increase the complexity or precision of their searches), use of feedback (series of searches in which students use the results of previous searches to modify or improve searches), and mode of search modification (incremental modifications or jumps from one strategy to another). The researcher used the categories to produce descriptive statistics about how students used their independent search time.

 

Results

 

Usefulness of ILI Sessions

 

The Web survey asked students to rate the usefulness of the different aspects of the ILI session from most useful (1) to least useful (9). Students rated individual help from the librarian as the most useful aspect of the ILI session with a rating of 3.89 (see Table 1). The second most highly rated aspect of the session was discussions of how to use resources (4.46). After the top two rated aspects of the session there was little variance between ratings. The author had hypothesized that students would rate time to work independently as the most valuable aspect of the ILI session. Instead, time to work independently was tied for the eighth most useful aspect of the session. Rather than simply valuing the time to work, students rated the one-on-one help they received during that independent search time as more useful. While this survey only asked for student perceptions of usefulness by comparing different pedagogical activities, students valued the scaffolding that took place in the session. As the results from the Web survey were too small to run a test for statistical significance, the ratings should be treated with caution and more research is needed to confirm these findings.

 


Table 1

Average Rating of Aspects of ILI Sessions from Most Useful (1) to Least Useful (9)

Answer Options

Rating Average

Individual help from librarian

3.89

Discussion of how to use the resources

4.46

Discussion about search difficulties

4.86

Citation discussion

4.93

Individual help from class instructor

4.94

Discussion of the resources to use

5.00

Time to work independently

5.23

Working/discussing with class mates

5.23

Explanation of the assignment

5.75

 

 

Table 2

Evidence, from Three Different Students, of Using What Was Taught in Class

Evidence of using what was taught in class

Use (%)

N=77

Example of evidence

Clear evidence

43 (56%)

Bipolar Disorder in MM, review articles, linked full text

Unclear evidence

21 (27%)

Bipolar Disorder, youth

No evidence

13 (17%)

Bipolar Disorder

 

 

The researcher examined the research logs to determine if students used what was taught in class. Of the 83 research logs included in the study, 77 (93%) contained recorded searches; 43 of those 77 (56%) showed clear evidence of using what had been taught, while 21 (27%) showed some evidence, and 13 (17%) showed no evidence (see Table 2). There is evidence that students understood the content of the lesson well enough for them to use it to search during independent searching within the context of the ILI session. (Table 2)

 

How Students Are Searching

 

The author also analyzed the research logs for evidence of how students search during the time given for independent searching. In total 237 searches were recorded, representing an average of 3.1 searches per research log (n=77). Searches averaged 3.7 words per query. Of the 77 research logs containing searches, 52 (68%) included use of Boolean operators, 23 (30%) included use of truncation, 19 (25%) used phrase searching, and 16 (21%) recorded the use of a search limit. Students used Boolean operators most frequently of the search strategies taught during the ILI session. Of those using Boolean operators, 1 (2%, n=52) used them incorrectly; for those using truncation, 4 (17%, n=23) used them incorrectly; and for phrase searching, 6 (32%, n=19) used them incorrectly (see Table 3). The percentage of incorrect uses for each of the search strategies may indicate that students are most comfortable using Boolean operators and least comfortable using phrase searching.

 

Table 3

Search Strategy Use and Examples of Incorrect Use

Search Strategy

Use (%)

N=77

Inappropriate Use (%), N

 

Example of Incorrect Use

Boolean

52 (68%)

1 (2%), n=52

Divorce and children and childhood

 

Truncation

23 (30%)

4 (17%), n=23

 

Immigration and poverty and Canada*

Phrase

19 (25%)

6 (32%), n=19

 

abuse and elder, specific to 65+1

1 Referring to database-specific age limit

 

 

Table 4

Search Complexity and Examples of Complexity

Search Complexity

Use (%) N=77

Example of search complexity

Complex search

34 (44%)

Globalization and relig* and identity

Not complex search

23 (36%)

Walmart and globalization

Elements of complexity

15 (19%)

Eat* local*

 

 

Searches were also rated on their complexity. A complex search had more than two ideas, or had two ideas in addition to employing specific search strategies Boolean operators, truncation, subject heading searches, etc. The author found that 34 of 77 (44%) research logs included complex searches, that 28 (36%) did not have complex searches, and that 15 (19%) had elements of complex searches but could not be fully categorized as complex (see Table 4). When looking at the number of students that used more than two ideas in their search and different search strategies, the author discovered that the overall searches were relatively simple. While different assignments required differing levels of search complexity, many of the topics students were exploring would retrieve results too great in number or lacking in precision.

 

The researcher also examined how searches were modified, whether in increments, by making small modifications to search strategies or an aspect of a term, or in jumps,

such as by changing vocabulary, topics, or resources entirely. Of the 60 research logs with multiple searches, 37 (62%) made search modifications using increments, 13 (22%) made search modifications using jumps, and 10 (17%) made search modifications using both. In making incremental modifications, students were searching in a more focused way, testing how small changes to a search will affect the search results. In making jumps in search query modifications, students were searching in a broader way, by exploring what is available or exploring their topic.

 

The researcher examined search modifications (changes made to search queries and/or resources in which the search was carried out over successive searches) to determine the specifics of how such changes were made: keywords used, resources used, and search techniques. Changing the keyword used was the most common modification, followed by adding or subtracting keywords, then changing the resource in which the search was carried out, followed by changing search techniques, such as using operators, truncation and/or phrase searching (see Table 5). Of 60 research logs with recorded search modifications, 19 (32%) revealed the use of multiple search modifications during the search. The data suggest that students view keyword terms as the primary way to change their searches.


 

Table 5

Types of Search Modifications by Number and Percentage of Research Logs

Search Modification

Number (%)

(n=60)*

Changing keywords

46 (77%)

Adding or subtracting keywords

37 (62%)

Changing resource used

22 (37%)

Putting on or taking off database limits

9 (15%)

Adding or subtracting Boolean operators

5 (8%)

Adding or subtracting truncation

3 (5%)

Adding or subtracting phrase searching

2 (3%)

Other

2 (3%)

*Multiple search modifications could be used in one research log.

 

 

Student Preparation to Search Independently

 

At the end of the ILI session students filled out the Web survey, rating their preparedness to do research before and after the session. Looking back, 41% of students rated their preparedness before the session as prepared or somewhat prepared, while 100% of students rated their preparedness after the session as prepared or somewhat prepared (n=73). The author examined research logs to see if this perception was corroborated in behaviour.

 

Examining changes in the research logs allowed the researcher to observe how students adapted their searching during time given in class to search independently. Searches were examined to determine if they showed progress, which was defined as a series of searches in which students increased the complexity or precision of their search. An example of a student search that demonstrates progression is shown in Table 6, while an example that does not demonstrate progression is found in Table 7. A total of 60 research logs contained more than one recorded search and were examined for search progression. Of those 60 research logs, 41 (68%) showed a progression while 19 (32%) showed no progression. That evidence of progression appeared in the majority of research logs with multiple searches indicates that students were able to use their independent search time to adaptively change their searches.

 

Effective changes must use feedback from the results of previous searches. Evidence of use of feedback was defined by a series of searches in which students used the results of previous searches to modify and improve their searches, as evidenced by discussion of changes in the Results or Notes to Self fields or the modification of search terms. Of the 60 searches that had multiple recorded searches, 29 (48%) showed evidence of use of feedback, while 31 (52%) showed no evidence. Approximately half of the students gave evidence that they used the previous searches to inform their subsequent search choices, indicating that students were learning while they used the independent time to search. A closer look at Table 6 and Table 7 reveals differing use of feedback.

 

Both tables show examples of two different research logs from the same class. The examples can be examined for of several types of searching: progression, use of feedback, demonstrating use of what was taught in class and complexity. In the class from which these research logs come, students were taught to use MeSH to search for neurological disorders. The student in Table 6 demonstrates progression through their use of the search terms: starting with the name of the disorder, checking it in MeSH, using the MeSH term as a major subject heading, and then continuing to add words and limits to the search until the student reaches what s/he determines to be a useful search. The student in Table 7 does not show progression. The second search used has more ideas, making it more complex; however, there is no indication of how the student arrived at the search or whether the student tried other searches that were more or less successful. The example in Table 6 demonstrates the use of feedback from previous results, making comments in the Notes to Self about how the search could be changed, which are then reflected in the searches and the resources found. The example in Table 7 demonstrates no use of changing based on previous results. The first example demonstrates what was taught in class subject headings, subheadings, and the use of limits while the second example indicates some use example has of the language discussed in class (e.g., etiology) but has used keywords in Medline and used no Boolean operators or phrase searching while in ScienceDirect. Also, the first example includes a complex search using many ideas and limits together, while the second example also has both a more and less complex search. The first recorded search is simple in query terms, though it might also include limits making it more complex, whereas the second search has more ideas included despite problems with the search construction.

 

 

Table 6

Example of Student Searching Demonstrating Progression in Searching

Resource Used

Search

Resources Found

Notes to Self

Medline

Guillain-Barre Syndrome

 

Overwhelming, 5,000+ articles

Use MeSH

Medline

(MM Guillain-Barre Syndrome)

 

1,800+ articles, but interesting subset headings

Advanced search etiology

Medline

(MM Guillain-Barre Syndrome) AND etiology

500+ articles

Advanced search check review articles

Medline

(MM Guillain-Barre Syndrome) and etiology + review articles

 

70 results. There are 7 solid articles I can use on first page

But there are

NO COPIES!!??

ScienceDirect

Advanced Review Articles, Title/Abstract/Keyword, Guillain-Barre Syndrome

72 results, 4-5 articles I can use

ScienceDirect actually has copies!

 

 

Table 7

Example of Student Searching not Demonstrating Progression in Searching

Resource Used

Search

Resources Found

Notes to Self

Medline

 

Narcolepsy and etiology keywords

Many results found reviewed, emailed

Link full text. Recent articles.

ScienceDirect

 

Narcolepsy sleep disorders REM sleep

e-mailed 8 articles for further review

Sleep disorders, etiology, sleep MRI, sleep EEG

 

 

The student whose search is represented in Table 6 demonstrates search progression, use of feedback, use of what was taught in class, as well as a complex search. The student whose search is represented in Table 7 partially demonstrates what was taught in class and a complex search. Without data triangulation through examining finished work it is not possible to determine if students recording of their searches was truly indicative of how they searched or of what they wanted to record for later use.

 

Discussion

 

Usefulness of ILI Sessions

 

The Web survey results indicate that students feel more prepared after the ILI session than before, and they rate the help from the librarian as the most useful part of the session. Counter to expectations, students rate the scaffolding they receive during the independent time to work much more highly than they rate the time they are given to work independently. Without further information about students perceptions and expectations, it is not possible to determine whether the ratings were due to perceived usefulness or prior expectations. Students may expect individualized help during ILI sessions. Overall, these results indicate that students do find ILI sessions useful in helping them to feel prepared to complete their research assignment. The data indicate that independent time to search is useful so far as it allows scaffolding to take place, as one-on-one help is viewed as more beneficial.

 

How Students Are Searching

 

Student searches were not very complex, with searches containing on average 3.7 words, and less than half of the research logs rated as having complex searches. Despite this, most students used Boolean operators and were able to use them appropriately. Students used truncation and phrase searching less frequently, and almost one-third of those using phrase searching could not use this technique appropriately. The majority of students are able to use, at least in part, what was taught in the ILI session. While this is encouraging, Cmor, Chan, and Kong (2010) found that while the majority of students could complete information literacy-related exercises in ILI sessions, few were able to demonstrate the ability to use new tools and search strategies or incorporate new knowledge into projects.

 

The researchers assumption that it is beneficial to give students hands-on time to search independently during the ILI session was partially substantiated. While students demonstrated that they could achieve many things during the time given to work independently, including formulating more complex and/or precise searches, using feedback to improve searches, and using what is taught in class, students most valued the individual help they received from the librarian. The research logs revealed several other aspects in which students could use instruction. For example, students searches were not very complex, and while not all search topics require complex searching, putting together a search strategy that increases both precision and recall is important. More instruction on search query formulation could be beneficial. In addition, more instruction about truncation and phrase searching could be beneficial; the number of incorrect uses indicates that students may not know what phrase searching will do.

 

Student Preparation to Search Independently

 

The majority of students who performed multiple searches showed evidence of search progression. This indicates that students are able, even within the course of an ILI session, to increase the complexity or precision of their searches. That fact that students are able to demonstrate this during class suggests that they may do the same in the searching that they engage in on their own time. Also encouraging was that almost half the student research logs contained multiple searches, evidence which suggests that students were using feedback to modify their searches, something not explicitly taught during ILI sessions. Additionally, the fact that students use feedback to make search modifications also lines up with the incremental changes seen in most research logs. Vocabulary and resources are the most commonly used ways to change searches, with little experimentation of search techniques. While changing terms is one of the best ways to modify a search query, students are not modifying search queries using search techniques, a topic that does not receives much attention during the ILI session.

 

Limitations

 

There were several limitations to this study, many of which stem from incorporating research into one-shot ILI sessions, which are particularly constrained. As the Web survey was completed at the end of the session, it had a lower response rate than the research logs, and the timing of the survey may also impact the results. This lower response rate may have been due to students rushing to finish class, which may also have impacted the survey results if students guessed at answers or chose answers which appeared first. In addition, the survey asked for usefulness of pedagogical tools in comparison with one another, making it more difficult to assess how these tools impacted student learning. The survey asked about the general preparedness of students, rather than preparedness related to specific tasks, which may have been less sensitive to differences in students levels of preparation and could have been affected by student interpretations of the question.

 

All data in the study were recorded by students. From in-class observations it was clear that some students were doing more searches than they were recording. From markings on the carbonless paper research logs it was clear that some students were also recording searches other places. Some students may have found recording their searches onerous. Some students might have difficulties performing the searches and making accurate recordings. This may have been the case particularly for students with less search experience, as recording searches adds another task and could increase the mental effort required to complete the work. While recording searches with research logs may have made the task more difficult, it has potential benefits as a pedagogical tool to help students think about their search process.

 

In addition to issues around students recording, the process of data analysis added limitations to the study. The researcher examined research logs together, by collapsing the classes into one group for comparison. Collapsing the classes has potential validity issues, though it was the most appropriate way to analyze the data collected. Since the research logs did not capture demographic data, other than class, and since the number of participants for each class was low, this means that group divisions were not meaningful. Further to issues of validity and reliability, the author categorized the research logs. In the future, to increase reliability and validity of the findings, more than one person should categorize the data.

 

Originally, the research project included an additional component to help triangulate the data, to gain further insight into student behaviours and to address whether the ILI had impact beyond the classroom. Those participants who completed the Web survey were asked if they would be interested in being contacted for a follow-up interview. Five participants indicated interest; however, only one participant took part in the semi-structured interview. Because of this, data collection was limited to the classroom and no follow-up information could be gathered. This lack of follow-up data means the results are limited to the quantitative results, and understanding reasons for participant behaviour is limited.

 

Implications for Future Practice and Research

 

From the progression seen in students searches produced during independent search time, it appears that students are making good use of that time. From the students point of view, that individual help from the librarian is the most useful part of the ILI session. Students find that independent time to search, when combined with scaffolding tailored to students individual needs, is beneficial or that it meets their expectations of an ILI session. Based on this evidence the author will continue to provide time for students to work independently and provide help on a one-to-one basis. The Web survey should be given to more students to determine how prevalent the view that the most important aspect of the session is one-on-one help with the librarian, in addition to what expectations students have of the session.

 

Recording searches can be challenging, particularly for students who are less familiar with searching and whose cognitive processing space is being used in doing the actual searches. To improve future research, as well as to help students keep track of their search process, new ways for students to accurately keep track of their search process should be explored. Database features such as search history, citation management tools, or instruments could be used for this purpose.

 

The majority of students demonstrated in their research logs what they learned in class. However, students are not demonstrating some of the important aspects of the research process. The author hoped for more evidence of using feedback to improve searches and metacognition about the search process. It is difficult to know whether students are not engaging in these activities, or whether the limitations of the situation (little time, computer lab environment, pressure to get work done, research log limitations) contribute to what students do or not do during class time. In addition, these types of higher-level thinking skills are not explicitly taught during class. Students are able to demonstrate activities taught during the ILI lesson during time given to search independently. It is possible that explicitly teaching skills such as thinking metacognitively and how to use feedback could also elicit those behaviours during independent search time. Future research will explore metacognitive aspects of the research process, both what aspects students engage in and how metacognitive thinking can be enhanced.

 

While students demonstrate that they can use what was taught in class, it is unknown if students can take that learning beyond the classroom and some research (e.g., Cmor et al., 2010) suggests it is doubtful. In addition, it is difficult to know whether students can extrapolate their learning in class to a greater understanding of research as a process. Librarians may want students to engage in metacognitive thinking and learn that research in a complex process, part of which involves library research. However, in one class it is unrealistic to expect students to become information literate. One-shot ILI sessions, while they may be effective, should be only one part of an overall library instruction program (Webster & Rielly, 2003). Because librarians often have only one class, working with instructors who have the semester and with programs that set four-year curricula becomes more important. ILI sessions need to go beyond teaching skills and into authentic student learning, requiring collaboration with classroom faculty (Wakimoto, 2010). Integrating ILI into what is done at the class and curricula level is necessary for students growth in information literacy.

 

Conclusions

 

Simply by examining what students do during a single ILI session, librarians can learn a lot about their own teaching, student information behaviour, and student learning. Students report that information literacy instruction sessions help them feel more prepared to do research. More than just time to work independently, students indicate scaffolding, the one-to-one instruction from the librarian, as being valuable. Within class, students are able to demonstrate generally correct use of skills taught during the ILI session, though their search queries are rather simple. While ILI sessions are only one part of a larger plan for developing IL skills, students report finding them useful. Also, demonstrating the usefulness of ILI sessions is the fact that many recorded student searches show increasingly complex searches or the use of feedback from previous searches to create more precise search queries. While skills are of immediate importance to students doing research assignments, these are important to students learning throughout their degrees. If students are not already demonstrating these understandings in their searches, they should be the focus of instruction. Again, this instruction cannot take place in a one-shot session as they are complex and take time to develop. Integrating more into courses and curriculum is important if we wish our students to attain these skills.

 

Acknowledgements

 

Support and funding for this research were provided by the Institute for the Scholarship of Teaching and Learning at Mount Royal University. A special thank you to Dr. Richard Gale, Dr. Karen Manarin, Dr. Deb Bennett, Dr. Miriam Carey, and the 2010 Scholar cohort. This research was first presented as a paper at the 6th International Evidence Based Library and Information Practice (EBLIP6) Conference, Manchester, UK, June 2011.

 

 

References

 

Baker, L. (2006). Library instruction in the rearview mirror: A reflective look at the evolution of a first-year library program using evidence-based practice. College & Undergraduate Libraries, 13(2), 1-20. doi: 10.1300/J106v13n02_01

 

Bates, M. J. (1989). The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5), 407-424.

 

Bren, B., Hillemann, B., & Topp, V. (1998). Effectiveness of hands-on instruction of electronic resources. Research Strategies, 16(1), 41-51. doi:10.1016/S0734-3310(98)90005-2

 

Buhay, D., Best, L. A., & McGuire, K. (2010). The effectiveness of library instruction: Do student response systems (clickers) enhance learning? The Canadian Journal for the Scholarship of Teaching and Learning, 1(1). Retrieved 4 Dec. 2012 from http://ir.lib.uwo.ca/cjsotl_rcacea/vol1/iss1/5/

 

Cmor, D., Chan, A., & Kong, T. (2010). Course-integrated learning outcomes for library database searching: Three assessment points on the path of evidence. Evidence Based Library and Information Practice, 5(1), 64-81. Retrieved 4 Dec. 2012 from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/6509/6449

 

Dervin, B., & Reinhard, C. D. (2006). Researchers and practitioners talk about users and each other . Making user and audience studies matter paper 1. Information Research, 12(1). Retrieved 7 Dec. 2012 from http://informationr.net/ir/12-1/paper286.html

 

Fain, M. (2011). Assessing information literacy skills development in first year students: A multi-year study. The Journal of Academic Librarianship, 37(2), 109-119.

 

Griffiths, J. R., & Brophy, P. (2005). Student searching behavior and the web: Use of academic resources and Google. Library Trends, 53(4), 539-554.

 

Johnson, C. M., Anelli, C. M., Galbraith, B. J., & Green, K. A. (2011). Information literacy instruction and assessment in an honors college science fundamentals course. College & Research Libraries, 72(6), 533-547.

 

Koufogiannakis, D., & Wiebe, N. (2006). Effective methods for teaching information literacy skills to undergraduate students: A systematic review and meta-analysis. Evidence Based Library and Information Practice, 1(3), 3-43. Retrieved 4 Dec. 2012 from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/76/153

 

Kuhlthau, C. C. (2005). Kulhthaus information search process. In K. E. Fisher, S. Erdelez, & L. E. F. McKechnie (Eds.), Theories of information behavior. (pp. 230-234). Medford, NJ: Published for the American Society for Information Science and Technology by Information Today, Inc.

 

Kuhlthau, C. C. (2004). Seeking meaning: A process approach to library and information services. Westport, CT: Libraries Unlimited.

 

Larkin, M. J. (2008). Scaffolding. In N. J. Salkind (Ed.), Encyclopedia of educational psychology. (pp. 863-864). Thousand Oaks, CA: Sage Publications.

 

Lau, E. P., & Goh, D. H. L. (2006). In search of query patterns: A case study of a university OPAC. Information Processing & Management, 42, 1316-1329. doi:10.1016/j.ipm.2006.02.003

 

Marcus, S., & Beck, S. (2003). A library adventure: Comparing a treasure hunt with a traditional freshman orientation tour. College & Research Libraries, 64(1), 23-44.

 

Markey, K. (2007a). Twenty-five years of end-user searching, part 1: Research findings. Journal of the American Society for Information Science and Technology, 58(8), 1071-1081. doi:10.1002/asi.20462

 

Markey, K. (2007b). Twenty-five years of end-user searching, part 2: Future research directions. Journal of the American Society for Information Science and Technology, 58(8), 1082-1099. doi:10.1002/asi.20601

 

Mount Royal University. (n.d.a). Fast facts. Retrieved 5 Dec. 2012 from http://mtroyal.ca/AboutMountRoyal/FastFacts/index.htm

 

Mount Royal University. (n.d.b). Inspiring learning: Mount Royal University academic plan 2012-2017. Retrieved 5 Dec. 2012 from http://www.mtroyal.ca/wcm/groups/public/documents/pdf/academic_plan.pdf

 

Novotny, E. (2004). I dont think I click: A protocol analysis study of use of a library online catalog in the Internet age. College & Research Libraries, 65(6), 525-537.

 

Portmann, C. A., & Roush, A. J. (2004). Assessing the effects of library instruction. The Journal of Academic Librarianship, 30(6), 461-465. doi:10.1016/j.acalib.2004.07.004

 

Smith, T. G. (2001). Keeping track: Librarians, composition instructors, and student writers use the research journal. Research Strategies, 18, 21-28.

 

Tabatabai, D., & Shore, B. M. (2005). How experts and novices search the Web. Library & Information Science Research, 27(2), 222-248. doi:10.1016/j.lisr.2005.01.005

 

Urquhart, C., & Rowley, J. (2007). Understanding student information behavior in relation to electronic information services: Lessons from longitudinal monitoring and evaluation, part 2. Journal of the American Society for Information Science & Technology, 58(8), 1188-1197. doi:10.1002/asi.20562

 

Warner, D. A. (2003). Programmatic assessment: Turning process into practice by teaching for learning. The Journal of Academic Librarianship, 29(3), 169-176.

 

Wakimoto, D. (2010). Information literacy instruction assessment and improvement through evidence based practice: A mixed method study. Evidence Based Library and Information Practice, 5(1), 82-92. Retrieved 4 Dec. 2012 from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/6456/6447

 

Wang, P., Berry, M.W., &Yang, Y. (2003). Mining longitudinal Web queries: Trends and patterns. Journal of the American Society for Information Science and Technology, 54(8), 743-758. doi:10.1002/asi.10262

 

Warner, D. A. (2003). Programmatic assessment: Turning process into practice by teaching for learning. The Journal of Academic Librarianship, 29(3), 169-176.

 

Webster, J., & Rielly, L. (2003). A library instruction case study: Measuring success from multiple perspectives. Research Strategies, 19, 16-32. doi:10.1016/j.resstr.2003.11.001

 

Willson, R., & Given, L. M. (2010). The effect of spelling and retrieval system familiarity on search behavior in online public access catalogs: A mixed methods study. Journal of the American Society for Information Science & Technology, 61(12), 2461-2476. doi:10.1002/asi.21433


 

 

Appendix A

Web Survey

Study Title: The Impact of One-Shot Library Sessions on Student Research

[] I have read the study information and consent to have my Research Log used anonymously in Rebekah Willsons study, The Impact of One-Shot Library Sessions on Student Research

 

1.       Have you ever had a library session for any other class?

a.       [] Yes [] No [] Prefer not to respond

2.       If yes, how many sessions have you had?

 

3.       Please rate the following from most useful (1) to least useful (9)

a.       Explanation of the assignment

b.       Discussion of the resources to use

c.        Discussion of how to use the resources

d.       Time to work independently

e.        Working/discussing with class mates

f.         Individual help from librarian

g.       Individual help from class instructor

h.       Discussion about search difficulties

i.         Citation discussion

j.         Other

 

4.       How prepared did you feel to do research before this session?

a.       [] Unprepared [] Somewhat Unprepared [] Somewhat Prepared [] Prepared

5.       How prepared do you feel to do research after this session?

a.       [] Unprepared [] Somewhat Unprepared [] Somewhat Prepared [] Prepared

6.       Are you interested in possibly participating in an interview to follow up on this survey? If so, please open a new window or tab in your web browser and copy and paste the address below into your address bar. This will allow me to get your e-mail address to contact you without attaching your e-mail to this survey.

 

 

Appendix B

 

Study Title: The Impact of One-Shot Library Sessions on Student Research

[] I have read the study information and consent to have my Research Log used anonymously in Rebekah Willsons study, The Impact of One-Shot Library Sessions on Student Research

 

Research Log

Research logs allow you to keep track of your research the searches you have done, the resources you have used and to plan for what you need to do next. They also help prevent you from duplicating the work youve done.

 

Topic/Thesis Statement:

____________________________________________________________________________________________________________

____________________________________________________________________________________________________________

 

Ideas/Concepts (and synonyms and words related to ideas/concepts):

____________________________________________________________________________________________________________

____________________________________________________________________________________________________________

 

Todays Date

Resource Used (e.g. Academic Search Complete)

Keywords/Search (e.g. global warming and ocean* in keywords)

Resources Found (e.g. Good results, e-mailed Smith & Jones article)

Notes to Self (e.g. try synonyms for global warming)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Did you ask questions during the session? [] Yes [] No

What type of question(s) did you ask? [] My topic [] Vocabulary [] Articles [] Books [] Problems Searching

[] Technical Problem

Did your question get answered? [] Yes [] No

Do you still have questions? [] Yes [] No

What questions do you think may come up later?

____________________________________________________________________________________________________________

 

 

 

 




Evidence Based Library and Information Practice (EBLIP) | EBLIP on Twitter