Linking Information Seeking Patterns with Purpose, Use, Value, and Return On Investment of Academic Library Journals
Donald W. King
Honorary University Professor
Smithfield, Rhode Island, United States of America
University of Tennessee
Knoxville, Tennessee, United States of America
University of Tennessee
Knoxville, Tennessee, United States of America
2013 King and Tenopir. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.
Objective – To demonstrate the power of the critical incident method in studying the information seeking patterns of university faculty.
Methods – Faculty at five U.S. universities participated in a study concerning their information seeking and reading patterns involving scholarly journals. The surveys relied on a critical incident method of asking questions concerning the last journal article read. This method allows analysis of the relationships among the purposes of reading articles, ways in which faculty first learned about the articles, where they obtained them, aspects of their use, and the value or impact of the information read.
Results – Results show that journal articles were by far the most used source of the last substantive piece of information used for work. Over half of article readings were from articles provided by libraries (52%, compared with 32.6% from personal subscriptions), and journal articles were the most frequent way faculty became aware of information prior to reading about it (33.9%, compared with 19.4% from informal discussions).
Conclusion – This project has shown that articles read for the purpose of research, found by searching, and obtained from the library collections have the highest value to faculty by many measures. Library provided articles save faculty time and effort, which can be quantified using contingent valuation. The return on investment (ROI) for library collections can be calculated by measuring all library costs and establishing the monetary returns to faculty members through contingent valuation. Library journal collections are estimated to have an ROI of between 3.3 and 3.6 to 1.
This paper is based on data gathered from an Institute of Museum and Library Services (IMLS) sponsored study “MAXDATA” involving surveys of university faculty concerning their use of academic library scholarly journal collections. The emphasis of this paper is to demonstrate the relationship of how purposes of reading scholarly journals (e.g., research, teaching, current awareness, etc.) lead to the information seeking patterns used by them (e.g., how they identify articles that are read, where they obtain them, etc.), which dictates certain aspects of use (e.g., how much is read, age of articles read, format of the articles, etc.), which is related to the positive outcomes or value of reading (e.g., increased productivity, improved research or teaching, saving readers’ time or money, etc.), which serves as return components of the ROI of academic library journal collections.
The paper gives an example of how the method of observing a critical incident of the last reading can be used to determine the above relationships. Estimates are made for the number of readings made for research; of these, the number identified from searching; and of these, the number obtained from library collections, the age of the readings and whether from print or electronic versions of the articles, the amount of time spent obtaining and reading the articles (as an indicator of what readers “pay” for the content read), the ways in which the reading affected research (whether the article read is eventually cited), and the “contingent value” of the articles read from the library collection.
Contingent value is an economic method used to assess the benefits of non-priced goods and services, by examining the implications of not having that product or service. In the example presented here, an estimate is made of how much more it would cost readers to obtain the article if there were no library collection. This value is compared to the relative cost of the library collection and the cost to the reader to estimate the ROI. Of course, this is only one way to do so. The University of Tennessee and other participants are currently conducting another IMLS study (LIBVALUE) to develop additional measures of “value” and “ROI” of all academic library services, in addition to those provided by their journal collections.
Over the years there have been hundreds of studies that provide estimates of the value of all types of libraries and more recently on the ROI in libraries. The University of Tennessee, School of Information Sciences and the U.K. research team CIBER (www.ciber-research.eu) were funded by the Institute of Museum and Library Services to examine how to maximize library investments in digital collections through better data gathering and analysis. The study focused on alternative means of collecting journal and article usage metrics including readership surveys, local server hits and downloads, data provided by various vendors, and deep log analysis of electronic journal usage data collected by CIBER from OhioLINK. The findings were intended to enable librarians to learn what conclusions can be drawn from each metric, the strengths and weaknesses of each one, how they complement one another, and what conclusions can be drawn if only one source of data is available.
This paper addresses the survey of faculty at five US universities that participated in the study concerning their information seeking and reading patterns involving scholarly journals (King, Tenopir, Choemprayong, & Wu, 2009; Tenopir, King, Edwards, & Wu, 2009). The surveys relied on a “critical incident” method of asking questions concerning the last journal article reading. This method allows analysis of the direct relationship of faculty purposes of reading articles, ways in which they first learned about the articles, where they obtained them, aspects of their use, and the value or impact of the information read. This information is used to establish the role academic libraries play in achieving value from their collections and, ultimately, a value contribution to the return component of journal collection ROI.
Examples of value include “purchase” value in terms of how much readers pay for the information in their time and money to obtain and read articles and the “use” value in the consequences of reading the information such as saving readers’ time in doing their work, improving their productivity, inspiring new thinking or ideas, improving their work, resulting in collaborations, and so on. One indicator of the value of journal collections is how much more it would cost readers to obtain the same information, if the journal collections were not available to them. The investment component of ROI is based on the relevant library cost and the cost to faculty in their time and money for browsing, searching, printing, and photocopying (King, Aerni, Brody, Herbision, & Kohberge, 2004a; King, Aerni, Brody, Herbision & Knapp, 2004b). A current IMLS sponsored study is developing additional measures of “value” and “ROI” of all academic library services to all who benefit from these services in a variety of ways.
Context is given for the role academic library journal collections play in achieving value. It is emphasized that it is “information content” that achieves value from reading and not the journals or articles, and that academic library services facilitate access to the information content in various ways. One aspect of this context is that faculty use many information sources to do their work and journal articles are collectively only one such source. A second context is that readers can obtain articles they read from many article sources such as personal subscriptions, article reprints, colleagues, authors, free web journals, and so on – library collections serve as only one of these article sources. A third context is that article information is often known (or partially known) before an article is read, and since the surveys focus on the last article read, it is possible that the reading is only the most recent of many past readings of the article by a faculty member. These contexts for assessing academic scholarly journal collections are discussed in the next section.
A section of this paper is also devoted to the purposes for which information is read by faculty, including research, teaching, writing, keeping up with new information, continuing education, and so on. Information seeking patterns are closely related to those purposes and form the focus of that section. Such patterns include ways in which readers learn about the articles they read (e.g., browsing, searching, being told, etc.) and the article sources used (e.g., personal subscriptions, library collections, etc.). An example is given in which articles primarily read for research are identified through searching and subsequently obtained from library collections.
The section following purposes for reading journal articles discusses various aspects of the use of article information, such as age and format of the articles. These aspects are given for those articles read for research, which have been found by searching and obtained from library collections.
The section following that deals with the outcomes or value of information found in journal articles. An example is given for value of information read for research, identified by searching, obtained from library collections, and the age of the articles. Finally, the collection ROI achieved for universities is estimated for these articles and compared with other reasons for reading, information seeking methods, and use aspects.
The Contexts for Assessing Academic Scholarly Journal Collections
In the broadest context, one can think of all activities in research communication. For example, various authors have described the flow of research information from discovery through oral or written reports over time, with journal articles being somewhat down the chain in time. (Garvey, Lin, & Nelsen, 1970; Crawford, Hurd, & Weller, 1996).
One can think of information as being a
resource that faculty use to perform their work and, for that matter, the
principal output from their work. Faculty can choose from many information
sources such as journal articles, books, personal contact, and so on. The
survey asked a question about sources used by faculty, which provides an
indicator of the relative importance of such sources. The question asked: “What
sources did you use for the last
substantive piece of information you used for work?” (King et al., 2009 shows figures; figures are also included in the 2010 Library Assessment Conference Proceedings version of this paper). The results demonstrate the relative importance of information found in articles compared with other sources of information and supports a reason for examining journal articles further.
There are a number of sources of journal articles that are read, including personal subscriptions; library collections (e.g., central and department library subscriptions/databases, interlibrary loan or document delivery services); preprint or reprint copies; copies from colleagues, authors, etc.; and repositories. It is useful to know the relative use of library collections in order to assess the importance of them and to examine why one source is chosen over others. As shown later, one aspect is purpose of reading and another aspect is the way articles are identified. When looking at the sources used to obtain the last article read by faculty, the results show that library provided articles are found to be the prevalent source (King et al., 2009).
Another context addresses the fact that a scholar often knows about the information in an article prior to reading it the first time. The surveys revealed that nearly half of the articles last read contained at least some information previously reported (King et al., 2009). There are many possible reasons that an article is read even though some of the information is known. For example, the reader might have heard about it at a conference and then waited for it to be edited and sent by a publisher to be refereed. The article might have been cited in another article in which case only some information found in the current reading is revealed.
It is abundantly clear that information found in articles is important to faculty work, libraries are used most often as a source of articles, and information cited in journal articles frequently leads to the entire information in articles being read. The importance of journal articles is well established.
Purposes for Reading Journal Articles
Information in journal articles is read for many purposes. Survey respondents were asked: “For what principal purpose did you use, or plan to use, the information obtained from the last article you read? (Choose only the one best answer)” (King et al., 2009). Research is the purpose most often given by faculty. Faculty average 240 readings annually so that about 114 readings a year are addressed to research in their work. Below the information seeking patterns used to obtain those 114 readings are described, as well as how these patterns compare with other purposes for reading.
It is shown that the purposes for reading determine to a large degree how readers become aware of the articles and where they obtain them. A detailed example is given for articles that are read for research, identified from searching, and obtained from libraries to show how the critical incident of the last reading can be applied.
In examining how faculty became aware of the article read for research, we see about an equal proportion of readings are found by browsing or searching and less from citations or another person. The question was worded as follows: How did you become aware of this last article you read? Specific options to the response “Found while I (or someone on my behalf) was searching (i.e., by subject or author’s name)” included: web search engine, electronic indexing/abstracting service, print index or abstract, online journal collection, current awareness service, and preprint/e-print service. The estimated number of readings done for research and found by searching is about 34 of the total 240 readings. Browsing is used much more for teaching (44.1% vs. 30.9% of readings for research) and current awareness (64.7%). Articles found through citations are used much less for teaching (12.2% vs. 21.2% of readings for research) and current awareness (only 2.5% of these readings).
About 42.3% of readings for research and found by searching involves information discussed in the article that is previously known. This compares with 56.9% of readings done for other purposes. Thus, even when information is known it is necessary to search at some level for many articles read for research, but less than readings for other purposes (56.9% of these readings). Faculty were asked if they had previously read the article. Answers did not vary much by purpose of reading and method of identifying the article. That is, about 17.0% of readings for research and found by search were re-readings compared with 19.6% of all other readings.
Across all readings (i.e., 240 annual readings per faculty), faculty tend to obtain articles most frequently from library provided sources (52.0% of readings) or personal subscriptions (32.6%) (King et al., 2009). These article sources depend a great deal on the purpose of reading and how the articles were initially identified.
Nearly two-thirds of articles read for research are obtained from libraries (compared with 52.0% for all readings). The 114 readings done for research are often found by browsing (30.9% of these readings). However, they are also often identified through searching (26.7% of these readings), citations (15.8%) and occasionally being told by another person (less than 17.6% since that person may do this by providing a copy of the article). In these instances and, sometimes when found by browsing, the readers must look for a place to obtain the articles. Most (76.9%) of the readings for research and found by search were obtained from a library provided article source (King et al., 2009).
These articles tend to be older and, therefore, are more difficult to obtain, which is why libraries play an important role. In the next section, we show that article use aspects such as age and format are dependent on information seeking patterns, particularly on how articles are identified and where they are obtained. Later it is shown that value of information read is also dependent on the purpose of reading and information seeking patterns.
Aspects of the Use of Article Information
The average age of articles read is 4.1 years old with the distribution of age being highly skewed, much like a nuclear decay curve. About half of article readings are made in the first year following publication or posting, but 2.8% of the readings are over 25 years old and 2 readings observed in the surveys were published in 1943 and 1947. The age of articles read depends a great deal on how the articles were identified and where they were obtained. For example, the average age of the articles found by browsing is 1.8 years, but much higher for those found by searching (4.7 years), from citations (8.0 years) and those mentioned by another person (3.5 years). Citations appear to identify particularly old articles, and contribute to 25.6% of readings from citations over 10 years old.
Browsing is done from all article sources: 51.7% of browsed articles are from personal subscriptions, 37.6% from library provided articles, and 10.7% from other articles sources. The average age of readings from personal subscriptions is 1.9 years, which suggests that the age of browsed articles from other article sources must also be low since the overall average age from browsing is 1.8 years. On the other hand, articles identified by searching and from citations largely come from library provided sources, which have an average age of 4.8 years. The average age of articles identified by searching is 4.7 years and three-fourths of these are provided by libraries (only 7.8% from personal subscriptions and 17.2% from other article sources). Similarly, 61.9% of cited articles are obtained from libraries and average 4.1 years old. It seems clear that there is a strong age of reading relationship between the way read articles are identified and the article sources used to obtain them.
About 24 of the 240 average readings are for research purposes, identified by search and obtained from libraries. The age of these articles becomes even older at 6.2 years (compared with 4.0 years for the rest of the readings). The information from them is shown later to have greater value to the readers.
Another aspect of use of articles is the format of the articles when read, which is also somewhat dependent on information seeking patterns. For example, across all readings about 54.4% are from electronic formats with 62.5% of readings for research from electronic formats. Only 12.6% of articles obtained from personal subscriptions are in electronic format while 71.2% of library provided articles are electronic and 68.5% from other article sources are as well. It is clear that libraries are a major source of electronic journal articles and provide a substantial number of readings in this format (82 of 240 average readings by faculty). Very few of the library provided readings take place in the library (5.4% of readings) and most of these are from print versions (84.2%), largely from browsed journals in periodical rooms. Average age of print journals is 4.4 years and electronic journals 3.9 years, where most of the difference is in articles over 10 years old. About three-fourths of articles read for research, identified by search and obtained from libraries are electronic compared with 51.5% for all other readings.
The purpose of readings, information seeking patterns, and aspects of use all have a bearing on the value of information read in articles. In turn, the value is the return component of the ROI of academic library journal collections.
The Value of Information Provided by Journal Articles
For this paper we differentiate value as:
- Purchase or exchange value: What one is willing to pay in time and/or money for information found in journal articles.
- Use value: The favorable consequences derived from reading and using the information. (Machlup, 1979)
A paradox that demonstrates the above concepts is that gems have high purchase value but low use value. On the other hand, air has low purchase value but high use value. Thus, the use value of information in articles generally is much higher than the purchase value.
The purchase value of journal information
Faculty pay for journal information through their time (and effort) in information seeking and reading journal articles and the price paid for personal subscriptions. Faculty on average spend about 150 hours per year in information seeking and reading, or an average of 37.5 minutes per reading (based on 240 readings per year). This sub-divides into 33.1 minutes reading and 4.4 minutes in information seeking.
The reading time is a good indication of the value of articles since readers would not choose to use this valuable time if the information was not of value to them. The reading time is related to purpose of reading, source of articles read, age of articles, and format of articles as discussed below. For example, the average time spent reading for various purposes has been documented (King et al., 2009). The information used for research and writing has greater purchase value than that used for other purposes such as teaching and current awareness.
Considering the source of articles, library provided articles average 35.4 minutes per reading compared with personal subscriptions (27.9 minutes), and other sources (34.0 minutes), which suggests that library provided articles have greater purchase value. Older articles tend to have greater purchase value (30.1 minutes for articles one year old, and 41.1 minutes for those over 5 years old). Print and electronic versions of articles have about the same value (32.9 and 33.4 minutes respectively). Both personal subscriptions and library provided articles gain value the older they are. That is, personal subscriptions go from 26.8 minutes for 1-year-old articles to 38.5 minutes for those over 5 years old and library provided articles increase from 31.8 to 42.8 minutes. The time spent reading articles for research found by searching and obtained from libraries is 39.1 minutes, which suggests these articles have greater value.
Interestingly, the average time spent browsing per article read is greater than that spent searching (6.9 vs. 5.3 minutes respectively). Generally, browsing from electronic sources takes less time than browsing from print versions (6.7 vs. 7.1 minutes respectively). This is affected by the fact that browsing electronic personal subscriptions takes more time than browsing print copies (7.6 vs. 6.8 minutes) and, on the other hand, browsing electronic library subscriptions is somewhat less time consuming (6.4 vs. 8.0 minutes).
Use value of information found in articles
In a sense the purposes of reading are an indication of the value of information in articles, but a better set of indicators is the outcome or consequences of reading the information. Survey respondents were asked: “In what ways did the reading of the article affect the principal purpose? (choose all that apply).” Outcomes vary by purpose of reading, how articles are discovered, and source of articles. It appears that articles read for the principal purpose of research, which were identified from searching and obtained from the library, yield greater value than the other readings in terms of outcomes. Faculty were also asked: “How important is the information contained in this article to achieving your principal purpose.” Coded responses are that the information is not at all important, somewhat important, or absolutely essential. Across all readings, 37.6% were said to be absolutely essential compared to 45.8% of the research readings, presenting further evidence of the value of the information to research. Additional evidence is that information from more readings done for the purpose of research were said to be cited in a paper or report. Over half (51.4%) of articles read for research are likely to be cited, while 25.2% may be cited, and 21.5% already have been cited. In contrast, about half of readings for other purposes will never be cited.
In the past two years, faculty averaged being an author or co-author of 3.41 articles in refereed scholarly journals; 1.21 non-refereed articles; 1.07 chapters in books, proceedings, etc.; and 0.11 books. Only 14.3% of faculty had not authored any publication. Most faculty (74.4%) had authored at least 1 article. Another indicator of value of article information includes the productivity given to readers by authorship. It is observed that those who read more are more likely to publish. For example, those who published a refereed article in the past 2 years read an average of 28.2 articles in the past month compared with 22.7 readings for those who did not publish.
Contingent valuation is an economic method used to assess the benefits of non-priced goods and services by examining the implications of not having that product or service. An indicator of the value of the library journal collection is to estimate what it would cost readers in time and/or money to obtain the information read from the collection if there were no collection. This is found by asking the following multi-part question pertaining to analysis performed only on readings from library provided articles:
Thinking back to the source of the [last read] article, where would you obtain the information if that source were not available?
(a) I would not bother getting the information.
(b) I would obtain the information from another source. If (b) is checked:
In order to obtain the same information, if this source were not available, I would expect to spend _______ minutes of time and/or $ _______. (If the answer is zero, answer “0” instead of leaving blank)
Results are given below from a survey conducted at the University of Pittsburgh where there was an average of 125 readings from library provided articles (King et al., 2004a; King et al., 2004b). The faculty indicated they would look for alternative sources of information for 99 of these readings. They averaged 3.0 hours per year searching, 3.4 hours browsing and 6.4 hours in obtaining useful citations, and articles from elsewhere, as well as photocopying, downloading, and printing articles. That is, about 12.8 hours were taken to search, browse, and identify articles from citations, other persons, etc. At about an average $55 per hour in salaries, benefits, etc. the cost to faculty was $704. In addition it cost the library and other facilities about $65 per faculty in photocopying, downloading, and printing for a total of $769 per faculty member.
The cost of obtaining alternative sources of the information is 59 hours in time ($3,295) and $990 for subscriptions, travel, communications, etc. Therefore, the additional cost to readers is 46.2 hours of time (59-12.8 hours) or $2,641 and $425 in other costs ($990-$65) or a total of $3,466 per faculty member.
A similar analysis was done for readings used for research, found by searching, and obtained through library sources. Here there were about 26 readings in which faculty would seek alternative sources and spend 2.3 hours searching, about 1.5 hours downloading and printing and/or photocopying at $60 per faculty. The time spent going to alternative sources is 14.8 hours and $538 in other costs. The net cost of the alternatives is 11.0 hours (14.8-3.8 hours) at $605 and $478 in other costs ($538-$60) or a total of $1,083 per faculty.
Therefore, one indicator of the value of the library collection is that it saves faculty on average about $3,466 annually. When reading is done for research, found by searching and obtained from the library collection, the savings is about $1,083. These values can be considered a return dollar component of the ROI in the library collection along with other “value” components mentioned earlier.
Return on Investment in Academic Library Journal Collections
The university investment in the library journal collection includes all of the library costs of purchasing and maintaining print and e-journal collections (allocated to an appropriate amount of reading) and the cost to readers in their salaries, benefits, etc., and other costs. For readings obtained from the library collection the library cost is about $283 per faculty member, and the cost attributed to readers is $769 or $1,052 total investment by the university in that portion of the library collection. Therefore the ROI of library journal collections is $3,466 ÷ $1,052 or 3.3 to 1.
The library cost to serve reading done for research and found by searching is $75 per faculty member. This added to the cost to users of $228 results in an investment of $303. Therefore, the ROI is $1,083 ÷ $303 = 3.6 to 1.
This article shows that journal articles are by far the most used information source of the last substantive piece of information used for work (articles 92.4% of the time, books 51.5%). Over 50% of article readings are from articles provided by libraries (libraries 52.0%, personal subscriptions 32.6%), and journal articles are the most frequent way faculty became aware of information prior to reading about it (articles 33.9%, informal discussions 19.4%). It is clear that journal articles play an important role in faculty communication.
Scholarly articles are read by faculty for many purposes, including research, teaching, writing, and current awareness. They find articles in many ways, including browsing and searching, and they obtain articles from a variety of sources, including library collections and personal subscriptions. Readers benefit in many ways from using information contained in articles. Every article reading is unique in its combination of purpose, ways of identifying articles, sources used to obtain articles, and benefits gained from reading them. By using the critical incident observation of the last reading of articles, the complexity of article information seeking and reading patterns can be sorted out. The purpose of this article is to demonstrate the power of the critical incident method. It also serves as a baseline for future studies.
Studies of this kind can also be used by a variety of audiences, such as readers to know if their information seeking can be improved, publishers to understand their market better, and librarians to make decisions on how best to serve their users. This article focuses most on the library issues. There are several ways to use this information to measure the value of article reading and the value of library access.
Faculty members spend considerable amounts of their time on discovering, obtaining, and reading articles, thus demonstrating the value of libraries for their research and teaching. The outcomes of reading are an even better indicator of value in supporting the principal purpose of reading, and a majority of articles are said to be essential or important to the principal purpose.
This project has shown that articles read for the purpose of research, found by searching, and obtained from the library collections have the highest value to faculty by many measures. Library provided articles save faculty time and effort, which can be quantified using contingent valuation. If the library collections were not available, the cost in time and other costs to faculty would be high.
The ROI for library collections can be calculated by measuring all library costs and by establishing the monetary returns to faculty members through contingent valuation. Library journal collections are estimated to have a return of between 3.3 and 3.6 to 1.
Academic librarians may be tempted to ask, why does this all matter? Libraries need to consider ways to measure their value in order to prioritize services in an era of new possibilities and declining or steady-state resources. And in an era of accountability and competition, academic libraries need to employ multiple ways to demonstrate their value to their constituents and funders. Measuring the use and value of scholarly journal article collections is one finding of the studies reported here, but is just one piece of the value picture. The LibValue project continues on this work and examines ways to measure other library products and services (www.libvalue.org).
Crawford, S. Y., Hurd, J. M., & Weller, A. C. (1996). From print to electronic: The transformation of scientific communication. Medford, NJ: Information Today.
Garvey, W. D., Lin, N., & Nelsen, C. E. (1970). Communication in the physical and social sciences. Science, 170, 166-173.
King, D. W., Aerni, S., Brody, F., Herbision, M., & Kohberge, P. (2004a). Comparative cost of the University of Pittsburgh electronic and print library collections. Pittsburg, PA: The Sara Fine Institute for Interpersonal Behavior and Technology. Retrieved 7 June 2013 from http://web.utk.edu/~tenopir/research/pitts/Pitt_Cost_Final.pdf
King, D. W., Aerni, S., Brody, F., Herbision, M., & Knapp, A. (2004b). The use and outcomes of university library print and electronic collections. Pittsburgh, PA: The Sara Fine Institute for Interpersonal Behavior and Technology. Retrieved 7 June 2013 from http://web.utk.edu/~tenopir/research/pitts/Pitt_Use_Final.pdf
King, D. W., Tenopir, C., Choemprayong, S., & Wu, L. (2009). Scholarly journal information-seeking and reading patterns of faculty at five US universities. Learned Publishing, 22(2), 126-144. doi:10.1087/2009208
Machlup, F. (1993). Use, value, and benefits of knowledge. Science Communication, 14, 448-466. doi:10.1177/107554709301400407 (Original work published 1979)
Tenopir, C., King, D. W., Edwards, S. & Wu, L. (2009). Electronic journals and changes in scholarly article seeking and reading patterns. ASLIB Proceedings: New Information Perspectives, 61(1) 5-32. doi:10.1108/00012530910932267
Evidence Based Library and Information Practice (EBLIP) | EBLIP on Twitter