Research Article

 

A Comparison of Traditional Book Reviews and Amazon.com Book Reviews of Fiction Using a Content Analysis Approach

 

Christy Sich

Research & Instructional Librarian

Western University

London, ON, Canada

Email: csich@uwo.ca

 

Received: 29 June 2015  Accepted: 14 Feb. 2017

 

 

cc-ca_logo_xl 2017 Sich. This is an Open Access article distributed under the terms of the Creative CommonsAttributionNoncommercialShare Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

 

 

Abstract

 

Objective - This study compared the quality and helpfulness of traditional book review sources with the online user rating system in Amazon.com in order to determine if one mode is superior to the other and should be used by library selectors to assist in making purchasing decisions.

 

Methods - For this study, 228 reviews of 7 different novels were analyzed using a content analysis approach. Of these, 127 reviews came from traditional review sources and 101 reviews were published on Amazon.com.

 

Results - Using a checklist developed for this study, a significant difference in the quality of reviews was discovered. Reviews from traditional sources scored significantly higher than reviews from Amazon.com. The researcher also looked at review length. On average, Amazon.com reviews are shorter than reviews from traditional sources. Review rating—favourable, unfavourable, or mixed/neutral—also showed a lack of consistency between the two modes of reviews.

 

Conclusion - Although Amazon.com provides multiple reviews of a book on one convenient site, traditional sources of professionally written reviews would most likely save librarians more time in making purchasing decisions, given the higher quality of the review assessment.

 

Introduction

 

Collection development training often promotes the use of book reviews to assist librarians in choosing fiction for library collections. Book reviewing as a systematic evaluation of literature first began with the publication of the Journal des Savants in 1665 (Boaz, 1958). Traditional book review sources for libraries include such periodicals as Book Review Digest, Booklist, Book World, Kirkus Reviews, and Library Journal. Other traditional book review sources include Saturday Review, Observer, New York Times Book Review, and The New Yorker.

 

In recent years, with the decline of the newspaper industry, the place for book reviews has come into question. Many stand-alone publications have folded and book review sections have been amalgamated with other sections or they have been slashed completely (Ciabattari, 2011). At the same time, with the rise of the web, independent book reviewers have cropped up in various online manifestations including Google Books, Goodreads, LibraryThing and Amazon, just to name a few. These new online platforms are convenient and allow readers and librarians quick, point-of-need access to reviews.

 

Not surprisingly, those in the formal book reviewing industry scorn these types of reader review and ratings systems, criticizing reviews published on Amazon for being anonymous or sometimes even fraudulent with claims made that authors seek out friends and family to write glowing reviews (Harmon, 2004). Pool (2007) argues that “Amazon has created a system that not only allows but encourages ethical and literary standards far lower than those we find in print reviewing” (p. 100).

 

Still, despite the criticism from the book reviewing industry, Amazon seems like a quick and easy way for librarians to get a sense of a book’s quality and whether or not it might be worthy of selection. In order to test the validity of the anecdotal claims about the flaws with Amazon reviews, and to be able to recommend review sources as tools for collections librarians, this study aims to measure the quality of reviews found on Amazon and compares the results to the quality of reviews in traditional publications, using a content analysis approach. By examining the texts of the reviews, the researcher assessed the quality and helpfulness of reviews found on the Amazon.com website to test the claims about the poor quality of Amazon reviews.

 

In order to determine the quality, the author scored each individual review using a checklist of 12 categories developed for this study (see Appendix). The categories of analysis were adapted from John E. Drewry’s Writing Book Reviews (1974), using elements that make up a helpful review. The 12 categories that the researchers looked for in each review included character source, character treatment, plot elements, plot devices, plot summary, theme, setting, four categories of style, and the inclusion of an evaluative statement about the book. In this study, quality was determined by looking at the total helpfulness score. Results of the study will help librarians responsible for collections of fiction decide whether or not to make use of Amazon as a collections development tool.

 

Literature Review

 

The deluge of the number of books published is something that librarians and readers alike must grapple with. The impetus and context for this study was formed, in part, through an analysis of some of the literature on book reviewing. In particular, this research was motivated by the apparent anxiety of critics, researchers, and writers of reviews who have attempted to make a case for formal, professional book reviews by condemning the free, online book review sources for encouraging amateur book reviewing. The literature reviewed in this study can be organized thematically by the role or purpose of book reviews, authority and anonymity of reviews—especially in online formats and the quality of reviews.

 

Role or Purpose of the Book Review

 

There has been tension between critics and writers regarding the purpose of the book review from the early days of periodical publishing. In her essay, Reviewing (1939), Virginia Woolf unabashedly unleashed her contempt of book reviews and the critics. Woolf proposed a new system and questioned the purpose of reviews: “Why bother to write reviews or to read them or to quote them if in the end the reader must decide the question for himself?” (p. 12).

 

Interestingly, when Reviewing was published, Woolf’s husband contributed his own contradictory commentary, defending the role of reviews to some degree, by suggesting that with the burgeoning of readers and the number of books published that the function of the reviewer is “to give to readers a description of the book and an estimate of its quality in order that he may know whether or not it is the kind of book which he may want to read” (Woolf, 1939, p. 29).

 

Just as with readers needing some guidance on what books to buy and read, librarians responsible for building large collections of fiction in both academic libraries and public libraries can also benefit from tools to help make purchasing decisions, especially when faced with dwindling budgets. Some articles, including Natowitz and Carlo (1997), Palmer (1991), Burchette (1992), Greene and Spornick (1995), touch on the impact of reviews on collection development in libraries as well as on individual purchasing decisions. In particular, Natowitz and Carlo examined book reviews published in Choice, Journal of American History, and American Historical Review to determine the degree to which book reviews provide assessments to help with acquisition choices. Natowitz and Carlo’s research showed that different variables involved in book reviewing depend upon the journal the review is written for, making the case that awareness of different types of reviews in journals allows librarians to make informed collections decisions.

 

In reviewing the literature on book reviewing, Blake (1989) covered the role of the review extensively and noted that there is a connection between the number of book reviews of a single title and the number of library collections in which that title can be found. Blake indicated that research on the reviewing of non-print materials had largely been neglected at that time and was an area that needed further study. In another study, Palmer (1991) pointed to the inconsistent role that reviews play in collection development in libraries by demonstrating that reviews are relied upon heavily by some libraries and not at all by others.

 

Pinfold (2007) focused on book reviews as a tool for collections librarians, explaining that, despite the prevalence of approval plans whereby libraries automatically receive books based on pre-set parameters rather than reviews, he heavily relies on reviews to help with selection decisions. Pinfold also commented on reviews from Amazon, stressing that they should be ignored because of the lack of peer review.

 

Authority and Anonymity

 

As revealed in Pinfold’s paper, a major critique of book reviews from online platforms such as Amazon is that they lack authority and the peer review process. Ciabattari (2011) explored this further by discussing the resurrection of the book review, arguing that just as there is a proliferation of books being published each year, so too is there a proliferation of reviewers commenting on books in the online world.  Reliance on these reviews then is all the more difficult because of the amount of reviews available. Ciabattari notes that “readers can find book news and reviews in formats ranging from a hundred forty characters to six thousand words and up, online and in print. … Despite the flood of friendly recommendations coming from Amazon and the social networking sites, many readers still turn to familiar gatekeepers for curatorial guidance” (2011, p. 122).

 

Pool (2007) emphasized the lack of quality control in the free online world: “In self-published reviews on the Web … critical failings are and are bound to be exacerbated. … Unscreened, anonymous, and unedited, self-published reviewers can be—and often are—as biased, uninformed, ungrammatical, and critically illiterate as they like” (p. 122). To highlight the uncritical aspect of online reviews further, Harmon (2004) reported on a glitch that occurred in 2004 when Amazon reviewers’ real names were displayed for a short time, revealing that some authors had given their own works and works of friends glowing, five-star reviews.

 

Quality of Reviews

 

In a study on the sunny book review, Katz (1985) discusses the disproportionate number of positive reviews to negative reviews. Katz suggests that doubling the number of reviews produced with an emphasis on negative reviews would be more beneficial to librarians.

 

Some studies have focused on the evaluative aspects of reviews. Bilhartz (1984) looks at the changes over time by reviewing works about history and notes that certain periods seemed to produce more critical reviews than other periods. Authors before 1960 could expect uncritical praise, but by the sixties, a new breed of reviewers had arrived. They were not so accepting and who began to write much more critically.

 

Boaz (1958) similarly points out that the nature of book reviews changes over time but suggests that this change has to do with an increase of educated readers. Book reviews were quite scathing in the earlier days of reviewing, speaking to a small educated few. By the mid-Twentieth Century the number of educated readers increased and so too did the volume of book production. At this time, book reviews became more favourable as the book reviews served increasingly as a means of notifying readers of books and their contents. It was left to the reader or librarian to make the ultimate decision about reading or purchasing the work.

 

In a more recent study, Hartley, Cowan, Deeson, and Thomas (2016) looked at the quality of individuals' writing to see if any changes occur over time. The researchers compared the writing styles of 5 different academic writers of book reviews over a 20-year span to see if there were any changes in style over time. They used automated measures of spelling, grammar, and readability found in Microsoft Word 8. They found that, generally, each reviewer remained relatively consistent over time.

 

Interestingly, Witucke (1982) compares reviews from several traditional reviewing publications, examining various aspects, such as coverage and promptness, and concludes that selectors cannot rely on a single reviewing publication to get a comprehensive view of a given book.

 

Liu, Chen, and Chiu (2013) propose a book review recommendation system that scrapes the web for all available reviews and systematically presents the reviews to the user in a ranked order based on quality. In order to evaluate the quality of the reviews for this proposed system, the researchers discuss two approaches. One was to look at textual features of the reviews and the other was to look at non-textual features of reviews. Liu and co-researchers chose to focus their study on non-textual features. These included book review length, time factor, book rating, and reviewers' reputation as ways to measure the book review quality. This author would argue that there are some flawed assumptions built into the features used to score reviews. For instance, the study gives more credit to lengthier reviews, based on comments made from an earlier study (Jurca, Garcin, Talwar, & Faltings, 2010) that looked at product reviews and found that lengthier reviews were perceived by other users as being written by someone with more authority on the product. It is unclear as to how the length would necessarily contribute to the quality of a review. Similarly, the timeliness of the review was given a higher score stating that earlier reviews are more influential. Again, it is unclear how a review's date can indicate its level of quality.

 

Aims

 

This study looks at the textual features of reviews and provides an alternative approach to assessing the quality of reviews. The purpose is to assess the quality of the reviews found on Amazon.com and determine whether or not Amazon is an appropriate tool for librarians, who are responsible for building collections of fiction to use to inform purchase decisions. The study compares book reviews posted on Amazon.com with book reviews published in more traditional book review sources, such as periodicals and newspapers. The author hypothesized a significant difference between the quality of reviews published in traditional sources and reviews found on Amazon.com. Therefore, the null hypothesis is no significant difference between the quality of traditional reviews and the quality of Amazon reviews. This study was undertaken with several research questions in mind:

 

  • Is there a significant difference between the quality of reviews found in traditional reviews and reviews published on Amazon?  
  • Is there a difference in rating agreement across the two types of reviews?
  • Does one type of source have a higher number of favourable, neutral, or unfavourable reviews than the other?  
  • Is the average length of reviews from one type of source an indication of the quality of the reviews from that type of source?

 

To this author’s knowledge no studies have yet compared the quality and helpfulness of traditional book review sources with online user rating systems such as Amazon.com. A comparison of the two modes of reviews will help to clarify anecdotal claims that one mode may be superior to the other. This has the potential of assisting library selectors in identifying appropriate sources for reviews, saving time, and making the most of library collections budgets.

 

Methods

 

For this study, the author determined that analyzing reviews of the same titles to look at rating agreement across the different modes of reviews would be important. As such, randomly selecting reviews from each type of review source was not possible. Similarly, selecting titles at random was difficult because often not enough reviews of the randomly selected titles existed in either the traditional book review sources or on Amazon. Around the time of this study’s inception, the Lost Man Booker Prize was announced. The purpose of this special Booker prize was to acknowledge retrospectively novels that had been published in 1970 and had missed an opportunity to compete when the Booker Prize changed its rules in 1971 to look at works of fiction in the current year only. This seemed to be an opportunity to make use of a selected list of books that would most likely have been reviewed in the traditional review sources.

 

Two types of reviews were selected for analysis: published reviews that had been indexed in Book Review Index and Amazon.com reviews that were written, in order to avoid bias, prior (pre-2008) to the announcement of the Lost Man Booker Prize. This study used reviews from seven books long-listed for the Lost Man Booker Prize because they each had six or more reviews from each type of review source. Table 1 shows the books and number of reviews used in the study. In total, 127 reviews from traditional published review sources and 101 reviews from Amazon.com were analyzed.

 

 

Table 1

Books Reviewed

Title

Total number of reviews published in traditional sources

Total number of Amazon reviews

Bomber

22

11

Troubles

8

16

The Bay of Noon

22

6

I’m the King of the Castle

8

13

A Fairly Honourable Defeat

23

15

Fire from Heaven

24

34

The Driver’s Seat

20

6

Total

127

101

 

 

In order to assess the quality of each review, the review’s helpfulness was scored using a coded checklist developed for this study that involved 12 categories (see Appendix). The higher the helpfulness score, the greater the quality of the review, and the more helpful it is to a librarian making a purchase decision. The categories were developed and adapted from a list of recommended elements that reviewers should address when reviewing fiction from John E. Drewry’s Writing Book Reviews (1974). Written as a guide for review practitioners, Drewry's work was selected for this study because it was thorough and included a chapter on reviewing fiction. The goal of Drewry's book was to encourage competent reviewing. Drewry was a former Dean of the University of Georgia's School of Journalism and not only wrote reviews for various publications but also taught a course in book reviewing.

 

For this study, one point was given to a review for each of the defined categories that it addressed. The categories included character source (fictional or historical), character treatment (or character development), plot elements (how elements are handled, e.g., introduction, suspense, climax, conclusion), plot devices (e.g., catastrophe, accident, fate, mystery, sub-plots etc.), plot summary (the plot is described), theme (a comment about the overall theme is made), setting (a comment is made about the setting or background), four categories of style (clarity, sentence structure, emotional qualities, and narrator perspective), and inclusion of a statement of judgment about the book. The length (word count) of the review and the evaluative rating—whether the review was favorable, unfavorable or mixed/neutral—were also documented for analysis. The coders looked for specific words such as recommended or not recommended to determine a rating.

 

Best practices in content analysis encourage inter-coder reliability testing. To follow these principles of best practice and to establish objectivity in content analysis, two coders were used in the study. The author coded the entire sample of reviews, and a second coder, a librarian colleague, was enlisted and trained. The author and second coder met several times. In two training sessions, they coded together reviews not included in this study, using the checklist. Where they disagreed, they made changes to the checklist. They met a third time after independently coding 10 non-study reviews, and they tweaked the chart again.

 

The second coder ultimately coded a random sample of 50 of the 228 reviews using the checklist. Initially, the author supplied the coder with 10 reviews from the study and did a preliminary calculation of percent agreement to make sure that an acceptable level of agreement was being met. At this point, the coder then coded an additional 40 reviews from the study. A free, online program called ReCal was used to establish inter-coder reliability by calculating percent agreement. Cohen’s kappa was selected for this study, as it takes chance agreement into consideration. According to Lombard, Snyder-Duch, and Campenella-Brachen (2010), a coefficient of .70 agreement is appropriate for some studies because some indices, such as Cohen’s kappa, are known to be more conservative; therefore, lower criteria can be used. In this study, acceptable levels of agreement were achieved in 10 of the 12 categories (see Table 2). Additional training may have improved the agreement in the Theme and Clarity categories.

 

Results

 

Quality

 

The quality of a review was determined by how well a review scored on the helpfulness checklist (see Table 3). To compare traditional reviews with Amazon reviews, a two-sample t-test was performed that does not assume equal variances. The test revealed that the probability of variance is less than 0.05, therefore we can reject the null hypothesis, that there is no significant difference between the quality of traditional reviews and the quality of Amazon reviews. The mean score for helpfulness for the traditional reviews (M = 6.57, SD = 2.399, N = 127) is significantly higher than the scores for the Amazon reviews (M = 4.81, SD = 1.999, N = 101), using the sample t-test for unequal variances, t = 5.9, p < 0.0001.

 

 

Table 2

Inter-Coder Reliability Testing

Elements of a helpful review

Cohen’s kappa

Character source

.88

Character treatment

.89

Plot elements

.70

Plot devices

.78

Plot summary

.92

Theme

.60

Setting

.87

Sentence structure

.80

Clarity

.53

Emotional qualities

.70

Narrator perspective

1.0

Judgement

.81

 

 

Table 3

Review Helpfulness Score by Book

Title

Averagea score for traditional reviews

Averagea score for Amazon reviews

Bomber

7.0

4.8

Troubles

6.3

6.1

The Bay of Noon

7.2

5.7

I’m the King of the Castle

5.0

3.4

A Fairly Honourable Defeat

6.2

4.5

Fire from Heaven

6.1

4.4

The Driver’s Seat

7.3

6.5

Weighted average score

6.6

4.8

 

 

Review Length

 

A Pearson product-moment correlation coefficient was computed to assess the relationship between the length of a review and the quality of a review as measured by the helpfulness score. There was a positive correlation between the two variables, r = 0.326, n = 228, p = 0. Increases in review length were correlated with an increase in the helpfulness score. Length of review was determined by an individual word count. Interestingly, despite the unlimited space available on its online platform, Amazon reviews tend to be more succinct on average than traditional reviews. The average word count for traditional reviews was 394 and the average word count for Amazon reviews was 244. A t-test reveals that this difference is significant (p = 0.0089). There is a definite tendency toward shorter reviews on Amazon, with well over half being 300 words or less. The traditional reviews that were analyzed had a fairly even distribution of varying lengths. Some of the publications containing the traditional reviews reserved very little room for reviews, while others had lengthier essays.

 

Rating

 

Further research questions that were asked in this study include the following: Is there a difference in rating across the platforms? Does Amazon have a higher percentage of positive ratings than the traditional sources? On average, do the two types of sources agree on rating?

 

A chi-square test was performed to look at whether a difference exists between ratings across the two modes. A significant difference was noted between the number of unfavourable reviews and favourable reviews, depending on whether the review was from Amazon or a traditional source (see Tables 4 and 5). Traditional sources had more unfavourable reviews and fewer favourable reviews than expected. The opposite relationship was observed for reviews on Amazon, which had more favourable reviews and fewer unfavourable reviews than expected, No significant difference existed between the expected number of unfavourable and neutral/mixed reviews for either modes, Also, the sources had no significant difference between the expected number of favourable and mixed reviews,

 

Ranking each book by the highest percentage of favourable views shows disagreement between the two types of reviews. The Driver’s Seat had the highest percentage of favourable reviews on Amazon. Conversely, this novel had the lowest percentage of favourable reviews among the traditional review sources. Instead, The Bay of Noon received the highest percentage of favourable reviews in the traditional review sources. Interestingly, the novel that eventually won the Lost Man Booker Prize, Troubles, was given the second lowest percentage of favorable reviews by Amazon raters and had only the fifth highest number of favourable reviews among the traditional review sources.

 

 

Table 4

Amazon Review Ratings

Title

Number of reviews by rating

Total

Favourable

Mixed/neutral

Unfavourable

A Fairly Honourable Defeat

11 (73%)

3 (20%)

1 (7%)

15

Bomber

9 (82%)

2 (18%)

0 (0%)

11

Fire From Heaven

26 (76%)

6 (18%)

2 (6%)

34

I’m the King of the Castle

8 (62%)

4 (31%)

1 (8%)

13

The Bay of Noon

5 (83%)

1 (17%)

0 (0%)

6

The Driver’s Seat

6 (100%)

0 (0%)

0 (0%)

6

Troubles

12 (75%)

3 (19%)

1 (6%)

16

Total

77

19

5

101

 

 

Table 5

Traditional Review Ratings

Title

Number of reviews by rating

Total

Favourable

Mixed/neutral

Unfavourable

A Fairly Honourable Defeat

7 (30%)

9 (39%)

7 (30%)

23

Bomber

10 (45%)

6 (27%)

6 (27%)

22

Fire From Heaven

16 (67%)

6 (25%)

2 (8%)

24

I’m the King of the Castle

2 (25%)

5 (63%)

1 (13%)

8

The Bay of Noon

17 (77%)

0 (0%)

5 (23%)

22

The Driver’s Seat

4 (20%)

12 (60%)

4 (20%)

20

Troubles

2 (25%)

3 (38%)

3 (38%)

8

Total

58

41

28

127

 

 

Discussion

 

Although Amazon has many benefits as a review source and some quality reviews do exist, this study shows that the best quality reviews are found in traditional book review sources. Based on the results of this study, the null hypothesis that no significant difference exists between the quality of traditional reviews and the quality of Amazon reviews can be rejected. The findings of this study reveal that the mean score for helpfulness for traditional reviews is significantly higher that the score for the Amazon reviews; therefore, collections librarians would be better equipped for making purchasing decisions if they avoid Amazon reviews and read reviews found in the traditional sources, such as those indexed by Book Review Index. Although tracking down these reviews may take more time, the product is more helpful for making purchasing decisions than reviews found on Amazon.

 

Findings in this study also reveal a positive correlation between length of review with the quality of review. It makes sense that if 12 elements are needed for a high-quality score, a minimum length is necessary to achieve this. This study also found that despite having no limits on length in the online environment, the average word count of the Amazon reviews was significantly less than the average word count of the reviews from traditional sources.

 

One limitation of the study is that the traditional book review sources were written decades before the reviews on Amazon.com. As Bilhartz (1984) pointed out, times change, and so do the ways in which writers write reviews. A follow-up study that compares reviews on Amazon.com to online sources of contemporary reviews found in proprietary or subscribed sources would offer additional insight.

 

Conclusion

 

Amazon is a tempting source for librarians to use in book selection largely due to its ease of use and the plethora of reviews that are often available. With the significant difference in quality that was found in comparing the Amazon reviews to the available traditional reviews, this researcher would urge selectors to use Amazon in a limited fashion, and if used, used in conjunction with more traditional sources. As with other studies, this research shows that agreement is not consistent even across the traditional sources; therefore, no single source should be relied upon. This researcher would recommend that collections librarians seek out traditional sources of reviews that involve at least an editor in the publication process. Full text databases that index book reviews from many different traditional sources are a good alternative to Amazon.com.     

 

Acknowledgments

 

The author would like to thank Dan Sich for his assistance coding reviews for the interrater reliability testing, Ken Meadows for his assistance with the data analysis of the study, and Lise Doucette for reviewing a draft of this manuscript.

 

References

 

Bilhartz, T. D. (1984). In 500 words or less: Academic book reviewing in American history. History Teacher, 17(4), 525–536. Retrieved from https://eric.ed.gov/?id=EJ304820

 

Blake, V. L. P. (1989). The role of reviews and reviewing media in the selection process. Collection Management, 11(1-2), 1–40. http://dx.doi.org/10.1300/J105v11n01_01

 

Boaz, M. (1958). Some historical sidelights on book reviewing. In L. C. Merritt (Ed.), Reviews in library book selection (pp. 179-182). Detroit: Wayne State University Press.

 

Burchette, R. B. (1992). An examination of children’s book review media (Masters thesis). University of North Carolina at Chapel Hill, Chapel Hill.

 

Ciabattari, J. (2011). Back from the Dead: The state of book reviewing. Poets & Writers, 39(5), 121. Retrieved from https://www.pw.org/content/back_from_the_dead_the_state_of_book_reviewing_0?cmnt_all=1

 

Drewry, J. E. (1974). Writing book reviews. Westport, Conn.: Greenwood Press.

 

Greene, R. J., & Spornick, C. D. (1995). Favorable and unfavorable book reviews: A quantitative study. The Journal of Academic Librarianship, 21(Journal Article), 449–453. http://dx.doi.org/10.1016/0099-1333(95)90088-8

 

Harmon, A. (2004, Feb 14). Amazon glitch unmasks war of reviewers. The New York Times. Retrieved from http://www.nytimes.com/2004/02/14/us/amazon-glitch-unmasks-war-of-reviewers.html

 

Hartley, J., Cowan, J., Deeson, C., & Thomas, P. (2016). Book reviews in time. Scientometrics, 109(3), 2123–2128. http://dx.doi.org/10.1007/s11192-016-2114-z

 

Jurca, R., Garcin, F., Talwar, A., & Faltings, B. (2010). Reporting incentives and biases in online review forums. ACM Transactions on the Web, 4(2), 1–27. https://dx.doi.org/10.1145/1734200.1734202

 

Katz, B. (1985). The sunny book review. Technical Services Quarterly, 3(1-2), 17–25. http://dx.doi.org/10.1300/J124v03n01_03

 

Liu, D. R., Chen, W. H., & Chiu, P. H. (2013). Recommending quality book reviews from heterogeneous websites. Internet Research, 23(1), 27–46. http://dx.doi.org/10.1108/10662241311295764

 

Lombard, M., Snyder-Duch, J., & Campenella-Brachen, C. (2010). Practical Resources for Assessing and Reporting Intercoder Reliability in Content Analysis Research Projects. Retrieved from http://matthewlombard.com/reliability/

 

Natowitz, A., & Carlo, P. W. (1997). Evaluating review content for book selection: An analysis of American history reviews in Choice, American Historical Review, and Journal of American History. College & Research Libraries, 58(4), 323–336. http://dx.doi.org/10.5860/crl.58.4.322 

 

Palmer, J. W. (1991). Fiction selection in Ontario public libraries—How important are reviews? Public Library Quarterly, 10(4), 39–48. http://dx.doi.org/10.1300/J118v10n04_04

 

Pinfold, J. (2007). Reviews: A librarian’s view. African Research & Documentation, (102), 31–35.

 

Pool, G. (2007). Faint praise: The plight of book reviewing in America. Columbia, Mo.: University of Missouri Press.

 

Witucke, V. (1982). The performance of juvenile book review media. Serials Review, 8(1), 49–55. http://dx.doi.org/10.1016/0098-7913(82)90030-2

 

Woolf, V. (1939). Reviewing. London: Hogarth Press.

 

 

Appendix

 

 

Helpfulness Code

Category Descriptiona

Score

1

CHAR_SOURCE

Comment on sources of characters—made up? historical?

2

CHAR_TREATMENT

Comment on treatment of characters, comment on attitude of writer toward his characters, character development? simple/complex? comment on character personality

3

PLOT_ELEMENTS

Comment on how (one or more) elements of the plot are handled, e.g., introduction, conflict, suspense, climax, conclusion (not just mentioned), tension, action

4

PLOT_DEVICES

Comment on plot devices generally used (e.g., catastrophe, accident, fate, mystery, sub-plots, journey), is plot primary or secondary? (letters, diaries, flashbacks, etc.)

 

5

PLOT_SUMMARY

Plot is described or outlined

 

6

THEME

Comment on overall theme, over-arching ideas in the novel, novel’s message, or purpose of the novel, e.g., betrayal, love conquers all, good vs. evil

 

7

SETTING

Comment on setting (historical/local, occupational/institutional, ethereal/esoteric), background, atmosphere, locale, scenic effects

 

8

STYLE_ELEMENTS

Comment on elements of style, e.g., words, prose, language used, figures of speech, sentence structure, paragraphs, tone, allusions, metaphors, symbolism, pace, aphorisms, truisms

 

9

STYLE_INTELLECT

Comment on intellectual qualities such as simplicity (e.g., written for children) or clearness of writing (e.g., triteness, clichés, satire)

 

10

STYLE_EMOTIONAL

Comment on emotional qualities, e.g., pathos, humour, force, tragedy, pity, horror, terror, darkness, irony, sarcasm, mysterious, quirky

 

11

STYLE_PERSPECTIVE

Comment on narration or narrator perspective

 

12

JUDGEMENT

Some kind of judgement about the book is made—either favorable, neutral/mixed, or unfavorable 

 

13

RATING

Overall review's rating of the book (favourable, neutral/mixed, or unfavourable)

aBased on the elements that make up a helpful review from Writing Book Reviews (Drewry, 1974)

 






Evidence Based Library and Information Practice (EBLIP) | EBLIP on Twitter