Citation Indexes: More than Just Web of Science

March 4, 2009 at 9:38 am

Until recent years, ISI’s suite of citation indexes, now known as the Web of Science, was the only real option for finding information about who cited what and when.  It is because of the Web of Science and its print predecessors (Science Citation Index, Social Science Citation Index, and Arts & Humanities Citation Index) that citedness and impact factor are now commonly used as measures of research output and quality.  After almost 40 years of ISI market monopoly, in late 2004, the citation index landscape shifted dramatically with the introduction of two new databases poised to compete on a level with or higher than Web of Science.  Scopus, an Elsevier database with integrated citation searching, was released on November 10, 2004[1], Google Scholar (GS), a free online database also seamlessly incorporating citation indexing, was released November 18, 2004[2].

In the wake of the release of Scopus and particularly Google Scholar, a flurry of activity has taken place as researchers, librarians, and information scientists have rushed to evaluate these new products and compare them to the gold standard, ISI’s Web of Science.  Though Scopus is generally agreed to be an excellent research tool, especially in its intuitive design, ease of use, and size and scope, Google Scholar has faced both virulent criticism and high praise.  Nevertheless, general consensus is that in order to do the most complete citation search, all three databases must be searched.  In fact, in one study where one article had 11 citations each found in Web of Science, Google Scholar, and Scopus, the 11 citations were not the same; Web of Science and Scopus each had one unique citation, and Google Scholar had four [3].  In an even more dramatic example, Peter Jacso found that for a sample classic citation where Web of Science found 83 citations, Google Scholar found 82 citations, and Scopus found 76 citations, only 33 citations were found in all three [4].

Google Scholar Pros and Cons

Google Scholar, developed by Google engineer Anurag Acharya [5], stems from Google’s involvement with the CrossRef project, a multi-publisher citation linking system [6,7].  As its crux is the same Google technology that ranks results by examining each document’s relationship to others via hyperlinks [8].

In Google Scholar (GS), instead of hyperlinks, the relationship is based on citations.  Frequently cited papers will rise to the top of any search’s results, making GS a great resource for finding classic citations and for examining citation data.  GS is a multidisciplinary database attempting to index the world’s scholarly literature.  To Google, this appears to include journal articles, preprint and e-print repositories, higher education web sites, conference proceedings, and books.  It is particularly strong, in comparison with Web of Science (WOS) and Scopus, in indexing and providing citation data for the non-journal article materials it indexes.  Because it indexes conference proceedings, books, etc., GS is often able to provide more references that cited a particular work than WOS or Scopus, though generally only for materials published in the past 20 years [3-5,9].  As a multidisciplinary database, it covers more fields than Scopus, which is restricted to primarily the sciences and some social science, making GS a great tool for finding citations across multiple disciplines.  Furthermore, GS’s indexing of preprints also means that citation information in GS may be months ahead of WOS or Scopus, which only index published materials [5].  Lastly, unlike WOS, you can search not only by cited author, year, or journal, but also by article title, a considerable bonus.

Though GS does have a number of features that make it an excellent citation index, it has been critiqued for a number of reasons.  In a lengthy review of the comparative strengths of GS, WOS, and Scopus, Jacso found that GS may grossly misinterpret citations, may present misleading or erroneous figures as to how many times an article is cited, and may produce results that contain redundant entries [4].  In an earlier review of GS, Jacso came to the same conclusions, finding that GS may in fact attribute citations to completely wrong articles, and further noted that it is only possible to see the first 1,000 results for any search, even when an article is cited over 1,000 times [10].  Though most of Jacso’s conclusions are valid, one study found that WOS produced far more redundant citations than GS [9], and many studies have found that the benefits of using GS outweigh its annoyances.  Jacso’s studies do teach GS users not to take its citedness scores at face value.

Another criticism of GS is that it only indexes materials in certain languages.  For example, one study examined the amount of Persian language or Chinese language documents in GS, finding none [5].  This, however, has changed — GS now contains Traditional and Simplified Chinese-language materials.  As with Google itself, it can be expected that more and more languages will be available in GS in the future.  At this time, however, GS contains almost solely European languages, an inherent bias.

How to Use Google Scholar as a Citation Index

To search to find out who cited a particular article, type in the information you know about the article.  This may include author, title, and/or journal.  In the example, below, the author (najarian) is entered along with some title keywords (initial graft function cadaver kidney).

google scholar

The results will display 10 at a time, with the results ranked in order primarily in descending order of citedness (most cited articles first).  If you cannot find your article, modify your keywords.  [HINT: You can also use the Advanced Scholar Search page to hone your results.]

In the results, if GS has found materials that cite the article, there will be a “Cited by” link underneath the citations (see above). Click on the link to see the citing articles.


The results list will show that a cited reference search has taken place.  The citing articles are again generally arranged in order of citedness and displayed 10 per page.  Often, clicking on the title of a citation may take you to the full text or an abstract.  Try clicking on the “group of” links to find full text links when not available otherwise.  [HINT: To access Mayo-owned online journals using SFX, change your preferences in GS to associate yourself with Mayo.  Then, click on the SFX services link when available.]


Searching to see how many times a particular author has been cited is slightly more difficult.  To search by author, either use the Advanced Scholar  Search or attach the author name to the author tag (author:).  If using initials and last name, put them in quotes.  For example, to search for publications by Kirk E. Smith, use the search term author: “ke smith”.


Scopus Pros and Cons

Scopus is an Elsevier database created by combining MEDLINE, Embase, Fluidex, Compendex, World Textile Index, Biobase, and Geobase [1].  Its primary scope is life and health sciences, followed by hard sciences, social sciences, and earth and agricultural sciences. Overall, Scopus indexes approximately 14,000 journals, compared to WOS’s 9,000.  To the data found in these databases, Scopus has added a significant amount of citation data, but primarily from 1996 to the present.  Older citation information is scattered and incomplete [1].

Scopus is a truly international database, indexing far more foreign language and regional titles than WOS or GS.  For example, where GS indexes primarily European language titles plus Chinese titles, Scopus indexes materials in Persian, Afrikaans, Thai, Japanese, and dozens more.  Though Scopus does have a larger base and range of material than WOS, citedness results are generally quite comparable to WOS in both number of citations and in the uniqueness of those citations, at least for the 1996-2005 period [3,4,11].  WOS is far superior in finding citations in and for older materials.

Scopus does, however, have a few features that put it a notch above WOS.  First Scopus displays full cited reference information, unlike WOS, which only displays first author, journal title, volume, first page number, and year.  In addition to displaying this information, Scopus also makes this information searchable, so, like GS, it is possible to search just by article title.  Secondly, Scopus appears to correct mistakes in articles’ bibliographies, meaning it may be easier to find all citations to a particular article than using GS or WOS [11].  Searching for all cited articles from a particular journal is also easier using Scopus than WOS and especially GS; Scopus provides a handy list of all journal users to simply check all variants of a journal title [12].  Scopus also parses in press citations more accurately than WOS [11].  Lastly, Scopus is better at finding citations in open access journals and materials than WOS [3,12].

How to Use Scopus as a Citation Index

Scopus is accessible from the left-hand links on the Mayo Clinic Libraries intranet home page.  To search for articles that cited a particular article, first search for the article using known information such as author, title keywords, or journal.  In the example below, the phrase “aging driver” is searched for.  The results will display 20 at a time.  Use the “Refine Results” box to restrict your search to particular years, journals, or authors if necessary.  If you cannot find the article you are looking for, try modifying your keywords.


The far right column lists the number of articles each result is cited by.  Click on this number to see the citing articles.  The cited references display is nearly identical to the normal results list, except that the article that was cited is displayed on the top of the screen.


To find out who cited articles not indexed normally in the Scopus database, use the advanced search option.  Options to search by cited reference source (REFSRCTITLE), cited author (REFAUTH), and cited reference date (REFPUBYEAR) are available.


Melissa Rethlefsen, Librarian – LRC, Mayo Medical School


  1. Deis LF. Web of Science (2004 version) and Scopus. The Charleston Advisor 2005;6(3).
  2. Sondermann TJ. Hi. [Blog].  2005. [cited April 5].
  3. Bauer K, Bakkalbasi N. An examination of citation counts in a new scholarly communication environment. D-Lib Magazine 2005; 11(9).
  4. Jacso P.  As we may search – Comparison of major features of Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases.  Current Science 2005 November 10;89(9):1537-47.
  5. Noruzi A.  Google Scholar: The new generation of citation indexes. [PDF]. E-LIS, 2006.  [cited March 4].,_The_New_Generation_of_Citation_Indexes.pdf.
  6. Sadeh T. Google Scholar versus metasearch systems. High Energy Physics Libraries Webzine 2006 March (12).
  7. the reference linking backbone. [HTML].  2005. [cited March 1].
  8. Brin S, Page L. The anatomy of a large-scale hypertextual web search engine. [HTML]. 1998. [cited March 4].
  9. Belew RK.  Scientific impact quantity and quality: Analysis of two sources of bibliographic data. [PDF]. arXiv, 2005. [cited March 4].
  10. Jacso P. Google Scholar and The Scientist. [HTML]. 2005.  [cited March 1].
  11. Iselid L. Errors in cited references of Scopus and Web of Science – corrected or not? [Blog]. 2006. [cited March 1].
  12. Jacso P.  Scopus. [HTML]. Gale Group, 2004. [cited March 1]

(Originally published in Quarterly Newsletter of the Mayo Clinic Libraries, Winter Issue No. 21, March 2006).

Entry filed under: From the Archives.

Image is Everything: Finding and Using Medical Images on the Web New Features for Mayo Clinic Proceedings

%d bloggers like this: