Impact Factor for IAOR vs. Google Scholar

by Graham Rand

Literature review is fundamental to research. Reviews allow you to improve your understanding of a subject, to gain insight into promising research directions, to focus the scope of the project, and to ensure that the research conducted is not redundant with prior published work. An in-depth, thorough review requires access to a corpus of source documents that is encompassing of the subject and to a search engine that accurately, efficiently, and comprehensively retrieves the appropriate documents. What resources are available to the ORMS researcher?

Google Scholar (GS) is an obvious resource for a researcher to consider. GS indexes the full text of scholarly literature froma wide range of disciplines, including most online peer-reviewed journals. The GS corpus includes conference proceedings, scholarly books, non-peer reviewed journals, patents, class notes—essentially any documents that can be crawled on the web.

An alternative, specific to our profession, is International Abstracts in Operations Research (IAOR), a subscription journal of IFORSpublished by Palgrave-Macmillan. IAORprofessional editorial staff collectand classifyabstracts from over 180 journals covering OR, MS, and closely related disciplines.IAOR Online is the gateway to a continually growing database of over 60,000 abstracts dating back to 1989.

So which should a researcher choose? Specifically, does IAOR add any value? Lawrence Bonczar and Pres White (editor of IAOR), at the University of Virginia,looked at these questions, by giving each resource the same queries from the subject area ofhealthcare simulation modeling. This topic was chosen for a couple of reasons. First, the researchers have considerable experience in this field. They are currently developing patient flow and scheduling models for the University of Virginia Health System. Second, as an application of operations research, articles written about it may appear in journals and other sources that are not directly linked to OR, which will make it a challenge for the IAOR database to detect such documents. Conversely, the GS database may draw results from too wide a sample, giving many irrelevant results.

A fuller description of the study is available at http://www.palgrave-journals.com/iaor/index.html so the authors’ main conclusions may suffice for now. First, and most importantly, the results of IAOR and GS searches are complementary. There is comparatively little overlap in the relevant documents retrievedin response to the same query. Using both together is highly advantageous.

Second, subject overviews or introductions are highly ranked in GS retrievals. This is likely to be because of the high weight given to citation counts and because surveys are widely cited. This is a useful property for an initial review of a new research topic, allowing subsequent traditional searches of the typically large number of references provided by overviews and the foundational papers often cited in subject introductions.

Third, IAOR may be superior to GS if the user is determining input strings based on limited prior knowledge. Shorter query strings yield more relevant documents in IAOR. Conversely, the more terms that appear in a GS-based query string—the more bounded and focused are the results. These characteristics almost certainly can be attributed to the effect of expert opinion. Since the documents that constitute IAOR’s database are screened and classified before entry in the database, the quality and relevance of an article to an ORMS topic are almost guaranteed to be higher than a sample article from a GS query.

Fourth, longer strings of conjunctive (“AND”) queries naturally tend to have lesser recall. For IAOR, this opens the potential for queries that are overly specific, owing to the more limited corpus of source documents and the potential that an appropriate keyword may not correspond to an IAOR classification. The primary keywords applied in IAOR classification are published in each issue and can be used to create initial queries. Additional terms also are listed when applied as appropriate to a specific document. These terms are included in a large and extensible database of keywords.

Fifth, the vastness of the GS corpus is both a strength and a weakness. In particular, conference proceedings are an especially important resource. In the study, documents retrieved only by GS included many relevant papers from the Proceedings of the Winter Simulation Conference. While necessarily less thoroughly and stringently reviewed, proceedings papers are more encompassing of the potentially pertinent literature and do not suffer the delays in publication for many scholarly journals. This observation reinforces the complementary nature of the two search engines.

So why not try IAOR and encourage your students to do so? INFORMS members can access INFORMS Online beginning at http://www.informs.org/Find-Research-Publications/Searchable-Databases/International-Abstracts-in-Operations-Research-IAOR-Database and ORS members can go to http://www.theorsociety.com/Pages/Publications/IAOR.aspx