TUTORIAL: Peer Review: Peer Review Basics

You will encounter many types of articles and it is important to distinguish between these different categories of scholarly literature.  Keep in mind the following definitions.

Peer-reviewed (or refereed):  Refers to articles that have undergone a rigorous review process, often including revisions to the original manuscript, by peers in their discipline, before publication in a scholarly journal.  This can include empirical studies, review articles, meta-analyses among others.

Empirical study (or primary article):  An empirical study is one that aims to gain new knowledge on a topic through direct or indirect observation and research.  These include quantitative or qualitative data and analysis. In science, an empirical article will often include the following sections:  Introduction, Methods, Results, and Discussion.

Review article:  In the scientific literature, this is a type of article that provides a synthesis of existing research on a particular topic.  These are useful when you want to get an idea of a body of research that you are not yet familiar with.  It differs from a systematic review in that it does not aim to capture ALL of the research on a particular topic.

Systematic review:  This is a methodical and thorough literature review focused on a particular research question.  It's aim is to identify and synthesize all of the scholarly research on a particular topic in an unbiased, reproducible way to provide evidence for practice and policy-making.  It may involve a meta-analysis (see below). 

Meta-analysis:  This is a type of research study that combines or contrasts data from different independent studies in a new analysis in order to strengthen the understanding of a particular topic.  There are many methods, some complex, applied to performing this type of analysis.

TIP: Review articles and Peer-reviewed articles are not the same thing! Review articles synthesize and analyze the results of multiple studies on a topic; peer-reviewed articles are articles of any kind that have been vetted for quality by an expert or number of experts in the field. The bibliographies of review articles can be a great source of original, peer-reviewed empirical articles.

Watch this 3 minute intro to peer review video by North Carolina State University or read this short introduction by the University of Texas at Austin Library.

There are a couple of ways you can tell if a journal is peer-reviewed:

• If it's online, go to the journal home page and check under About This Journal. Often the brief description of the journal will note that it is peer-reviewed or refereed or will list the Editors or Editorial Board.
• Go to the database Ulrich's and do a Title Keyword search for the journal. If it is peer-reviewed or refereed, the title will have a little umpire shirt symbol by it.
• BE CAREFUL! A journal can be refereed/peer-reviewed and still have non-peer reviewed articles in it. Generally if the article is an editorial, brief news item or short communication, it's not been through the full peer-review process. Databases like Web of Knowledge will let you restrict your search only to articles (and not editorials, conference proceedings, etc).

They both come out once every month. They're both in English. Both published in the United States. Both of them are "factual".They both have pictures. They even cover some of the same topics.

The difference is that one--Oceanography--is peer-reviewed, whereas National Geographic is a popular-press title. 

Peer review is scientists' and other scholars' best effort to publish accurate information. Each article has been submitted by a researcher, and then reviewed by other scholars in the same field to ensure that it is sound science. What they are looking for is that:

• The methodology has been fully described (and the study can thus be replicated by another researcher)

• There are no obvious errors of calculation or statistical analysis

• Crucially: The findings support the conclusions. That is, do the results of the research support what the researcher has said about them? The classic problematic example is a scientist claiming that hair growth causes time to pass: The correlation is clearly not causation.

It isn't a perfect system: Scientists make errors (or commit fraud) as often as any other human being and sometimes bad articles slip through. But in general, peer-review ensures that many trained eyes have seen an article before it appears in print.

Peer-reviewed journals are generally considered "primary source" material: When a new scientific discovery is made, a peer-reviewed journal is often--but not always--the first place it appears.

Popular and trade publications are not peer-reviewed, they are simply edited. That does not mean they are any less potentially truthful or informative--most popular and trade publications take pride in careful fact-checking.* But when the topic is scientific research, the information is generally "secondary": It has already appeared elsewhere (usually in a peer-reviewed journal) and has now been "digested" for a broader audience.

Peer-reviewed journals will always identify themselves as such. If you want to verify that a journal is peer-reviewed, check Ulrich's Periodical Directory.

Some sources of peer-reviewed articles: