The Rating System

Financial Information

Political Independence

Contact

The Organizational Structure and Methodology of FactReview

When analyzing the enormous volume of information circulating daily on social media and across different news media, fact-checking organizations around the world use a range of tools and methods to determine whether a given claim is substantiated.

Here we explain how we organize our research and editorial team, both to identify and debunk potentially false claims and to check the quality of our work before an article is published. We also examine the methods and tools we use at FactReview to verify the accuracy of each claim we encounter.

Before proceeding, it is useful to understand that these guidelines and tools form a very general model. Tools are constantly updated, and each fact-checker or fact-checking team uses different workflows to do their work effectively, taking into account, among many other factors, the particularities of their geographic location and the socioeconomic and political circumstances. In addition, our policies are reviewed every year for their effectiveness and are updated or expanded according to the findings.

Organizational Structure and Editorial Review

Collecting Claims and the Topics We Cover

The first step in the fact-checking process is to review news coverage and claims circulating online from media outlets, blogs, and social media accounts, and to identify cases of claims that may not be well-founded. However, the volume of content produced online at any given moment is far too large to examine manually.

This is where software is used to group new articles and social media posts in a way that allows us to review news coverage or social media posts much more easily, using filters that enable grouping.

This type of software is called “aggregators” and is one of the most basic tools in fact-checking. For articles published by media outlets and blogs, there are thousands of applications and websites that allow their content to be grouped by category.

However, while technological solutions for automating the discovery of potential topics for review are important tools, the most valuable resource is the public itself. Readers who use social media every day and come into contact with thousands of different pages and users may identify claims or articles that appear unfounded or inaccurate. They can then notify us by sending the link to the article or public post they found so that we can check the accuracy of the claim in question. Because, as noted above, the volume of information circulating online is impressively large, direct tips from readers greatly support our work.

Therefore, if you want to help us by sending topics you have identified and believe require examination, you can send us a message by email here, or by pressing the button at the bottom right to send a direct message to our Facebook page through Messenger.

To be effective in our work, and given the large amount of material to examine, we carefully select the topics we will cover. Specifically, we give priority to more serious or urgent misinformation issues, with emphasis on topics that may put people at risk, such as medical claims, target minority groups, or require immediate debunking, such as scams involving online contests that may disappear within a few days.

We do not debunk personal positions or opinions, nor cases of satire, but only specific claims. In addition, to maintain balance in our coverage of political topics or claims made by political figures, we try, as far as possible, to examine claims from across the political spectrum and avoid focusing on one party or one part of the spectrum. When any direct or indirect conflict of interest exists in an article, whether political or general, involving an author, an expert we consult for clarification, or a person from whom we request comment, this is stated explicitly in the interest of full transparency.

Topic Triage and Assignment

Once potential topics for coverage have been identified, they are reviewed by the editor-in-chief and the site’s senior editor, and then assigned according to workload and each writer’s interests. Once an article has been completed, it is checked by at least two people other than the author, usually the editor-in-chief and the senior editor, for both textual coherence and the accuracy of the sources used to document the debunking. Each article is published on our site only after it has passed all relevant checks.

Organizing Claims and Contacting Relevant Parties

Each writer has their own process for researching a topic and organizing their work, but in general a common strategy is followed during a fact-check.

Once we have selected the topic we will cover, we examine how many times it has been reproduced and by which outlets, so that we have as complete a picture as possible of the extent of its spread. These data may also be used later in research on misinformation trends in specific topic categories, such as misinformation about the climate crisis.

This information, together with a list of all claims made by the publication under review, is entered into a draft and examined separately. At this stage, once we have a clear draft of how we will organize the topic, we often need to contact the person who wrote the article under review or one of the people mentioned in it. It is especially important to have a complete picture of the issue, which is why collecting information from directly involved parties is a core part of the research process.

It is also important to give directly involved parties an opportunity to comment when this is appropriate and/or feasible.

Complaints and Corrections Policy

Like any other outlet, despite the detailed review procedures we have established for checking our material before publication, mistakes and/or omissions may still occur in our work. Recognizing this reality, we have created a separate policy for reviewing correction requests.

Our website has a dedicated section where we encourage every reader to contact us if they identify an error in an article or other material of ours. In this context, the reader should point out the place or places where they believe an error has been made, providing a relevant explanation and, where necessary, suitable evidence.

The editor-in-chief together with the senior editor will review the request and, if an error is indeed found, we will proceed with an immediate correction. A record of the error will then be created in a separate section of our site, which we maintain to record our mistakes in the interest of full transparency. These records are organized by date, and each record includes the link to the article or other material that was corrected and a short description of exactly what was corrected.

We follow a similar policy for complaints. In the same form used for correction suggestions, readers can send both improvement suggestions and objections regarding topics where they believe we made an error or disagree with how we covered an issue.

All requests are carefully reviewed by our team, provided that the communication is respectful and does not use offensive language of any kind toward the FactReview team. Where appropriate, we implement the suggestions we receive or make changes to points where criticism is raised, always informing the sender accordingly.

Verification Methodology and Examples of Fact-checking Tools

Analysis of Audiovisual Material

One of the most common forms of misinformation involves the alteration and use of audiovisual material, namely videos, images, or audio recordings, to spread unsubstantiated claims. When we have a post or article that uses an image or video as the basis for a claim, the easiest way to examine the accuracy of that claim is to identify the original source of the video or image under review.

Finding Earlier Uses of an Image

Finding Earlier Uses of a Video

For this purpose, we use a technique known as “reverse image search”. In practice, this is a search like the one we perform for a topic in search engines such as Google, except that instead of text we search based on an image.

There are several services for this purpose, but the four best known are Google, Yandex, TinEye, and Bing Image Search. This process can be done manually or automatically. If we want to search manually, we download the image we are interested in and upload it to each service separately. The same process can be done automatically with a dedicated browser plugin, which allows us to search for the image we are interested in simply by right-clicking and selecting the service we want to use.

You can find all these services in FactReview’s fact-checking toolkit, a section of our website where we have organized information-checking tools by category. You can find the toolkit here. You can find a relevant example of an article in which we identified the reproduction of a false claim using an older image as if it were recent, using reverse image search, here.

When we try to identify a possible earlier use of a video, we follow a different process. To date, there are no freely accessible tools for automatic searching based on a video in the way there are for searching based on an image. Therefore, to find an earlier use of a video, we must divide it into its individual frames, meaning small snapshots from each second, and search based on those images. In practice, this is a variation of reverse image search.

Although we can do this manually by taking screenshots from the video ourselves, there is a way to automate the process. This is done through a platform called InVid, which is a collection of tools for analyzing audiovisual material. One of these tools separates the frames of a video, which we can either upload to InVid as a file or provide as a video link. The platform then automatically divides the video into individual frames and allows us to immediately run a reverse image search with a relevant service, such as Google Images.

InVid is also included in FactReview’s fact-checking toolbox, which you can find here. You can find a relevant example of an article in which we identified the reproduction of a false claim using an older video as if it were recent, using InVid, here.

Examining Data from Reliable Sources

A common denominator in every different topic we examine at FactReview is the effort to use primary evidence and sources. Where this is not feasible, we choose reports and reviews of evidence from official bodies, always providing references for every claim made in our articles.

Claims Involving Scientific Content

In scientific topics, scientific studies are considered primary sources, while reports by official scientific bodies that use scientific studies as sources are considered secondary sources. When we examine a scientific claim, for example that homeopathy can treat a serious disease, we use official and, as far as possible, primary sources to support our analysis. These sources are always available within our articles so that each reader can see the evidence we used.

Specifically, in the case of scientific studies, we can follow a specific process to find evidence presented in the publication under review:

Finding the Study

This is usually one of the most demanding steps, because many publications refer to findings they claim came from a study but rarely provide the study title, and even more rarely provide a link to the study. Therefore, to find the source, we must search for it manually. This means taking information from the article under review and using it as keywords in a search engine. For example, if a hypothetical article states that “according to a study by a university in China, 25% of those who received childhood vaccines developed autism,” we can search for the terms “China” + “study” + “child” + “vaccines” + “autism.” If we know approximately when the study was published, we can include that in the search.

Assessing the Quality of the Journal

This is an important, though complex, step because in many cases it requires relevant experience. However, the easiest step is to search for the journal in one of the existing lists of unreliable journals, known as predatory journals.

Journals of this type publish practically any study for the corresponding fee, with minimal or no checks on the reliability of the data and methodology. The best-known list, with data up to December 2021, is Beall’s List, which you can find here.

Examining the Study Itself

Obviously, it is not feasible for everyone to devote time to reading a scientific study, since beyond the technical terminology that complicates the process, it is time-consuming. However, there are ways to dramatically reduce the time needed to find the information we need. While at FactReview we must read in detail the studies we examine when analyzing claims, someone who simply wants to obtain specific information can do so by reading targeted parts of a study.

The first is the study abstract, which briefly describes what the study examines and what it found. The second and most important point is the conclusions, which in most studies are titled “Conclusions” and/or “Discussion.” These sections provide a somewhat more detailed summary of the study and its conclusions, usually without technical definitions. The simpler language used in these sections makes it easier to find the information we need quickly and without a major time investment in analysis.