Reviewing documents with one or more reviewers

In Sysrev, the review of documents can be done by a single reviewer or collaboratively. For collaborations, Sysrev makes possible blind review (where reviewers do not see the decisions or labels of other reviewers), and provides different review modes to facilitate single or double screening of documents. It also makes it easy to identify where conflicts between reviewers exist, so those conflicts can be resolved by discussion, consensus or a third reviewer.

Review options and settings

There are a number of settings that you can adjust to manage your screening and review process. These are found under Manage -> Settings, in the Options box.

• Article Review Priority: When reviewers start to look at articles in the Articles tab, they will be automatically presented with an article. The Article Review Priority will determine if the articles provided to reviewers are not yet screened by anyone (Single), screened by one reviewer already (Full), or a mix of these two (Balanced). In this way, you can manage a collaborative review with either single or double screening.
• Allow Unlimited Reviews: This setting (Yes/No) can allow for reviews in which you want more than two reviewers (i.e. unlimited reviewers) per document.
• Automatically Save Review Labels: This option (Yes/No) can be used when reviewers prefer to not have to click Save after completing the required labels. If Yes is chosen, once all REQUIRED labels are completed, they will automatically be saved and the next record will appear. This is probably not a good option if you have some optional labels, or a complex label structure.
• Label Blinding: Turning this label on will make the decisions of other reviewers invisible. This is a good option for most reviews in which double screening is carried out. This setting can be turned off during conflict resolution.

We will cover the other review Options in other sections of this guide. Once you have adjusted your review settings, don't forget to click the Save Changes button at the bottom of the Options box!

How to review documents

Any member of the project can review documents (i.e. assign labels to documents). To do so, from the main project page, click the Review tab. On the left-hand side, all of the enabled labels will appear. Once all required labels are answered by the reviewer, the blue Save button will become available. Click Save to record label answers. The next record for review will automatically appear in the window. Reviewers can also choose to skip the record.

Note that depending on the Article Review Priority settings described above, the records will be prioritized by whether or not they have already been reviewed by another reviewer. Other than this sorting, the records are randomly assigned.