AHRQ Online Critiques & Scores

Overview of the AHRQ Scoring Process

The Agency for Healthcare Research and Quality (AHRQ), NIH, and the Centers for Disease Control and Prevention (CDC) follow the same basic procedures for submitting scores and critiques for non-SRF applications.

Reviewers enter a 1-9 score for five main review criteria: Significance, Investigator(s), Innovation, Approach, Environment — all are scored.

Open and Complete the Critique

Follow these steps to complete the online critique:

1. Go to the List of Applications or List of My Assigned Applications screen in Internet Assisted Review.
2. In the Application Number column for the application you are critiquing, click the three-dot ellipses icon and select the Submit Critique option.

3. Optionally, review the Important Reminders heading by clicking on the heading text Important Reminders in the blue bar. This might contain instructions, set by the SRO, that are relevant for the current meeting.
4. Click the blue or green headers in whatever order suits you and begin filling out the critique. Use the tab key to jump from field to field.

5. For each review criteria, read all guidance at the top of the editing area by clicking the read more link, which shows all available guidance for the criteria item.
6. If applicable, select scores from the Score dropdown menu. See the next section for details.
7. Select a radio button from each radio button group.
8. Enter your comments in text fields. See AHRQ Online Critiques & Scores. As a best practice, to ensure you don't forget a field unintentionally, enter N/A or none in the field if you have no comments for the field. This reduces warnings about empty fields when you submit the critique.

Editing in a Critique Text Field

When you click in a field, a toolbar appears with formatting tools:

The first six of these are familiar to most users, and are bold, italic, underline, strikethrough, subscript, and superscript. You can hover your cursor over any editing tool to see its name and its keyboard shortcut keys if available.

The less familiar icons are explained below.

Formatting Button	Action
	Removes all formatting from selected text.
	Numbered List/Bulleted List tools. Bullet or number the selected text.
	Special character tool. Opens a Select Special Character window, from which you can choose special characters. While you can copy and paste from other applications to the Online Critique screen, special characters might not be rendered correctly in the final critique if you do so. It is recommended that you use the special character editing tool for inserting special characters to ensure correct rendering of special characters.
	Undo and redo tools.

Spelling and Grammar Checker

The Online Critique text boxes have an advanced spelling and grammar checker. The status of the checker is indicated by a colored dot in the bottom right corner of the text field.

The status colors are:

• Orange – the checker is enabled
• Red – there is a least one suggestion found within the text.
• Gray – the checker is disabled

Spelling suggestions are underlined with a red line, while grammar suggestions are underlined in blue.

Hovering over a suggestion provides a list of options.

The suggestions can be ignored by clicking the Ignore all button, or the word can be added to the local dictionary by clicking the Add Word button.

The three-dot More options icon provides three additional selections.

• Proofread in dialog – opens a dialog box in a separate window. The window shows the current issue and possible suggestions. At the bottom of the window, the number of suggestions is shown and navigation buttons let you scroll to other issues/suggestions.

  

• Settings – Opens a dialog box with a variety of options that can be selected or toggled on and off using the corresponding toggle switch.

  

• Disable – turns the checker off and set the status color to gray.

Clicking the status dot provides the same options as described above:

• The power switch icon disables/enables the checker.
• The gear icon opens the Settings dialog box.
• The double arrow icon opens the Proofread in dialog window.
• The numbered circles represent the number of suggestions.

When all the suggestions have been reviewed and updated, a circled checkmark appears in the bottom right corner.

Submit the Critique

After you fill out the critique completely, you are ready to submit. Make sure you are on the Online Critique screen and do the following:

• In the gray summary box at the top of the Online Critique screen, click the Submit button.

1. If any entries are not completed, a warning dialog lists the incomplete fields.

2. Click Continue with Submission only if you know the omissions to be valid and intentional.
3. Otherwise, click Go Back and Edit.

4. Fill out the incomplete entries if necessary; if you are sure you want them blank, then you can leave them blank and submit regardless.
5. Click the Submit button again, and if necessary, click the Continue with Submission button if you want to leave blanks.

6. Review the information and click the Confirm button to complete the online critique submission.
   A message appears at the top of the screen indicating successful submission.
   

At this point, a submitted critique is saved into the system. If you later decide you want to edit the submission, you can do so; see Make Changes to a Critique After Submitting.

You can view your own critiques and scores as well as those of other reviewers (when permitted) once you have submitted them. The process by which critiques are viewed differs from phase to phase. Refer to Viewing Critiques for a description of the phase-specific method of viewing critiques.