THE UPSTATE BIAS CHECKLIST

The Upstate Bias Checklist is a free, publicly available tool that anyone can use when developing or reviewing content for learners at any level in the health professions. It is designed to avoid burdening learners with the responsibility to call attention to biased material, although it can and has been used by learners to provide feedback on content and educational experiences. It is expressly not intended to be punitive toward educators, but instead to promote self-reflection, faculty development and quality improvement in education, while also preventing the harm that comes when biased content reaches learners—harm that not only impacts our learners but also impacts their and our future patients.

A public version of the Checklist is available for anyone to use here: https://tinyurl.com/upstatebiaschecklist

Frequently Asked Questions

1. Who should use the checklist? 
2. What types of health professions education content can I review with the checklist?
3. Can I use the checklist to review more than one educational session at once?
4. When should I complete the checklist? 
5. What happens if the checklist picks up bias in my content? 
6. Why should I use the checklist? 
7. Some of the terms in the checklist are confusing. Where can I find more information?
8. What happens to the data submitted through the checklist?
9. How was the checklist developed?
10. Has the checklist been published? 
11. I am at another institution. How can I adapt the checklist for our own use and/or access my faculty’s data? 

1. Who should complete the checklist? 

Anyone can complete the checklist, including content creators (e.g., faculty writing and giving lectures, designing case-, team- or problem-based learning sessions, etc.); curriculum supervisors (e.g., unit, course or clerkship directors; evaluation and assessment leaders; deans); session facilitators (e.g., small group facilitators); and learners themselves. 

The more frequently both the content creator and unit, course or clerkship director (or other supervisor) complete a checklist for the same content, the better the checklist validation, so our team really appreciates it when supervisors have time to assess content in their courses or programs. We also find learner feedback extremely useful. 

The unit, course or clerkship director can be the only person to complete the checklist; however, we still recommend completing the checklist before the lecture is presented so that you can share any recommendations for changes to be made with the faculty member implementing the content. 

The checklist does ask about the user’s role, specifically whether they are reviewing the content in a course director role or as the person creating and presenting the content.

2. What type of health professions education content can I review with the checklist?

It is applicable to a variety of types of health professions education content, including but not limited to: lecture slides or notes, clinical vignettes, multiple-choice questions, case-based learning materials, objective structured clinical examinations (OSCE), and standardized patient encounter (SPE) scripts. 

It is also applicable to all health professions, at all levels of training, and has been utilized in undergraduate, graduate and continuing medical education, nursing education, and physician assistant programs, and has even been adapted for use in clinical laboratory sciences and veterinary education. 

At Upstate, all formal curricular sessions in any unit, course or clerkship in the MD program must be reviewed using the checklist.

3. Can I use the checklist to review more than one educational session at once?

Faculty who teach multiple sessions (including multiple lectures) can choose whether to submit one checklist or separate checklists. The checklist does ask whether the user is evaluating one session or multiple sessions, and how many contact-hours are covered. 

We recommend a single checklist only if the lectures or sessions are closely related; a lecturer who gives two lectures on very different topics within the same unit (for instance, one that is very biochemistry-heavy and one that is a clinical correlation) would be better served by completing two checklists. 

Multiple cases should be evaluated individually, using the checklist, and collectively using the “Clinical Case Tracker” grid embedded in the checklist (under “Clinical Vignettes”).  

4. When should I complete the checklist? 

The checklist is most effective when completed in advance of the educational session, with enough time allotted to allow for changes to be made to the content. However, it can also be completed retrospectively, as part of a review prior to the next iteration of the unit, course or clerkship. 

5. What happens if the checklist picks up bias in my content? 

Congratulations! You used the checklist successfully.

If the possibility of bias is detected, you will see a box telling you that your content is at risk for bias and recommending you consider making a change. The checklist does NOT tell you HOW to change your content. (Sorry, we’re physicians and social scientists, not AI experts.) Changes might include:

• Removing the content entirely (e.g., an image that promotes stereotypes of certain patient groups or an inappropriate joke)
• Replacing the content (e.g., replacing some slides of white skin with more representative slides of many skin colors, replacing outdated or offensive terminology with more appropriate language)
• Adding additional material to the content (e.g., including women and people of color in a lecture on the history of medicine, discussing why a race-based disease association might exist)
• Attaching an apology or disclaimer to the content (e.g., acknowledging that race-based GFR corrections are not based in science but may appear on standardized tests—please note that this is a last resort if none of the other approaches can be applied).

You may not be sure what type of change to make or even if a change is definitely needed. That is completely understandable and expected. Please feel free to reach out to Dr. Caruso Brown (brownamy@upstate.edu) for additional assistance, or consult your unit, course or clerkship director. Student feedback can also be very helpful in these situations.

6. Why should I use the checklist? 

Bias in health professions negatively impacts learners by creating a learning environment that is unsupportive and even hostile to learners from traditionally underrepresented backgrounds, hindering their success. However, it has an even greater effect on learners’ future patients. Health professions students and trainees who learn biased material (for instance, suggesting that race is a biological, rather than social, construct) are more likely to treat their patients differently based upon their social identities—missing diagnoses that don’t fit stereotypes, under-managing pain and other symptoms, leaving patients feeling unheard amd disrespected, and increasing mistrust in the healthcare system. Every interaction between health professions educators and learners is an opportunity to begin to dismantle the bias and structural oppression embedded in our society. What you teach today—even if it seems very far removed from clinical care—may change a patient’s life tomorrow. 

7. Some of the terms in the checklist are confusing. Where can I find more information?

Some definitions are embedded into the checklist, below the answer choices. A glossary is also available here.

7. What happens to the data submitted through the checklist?

By utilizing the REDCap version of the checklist, you are agreeing to allow Dr. Caruso Brown and the Upstate Bias Checklist team to retain your responses for further refinement of the checklist. Identifying information is not collected.

At Upstate, pooled data on checklist utilization (number of checklists completed/number of hours of content assessed, number of changes prompted, number of changes made) for each unit, course and clerkship will be provided to the directors prior to the annual review each year.

8. How was the checklist developed? 

The checklist was designed by Dr. Amy Caruso Brown at SUNY Upstate Medical University, and she holds the copyright to this tool. Please feel free to contact Dr. Caruso Brown (brownamy@upstate.edu) with any questions or feedback.

It was informed by a review of the literature, synthesized with three years of student evaluation data from SUNY Upstate, and is regularly revised to include new material. The current checklist includes 13 domains identified as being at risk for bias or promotion of shame, stereotype or stigma: Race and Ethnicity, Gender, Sexual Orientation and Sexuality, Disability, Mental Health Including Substance Use, Weight, Immigration Status, Poverty, Religion, Prisoners, and Interprofessional Communication; and two types of content which are especially prone to bias: visual images and clinical vignettes.

9. Has the checklist been published? 

Caruso AB, Hobart TR, Botash AS, Germain LJ. Can a checklist ameliorate implicit bias in medical education? Medical Education. 2019;53(5):510. https://europepmc.org/article/med/30856280

10. I am at another institution. How can I adapt the checklist for our own use and/or access my faculty’s data? 

Please contact Dr. Caruso Brown at brownamy@upstate.edu about creating a Friends of Upstate REDCap account and a REDCap project with the checklist for your institution, which will give you both the ability to edit the checklist and access to data from your own faculty. Dr. Caruso Brown is also available to lead faculty development workshops and can provide additional information regarding institution- or program-wide implementation and monitoring.