LEAP

In support of our focus on equity-minded assessment practices, the Office of Academic Assessment adopted a modified version of the ATLAS Data Looking Protocol in 2023. 

The tool, developed by Eric Buchovecky, is based in part on the work of the Leadership for Urban Mathematics Project and the Assessment Communities of Teachers Project. The tool also draws on the work of Steve Seidel and Evangeline Harris-Stefanakis of Project Zero at Harvard University. The ATLAS Data Looking Protocol is available for institutions to use and adapt to meet their data analysis needs.

The protocol implemented by Emerson is modified from a version created by Ruth Slotnick at Bridgewater State University. Dr. Slotnick adapted the protocol by adding particular equity-minded questions to the facilitation process.

LEAP Canvas Outcomes Data-Looking Protocol

Adapted from the ATLAS Data-Looking Protocol

Meeting 1

OAA Staff:

• Present overview of Canvas Outcomes process 
• Present overview of data collected from the previous semester (raw data & visualizations)

LEAP Project Partners:

• Brainstorm desired demographic visualizations they’d like to see (e.g. race, pell-eligibility, US domestic vs. international)
• Questions

Between Meetings

LEAP Project Partners:

• Step 1: Describe the data: What do you see?
  • Describe what you see when you look at the data, first avoiding judgments about quality or interpretations. It is helpful to identify where the observation is being made – e.g., “In the visualization on slide 3…” or “In Column C of the data…”
• Step 2a: Interpretations and Wonderings
  • What does the data suggest is going well?
  • What does the data suggest needs attention and/or improvement?
     

Meeting 2 

OAA Staff:

• Facilitate discussion/analysis of data
• Take notes to supply to LEAP fellow for the report

LEAP Project Partners:

• Brief overview of findings from Step 1 and Step 2a
• Step 2b: Interpreting and Wonderings: What does the data suggest through an equity lens? 
  • Try to find as many different interpretations as possible and evaluate them against the kind and quality of evidence.
  • Carefully review the data and look for quantitative evidence that may signal inequities.

• Step 3: Implications for Practice: (What does this means in terms of our work: institutional, programmatic, classroom)
  • What steps could be taken next?
  • What strategies might be most effective?
  • What else would you like to see happen?
  • What are the implications for equity, diversity, and inclusion?