How aligned are e-marketplace and AI activities, items, questions, and/or tasks to the grade level or content area standard being addressed and assessed in your school? Also, how could Depth of Knowledge (DOK) determine the degree of alignment or consistency between academic standards and these marketplace or AI activities and assessments?
This is the second part of a two-part blog. You can read the first part, which covers the important role of alignment in teaching and learning and how teachers should approach it here.
How does Depth of Knowledge serve as a criterion for alignment studies?
Depth of Knowledge can help with checking and confirming the alignment between standards and the curricular activities, assessment items, and AI resources that address and assess them. Knowledge is one of six criteria for analyzing the alignment of the content focus between expectations (standards) and assessments.
According to Webb (1999), standards and assessments are aligned—or consistent—for Depth of Knowledge “if what is elicited from students on the assessment is as demanding cognitively as what students are expected to know and do as stated in the standards” (pp. 8–9).
He also developed the DOK levels as a coding system teachers and test makers could use to categorize and compare the degree of alignment—or DOK consistency—between grade-level standards and assessment activities, items, or tasks.
DOK Scale of Alignment and Consistency
Table 1 features the DOK Scale of Alignment and Consistency from my book Deconstructing Depth of Knowledge: A Method and Model for Deeper Teaching and Learning (2022), published by Solution Tree. These scales of consistency expand on the original measures first developed by Webb, which checked and confirmed the consistency between standards and assessments more comprehensively and completely.
This scale focuses more on analyzing the complexity and consistency between a specific standard and an individual activity, item, question, problem or task that addresses or assesses it.
| Insufficient | An activity, item, or task addresses or assesses one aspect of the specific standard BUT NOT at the DOK level demanded by the standard’s learning intention and/or its most cognitively demanding objective. |
| Acceptable | An activity, item or task addresses and assesses at least 50% of the specific standard’s learning intention OR expects students to understand and use their learning at least one (1) DOK Level below the overall level of Depth of Knowledge demanded by the standard or its most cognitively demanding objective. |
| Full | An activity, item, or task addresses and assesses the learning intention of a specific standard completely OR the most cognitively demanding objective of a specific standard (i.e., the one that designates its overall DOK Level) |
| Beyond | An activity, item, or task demands students to understand and use their learning beyond the DOK Level of the specific standard being addressed and assessed and/or its most cognitively demanding objective. |
Using DOK levels to guide assessment design
The DOK Level of an academic standard clarifies and communicates what Hess (2018) describes as the “ceiling” of assessment—“the highest potential depth-of-knowledge level to be assessed, as well as assessing DOK levels up to the ceiling” (p. 614). When we determine the level of Depth of Knowledge demanded by the specific standard being addressed and assessed, we understand that this is the deepest level students could and should be evaluated on a standardized assessment.
We also become aware of the range of DOK curricular activities and assessment items that could and should evaluate student learning. For example, if a specific standard is determined to be a DOK 3, that establishes the deepest level assessment items could and should evaluate student learning.
How to conduct alignment studies with Depth of Knowledge
Conducting alignment studies starts with determining and designating the DOK Level of the specific standard being addressed and assessed. This involves deconstructing the learning intention of the standard to determine the following:
- What exactly must students learn?
- How deeply must students understand and use their learning?
- How many objectives must the students achieve to demonstrate proficiency?
Table 1 features four separate standards that have been deconstructed into their individual objectives. Notice how each learning intention of the standards contains multiple objectives students must achieve to demonstrate grade-level proficiency. Each of these objectives, individually or collectively, can establish the expectations for a curricular activity, assessment item, or AI resource that addresses and assesses the specific standard.
| Standard | Objectives |
| Multiply a whole number of up to four digits by a one-digit whole number and multiply two two-digit numbers, using strategies based on place value and the properties of operations. Describe the strategy and explain the reasoning. |
|
| Determine the theme of a story from details in the text and explain how the details support the theme. |
|
| Develop and use a scientific model to explain the function of a cell and the ways the parts contribute to its function. |
|
Mastering DOK alignment
Once the DOK levels have been determined and designated, we need to check and confirm how closely the activity, assessment item, or AI resource addresses and assesses the standard’s learning intention and individual objectives. If it addresses or assesses:
- The specific standard completely or the DOK level of its most cognitively demanding objective, then it is fully aligned.
- For example, a math activity, assessment item, or AI resource must address and assess the DOK 2 objectives of the standard to be deemed fully aligned.
- At least 50% of the specific standard or one level below the DOK level of the standard’s most cognitive demanding objective, then it’s acceptably aligned.
- For example, an activity, assessment item, or AI resource that demands students to determine a theme of a story from details in the text is acceptably aligned because it addresses and assesses one of the two objectives of the standard.
- An aspect of the specific standard but not at its overall DOK level, then it is insufficiently aligned.
- For example, if you provided students with a graphic of an animal and plant cell and directed them to label the parts of a cell, then the activity, assessment item, or AI resource would be rated insufficiently aligned because it only addresses and assesses one aspect of the standard (parts of a cell).
- We could justify that the animal and plant cells are scientific models the students must use to complete the task. However, it’s still insufficiently aligned because it addresses and assesses only one of the four objectives.
- Student learning beyond the DOK level demanded by the specific standard is beyond aligned.
- For example, an activity, assessment, or AI resource that demanded students determine the themes of two or more stories and explain how details in the texts support the themes would be concluded as beyond aligned.
- However, an activity, assessment item, or AI resource that demanded students only determine the theme of two or more stories would be deemed acceptably aligned because it still only addresses and assesses 50% of the standard’s learning intention.
Where do we go from here with Depth Of Knowledge?
While technology has made instructional resources and teaching tools easier and simpler to access and afford, we need to be very careful and cognizant about what we are administering and assigning our students. Just because the e-resource or AI tool states it addresses and assesses a standard or demands students to comprehend, communicate, or consider their learning at a defined or designated DOK level, we cannot and should not trust this blindly.
We still need to conduct both an item analysis to determine whether the level of complexity and demand designated is accurate. We also need to check how closely the academic standard is being addressed and assessed and the curricular activity, assessment item, or AI resource we’re using to educate and evaluate our students.
Key checks for activity alignment
So when you access that engaging activity or assessment from an e-marketplace or through an AI tool that claims to address and assess the standard you’re teaching and teaching, check and consider the following:
- What exactly does the standard demand?
- How deeply does the standard demand students understand and use their learning?
- How many objectives does the standard demand students achieve? Can they demonstrate proficiency?
- Which aspect or objective of the specific standard does the activity, assessment, or AI resource address?
Knowing the answers to these questions will not only establish the degree of alignment—it will also drive your decisions regarding how you will use that activity, assessment, or AI resource to educate, evaluate, and encourage your students to learn.
Looking to deepen your own understanding of student learning, how to design your curriculum to support comprehension, and what questions to ask when you’re assessing student growth? Check out these hand-picked resources from our blog to learn more—and/or subscribe to our blog using the dark blue subscription box to the right.
About the educator
Erik M. Francis is an international author, educator, and presenter. He provides professional development on how to plan and provide engaging teaching and learning experiences that are academically rigorous, socially and emotionally supportive, standards-based, and student responsive. He is the author of Deconstructing Depth of Knowledge and Inquiring Minds Want to Learn.
