7.2: Criteria for Evaluating Web Resources in Education

Section Learning Objectives

• Explore key criteria for evaluation of educational technology: accuracy, authority, objectivity, currency, relevance, equity, and accessibility.
• Consider the specific needs of students and instructional goals when assessing web resources.

Evaluating Online Instructional Resources

When selecting online educational technology and resources to use for instruction, you should thoroughly evaluate potential materials to ensure they meet key criteria. Some criteria to consider include accuracy, authority, objectivity, currency, relevance, equity, and accessibility.  As we have discussed earlier, ease of use and visual interest are also very important, and while somewhat subjective, must be part of your evaluation. Before we continue our exploration of each of these criterion, watch the following TedxTalk from Dr. Shirin Shafiei Ebrahimi, who shares her research in the use of technology in education.

Use Of Technology In Education. [Video 12:02. Closed Captioned]

Accuracy

Evaluating accuracy means ensuring that the information in the resource is factual and error-free, by verifying equations, formulas, dates, and event details in math, science, and history materials. With the wealth of online educational resources available, evaluating accuracy takes on increased importance. Follow these guidelines when analyzing instructional tech for accuracy (Young, n.d).:

• Check facts, data, definitions against authoritative academic sources. Many websites post information without oversight. Verify accuracy through cross-reference.
• For instructional videos, independently confirm the creator's credentials and expertise. Evaluate claims and instructional methodology against accepted best practices.
• Scrutinize math and science tutorials closely. Check equations, formulas, sample problems, theorems, and laws of nature match established facts. Evaluate logical validity.
• With simulations and virtual environments, check that details match real-world evidence. Analyze if modeling of processes and systems is scientifically/historically accurate.
• For quiz/practice sites, verify question banks reflect updated facts and current consensus. Ensure scoring algorithms are free of errors.
• Review accuracy of translations for English learners. Check reading passages, audio narrations, captioning matches source content meaning.
• With AI tutors, continually evaluate responses against changing knowledge. Audit for outdated or inaccurate information periodically.

Authority

Evaluating authority means assessing the expertise of the author or creator to determine if they are knowledgeable in the subject matter. With online resources, authority is determined largely by the creator's credentials and reputation. Use these tips when evaluating authority (Building Technology Infrastructure for Learning, 2023).:

• Authority stems from relevant expertise and providing quality, trustworthy information. Favor instructional technology designed purposefully by credentialed professionals grounded in education research.
• Research the author, organization, and publisher behind the technology. Look for bios, qualifications, mission statements.
• Give more weight to resources created by educational institutions, government agencies, and established educational companies. 
• Be wary of anonymous or crowd-sourced material with no clear owning authority. Analyze who curated and filtered content.
• Examine evidence of credibility like citations, references, testimonials. Note awards, endorsements, or reviews by education experts.
• For apps and games, research developer credentials. Prioritize education specialists from academia or industry.
• For user-generated content like online worksheets, favor those with oversight mechanisms like ratings or expert verification.

Objectivity

Evaluating objectivity means assessing the balance and potential bias in the presentation of information. With endless resources available online, evaluating objectivity helps avoid biased or skewed information. Consider the following when reviewing the objectivity of instructional technology tools (Online Equity Rubric).

• Watch for imbalanced coverage of controversial issues. Resources should present multiple perspectives without obvious bias.
• Scrutinize language used. Favor neutral vocabulary over emotional or one-sided rhetoric.
• Check that statistics, facts, and images aren't cherry-picked to support one viewpoint over balanced presentation.
• For surveys and polls, analyze the sampling methodology. Determine if questions are neutral versus leading.
• With AI conversational agents, analyze if responses reveal bias through word choices, framing, or omissions.
• For simulations, examine if potential variables and complexities are sufficiently modeled. Or is a simplified narrative presented?
• With educational games and apps, assess if certain identities, roles, or professions are portrayed in stereotypical ways.

Currency

Evaluating currency means considering the publication date of resources to ensure they are up-to-date, especially in fields like science and technology. Publication date and version are especially important when evaluating educational technology tools. While some content is relatively static, topics like science, technology, and current events require up-to-date resources (Building Technology Infrastructure for Learning, 2023).

• Review dates on apps, sites, and platforms. Prioritize most recent versions and frequently updated content.
• In fast-changing fields like computer science, newer resources are crucial. However, older materials may represent “classics” with enduring relevance.
• For practice sites or apps with static question banks, check that content aligns to current standards, facts, terminology, and methods.
• With AI-based tools, assess if the knowledge model reflects current information. 
• For STEM simulations, analyze if models represent established scientific consensus and current data. 
• For topics like policies, events, and social issues, ensure resources use latest facts, statistics, and developments for contextual relevance.

Figure 7.2.1: Evaluation Criteria by Renee Albrecht. Licensed under CC BY.

Relevance

Evaluating the relevance means ensuring that resources directly align with specific curriculum goals, unit objectives, and student needs. Evaluating relevance ensures purposeful resource integration, not just flashy bells and whistles. Well-aligned resources enhance curriculum to help support student growth (Young, n.d).

• Align tools tightly to required learning standards and skills that will be assessed. Resources should directly support mastering objectives.
• Consider student prior knowledge, skills, and interests. Will the resource engage students at an appropriate level without boring or confusing them?
• For struggling students, favor adaptive, scaffolded resources that provide remediation and differentiated instruction.
• When evaluating apps and games, analyze if activities and problem formats match instructional goals, not just entertainment value.
• With simulations, determine if models effectively demonstrate the scientific principles or skills being taught.
• For AI tutors, assess whether dialogue, feedback, and activities scaffold student learning paths in a meaningful sequence.
• Examine whether assessments match classroom-based tests and high-stakes exams. Do question formats align?

Accessibility

Evaluating for accessibility requires some knowledge of accessibility requirements. If this is an area that you are not familiar with, it would be a good time to brush up on the information, or collaborate with someone who can help you in this area. Educational technology presents new opportunities to increase accessibility, but only if consciously designed. Prioritizing accessibility reduces barriers to participation and creates more inclusive learning environments. Consider the following when evaluating accessibility (Advancing Digital Equity for All, 2022):

• Review compliance with web content accessibility guidelines. Check for captions, alt text, keyboard navigation, color contrast.
• For video and audio, ensure accurate closed captioning and transcripts are available.
• Assess the reading level of content. Offer options to adjust text complexity, font sizes, and background colors.
• Evaluate visual design. Icons and buttons should include descriptive labels. Information should not rely on color alone.
• For voice interfaces, consider providing alternatives like chatbots and text input.
• Check for availability of resources in languages spoken by English learners. Offer machine translation if full translations are not feasible.
• Support learning disabilities through tools like text-to-speech, dictation, digital manipulatives.

Equity

Equity is the principle of ensuring fairness, justice, and impartiality in the distribution of resources, opportunities, and benefits, with a focus on addressing and mitigating systemic disparities to provide equal access and outcomes for all individuals, regardless of their background or circumstances. It seeks to eliminate discrimination, promote inclusivity, and empower marginalized groups, creating a level playing field for everyone (Advancing Digital Equity for All, 2022):

Equity should be a major consideration when selecting instructional resources. Selecting equitable instructional materials demonstrates valuing diversity, fosters positive identity development and creates a more just and inclusive classroom climate. Instructors should evaluate materials through an equity lens asking:

• Are various cultural and identity groups represented? Resources should reflect diversity including race, ethnicity, gender, disability, etc. Students should be able to see themselves represented.
• Are groups portrayed in fair, accurate, non-stereotypical ways? Look for balanced depictions that avoid tokenism. Authors and perspectives from non-dominant groups should be included.
• Analyze representation of diverse identities in content and imagery. Avoid stereotypes and provide positive portrayals.
• Does content promote inclusion, empowerment and equal access? Resources should use inclusive language and examples. Historical materials should depict the roles and contributions of marginalized groups.
• Are multiple perspectives present on issues of inequality and discrimination? Avoid single narratives around oppression or privilege. Include counter-narratives from marginalized voices.
• When evaluating relevance, also consider economic equity of access. Prioritize free or low-cost tools to avoid excluded students without means from benefiting. Technologies requiring purchase may seem cutting-edge but undermine equitable participation. Favor options allowing full access regardless of family income.

On the next page, you will explore another tool, focused on equity, that you can use in your teaching and learning practice.

References

Young, J. (n.d.). Identifying and evaluating instructional materials. Online Course Development Basics. CC NC SA 4.0

Peralta Community College District. (n.d.). Online Equity Rubric. Peralta Community College District.

Shafiei Ebrahimi, Dr. S. (2018, July 2). Use of Technology in Education. YouTube.

US Department of Education. (2022, December 22). Advancing Digital Equity for all. Office of Educational Technology. 

US Department of Education. (2023, August 7). Building Technology Infrastructure for Learning. Office of Educational Technology.