___________________

¹ The agenda and recordings from the December 5–6, 2023 meeting are available at https://www.nationalacademies.org/event/40702_12-2023_equitable-and-effective-teaching-in-undergraduate-stem-education-a-framework-for-institutions-educators-and-disciplines-meeting-5-and-public-input-session

² More information about the Roundtable on Systematic Change in Undergraduate STEM Education is available at https://www.nationalacademies.org/our-work/roundtable-on-systemic-change-in-undergraduate-stem-education

• Organizational submissions (17 groups and individuals within organizations [listed in Table A-1] that provide or advocate for undergraduate STEM education submitted comments).

Given the response rates and representation across institution types, roles, and respondent demographics, disaggregating any subgroup with statistical significance was difficult. However, this analysis did use disaggregation to identify broad themes.

The analysis of the open-ended responses on the questionnaire, meeting summaries, and organizational submissions took this approach:

1. Each response was coded in a matrix, organized by survey question and thematic categories.
2. Category responses were reviewed to discover prominent themes within categories.
3. Categories and themes were identified by survey question (e.g., What’s missing from Principle 3?).
4. Top overarching themes for the draft report, framework, and principles for systemic change in undergraduate science, technology, engineering, and mathematics (STEM) education were identified and synthesized across all questions.

Overall, the public feedback was positive, with no significant requests for changes to the content or organization of the Principles for Equitable and Effective Teaching, although there were suggestions to improve the draft.

This summary of the public input on the discussion draft includes five overarching themes, broad themes for principles and practices, suggested actions for departments and institutions, and recommendations for strengthening the Principles for Equitable and Effective Teaching in undergraduate STEM education. It also includes the Committee’s responses based on the feedback received.

FIVE OVERARCHING THEMES

1. Offer More Contextualization. Respondents asked for more emphasis on the critical importance of equitable and effective STEM education in the broader global context, both at the macro level and for individual students. They wanted more discussion about the current state of equity and effectiveness in undergraduate STEM teaching. They indicated a need to incorporate varied perspectives and partnerships (including with students) to improve teaching.

2. Improve the Framework. Respondents offered suggestions to improve the framework to achieve the goal of equitable and effective STEM education:
   • Go beyond the classroom. To drive meaningful change, take a bolder and broader approach to address out-of-classroom challenges, rather than focusing solely on classroom-dominant solutions.
   • Make the Principles for Equitable and Effective Teaching actionable—and provide examples. Convey what is realistic and attainable. Frame the Principles with clear context, supported by evidence-based theories, and include thorough references and illustrative examples. Provide more clarification on existing effective efforts, identify the stakeholders responsible for specific practices, and ensure the language is accessible to non-education stakeholders. Respondents repeatedly emphasized the need to illustrate concepts and practices with concrete examples of implementation across modalities and institution types, particularly for faculty; responsibilities of stakeholders at all levels; and ideas for how the examples can be scaled.
   • Integrate the evidence. Better connect the evidence and citations with the practices in the framework, rather than as a separate section. Survey respondents offered suggestions for more evidence and suggested that evidence must better represent diverse schools.
   • Convey the interconnectedness of the Principles. Acknowledge that the Principles do not operate in isolation and often overlap with each other. For example, integrate the messages about belonging to messages about engaging in disciplinary learning.
3. Address the Broader Ecosystem. Across all forms of public input, respondents indicated that it is imperative to address the broader ecosystem (including K–12, the workforce, disciplinary organizations, and nongovernment organizations):
   • Emphasize institutions and departments. Highlight the role of institutional and departmental policies in implementing equitable and effective teaching. Acknowledge that classroom practices alone will not adequately address this challenge. Clarify institutional and departmental responsibilities more broadly across the disciplines, while still reflecting discipline-specific needs.
   • Connect to K–12 education. Consider the connection to K–12 education (including Next Generation Science

3. • Standards [NGSS]) and partner with K–12 systems to provide guidance on how to address gaps in prior knowledge and recruit more students into STEM.
   • Increase focus on community colleges and career and technical education. Expand discussion on connections with community colleges, particularly the emerging role of community college prerequisite work, and the relevance of career and technical education.
   • Broaden organizational recommendations. Involve professional societies in enacting the recommendations and encourage federal agencies and accrediting associations to update standards that incentivize equitable and effective teaching practices.
4. Focus on Faculty. Respondents overwhelmingly emphasized the critical role of faculty in addressing equitable and effective STEM education. They suggested adding a faculty-specific principle that calls for directly supporting and providing professional development for all faculty types, informed by broader and more representative faculty input:
   • Broaden the traditional concept of faculty. Include teaching assistants, adjunct, and non-tenure-track instructors.
   • Align faculty incentives. Create a stronger alignment between equitable and effective teaching and career advancement. Reform teaching evaluations incorporated in review, promotion, and tenure decisions.
   • Emphasize effective faculty professional development and support. Consider enriching professional development and other opportunities (e.g., mentors), and include self-reflection and explicit instruction on equitable and effective teaching across all modes of instruction (including laboratory opportunities). Emphasize how student and faculty success are intertwined.
   • Consider workload and resources. Pay attention to faculty workload and increase faculty buy-in by ensuring they have the time, resources, and professional learning to support effective implementation. This is critical—and will require clear instructional guidance for institutions.
5. Coordinate Curriculum Reform. Respondents stressed a need for curricular reform, with student input, that emphasizes accessible and effective active learning with real-world opportunities. They suggested explicitly recognizing that core STEM content, skills, and knowledge are necessary for STEM success and that equitable and effective teaching should not sacrifice rigor:

6. • Address discipline-specific concerns. Make interdisciplinary connections, but attend to discipline-specific issues, particularly requirements to address the leaky STEM pipeline, such as math reform.
   • Personalize learning. Maintain high standards while emphasizing the need for pedagogical approaches that personalize learning to accommodate diverse needs. Incorporate progressive, evidence-based instructional techniques. Consider, but do not overemphasize, demographic identifiers.
   • Make more real-world and workforce connections. Add more overt links to the workforce and to real-world experiences and opportunities, including experiential and lifelong learning. Emphasize the importance of instructional labs for active learning and of ensuring quality instruction in all types of environments.

Committee Response

The final report offers more contextualization to demonstrate where the Principles for Equitable and Effective Teaching fit into the overall system of higher education. It includes a chapter on using the Principles to improve learning experiences (Chapter 5). The evidence for the Principles is no longer separated out in a separate section (Chapter 4) and additional language has been added about the interconnections between the principles. Additional material is added about the current state of undergraduate STEM education (Chapters 2 and 3). Additional examples of have been added as has material on community colleges and career and technical education, including discussion of transfer pathways.

Multiple recommendations are included that focus on faculty and other types of instructors. They call for directly supporting and providing professional development for all types of instructors. The definition of instructors used in the report has been explained to be clear that it includes VITAL and tenure track educators. Recommendations also call for academic units and institutions to provide professional learning and development and to support instructors in other ways including by rewarding efforts to improve teaching (and to improve how teaching is evaluated).

Extensive portions of the final report discuss the role of departments and institutions in implementing equitable and effective teaching and clearly state that classroom practices alone will not adequately address this challenge (Chapters 6–9). Recommendations for funders and disciplinary societies have been included.

• with a balanced and engaging curriculum and address barriers (e.g., whether math is an important gatekeeper across STEM disciplines). Make evidence-based efforts to standardize sections and courses to ensure continuity between professors and reduce duplicative efforts, where appropriate. Include broad perspectives in developing and reviewing curriculum, (e.g., alumni, students, faculty) and consider the outcomes for all students (STEM majors and beyond).
• Provide whole-student support, including advising, peer mentors, tutoring for prerequisite/foundational courses, and corequisite support. Approach each student as an individual and focus on the student-teacher relationship.

Committee Response

The report now includes the importance of sustained faculty development and the role of evaluations and rewards in accountability and promotion. The role of departments and other academic units in supporting and encouraging attention to policies and practices that support equitable and effective teaching is included. The need for them to analyze pedagogy and curriculum is also included.

Institutional Actions

Institutional actions that were suggested by respondents included

• Create incentives and broaden accountability. Offer robust faculty support and development for cultivating equitable and effective instructional practices, including aligning review, promotion, and tenure decisions with the integration of these Principles and ensuring thorough evaluation of course outcomes. Acknowledge and reward exemplary and equitable teaching practices and extend these efforts to non-tenure-track faculty, including adjuncts. Factor in faculty support and workload considerations throughout these initiatives.
• Optimize operations. Remove obstacles for departments and faculty to achieve effective and equitable teaching, which involves incentives, accountability, and resources (funding and time). Ensure that departments, faculty, and students have access to comprehensive support services and institutional policies promote equity and visibly demonstrate the priority of effective and equitable teaching (via aligned policies and accountability measures). Improve the connection to K–12 education via outreach and support.
• Strengthen communications, engagement, and leadership. Improve

• strategies to articulate a shared and coherent commitment to equitable and effective teaching, considering diverse perspectives from departmental leadership, faculty, and students, while actively involving leaders at all levels of the institution. Some respondents suggested that it is important to hold upper administrators accountable for a commitment to academic excellence.
• Foster cross-disciplinary and interdisciplinary collaboration. Prioritize a broad STEM foundation for all students and cultivate cross-disciplinary collaboration among faculty. Focus on building a strong interdisciplinary community committed to equity, with the goal of breaking down institutional silos and promoting collaboration both within and outside STEM disciplines.

Committee Response

The report discusses the important role of institutions and institutional leaders in setting priorities and influencing culture around instruction and student-centered approaches. It discusses the need to offer instructors support to improve their teaching and also the importance of holding people accountable for teaching (Chapter 10).

SUBMISSIONS FROM ORGANIZATIONS AND INDIVIDUALS

TABLE A-1 Submissions from Organizations and Individuals