Request for Quotes (RFQ): External Evaluator(s) for National Ocean Sciences Bowl Needs Assessment

Submission Deadline for Quotes: September 25, 2023

  • National Ocean Sciences Bowl (NOSB); www.nosb.org
  • Center for Ocean Leadership (COL)
  • University Corporation for Atmospheric Research (UCAR)

The National Ocean Sciences Bowl (NOSB) at the Center for Ocean Leadership (COL) is inviting evaluation consultants or firms to submit quotes to conduct a thorough programmatic needs assessment. Ideal candidates should have experience in STEM education (ocean and climate science preferred), informal education, diversity, equity and inclusion (DEI) practices and efforts, and workforce development. We are seeking evaluator(s) with experience in study design, effective evaluation methods, statistical analysis, experience and aptitude to work effectively with identified stakeholders, and demonstrated experience in culturally competent data collection aligned with guidance from the American Evaluation Association. Experience running projects where Institutional Review Board (IRB) approval for Human Subjects Research was sought and obtained is also desired.

Background

The NOSB is a vital education program and academic competition that has played a crucial role in promoting ocean literacy and inspiring the next generation of ocean scientists for the past 25 years. The competition (25 regional competitions and one Final competition per year) and ancillary activities (experiential field/award trips, webinars, career mentoring, etc.) provide a platform for students to learn about ocean science and the wealth of ocean science-related careers. Students gain skills to become future leaders in the field, contributing to sponsors’ efforts to promote ocean literacy, enhance public awareness of ocean issues, and foster the next generation of scientists and knowledgeable citizens. The NOSB has operated for 25 years in its current quiz bowl format without major changes. In that time, education and ocean literacy needs have evolved and the COVID-19 pandemic had myriad impacts on students and education. The NOSB program also recently transferred to the Center for Ocean Leadership (COL) at the University Corporation of Atmospheric Research (UCAR). Given these factors, it is a beneficial time to conduct a thorough needs assessment.

The NOSB strives to reassess and reimagine the program to better align with the current ocean science education, research, technology, and workforce development needs of the nation. The needs assessment will identify strategic changes to the competition and associated activities, ensuring the overall program better serves the evolving needs of participants (high school students and teachers), improving and increasing its engagement and participation of underserved students and teachers. The reimagined NOSB will provide innovative approaches to integrate and enrich the STEM education and create opportunities for students to make connections between what they learn and the actions they can take to ensure a healthy and sustainable ocean. The new format will address the national gap in earth and environmental sciences in public high school education, while continuing to focus on the NOSB’s four original, but still relevant and important, goals:

  • Develop knowledgeable ocean stewards that understand the ocean’s impact on daily life and the importance of scientific research
  • Foster the use of the ocean as an interdisciplinary vehicle to teach science and mathematics and encourage the inclusion of the ocean sciences in curricula
  • Encourage and support the involvement of under-represented and geographically diverse communities in the ocean sciences
  • Provide students interactive education that develops critical thinking and skills for the workforce and exposes them to ocean science professionals and career opportunities

To evaluate the effectiveness of the NOSB in achieving its goals and identifying areas where improvements can be made, an external evaluator will bring a fresh perspective and unbiased approach to identifying strengths and weaknesses, opportunities for improvement, best practices for implementation of changes, and demonstration of value and impact to program sponsors.

Scope of Work

The needs assessment is expected to commence in November 2023 and be completed by July 2025. The following are the expected objectives, outcomes and outputs of the needs assessment that will include action, input and/or support from the hired external evaluator(s).

Objectives

  • Identify the current strengths and weaknesses of the NOSB program, including competition format, ancillary activities, partnerships, engagement of students and teachers, and program communication and evaluation
  • Determine the best new format for the academic competition portion of the program at the national, regional, and/or combined level to ensure activities encourage student critical thinking, inquiry, problem solving, and solution-based thinking
  • Determine best methods for incorporating workforce development skills and career mentoring activities and events in the program
  • Determine the barriers and other issues that prevent underserved students and teachers from participating in the NOSB and ocean science / STEM education programs in general
  • Determine best methods for expanding engagement of underserved students and teachers
  • Determine opportunities for incorporating additional learning opportunities, outside of competition, for students and teachers, potentially including field- or laboratory-based experiences, internships, fellowships, or other activities

Outcomes

  • Identification of the educational needs of the various stakeholders involved in the NOSB, including students, teachers, regional coordinators, and sponsors
  • Identification of educational needs of underserved students and teachers
  • Identification of the programmatic structure and impact of the NOSB expected by current, recent, and prospective sponsors
  • Assessment of educational infrastructure and resources at UCAR available to assist NOSB in the needs assessment and implementation of recommended changes
  • Identification of potential partnerships or collaborations that could enhance the NOSB’s educational offerings
  • Recommendations for changes or improvements to the existing NOSB competition and ancillary activities
  • Identification of potential barriers to NOSB success

Outputs

  • Detailed needs assessment surveys or questionnaires to gather data from the NOSB’s variety of stakeholders
  • A list of key stakeholders for important interviews or focus groups conducted as part of the needs assessment
  • A workshop focused on diversity, equity, and inclusion (DEI) in the NOSB and how to better engage underserved students, teachers, and schools
  • A detailed report from the external evaluator that summarizes the findings of the needs assessment, including data analysis, conclusions, and recommendations for improvements
  • A SWOT analysis that identifies the strengths, weaknesses, opportunities, and threats of the recommendations for implementation

Expertise Required

The chosen evaluation consultant/firm will have experience in study design, effective evaluation methods, statistical analysis, experience in informal STEM education assessments, experience and aptitude to work effectively with identified stakeholders, and staff, demonstrated experience in culturally competent data collection aligned with guidance from the American Evaluation Association (ethically responsible engagement is honest, accurate and respectful of stakeholders, and considerate of the general public welfare; diverse voices and perspectives are honestly and fairly represented; and the evaluation theories are shaped by cultural values and perspectives), a strong reputation in the field, independent and unbiased, excellent communication, and ability to complete project within schedule and budget. A bonus would be assistance with workshop development and facilitation.

The evaluator(s) will be responsible for designing the evaluation study, defining inputs, goals, activities, and intended outcomes, developing data collection processes, analyzing the collected data, preparing a final evaluation report, and providing recommendations for changes or improvements to the existing NOSB competition and ancillary activities. NOSB staff will assist with the development of the study, communicating with partners and sponsors, discussing the overall needs and objectives of the evaluation, overseeing the progress of the needs assessment, including adherence to budget, timelines, and milestones, assisting the external evaluator with connections to stakeholders, and providing access to NOSB historical data and program materials. NOSB staff will also assist with research on competition formats, potential new partnerships, and prospective sponsors.

The NOSB’s long-term evaluators from the College of Exploration and Ashland University, who have conducted an annual study of program alumni since 2002, as well as special focus studies on NOSB’s impact on secondary tier students and instructional choices and team preparation, will also be involved. They will provide historical input on the program’s long-term evaluation, impacts of NOSB participation on enhancing learning and skill development, impacts on college and career paths, suggestions for potential program revisions based on past surveys and reports, and literature reviews on topics related to academic competitions and informal science and STEM education.

We understand that the entirety of the needs assessment work described above may be outside the scope of work of any one single consultant/firm. Therefore we will consider hiring an evaluation team from one firm, several consultants from multiple firms, or several independent evaluation contractors if necessary. Such decisions will be based on proposals and quotes from interested consultants or firms and we can determine each of their strengths in evaluations, especially as they relate to STEM, science (ocean and climate) education, informal education, DEI initiatives, and potential workshop facilitation. Therefore, consultants/firms that can only meet partial requirements should still submit a proposal detailing which requirements and metrics they can meet, with the understanding that funding may ultimately be split among multiple consultants/firms.

Please note, the final needs assessment plan agreed upon by COL, NOSB, and the evaluator(s) will require approval by the Human Subjects Committee (HSC), UCAR’s institutional review board or IRB. Therefore consultants or firms with experience running projects where IRB approval was sought and obtained are preferred.

Submissions

Submission Requirements

Interested consultants or firms are requested to submit their proposals by September 25, 2023. Proposals should include the following:

  • Cover letter expressing interest and summarizing the consultant/firm’s relevant experience
  • Details of the approach and methodology to be used in the evaluation
  • Details on the approaches to confidentiality and informed consent/consent processes
  • Details on experience running projects where IRB approval was sought and obtained
  • Proposed timeline for completion of the evaluation
  • Examples of previous similar projects or evaluations conducted by this consultant/firm
  • Names and qualifications of the key personnel to be involved in the project
  • Budget outlining all costs associated with the project, including consulting fees and other relevant expenses

Evaluation Metrics

Proposals will be evaluated, scored and ranked based on the following criteria:

  • Demonstrated experience in program evaluation and needs assessments (15%)
  • Evaluator(s) experience and qualifications (15%)
  • Proposed methodology and approach (10%)
  • Knowledge and experience in STEM, ocean, and/or climate science (10%)
  • Knowledge and experience in informal education (10%)
  • Knowledge and experience with DEI in education (10%)
  • Knowledge and experience in workforce preparation and development (10%)
  • Budget (15%)
  • Adherence to timeline (5%)

Expected Timeline*

  • Submission deadline for clarifying questions: September 18, 2023
  • Submission deadline for quotes: September 25, 2023
  • Proposal selection: October 28, 2023
  • Needs assessment kickoff: November 2023
  • Finalize data collection: November 2024
  • Submission deadline for evaluator(s)’ final report(s): February 2025
  • Completion of evaluator(s)’ participation: July 2025

*This timeline may shift slightly based on finalized study plans with hired consultant(s) an/or firm(s).

Contact

Please submit your questions for clarification and quotes to:

Melissa Brodeur
NOSB Program Manager
mbrodeur@ucar.edu (Subject line: NOSB evaluation submission)

We look forward to receiving your proposals and finding a suitable partner(s) to conduct the NOSB needs assessment. Thank you for your interest in our mission to promote ocean science literacy, STEM career exploration, and environmental stewardship.

The Center for Ocean Leadership (COL) manages, coordinates, and facilitates ocean research and education programs in areas of ocean exploration, ocean observing, and informal education. COL has a proven track record of shaping the future of ocean science and technology through discovery, understanding, and action.

UCAR is a nonprofit consortium of more than 120 North American colleges and universities focused on research and training in Earth system science. We bring together the Earth system science community to exchange ideas, discuss challenges, and share what we’ve learned. By connecting researchers and educators with each other, with cutting-edge resources, and with the private sector, we take research out of the lab into the real world for the benefit of society. UCAR manages the National Center for Atmospheric Research (NCAR) on behalf of the National Science Foundation. We have accelerated and amplified the impact of scientific discoveries at NCAR and our member colleges and universities since 1960.