Jill K. Singer, a State University of New York (SUNY) Distinguished Teaching Professor in the Geosciences Department at Buffalo State University, currently leads two Advanced Technological Education (ATE) projects that are developing new variations of the EvaluateUR Method.
The EvaluateUR Method measures student learning by providing statistically reliable assessments of student growth in a wide variety of outcome categories, including disciplinary and employability skills. A unique feature of the EvaluateUR Method is the growth of student academic self-awareness, often called metacognition, that the method fosters.
The EvaluateUR Method is implemented online and serves as a tool for students, helping them build that awareness and use it to improve their learning and problem-solving skills. The website resources include a series of short exercises designed to foster metacognitive practice.
The two current ATE projects examine the learning outcomes of aspiring technicians whose community and technical college experiences include independent or course-based research, internships, and academic competitions.
“The idea is to help the students really take the lead and responsibility about what they're doing and how they're doing it,” Singer said. The ATE-supported expansions of the EvaluateUR Method are designed to address students’ various experiential learning opportunities and the time constraints of community and technical college faculty who are preparing technicians for advanced technology careers.
A New Approach to Evaluate Student Learning Outcomes Resulting from Participation in Remotely Operated Vehicle Competitions (Award Abstract 1932929) has developed E-Compete for students participating in engineering design competitions. While initially developed to support MATE’s remotely operated vehicle (ROV) competitions, it has been expanded to support other collegiate competitions including the Society of Automotive Engineers Baja competition. In a Journal of Advanced Technological Education article, Singer and five co-authors reported “a primary benefit of the method is that it encourages students to become more aware of what learning strategies they employ to analyze and solve problems.”
Extending the EvaluateUR Method to Expand the Community of Users project (Award Abstract 2325397) is introducing new options that add flexibility to the method and eliminate barriers that in the past have limited two-year college faculty from using the EvaluateUR Method. (Read more about this project in the full ATE Impacts Blog post.)
This spring the project is launching a community forum where faculty can post questions about the EvaluateUR Method and receive advice from other educators who are using it. Singer encourages two-year college faculty who are interested in using the EvaluateUR Method to explore the program’s website and email her at [email protected].
--
Pilot Testing of New EvaluateUR-CURE (E-CURE) Options at Whatcom Community College
Faculty at Whatcom Community College, which has 10-week quarters, have been helping to pilot test some new options in E-CURE, one of the variants of the EvaluateUR Method designed for course-based undergraduate research courses (CUREs) in geology and engineering. One of the options involves only the students scoring the assessments and another option is designed for smaller scale or shorter duration research projects.
The Whatcom faculty members are assessing their students on the outcomes that best align with their course goals from the options listed on EvaluateUR and EvaluateUR-CURE Outcome Components. They can add course-specific options if they choose, as well. The participating students’ research projects incorporate both disciplinary knowledge and employability skills. Singer uses the term “employability skills” rather than “soft skills” to reflect the high value that employers place on workplace skills such as teamwork, communication, and problem-solving.
There is no charge for using E-CURE and faculty may request a course dashboard by completing a short online form. The project team also provides onboarding resources as well as general and technical support.
New Approach for Evaluating Internships
The project is also developing a variant for students participating in internships. This new variant recognizes that while industry employers welcome having interns from community colleges, they are sometimes uncomfortable assessing the students’ knowledge and skills using EvaluateUR-Internship’s online assessments.
Singer described how E-Internship, the latest addition to the Evaluate-UR Method, is being pilot tested at Hudson Valley Community College and Portland Community College.
Before starting their internships, students participating in the project answer a set of open-ended questions that encourage them to think about their internship experiences. Students then score themselves on the set of outcomes that the internship coordinator has identified. After submitting their responses, the students receive an automated email that lists things for them to think about as they embark on their internships. It also prompts them to schedule a meeting with their internship supervisor to talk about internship outcomes and workplace expectations. After that meeting, the students return to the Evaluate-UR online system to answer a few reflective questions. This process is repeated at the mid-point and at the end of the internship.
According to Singer, E-Internship helps students think about what they are learning and how information from their coursework is applied to address real-world problems. The design encourages students to have periodic conversations with their supervisor, but the built-in prompts prepare them for these conversations. The students are also encouraged to talk to other people they interact with during their internships.
The Origins of EvaluateUR
Singer began work on EvaluateUR more than 20 years ago when, as director of the Undergraduate Research Office at Buffalo State, she wanted to document the impact of the college’s investment in the students conducting research. She and Daniel Weiler developed the methodology that grew into EvaluateUR.
Successful use of EvaluateUR at Buffalo State led to grants for scaling its use at other higher education institutions and involved close collaboration with the Science Education Resource Center (SERC) at Carleton College where the EvaluateUR method is hosted. Sean Fox, the technical director at SERC, is responsible for developing the code that underpins the EvaluateUR method.
Articles about EvaluateUR include a chapter in The Cambridge Handbook of Undergraduate Research doi:10.1017/9781108869508.021 and an article in the Journal of Engineering Technology.