Evaluation is a new discipline. The American Evaluation Association was only founded in 1985.

As such, researchers and students in the interdisciplinary field come from a variety of areas including sociology, communications, psychology, statistics, health, and education. Private firms and government entities including the Department of Education also carry out program evaluation.

“Over the last two decades, the National Science Foundation and the National Institutes of Health began to require evaluation for all funded projects, but there’s no accreditation process for evaluators,” says Reid.

“We’re really at a point where everything is kind of like the Wild West – everyone’s doing something different,” Boyce adds.

SPEL traces its roots to fall 2015, when Boyce and Reid joined the Department of Educational Research Methodology as an assistant professor and graduate student, respectively. Under the auspices of the Office of Assessment, Evaluation, and Research Services, which she co-directs, Boyce created the lab in 2018.

The following year, Reid graduated and joined SPEL. They are both co-directors, and within their field, the researchers say, having two faculty running a laboratory is rare. The lab now includes another faculty member, Dr. Tiffany Tovey, as well as twelve graduate students and six undergraduates. That’s not all that distinguishes SPEL.

That orientation leads lab members to focus on culturally responsive evaluation, keeping the cultures of project participants in mind as they examine project methods and measure project impacts.

For example, researchers might assess a project’s anti-deficit orientation – a way of thinking about underrepresented groups.

“Instead of saying, ‘Oh, we need to help these poor people, what’s wrong with them?’ we say, ‘What’s wrong with our own culture so that certain people can’t succeed?’” Reid explains. “And we ask what strengths those groups bring versus saying, ‘Here’s all the things wrong with them.’”

The researchers express a deep sense of ownership and pride in the projects their lab helps advance.

A recent example is the $10 million NSF-funded STEM PUSH network, which connects and advances pre-college programs that increase the preparation of ethnically and racially underrepresented students for college STEM programs. PUSH also seeks to expand the ways college admissions offices assess potential students, diversifying beyond standardized test scores and grades.

With the shifts in admissions requirements due to the pandemic, it’s a perfect time to push for a reassessment of traditional metrics, says Reid. “We’re in a kairos – a critical moment – for admissions.”

Other large clients include the Purdue University-based Network for Computational Nanotechnology, which has been in operation for 20 years, and the Midwest Big Data Innovation Hub connecting 12 states and six research universities.

“When we started off we had smallish projects that had between like $50,000 and $100,000 over the life of the projects,” Boyce says. “The really exciting aspect is that, as more time has gone on, we have more opportunities for large-budget projects.”

Closest to home is the prestigious $1 million NSF ADVANCE grant recently won by a team of physical and social science faculty at UNCG. The project, which includes Boyce as a co-principal investigator and Reid as senior personnel, seeks to promote gender equity at the University.

Their aim, the researchers say, is to change culture, climate, and policies at UNCG – and not just within STEM fields.

This summer SPEL ran 13 active projects, each involving at least two graduate researchers and one undergraduate. Since the lab’s launch, 20 students have worked there, developing their research skills by conducting interviews, focus groups, document reviews, and surveys.

“Through SPEL, we are able to mentor the master’s and PhD students as faculty members,” Reid says. “And then they in turn get experience, not just in evaluation research and evaluation practice, but in mentoring undergraduate students.”

Those undergraduates are another feather in the SPEL cap.

In fact, there are no undergraduate programs in the young field of evaluation. Hence Reid and Boyce focus on mentorship of undergraduates from other fields. Some will hopefully move on to graduate programs in evaluation.

For those who don’t, says Reid, it’s still “a big win, when they can begin to make those connections with the work that they’re doing and transfer it to their own field – it brings evaluative thinking into another area.”

The lab offers a high-impact experience. In addition to learning evaluation skills, students create data visualizations, generate reports, and present to funders and at conferences.

All lab members take part in weekly three-hour lab meetings, which include an hour of professional development as well as project check-ins. Reflecting the SPEL’s values of social justice, anti-racism, and inclusivity, the group discussed “How to Be an Antiracist,” by Ibram X. Kendi, for some of their professional development hours.

With their emphasis on diversity and inclusion both in their work and within their lab, Reid and Boyce embody the values they espouse.

“We have students from all walks of life,” Boyce says proudly. “I mean ethnicity and race, but I also mean in terms of age, background experience, having children and not having children – they are in all kinds of places in their lives.”

That makes a difference to the work, says Reid. “I think we are transforming the landscape of STEM program evaluation because of who we are and how we’re doing it.”