Commonwealth Autism’s mission is to build the capacity of the autism and developmental disability service provider network through leadership, innovation, example and collaboration. After Executive Vice President and COO of the organization, Jessica Philips, took a class about nonprofit evaluation, it prompted her to question whether the organization was collecting the right kind of data to really evaluate the impact the organization was having on services for people with developmental difficulties and autism.
What we did:
To measure and guide the results for one of the organization’s key mission initiatives, helping strengthen public school services for students with autism, Knowledge Advisory Group recommended that they first create a logic model. With this graphical representation, the organization was able to identify relationships between the services they provided and the outcomes they were trying to achieve. More specifically, the logic model helped the organization remain realistic about what was achievable in the short term versus the long term. For example, while a teachers’ training session might immediately change knowledge and attitudes, changing teachers’ practices and the way they interacted with students with autism might require a different service and much more time.
With this foundation, KAG was then able to develop data sources and create data collection tools to assess Commonwealth Autism’s short- and long-term outcomes. We also helped the organization tweak existing surveys to save their staff time and minimize redundancy. KAG also recommended changing narrative response questions to those that used rating scales, thereby making the data easier to analyze and more meaningful to the organization.
KAG also streamlined the organization’s reporting process with a custom report template, which used graphic elements to visually depict findings and hone in on key recommendations. This improvement saved time not only in creating the reports, but also for the group’s customers/partners to review the reports.
What they learned:
Although the organization started with one key mission initiative, Commonwealth Autism has realized the value of developing an outcomes strategy for the entire organization and all of their key mission initiatives. “Investing time and resources into this process has helped us make more informed decisions about our evaluation needs at the program level,” says Philips.
Plus, understanding their impact on clients now and in the future has allowed Commonwealth Autism to be more intentional as they plan specific program activities. Over time, the organization has developed additional logic models, which they frequently reference to inform marketing efforts and to tailor messaging for different constituency groups, which they can then deliver consistently. These models have also served as a powerful tool to help Commonwealth Autism in departmental reorganizations.
“Knowledge Advisory Group has been instrumental in helping us develop the internal capacity to measure our impact. We’ve become more discerning about collecting and analyzing the right data, helping us more efficiently and effectively meet our mission.” —Jessica Philips, Executive Vice President and COO of Commonwealth Autism
