Adjust text size

September 13, 2013

Program Evaluation

The Center on Health Disparities Lunch at the Roundtable welcomes Dr. Allison A. Vanderbilt!

Dr. Vanderbilt is the Director of Assessment and Evaluation for the Center on Health Disparities and Assistant Professor of Assessment and Evaluation at VCU, School of Medicine. With years of experience conducting evaluations of federal grants, schools systems, and programming, Dr. Vanderbilt will bring her take on evaluation in community research to the table.

Friday, September 13, 2013

12pm – 1pm

Jonah L. Larrick Student Center – End Courtyard Ballroom B

Light refreshments will be served

 


Roundtable Recap

After introductions and background on Dr. Vanderbilt’s collegiate history and experience in program evaluation, Roundtable participants were encouraged to ask questions and respond to fellow participant requests. The following highlights the key outcomes from the September discussion.

 Question: How do we start?

How do we start collecting data?  I want to evaluate my programs, but how do we move beyond just “counting”? And, How can I better evaluate the success of our programs?

 Dr. Vanderbilt recommends:

  • Be concise. Focus your efforts on exactly what you want to know (your hypothesis or research question).
  • Be clear so that your materials are easy to read.Plan and edit your materials for the readability of your participants.
  • For special circumstances (e.g. remote locations, rural populations, long term studies), be sure to consider your sample size and attrition. You’ll need to plan for a larger sample size the longer your study is to account for people who will likely drop out of the study over time (attrition).
  •   Avoid common mistakes. It is very important to make sure that your research question, your data collection method, and measures all match. Don’t be afraid. We all make mistakes. Just be sure to learn from them, and be prepared. For example, a study that wants to test the success of a weight loss program asks participants and their family members to report their favorite colors (Are favorite colors really important to weight loss or your weight loss study? If not, this is probably not the best question to include. Were family members involved in the weight loss study or just the participants themselves? If not, this is probably not the right audience to complete your evaluation).
  •   Evaluation preferences. Quantitative styles are number oriented (think multiple choice and scales of satisfaction). Qualitative styles are open ended questions (think interviews and discussion groups). Both options have their pros and cons. The style or method you choose should depend on your audience and the amount of information you want to receive.

 

Question: What community resources are available to help us?

 

Dr. Vanderbilt assures that you are not alone! Several resources are available to support you.

  • Seek VCU/local university support. Search university websites for researchers doing work in your interest area and make contact.
  • Get involved with local community services. E.g. Virginia Department of Health.
  • Get key community leaders involved. Word of mouth can go a long way. Sometimes you never know who will be interested in what you are doing or who has friends in high places.
  • Contact me! Allison Vanderbilt (avanderbilt@vcu.edu)

 

Question: What does it really take to do a successful evaluation?

Dr. Vanderbilt reports:

  • Patience
  •  Theory based research question(s)
  • Unbiased and data driven evaluators (measurement personnel that are not part of your intervention or integrally tied to your outcomes)
  • Evaluators with prior experience or content knowledge (mentoring opportunities may be available to you at VCU)
  • Money? It depends on the detail of what you want to know, e.g. counting attendance vs. conducting complex formulas to determine measurable effect of each intervention strategy on project outcome(s). Research itself costs money (labor, materials, etc), but local opportunities are available to support community efforts! See VCU’s Division of Community Engagement ($20K opportunity) and other local initiatives. Get connected with VCU faculty – they almost always have their eyes open for the latest in funding opportunities.

Evaluation doesn’t have to be scary. Don’t wait until the end of your project to think about it. Plan ahead: Evaluation is as much a part of your research question as anything else that you do. And, if you need assistance, contact me.

I am happy to help. – Allison Vanderbilt, Ed.D.