Methods

Framing the Research – Center Directors and Evaluators Stakeholder Focus Groups

Consistent with the I/UCRC’s utilization-focused evaluation that provides actionable information to key stakeholders, project staff will host two telefocus groups. The first telefocus group will include up to six Center Directors and IAB members who will elaborate on the value of past exit interviews, potential additions to the semi-structured interview guide, and additional probes to uncover deeper motives that influence industrial sponsorship. Prior to the telefocus group, participants will receive a summary of the session goals and a few key questions to aid in recall and promote meaningful dialog during the session.

The second telefocus group will include up to six Center Evaluators who are familiar with both the I/UCRC evaluation project as well as qualitative research methods. This session will address the value of past exit interviews, potential improvements to a revised semi-structured interview guide, Center Director recommendations, analysis considerations, the depth of the transcriptions, and granularity of the analysis  (McLellan, MacQueen & Neidig, 2003).

Interview Guide Design

The results of these two telefocus groups help define the semi-structured interview guide evaluators will use during exit interviews as well as the analysis approach and interpretations of the interview data (Kvale, 1996; Patton, 1990). Collecting stakeholders’ knowledge and insights will not only help organize data reduction, analysis and interpretation but also insure that the research questions addressed are meaningful to stakeholders. To further aid in this effort, the interview guide will contain fields in which the Evaluator Interviewers can add their own notes, interview summaries, tentative interpretations and emerging hypotheses.

Data Collection

Every two months Center Evaluators will be reminded to complete interviews of all members leaving their Center via a centralized member exit reminder notification system created for this project. With permission of the local Center Evaluators, automated reminders may also be sent to Center Directors and Center Administrative Assistants via email asking them to let their local Evaluator and Project Staff know of any sponsor terminations. Once a terminating sponsor has been identified, Project Staff will follow up with Center Evaluators to ensure their compliance in completing this required evaluation component and check that their interview results are submitted into the project database.

Project staff will provide Center Evaluators a link to an on-line database for entering the interview data. This database, built by project staff, will contain the exit interview questions and follow-up probes. Center Evaluators will be able to enter categorical information, multiple choice and multiple selection feedback, and open-ended text. In order to ensure data security and to track which center evaluator is supplying the interview results, evaluators will be given a password and a unique identifier.

Combined with the automated reminder system, this central repository for exit interview data may be used as a permanent feature of the I/UCRC evaluation system.

Analysis & Reporting

Data will be analyzed using qualitative content analysis techniques (HyperRESEARCH will be the primary qualitative analysis software used). Miles and Huberman's (1994) model of qualitative data analysis (data reduction, data display and verification) will guide the qualitative analysis and help organize core themes and recommendations. Frequencies and categorical information will be analyzed using SPSS.

A preliminary report will be disseminated at an evaluator session to be held at the January 2010 Center Evaluators Conference in Washington, DC. Project staff will first present the results of this project followed by a discussion among Center Evaluators and other stakeholders to build an informed consensus on the future of the exit interview process. The final report will be submitted to NSF on or before February 19, 2010.