|
|
Ross L. Flynn, Doctoral Candidate
A growing body of literature underscores the importance of program evaluation for counselor education programs (Engels & Wilborn, 1984; Osborne & House, 1995; Sayers, Carroll & Loesch, 1996, CACREP Standards, 2001). Reasons for this changing climate include reduced resources (Osborne & House, 1995), increased accreditation requirements (CACREP, 2001), increasing demand for program accountability, improved decision-making and increased student/employer satisfaction (Hadley & Mitchell, 1995), and the need for quality data for program improvement (Osborne & House, 1995). There are no standardized measures in the professional literature designed to evaluate graduate counselor education programs and fulfill CACREP’s standards (Engels & Wilborn, 1984; Osborne & House, 1995; Sayers et al., 1996). CACREP’s 2001 Standards require formal follow-up studies of graduates, clinical site supervisors and program graduate employers at least once every three years “to assess their perceptions and evaluations of major aspects of the program” (Section VI.C.3, 4, 5). Only three comprehensive graduate and/or employer surveys exist in the professional literature (Engels & Wilborn, 1984; Osborne & House, 1995; Sayers et al., 1996). The first published survey was developed to assess the eight objectives of a particular counselor education program (at the University of North Texas) (Engels & Wilborn, 1984). A Survey of Graduating Students was administered at the end of a students’ last semester prior to graduation. A Graduate Follow-Up Survey was used to collect data from graduates at 1-, 3-, and 5-year intervals following graduation. The survey included 91 items in 36 questions, including eight demographic questions (Engels & Wilborn, 1984). The second survey in the literature was prepared by authors at Oregon State University (Osborne & House, 1995). The items in their survey derive generally from the major elements of accreditation standards (Sayers et al., 1996) as well as the objectives of their individual program (Osborne & House, 1995). The Graduate Survey included 93 items in 82 questions including ten demographic questions. The Employer Evaluation had 14 items. The authors mailed these surveys two years after graduation, suggesting this delay allowed both graduates and employers a base of experience to assess the value of their educational program. The survey was administered to graduates twice before publication, resulting in graduate response return rates of 57% and 70%. The employer survey return rates were 29% from community agency employers and 38% from school employers (ibid, p. 261). The third survey in the literature was developed utilizing an ACES grant by authors at the University of Florida at Gainesville using a “consensus approach,” collecting question items from approximately 50 graduate surveys from universities with CACREP-accredited programs (Sayers et al., 1996). The resulting 130 question items were submitted to 342 counselor educators who were asked to rate the items from 0 (not important to include) to 9 (very important to include) “to indicate the extent to which they believed it important to include each item listed in follow-up surveys for program graduates or for their employers” (Sayers et al., 1996, p. 181). Means and standard deviations of importance ratings were calculated to identify the most highly rated items in this consensus of counselor educators. The result was a Graduate Survey of 66 items and an Employer Survey of 46 items which was utilized with a greater than 75% return rate for both surveys. In the Spring of 2002, the authors at Oakland University collaborated to refine their graduate and supervisor/employer surveys by researching the literature and comparing graduate and supervisor/employer surveys utilized among CACREP-accredited universities. After discussion it was decided to develop and utilize a survey patterned after the Sayers’ et al (1996) “consensus” model. After review and input by the entire full-time counseling faculty, an instrument with 85 total questions was designed for graduates, and an instrument with 47 questions was designed for supervisors and employers. The Graduate Survey is included in the Appendix. The Graduate Survey was sent in March, 2004 to the 520 graduates of OU’s Masters in Counseling program since December, 1995. A cover letter from the department chair invited the graduates to give an enclosed Employer Survey and a postage-paid return envelope to their employers. The Supervisor Survey was mailed to 156 supervisors active since Spring, 2002. Respondents were assured in the cover letter that their responses would remain anonymous and confidential, and that only aggregate (i.e. grouped) data would be reported. A total of 103 graduates (20%), 27 employers (5%) and 17 supervisors (11%) responded as of August, 2004. The authors provided no means of identifying how many graduates gave their surveys to their employers as requested. Results were distributed to counseling program faculty, university administrators, graduates who requested copies, and the counseling library for review by students currently enrolled in the program and for review by prospective students. Benefits included curriculum changes and positive feedback from students about specific aspects of the program which improved faculty morale. Highlights of the Graduate Survey include: Ø Demographics of those who responded: mean age 39 years, SD = 10 years, range 25-63 years, 92% female, 97% white Ø How they identified themselves: 45% community agency or private practice counselor, 32% school counselor, 23% other (including social workers, career counselors, youth assistance caseworkers, “stay-at-home” mothers, etc.) Ø Months between graduation and start of job:
Ø Starting salary by identity for those working full-time (Univariate ANOVA F=3.198, df=12, p=.002)
Ø Current salary by identity for those working full-time (Univariate ANOVA F=8.217, df=11, p<.001)
Ø Current licenses/certifications held:
Ø Graduates’ Program Components (highest and lowest mean scores)
Ø Graduates’ Knowledge Attainment in Specific Curricular Areas and Graduates’ Skills Attainment While in the Program were also provided, and are available from the authors. Ø Major strengths of the program
Ø How could the program be improved?
Highlights of the Employer Survey and Supervisor Survey were also included in published reports. In summary, the surveys indicated: Graduates value highly the academic knowledge they received during their graduate program Graduates who responded were employed within a few months of graduation Employers rated graduates as dependable, conscientious, ethical, and professional (at the 97 to 99.6% level)(4.87 to 4.98 out of 5) School counselors’ starting salaries average 46% higher than community agency counselors’ starting salaries School counselors’ current salaries average 62% higher (> $20,000 higher) than community agency counselors’ current salaries 86% report being licensed as LLPC or LPC Graduates, employers and supervisors agreed on
Appendix A Graduate Survey
Oakland University, Department of Counseling Please provide the following information as appropriate: Graduates’ Demographic InformationAge: ______ Gender: _______ Race/Ethnicity: ____________________ The month and year you received your Masters in Counseling degree:
_____/_____ Please indicate your major emphasis of study:
What elective(s) did you take? ___ Significant Loss ___ Community Agency Cns ___ Intro to Couple & Family ___ Diagnosis & Treatment ___ Experiential Cns ___ Art Therapy ___ Intro to Chem Dep/Addictions ___ Other ___________________________ What specialization(s) did you complete? ___ Advanced Career Counseling ___ Mental Health Counseling ___ Child & Adolescent Cns ___ Couple & Family Counseling ___ Post-MA School Cns ___ Post-MA Teacher’s Cert for Cnslrs ___ Other _____________________ In regard to your world of work: How do you identify yourself? ___ Community Agency Counselor ___ School Counselor ___ Private Practice Therapist ___ Other ____________________ Current position title: ______________________________________________ Current employment setting (agency, school, institution name): _____________ _________________________________________________________________ Current agency/institution City/State location: ___________________________ _________________________________________________________________ Current primary counseling clientele: __________________________________ Current primary position functions/roles/job duties: _______________________ _________________________________________________________________ Hours worked currently: ___ fulltime ___ ¾ time ___ ½ time ___ ¼ time Post-graduation starting date and salary: Date _________ $______________ /yr Current income from counseling activities: Salary $______________ /yr
Current licensures and/or certifications held: ____________________________ _________________________________________________________________ Additional professional counseling activities other than your primary employment setting (e.g., private practice); please briefly describe your activities in this capacity: _________________________________________________________________ _________________________________________________________________ Graduates’ Program ComponentsPlease use a scale of 1 = low/poor to 5 = high/very good to indicate your personal evaluation of each of the following aspects of the program(s) from which you graduated: _____General evaluation of program curriculum _____General evaluation of academic knowledge received _____Self-evaluation of in-program skills development (e.g., Techniques) _____Evaluation of supervised field experiences (e.g., Internship) _____Evaluation of on-campus field experience (e.g., Practicum) _____General evaluation of instructional effectiveness _____Evaluation of faculty competence _____Evaluation of faculty accessibility/availability _____General evaluation of academic advisement _____Evaluation of facilities and resources _____Evaluation of supervisors’ competence/effectiveness _____Evaluation of faculty as mentors to you _____Evaluation of in-program student evaluation procedures (i.e., did you feel the procedures used to evaluate you were valid and effective?) _____Evaluation of program duration (i.e., 48-hour academic length of the program) Graduates’ Knowledge Attainment in Specific Curricular AreasPlease use a scale of 1 = low/poor to 5 = high/very good to indicate your personal evaluation of each of the following knowledge areas in your program(s). Use NA if the area is Not Applicable. _____Ethical and legal issues in your profession _____Small group dynamics and counseling _____Theories of counseling _____Career and lifestyle counseling _____Multicultural counseling _____Human growth and development _____General evaluation of practicum experiences _____General evaluation of internship experiences _____Large group dynamics and counseling _____School counseling _____Standardized (i.e., group) testing _____Crisis intervention/counseling _____Consultation _____Psychological clinical diagnosis _____Professional credentialing _____Theories of personality _____Family counseling _____Case management/planning _____Professional organizations _____Research and statistics _____Advocacy _____General evaluation of practicum supervisor(s) _____General evaluation of internship site supervisor(s) _____General evaluation of practicum site _____General evaluation of internship site Graduates’ Skills Attainment While in the ProgramPlease use a scale of 1 = low/poor to 5 = high/very good to indicate your personal evaluation of each of the following skill areas in your program(s). Use NA if the area is Not Applicable. _____Individual counseling skills _____Small group counseling skills _____Multicultural counseling skills _____Large group counseling/guidance skills _____Career and lifestyle counseling skills _____Crisis intervention/counseling skills _____Child and adolescent counseling skills _____Family counseling skills _____Consultation skills _____Case planning/management skills _____Psychological clinical diagnosis skills _____Assessment skills _____Couples/marriage counseling skills What were the major strengths of the preparation program(s) from which you graduated? In what ways could the preparation program(s) from which you graduated have been improved? What would you like to have added? Deleted? (Use back of sheet if necessary) Please return this survey by [DATE] to: Oakland University, Department of Counseling, 491B Pawley Hall, Rochester, MI 48309 References CACREP. (2001). The 2001 Standards, [internet]. Available: http://www.counseling.org/cacrep/2001standards700.htm. Engels, D., & Wilborn, B. (1984). Surveying graduating students and graduates of counselor education programs: Instruments, rationale and genesis. Counselor Education and Supervision, 23, 234-243. Hadley, R. G., & Mitchell, L. K. (1995). Counseling research and program evaluation. Pacific Grove, CA: Brooks/Cole. Osborne, J. L., & House, R. M. (1995). Evaluation of counselor education programs: A proposed plan. Counselor Education and Supervision, 34, 253-269. Sayers, R. D., Carroll, J. J., & Loesch, L. C. (1996). Follow-up survey for counselor preparation program graduates and their employers. Counselor Education and Supervision, 35, 179-189. |
|
|