The System by which Candidate Performance is Regularly Assessed

The UAB programs for the preparation of teachers, school counselors, and educational administrators have employed a variety of techniques for the assessment of programs and candidates for many years. In 2003/2004 as part of efforts to improve unit cohesion and performance, the unit developed a comprehensive assessment system that was aligned with the unit’s new vision, mission, and conceptual framework. The system needed to be able to accommodate outcomes of diverse programs (initial and advanced teacher education, school counseling, educational administration) while still allowing for the ability to aggregate and communicate unit level progress and improvement in clear and consistent ways. Initial steps toward implementing the comprehensive new system began during the spring of 2004. The new system was fully implemented for students entering all initial programs during the fall of 2004 and will be fully implemented for all advanced programs by the fall of 2005.

The unit is committed to preparing teachers, school counselors, and school administrators to be successful in diverse contexts, such as those found in the Greater Birmingham Metropolitan Area. Specifically, the faculty is committed to preparing professionals who are focused on the needs of the learner, trained in accordance with state and national standards, engaged in reflection to advance their craft and inquiry to advance understanding and the knowledge base, and who understand that change in learners and in professionals is a developmental process. As a result, the assessment of candidates considers: (a) the degree to which the candidate is responsive to the needs of learners, (b) the degree to which the candidate has demonstrated critical knowledge, skills, and dispositions expected by state and national professional organizations, (c) the candidate’s use of reflection to improve their craft and/or their use of systematic inquiry to enhance personal or generalizable knowledge, and (d) the candidate’s awareness of the developmental status of learners and of their own position along a continuum of professional development. The degree to which the candidate is learner centered is evaluated by the candidate’s documentation of their impact on the learner and by evidence of their responsivity to the needs of diverse learners. The degree to which the candidate has demonstrated standards-based knowledge and skills is evaluated by their performance in courses and field experiences as well as by high-stakes course-embedded assessments which directly reflect critical state and national standards in the professions (e.g., assessments within the eleven outcome areas of the teacher education programs relate to INTASC and state standards for teachers). The degree to which the candidate demonstrates expected dispositions is evaluated through observations during coursework and fieldwork. A discussion of the dispositional assessment system is included in the discussion of Standard II. The extent to which the candidate is reflective and is aware of field-specific methods of inquiry is assessed as a part of coursework and high-stakes assessments embedded in courses and field experience.  Furthermore, many candidates for advanced degrees must engage in systematic inquiry within their field in order to enhance their own professional development or to add to the knowledge base of the field. The candidate’s awareness of the development of learners and of themselves as professionals is evaluated as part of coursework and through high-stakes course embedded assessments and self-reflections during internship. The developmental continua of professionals and performance criteria at each level of the continua are explicitly represented in novice to expert progressions of knowledge and skills in each outcome area for each program. In order for candidates to complete their programs they must display performance level skills at criterion-levels in outcome areas.

The assessment system for candidates essentially provides information to facilitate decisions regarding admission, continuation, and recommendation for certification. Criteria and methods for admission to programs at initial and advanced levels are included in Section II and associated appendices. Once admitted to a program, professional education candidates are required to demonstrate knowledge, skills, and dispositions within program outcome areas. All programs use a rubric-based system for describing the various levels of professional progression of knowledge and skills from novice to expert levels within each outcome area. These rubrics convey the expectations of faculty and the professional community of teachers, administrators, and counselors as the candidate develops in various critical areas of the profession. The rubrics operationally define the most critical goals of the curricula. The rubrics for each outcome area for any program reflect the developmental continuum for knowledge or skill in that area ranging from unacceptable, to emerging initial (e.g., prerequisite mastery, readiness for field placement), proficient initial (e.g., basic mastery, readiness for initial certification), and one or more advanced levels (up to the equivalent of expert in the novice-expert continuum for that outcome area).

Professional education candidates are assessed on their mastery of knowledge and skills in all outcome areas within courses. All programs have mapped outcome areas onto courses. Instruction and assessments tied to outcome areas are conducted in specified courses. Some assessments of knowledge or skills within each outcome area have been determined to be of sufficient importance to be used for making high-stakes decisions about the candidate. These assessments have been labeled as “artifacts” within the unit assessment system. Within initial certification programs in teacher education and within the program in school counseling, a candidate must perform at “emerging initial” professional levels on course-embedded artifacts tied to every outcome area prior to being allowed to enter the internship experience. Candidates within the advanced certification programs in school administration are required by the state to take coursework simultaneously with internship. Therefore, these candidates must perform at emerging initial professional levels on course-embedded artifacts. Failure to “pass” the course-embedded artifact in any program prohibits the candidate from advancing in the program. Performance-based assessments by supervisors during internship experiences serve as the final set of artifacts for students receiving initial certification. Specifically, all initial certificate candidates in teacher education as well as advanced candidates in school counseling and educational administration must perform at the proficient initial professional level on all targeted outcome areas and demonstrate appropriate dispositions in order to pass the internship and be recommended for certification by the professional education unit.

Because of their high-stakes nature, a fundamental requirement of all artifacts is that the candidate has sufficient opportunity to remediate performance before the high-stakes judgment is made. Therefore, instructors and supervisors are required to evaluate preliminary submissions of artifacts and provide clear and direct feedback to promote remediation in a timely manner before the final submission is due. Midterm evaluation of the candidate is considered to provide this feedback during the internship.   

The artifacts represent critical evidence of the candidate’s mastery of expectations of the profession. The unit has adopted Livetext as a mechanism for developing candidate portfolios consisting of artifacts and other supporting evidence within initial certification teacher education programs and for electronically storing and aggregating performance evaluations across candidates within initial certification programs.

Some programs also require candidates to pass formal examinations tied to outcome areas in order to be recommended for program admission or certification. These exams will be discussed in more detail in Section II and the appendices.

As indicated above, candidates in the initial preparation programs must demonstrate criterion-level performance in all outcome areas at two points in their program.  Programs have adopted different expectations for candidate performance in advanced certification programs. The fundamental and common expectation of such programs is that candidate performance in at least fifty percent of the outcome areas will exceed the performance levels required for initial certification. As with the initial certification programs, course-embedded assessments (evidence and artifacts) reflecting the program’s outcome areas are mapped onto courses. Those assessments which are identified as artifacts are scored using the outcome rubric as well as whatever letter-based or percentage-based grading system that is traditionally used. Student performance on artifacts is electronically stored to assist with evaluations of the candidate and aggregated to inform program and unit evaluation.

Unit Assessment

In terms of assessment of unit performance, the assessment system is comprised of:

Data that we aggregate from candidate performance that provides information on the success of our programs and our school in critical outcome areas (e.g., percentage of candidates scoring at the proficient initial level on each outcome area during student teaching or internship, percentage of candidates identified as demonstrating inappropriate dispositions in courses prior to student teaching, percentage of students removed from TEP due to failure to demonstrate appropriate dispositions after remedial efforts)

Data that we gather and compile from other assessment sources that provides information on the success of our program and school within critical outcome areas (questions on alumni surveys that are directly tied to outcome areas, questions on employer surveys that are directly tied to outcome areas, candidate self-assessments on items that are directly tied to outcome areas, etc.)

Data that we gather and compile from other assessment sources that provides information about the general functioning and effectiveness of programs and the school (percentage of students passing the Alabama teacher test, percentage of students employed as teachers within 4 months of graduation, evaluations of faculty teaching performance, general analyses of strengths and weaknesses of candidate preparation provided by candidates, employers and alumni, Alabama higher education report card)      

Data that we gather and compile concerning processes and progress relating to goals for scholarship, technology, diversity, faculty governance, and administration of the programs and school (e.g., minority faculty and student numbers and recruitment efforts, evidence of plans for technology and minority recruitment, evidence of faculty productivity in service and scholarship).

Data that we gather and compile that demonstrates we are making and monitoring informed improvements to our programs and school.

Data that we gather and compile that demonstrates that degree to which we are responsive to community stakeholders.

Data that we gather and compile that evaluates the degree of success of all programs in achieving the mission and vision of the school.  

Data that we gather and compile that evaluates the degree of success of all programs in serving the conceptual framework.

Internal and external mechanisms are used to document and evaluate the success of programs and the unit personnel preparation program in general. Internal mechanisms include:

Aggregating artifact scores across candidates (within and across outcome areas) in order to determine strengths and needs of programs and candidates.

Examination of the relationship between course grades, artifact ratings prior to student teaching, and student teaching evaluations in order to assure the validity of judgments.

Candidate evaluations of courses

Exit surveys of candidates in initial certification teacher education programs

Documentation of faculty productivity regarding teaching, scholarship and service.

External mechanisms include:

Surveys of principals and employers

Surveys of alumni

State Department of Education evaluation of first-year teachers (PEPE and surveys)

Data from internal and external sources are collected, compiled, and reported to unit faculty and external stakeholders in order to monitor implementation of the themes and conceptual framework, evaluate candidate performance across outcome areas and faculty performance in teaching, research and service; and to facilitate program improvement. More detail about unit assessment and reporting are provided in Section II of the NCATE report.