|
Home
MSU
home page
MSU
College
of Education
About the
Visit
Institution Report
Conceptual
Framework
Continuous Assessment
Plan
Assessment Documents
Syllabi
Program Reviews
New MAT
Educator
resources
Information:Morehead
Contact
NCATE coordinator
Contact
Site editor
|
III. Standard 2: assessment and unit evaluation
The unit has an assessment system that collects and analyzes data on
applicant qualifications, candidate and graduate performance, and unit
operations to evaluate and improve the unit and its programs.
Introduction
Unit assessment has been on going since May of 1999 when the “Teaching
the Teachers of the 21st Century” Task Force was created to review
all aspects of the teacher preparation programs at Morehead State University.
The task force was comprised of fifty faculty members from diverse academic
disciplines. Ultimately each person was assigned to one of eight groups,
and each group was assigned the responsibility for reviewing and evaluating
its assigned teacher education program component (Sample Components: admission
requirements, clinical and field experiences, content studies, and alternative
routes to certification). Following the year-long evaluation, each group
produced a report that described its process and contained a list of recommendations
for change. The co-chairs produced the Summary of the “Teaching the
Teachers for the 21st Century” Task Force group reports. www.msucoe.org/teachingtheteachers.pdf
This summary report was shared with the President and Provost for their
review. To ensure the implementation of the recommendations cited in the
report the President decided to create a transition team, which evolved
into the Executive Committee for the Transformation of Teacher Education
– ECTTC. With the leadership of the Provost, this group has taken
responsibility for translating the Teaching the Teachers Task Force Summary
into an action plan referred to as the Transition Team Plan 2000. (Documents
Room: Transition Team Plans 1.6)
The Transition Team Plans are organized around 14 goals and 80 objectives
linked to an implementation time framework. The ECTTC has continued to
meet twice a month for the past four years for the purpose of: 1) prioritizing
the work for the group each year, 2) determining the process to be used
to accomplish the objectives, and 3) to monitor the progress toward the
fulfillment of specific objectives. A review of the annual Transition
Team Plans will clarify the progress that has been made toward accomplishing
the goals linked to the transformation of teacher education programs.
(Documents Room: Meeting Information ECTTE 1.7)
One of the goal areas included in the Transition Team Plans (TTP) is assessment.
Of the six tasks listed underneath objective 10 “Strengthen the TEP’s
Assessment functions” all of the original tasks have been accomplished;
including: 1) alignment of all programs with EPSB, NCATE, Praxis Test,
2) the review of the assessment systems in place at other institutions,
3) the appointment of an Assessment Coordinator to develop a data base,
and 4) Promote portfolio assessment according to a rubric, and 5) Develop
preparation and remediation strategies: PRAXIS content and PLT exams.
Fulfilling the tasks aligned with the fourth objective: Strengthening
the Teacher Education Program Curriculum was expedited by the Standards-Based
Education Project (STEP), which was funded in 1999. All educator preparation
programs have been aligned with state and national professional associations
and NCATE performance standards. This task has been accomplished and is
documented in the program reviews produced by academic departments. The
Continuous Assessment Plan (CAP) described below has connections to both
the STEP and Assessment objectives, which are part of the Transition Team
Plan (Documents Room: TTP 2002 1.6)
Element 1: Assessment System
The Continuous Assessment Plan (CAP) www.msucoe.org/capmsu.html
at Morehead State University was developed by the Standard 2 Committee
members and the Assessment Coordinator. The CAP is a summary of the elements
of the Unit's assessment system. As the CAP was being developed, draft
iterations were reviewed by the Executive Committee on the Transformation
of Teacher Education . The penultimate draft was placed on the Morehead
State University, College of Education, website to invite input from a
variety of stakeholders. All feedback was reviewed and considered by the
Standard 2 Committee and Assessment Coordinator. Changes were made in
the CAP to incorporate some ideas from stakeholders. The feedback process
was used to ensure that the final document was accurate, comprehensive,
and inclusive. http://www.msucoe.org/ttp.doc
Candidate Continuous Assessment
The CAP was designed to monitor candidates as they progress through their
performance-based programs. This monitoring is achieved through ongoing
assessment of the content knowledge, pedagogical and other professional
knowledge and skills, technological proficiency, and dispositions of the
candidates. The expectations for students are based on MSU's academic
standards, the MSU Conceptual Framework for Professional Educator Preparation
Programs http://www.msucoe.org/conceptframe.html
, the standards of accrediting agencies www.ncate.org
, learned societies, and the standards approved by the Kentucky Education
Professional Standards Board http://www.kyepsb.net/teached/Programs/DEFAULT.html.
Programmatic assessment frameworks are used to clarify and systematize
the on-going assessment of all initial and advanced candidates. The Undergraduate
(Table 2.1) and Graduate (Table 2.2) Transition Points Documents are included
to more specifically articulate the points in the program where candidate
progress is monitored.
Incorporating recommended assessment practices, MSU's plan for monitoring
and evaluating candidates changes focus as the candidates move through
an initial or advanced teacher certification, school counselor, or school
administrator program. Prior to admission, candidates are most often assessed
by external sources of data (e.g., grade point averages, standardized
test scores, faculty disposition evaluations). Post-admission assessment,
however, involves a blend of faculty, clinical practice teacher, and candidate
self-assessment (embedded in coursework). Exit assessment requires candidates
fulfill a combination of institution, state, and national performance
standards.
Portfolios are used to assist the candidates' reflective progression through
a cohesive, standards-based program (Documents Room: Candidate Work Exhibit).
Portfolio pieces are included by candidates to demonstrate knowledge,
understanding, and proficiency in relation to each program’s standards
(New Teacher, Experienced Teacher, School Counselor or School Administrator
Standards). Self-assessment through the portfolio development process
is strongly supported by the faculty. Additionally, reflective self-report
data are consistently sought from candidates--specifically immediately
after student teaching each semester--and are used to inform and improve
the teacher education program (Documents Room: Data Portfolio 2.1 Tabs
4, 5,and 6).
A survey of recent alumni also provided valuable data for program improvement.
The National Database for Teacher Education Follow-up instrument was used
in a web-based format (Facilitate.com) to collect data from alumni (August
2003). These data will be compared with data from other teacher education
institutions across the nation. This will permit a unique perspective
for program evaluation because the data is configured to so that the data
can be evaluated from an institution as well as a comparative perspective.
http://morehead.facilitate.com/ndb.shtml
To better incorporate the professional community at large in improving
teacher education, Morehead State University's College of Education is
in the process of creating a standing Unit Program Assessment Committee
(UPAC). The UPAC will include four Unit faculty members, one superintendent
and one building principal from a school district in the service region,
one MSU alumnus(a) currently teaching in a public school in the service
region, and the Unit's Assessment Coordinator, who will chair the committee.
This committee will meet on a monthly basis to identify, review, and distribute
data to improve the educator preparation programs.
Program Evaluation
Academic programs have been evaluated annually using the Nichols Five
Column Model as the framework. Each fall, by September 15th, data are
distributed to the department chairs. Each chair distributes certification
program specific data to the appropriate faculty, and they are responsible
for (1) using the data to determine if candidate performance meets the
criteria set forth in the previous year’s Assessment Plan and (2)
to determine what changes need to be made when candidate success falls
below the established criteria. Changes are noted in the assessment plan,
which the academic program faculty creates after analyzing current data.
Data are used to make decisions about program quality. www.msucoe.org/programs.html
- Appendix A: Program Assessment Plan 2000). Assessment reports are reviewed
by the appropriate Department Chair, the Dean of the College, and the
Provost. The documents supporting the Assessment Report and Assessment
Plan are maintained in the academic departments.
The institution also requires all academic programs to complete a Five-Year
Review. At the scheduled time, the Provost distributes report guidelines
and timelines to department chairs. Department chairs meet with the faculty
of each program in the department so that they understand all aspects
of the program review and reporting process and also make decisions about
the data collection process. This review requires faculty to consider
data such as: program admission, enrollment, retention, exit exam pass
rates and graduation rates. Five years of data are evaluated to determine
program trends, strengths, and weaknesses. Alignment with university,
state, SACS, and national standards is also required. Plans for improving
noted areas of weakness are required as is a timeline for implementation
and completion.
Five-Year Review Reports are delivered to the College Dean who reviews
each department’s report and documents to determine if the information
is complete and accurate. All review materials are then submitted to the
Provost who shares them with the University Academic Program Review Committee
(UAPRC). The members of this committee evaluate reports to see that the
alignment with the university strategic goals is evidenced and that the
evaluation of each program within the department is complete. Department
reports are not accepted until the members of the UAPRC are satisfied
(Documents Room: Five Year Program Reviews Spring 2003 2.
2)Faculty Evaluation
The faculty and their ability to teach affects candidate learning. Producing
high quality undergraduate and graduate students is a primary goal of
the university http://www.moreheadstate.edu/units/budgets/plan/mission.html
. In addition, the Transition Team Plan (http://www.msucoe.org/ttp.doc
) includes an objective “ . . . to have all faculty model good teaching.”
(Objective 3, Task e) Therefore, an annual system of faculty evaluation
has been established. All tenured or tenure-track faculty members are
required to develop a vita and portfolio to document their performance
in the areas of teaching, professional achievement, and service according
to expectations set forth in the Faculty Evaluation Plan. (Documents Room:
Faculty Portfolios 5.5) Non-Tenure track, adjunct, and fixed term faculty
are evaluated in the area of teaching. Each department has established
procedures for accomplishing and reporting success.
Prior to January 2003, Faculty Evaluation Plans (FEPs) were developed
and implemented by departments. However in January of 2003, the faculty
of the College of Education approved a college-wide Faculty Evaluation
Plan (FEP). www.msucoe.org/coefep.html The current Faculty Evaluation
Plan was developed by members of a College of Education Committee comprised
of two-faculty members from each department. This was done to bring consistency
to the faculty evaluation process. All faculty had opportunities to provide
feedback about the plan and voted to approve the final version of the
document prior to its implementation.
Each faculty member receives written feedback addressing individual strengths
and weaknesses. Strengths and weaknesses are based on the performance
expectations established in the FEP. When a weakness is cited, a process
or expectation for improvement is included. The annual Performance Based
Salary Increase (PBSI) for each faculty member is based on the results
of his or her evaluation (A more detailed description of faculty evaluation
and development is described in Standard 5: Element 6).
Element 2: Data Collection, Analyses and Evaluation
The entire university maintains a continuous assessment procedure of which
the Educational Services Unit assessment is a part. The Unit maintains
an assessment system, and it is used to provide regular and comprehensive
information on applicant qualifications, candidate proficiencies, competencies
of graduates, unit operations, and program quality. Using multiple assessments
from internal and external sources, the Unit collects data from applicants,
candidates, graduates, faculty, and members of the professional community
and uses digital technology to store and analyze that data.
The Unit maintains a number of different computer-mediated databases and
report forms. With guidance from the Standards II Committee, the University’s
Institutional Research and Computer Application department has consolidated
all the data into one user-friendly report form that recently became available
to all program assessment coordinators and department chairs through the
University’s AIMSWEB interface. With the aid of the Unit’s Assessment
Coordinator, the Unit is implementing its assessment system to provide
regular and comprehensive data on program quality, Unit operations, and
candidate performance at each transition point and at the end of the first
year of practice (Internship Year). Data are regularly and systematically
collected and compiled, summarized, analyzed, and shared for the purpose
of improving candidate performance, program quality, and Unit operations.
The Unit is developing and testing different information technologies
for collecting and analyzing data (e.g., facilitate.com).
The Continuous Assessment Plan identifies the candidate data that are
to be collected. The Undergraduate and Graduate Degree Transition Points
Documents provide a comprehensive overview of the timing of the data collection
points in the program, a list of the data that are to be considered at
each transition point, the person or group who is responsible for making
decisions about a candidate’s continued progress through a program,
and the identification of the system for communicating the decision to
the candidate.
Data are collected for initial teacher candidates upon admission, prior
to clinical practice, during clinical practice and upon completion of
the program. Experienced teachers, school guidance counselors, and administrator
data are collected upon admission, to determine eligibility to sit for
the exit exam(s), and at program completion, The standard used to determine
success in relation to each evaluation is articulated in the appropriate
Undergraduate (Documents Room: 1.3) or Graduate Catalogue www.moreheadstate.edu/units/graduate/grad-cat03/
, on an official candidate check sheet www.msucoe.org/programs.html,
and in the Teacher Education Program Policies Handbook http://www.moreheadstate.edu/units/colleges/education/esu/
.
At both the undergraduate and graduate levels there are multiple indicators,
multiple assessments, and multiple transition points used to determine
candidate eligibility to continue through a program. The Unit uses information
from internal sources. Course syllabi outcomes or assessments are aligned
with state or learned society performance standards and the NCATE Themes.
www.msucoe.org/syllabi.html
The formative and summative evaluation procedures described in course
syllabi, are in turn aligned with these objectives.
Candidate portfolios artifacts are selected from course assessments and
aligned with state standards as evidenced in the Exit Portfolios. (Documents
Room: Candidate Work Exhibit) The evaluative rubrics for candidate portfolios
are also aligned with appropriate state standards. www.moreheadstate.edu/colleges/education/esu/
The Unit also uses information from external sources such as state
licensure exams, PRAXIS scores, cooperating teacher evaluations of student
teachers, employers surveys, and Kentucky Teacher Intern Program evaluations
of graduates.
All data pertaining to initial candidates is currently maintained in the
Education Services Unit (ESU). A professional staff member enters data
when recommended by the UPAC. Once data are part of the system they are
collected on a regular cycle to permit both longitudinal and cross-sectional
analyses. The assessment coordinator oversees all aspects of data entry,
analysis, and distribution. The items selected for consideration during
transition point evaluation represent the candidate’s ability to
demonstrate knowledge, skills or dispositions that are critical for success
as a new or experienced teacher, school guidance counselor, or administrator.
Maintaining confidentiality of all individual data is a constant concern.
Data are reported only in aggregate format. Legal and ethical guidelines
shape the work of the Assessment Coordinator and UPAC committee.
The fairness, accuracy, and consistency of data derived from instruments
and rubrics that the MSU faculty members have designed have been assured
through the pilot study process. At the end of each pilot to develop instruments,
input about implementation, rubric descriptors, score sheets, and/or minimum
required scores has been solicited from faculty members. Comments, suggestions,
and problems that were communicated by the respondents were addressed
by the members of the committee that produced the instruments. Adjustments
were made and revised documents have been implemented during the next
evaluation/assessment cycle.
During instrument pilot studies, candidates’ or graduates’
scores are not used as criteria for evaluating progress through or exit
from the program. The following rubrics have been developed, tested and
implemented since the 1998 reaccredidation visit: Initial Preparation
- Disposition Rubric and Exit Portfolio Rubric. Individual Graduate Masters
Degree Programs have developed customized Disposition Rubrics, Program
and/or Practicum Portfolios Rubrics. These can be reviewed at http://www.msucoe.org/assess.html
NCATE Data Portfolio
The “NCATE Data Portfolio” is part of this report and contains
the following data tables and summaries:
1. Admission to Program Data Summaries, 2000 – 2003
2. Summary Descriptive Statistics of Disposition Ratings 2001-2003
3. PRAXIS scores 1999-2002
4. Standards-Based Evaluation of Student Teachers, Spring 2003
5. Standards-Based Student Teacher Evaluations of the Program 2000-2003
(Including an in-depth quantitative/qualitative study of technology use
in 2003)
6. Performance of Graduates on the Kentucky Internship Program Evaluation
1999-2001
7. Standards-Based Evaluation from the Educational Professional Standards
Board
8. National Database Teacher Education Follow-up Alumni Survey (data available
September 1, 2003)
9. Results of Exit Exams for Graduate Programs in the College of Education
1998-2002
10. Faculty Integration of Technology Data 2002
The following are some preliminary analyses of Admission and Completion
Data.
Undergraduate Candidates: Admission to Teacher Education Program Data
2001-2003
Tables 2.3 through 2.4 below present summaries of selected entrance data
for the Teacher Education Program admissions for the years 2001-2002 and
2002 -2003. The data in these tables include means and standard deviations
for grade point averages (GPA), means and standard deviations for the
number of course hours completed at the time of admission (HRS), and means
and standard deviations for ACT scores.
Undergraduate Candidates: Completion of Teacher Education Program
Data 2001-2003
Tables 2.5 through 2.6 below present summaries of some data for candidates
completing the Teacher Education Program for the years 2001-2002, 2002-2003
. The data in these tables include means and standard deviations for grade
point averages (GPAs) and means and standard deviations for ACT scores.
Instructional Leadership Exit Exams
Students pursuing certification as a school principal are required to
pass two exams; the School Leader’s Licensure Assessment (SLLA) and
the Kentucky Principal’s Test (KYPT). Morehead State University has
had a 100% pass rate on the SLLA. The pass rates for the last three years
on the KYPT are shown in the Table below.
Record of the Formal Process Used to Handle Candidate Grievances
Undergraduate Candidates
The process for recording and documenting candidates' formal complaints
is described on page 36 of the Morehead State University 2002-2003 Undergraduate
Catalogue. It is suggested that the candidate first contact the instructor
of the course. If the candidate feels that it is not practical to contact
the instructor, the candidate may present the complaint to the chair of
the department to which the instructor is assigned. Prior to action by
the department chair, the candidate will be required to complete a Student
Grievance Form, available in the Office of the Provost. The candidate
should then return the completed Student Grievance Form to the department
chair. The department chair and the college dean will review the grievance
and attempt to mediate a settlement. Records of all meetings, including
recommendations by the department chair and college dean are sent to the
Provost and to all parties concerned. The Provost is responsible for enforcing
the committee's decision.
Graduate Candidates
The procedure for academic grievance by graduate candidates is outlined
in four steps in the MSU 2002-2003 Graduate Catalog on pages 39-40. In
summary the steps are: (1) the student discusses the complaint with the
person involved (timelines provided). If the complaint is not resolved
at this stage the student has 30 days to file a formal complaint. (2)
If the question is not resolved at the instructor level, the student may
present the question to the chair of the department. Prior to any action
by the department chair, the student will be required to complete a Student
Grievance Form (available from the Provost's Office). The department chairs
and the college deans recommended solution must be considered by the student.
Records of the meeting between the student, chair, and dean are sent to
the Provost and to all parties involved. (3) If the recommendations presented
by the department chair and the college dean are not acceptable to the
student, he or she may appeal to the Graduate Committee. The Graduate
Committee's decision, after a fully documented hearing process, is forwarded
to the Provost. The Graduate Committee's decision is final.
Element 3: Use of Data for Program Improvement
The State of Kentucky and Morehead State University require the use of
standards-driven, performance-based assessments to determine the quality
of educator preparation programs. These data are collected, analyzed,
and presented to continually inform all teacher, school counselor and
school administrator preparation programs at MSU.
The Nichols Five-Column format is used to inform the annual program review
and reporting process. This two-step process requires the production of
an Assessment Report and an Assessment Plan. The Annual Report focuses
on the use of data to document the extent to which candidates fulfilled
that year's Assessment Plan. Assessment Plans are developed annually and
these reflect program changes that have been made to improve candidate
performance or new or revised standards.
The annual assessment reporting process begins when each department chair
receives data from the Certification Officer (September 15th is the target
date). The chair then distributes the data to the appropriate faculty
committee. The members of the faculty analyze the data to determine program
strengths and weaknesses in light of the assessment results set in the
Program Assessment Plan. When candidate performance falls below the target
assessment results the faculty develop a plan to implement a program change,
that when implemented, to improve candidate performance. The plan or change
is articulated in the next year’s Program Assessment Plan. Program
faculty members are expected to establish a timeline and process for implementing
all changes. All Assessment Reports and Plans are reviewed by the respective
department chairs and deans and then forwarded to the provost for approval.
www.msucoe.org/programs Appendix A
All initial certification program faculty review candidate performance
on the PRAXIS Content Area and Pedagogy of Learning and Teaching exams
as a means for determining program strengths and weaknesses. To provide
faculty with the most meaningful data, all candidates are now required
to bring a copy of their test results (including subscale scores) to the
Educational Services Unit prior to clinical practice. www.moreheadstate.edu/units/undergraduate/cat02-03/
This makes it possible to collect and distribute aggregated candidate
performance data for each section of each test. Thus, program faculty
members now have complete and meaningful performance data to consider
when determining the strengths and weaknesses of the program in preparing
candidates for success on PRAXIS. An additional benefit for this policy
relates to the feasibility of advisors meeting with candidates to create
a remediation process prior to leaving campus for clinical practice.
When a program's pass rate falls below the 80% pass criteria on one or
more PRAXIS Exam(s), the faculty are required to develop a report that,
among other things, explains the results, sets forth a plan to improve
candidate performance, and includes an implementation schedule. This report
is reviewed by the Dean of the College of Education, Department Chairs,
Program Faculty, and members of the Educational Professional Standards
Board (Document Room: PRAXIS Low Performance Reports 2.3).
Another example of how data were used to promote program change occurred
as part of the “Teaching the Teachers of the 21st Century” Task
Force work. Feedback from student teachers on the Student Teachers' Evaluation
in 1998 was considered when evaluating the initial teacher preparation
programs. The review resulted in recommendations to revise the Secondary
Professional Education Core Courses. The Executive Committee for the Transformation
of Teacher Education included an objective linked to this recommendation
in the Transition Team Plan. As a result, the Secondary Professional Education
Core has been revised and all changes will have moved through the University
curriculum revision process www.moreheadstate.edu/units/undergraduate/
by Fall 2003. (Documents Room: ECTTE Information 1.7)
Beginning Spring Semester 2003, each candidate's performance in relation
to each New Teacher Standard was evaluated by either the Student Teacher
Record of Performance or the Kentucky Teacher Intern Program (KTIP) Observation
Instrument by both cooperating teachers and university supervisors. The
data were not coded by program and disaggregation by program was, thus,
not possible. Aggregated data are presented in the Data Portfolio (Tab
4). The results will be shared with and reviewed by teacher education
program faculty. This data will provide an additional source for determining
program strengths and weaknesses and potentially lead to suggestions for
program change.
The Executive Committee for the Transformation of Teacher Education will
continue to meet until all of the goals and objectives linked to the transformation
of teacher education have been accomplished. In addition to meeting, this
group will continue to monitor its process and progress and revise the
annual Transition Team Plan accordingly to ensure that all aspects of
teacher education reform are in place.
It is the intention of the Unit to continue to refine all aspects of our
Continuous Assessment Plan. Expanding the types of data collected, aggregated,
analyzed, and distributed will increase the data available for use by
faculty to improve educator preparation programs. The goal is to discover
factors that contribute to optimal candidate performance. It is assumed
that strong preparation programs and faculty are essential. Continuous
assessment is central to on-going quality control.
Next: field experience
and clinical practice
|
