PERCEPTIONS OF FACULTY MEMBERS IN SELECTED CATHOLIC HIGH SCHOOLS REGARDING NORTH CENTRAL ASSOCIATION SCHOOL IMPROVEMENT MODELS

This article discusses a study which compared faculty members' perceptions regarding the North Central Association (NCA) evaluation process based on format used (conventional, individualized, or outcomes) and school type (diocesan or private). Data were collected on the independent variables of format, school type, and prior NCA experience. The dependent variables were respondents' ratings on a seven-point Likert-type scale regarding the mechanics of self-study, facidty ownership, relevance to local and professional concerns, improvement generated, and the results of participation. The author concluded with the following recommendations: (J) more time for conducting the self-sttidy; (2) more clariftcation of the evaluation process, especially the outcomes format; (3) additional examples ofexemplaty self-studies; (4) greater attention to designing the self-study, developing faculty leadership, sustaining faculty motivation, and developing speciftc action plans; and (5) increased emphasis on student learning.

T he approach of the year 2000; the rapidity of change in society; the knowledge explosion; and the recent outcries for school reform, accountability, and increased participation in decision-making have all contributed to renewed emphasis on school improvement. The locus of improvement has also shifted closer to the individual school. Within this context, the perceptions of faculty members at 12 Catholic high schools regarding the North Central Association's (NCA) site-specific, faculty-based model for school improvement were explored. The findings of this study support the NCA's revised school improvement process which focuses on improving student performance in specific areas.
The literature on organizational change and participatory decision-making emphasizes the link between the needs of the organization and the needs of its members (Beer & Walton, 1987;Woehl, 1989). It also stresses the importance of involving those people affected by changes in determining the content and method of change. The NCA school improvement process provides a mechanism for such linkage and involvement (Covin & Kilmann, 1990;Evans, 1993;Harvey, 1990;Mauriel, 1989).
The NCA has been the subject of a substantial number of research projects. Educators generally reported that the self-study was a valuable undertaking and that it was the most important part of the NCA evaluation process. However, concems regarding time were consistently expressed. Faculty ownership was found to be an important attribute of a successful self-study. Benefits of the self-study cited in the literature included both direct program improvements and indirect climate improvements (Angney, 1983;Armstrong, 1982, n.d.;Boersma & Plawecki, 1972;Boyd, 1976;Cordova, Kelly, & Tenorio, 1988;Hahn, 1989;Jordan, 1977;Leigh, 1971;Littrell & Bailey, 1976;Master, 1970;Mather, 1981;Shaw & Jordan, 1971).
The purpose of this study was to examine whether faculty members' perceptions of the NCA evaluation process varied according to the format used for the self-study and/or according to school type (i.e., diocesan or private). Participating schools used one of three formats: (1) a conventional format which reviewed all major aspects of the school and its programs; (2) an individualized format which explored in depth a limited number of issues or areas considered especially important by the school; or (3) an outcomes format which focused exclusively on a limited number of student leaming areas targeted for improvement. The current NCA model essentially requires schools to use a combination of the outcomes format and the individualized format.
The three formats differ in scope of areas studied, method of topic selection, and type of assessment procedures. The conventional format evaluates general areas such as philosophy and mission, school and community, staff and administration, facilities, student services, and all curricular subject areas. Each section of the conventional format concludes with a listing of major strengths, concems, and recommendations. The most significant recommendations are developed into a school improvement plan.
The individualized format involves faculty in designing a self-study around five key areas which they desire to explore in depth. Topics are generally broad and pervasive. Examples include communication, change processes, and technology. Like the conventional format, the individualized format culminates with identifying strengths, concems, and recommendations for each area studied and organizing the most significant recommendations into a school improvement plan. In contrast to the conventional format, many areas of the school are not studied in preference for focusing in depth on a few areas perceived as especially important. Both the conventional and individualized formats rely heavily on qualitative and subjective judgments in determining areas of strength and areas needing improvement.
The outcomes format focuses exclusively on five areas of student performance targeted by the faculty for improvement. Cognitive and affective areas of student achievement are evaluated. Examples of target areas include communication, caring for self and others, and problem solving and critical thinking. The faculty designs a method for assessing current student performance in each target area to provide a baseline for determining how much improvement occurs. Improvement plans are developed, and methods for documenting the implementation of strategies and obtaining interim assessments of their effectiveness during the implementation period are established. After improvement plans have been implemented for two to three years, student performance is again assessed and the degree of improvement is analyzed.

METHOD
Faculty members' perceptions of five aspects of the NCA evaluation process were tested: (1) the mechanics of the self-study (i.e., materials, time, difficulty); (2) faculty ownership of the process; (3) relevance of the evaluation to local and professional concerns; (4) improvement generated through the process; and (5) results of participating in the process (effects on communication, knowledge of the school, morale, etc.). Perceptions of faculty members were disaggregated by format used and school type.
The sample consisted of 340 faculty members from 12 St. Louis area Catholic high schools who were on staff at the time of their school's last NCA evaluation and were still on staff at the time of this study. Six of the selected schools were diocesan schools (i.e., archdiocesan, regional, or parish schools), and six were private schools.
Data were collected by means of a 35-item survey using a seven-point Likert-type scale constructed for this study. The response rate was 81.8% (278 surveys were returned). Content and construct validity were supported through reviews of the instrument by 11 educators who had extensive knowledge of and experience with the NCA evaluation process. The reliability of the survey was supported through a pilot study using the test-retest method. The correlation for the full survey of 35 individual items was significant at the .001 level (r=.68,/?<.0001).
The 35 items were clustered conceptually into five dimensions (mechanics, ownership, relevance, improvement, and results of participation). Responses for each group of seven items were summed to obtain a score for each of the five dimensions. Therefore, a score for a given dimension could range from a low of 7 to a high of 49. The test-retest correlation for the full survey of 35 items after related items were summed to obtain scores for the five dimensions was r=.78, /7<.OOO1.
The test-retest correlations and p-values for each individual dimension are reported in Table 1. A series of one-way analyses of variance (ANOVA) was used to test the null hypotheses that the mean ratings of respondents did not vary significantly according to which evaluation format was used. The Scheffe test was used as a post-hoc comparison procedure to determine which groups' means differed significantly when the overall ANOVA F statistic indicated a significant difference. T-tests were used to test the null hypotheses that the mean ratings on selected dimensions did not vary significantly according to school type.

SUMMARY AND DISCUSSION OF FINDINGS AND SUGGESTIONS FOR IMPROVED PRACTICE MECHANICS
The mechanics dimension explored perceptions about the logistics of completing the self-study; the clarity and difficulty of the task, adequacy of materials, volume of data required, extensiveness of reports, time needed, and interference with other responsibilities. Despite differences in the number and breadth of the areas studied in each format, the evaluation projects required by all three formats were perceived as siniilar in magnitude. There was no significant difference among users of the three formats. The statistical results are summarized in Table 2. Although the F probability was <.O5, the Scheffe post-hoc test concluded that no two groups were significantly different at the .05 level.
Among the individual mechanics items, the most positive ratings were given to the clarity of the task and the adequacy of materials by users of all three formats. The mean for the item related to clarity of task was 5.03, and the mean for the item related to adequacy of materials was 5.08. As in prior research, time emerged as the major concern, receiving the least positive rating of any item in the mechanics dimension. The mean for the item related to time needed to complete the self-study was 2.83.
Analysis of responses regarding mechanics suggests the following recommendations for improvement of the self-study experience: 1. Ensure that faculties have adequate time to conduct the self-study. 2. Continue to clarify the school improvement process, especially the current model, which places a strong emphasis on the use of objective data. 3. Develop more materials which exemplify self-study data presentations, documentation, and reports.

OWNERSHIP
The ownership dimension examined the faculty's sense of responsibility for the completion and quality of the self-study by considering their perceptions of selecting the topics studied, the importance placed on the evaluation, faculty commitment, cooperation, interest, contribution, and involvement. Those who had used the individualized format reported significantly greater ownership than those who had used a conventional format. The statistical results are summarized in Table 3. This finding is most likely due to faculty feeling that they were working on their own agenda rather than studying required areas. Although the outcomes format also involves faculty selection of target areas, faculties using it did not express significant ownership of the process. Possible reasons include discomfort with the interdisciplinary focus of the outcomes format, the protracted time frame, and lack of familiarity with the measurement, data disaggregation, and documentation aspects of the process. These factors could negatively impact faculty members' ability to sustain interest and involvement. As reported in the school improvement literature, teachers may also be more comfortable dealing with program and resource aspects of schooling than taking direct responsibility for improved student achievement (David & Peterson, 1984;Finn, 1984;Goodlad, 1984;Hord, 1989: Levine, 1991Rosenholtz, 1989).
Analysis of responses regarding ownership suggests the following recommendations for improvement of the self-study process: 1. Give greater attention to the design of the self-study and to the selection of target areas, especially in terms of faculty readiness. 2. Give greater attention to developing faculty leadership for the evaluation project. 3. Build upon the interest generated by faculty selection of the self-study topics. 4. Identify and address faculty concems regarding efforts to address improved student performance. 5. Explore ways to sustain faculty motivation and interest, especially as school improvement becomes an ongoing rather than a periodic endeavor.

RELEVANCE
The relevance dimension investigated the relationship of the self-study topics to school and faculty needs by exploring perceptions about professional significance, addressing professional concems, providing an opportunity for professional dialogue, promoting an enhanced knowledge of the school.
addressing issues specific to the school, and identifying strengths and needed improvements. Faculties using the individualized format reported significantly greater relevance than those who had used the outcomes format. The statistical results are summarized in Table 4. .0003 The focus of the outcomes format on the core purpose of student leaming was not sufficient to prompt a high perception of relevance.
The items related to accurate identification of strengths and areas needing improvement were rated low by faculties using the outcomes format. The means for these items were 4.98 and 4.92 respectively.
The relatively high rating of relevance by users of the conventional format may be attributable to the substantial attention given to individual subject areas. Those who had used the individualized format gave the highest ratings to items regarding increased knowledge of the school (mean = 5.07) and identification of strengths (mean = 6.04) and concems (mean = 5.62).
Analysis of responses regarding relevance suggests the following recommendations for improved self-study experiences: 1. Develop or enhance a school culture which focuses on student success and which promotes increased faculty acceptance of responsibility for student leaming. 2. Clarify the relationship between program or resource improvements and documented increases in achievement. 3. Build on faculty selection of target areas as a means to enhance relevance to individual school and faculty needs.

IMPROVEMENT
The improvement dimension considered changes in the school which were perceived to be positive and for which the NCA evaluation was the catalyst. It explored perceptions regarding the feasibility and helpfulness of recommendations, problem identification and solution generation, impact on teaching and leaming, addressing improved student achievement, stimulation of short-term improvement, and planning for long-term improvement.
Faculties who had used the individualized format reported significantly greater improvement than those who had used the conventional format. The statistical results are summarized in Table 5. This finding is likely due to the individualized format's focus on issues considered crucial and timely for the school. Those who had used the outcomes format did not report significant improvement, perhaps due to the format's measurement and documentation requirements. Assessments of improvement are data-driven, and change if student achievement is documented rather than assumed through subjective or intuitive judgments. In addition, faculties may not have had a clear understanding of how much improvement during an outcomes cycle should be considered a substantial accomplishment.
Analysis of the responses regarding improvement suggests the following recommendations for improved self-study experiences; 1. Ensure that the self-study design process seeks consensus about areas needing improvement. 2. Focus on the connection of evaluation activities to improve student learning. 3. Enhance planning for improvement by careful attention to specific action steps to implement and monitor changes.

RESULTS OF PARTICIPATION
The results dimension investigated the effects of participating in the selfstudy on school climate factors by exploring perceptions about interaction with faculty from other departments, improved communication and morale, effectiveness of leadership, increased appreciation of others' contributions and problems, involvement in decision-making, and facilitation of a shared vision. Faculties who used the individualized format reported significantly greater benefits from participation than those who had used the conventional format, most likely due to their greater sense of ownership. The statistical results are summarized in Table 6. Although the outcomes format requires substantial interdepartmental communication and the entire outcomes project focuses on teaching and leaming, it is possible that ambiguity about unfamiliar aspects of the process overshadowed other aspects of participation in the self-study.
Analysis of the responses about results of participation suggests the following recommendations for improving the self-study experience: 1. Ensure sufficient opportunities for faculty interaction during the selfstudy. 2. Develop strategies to capitalize on the potential of the evaluation process to positively affect school climate.

SCHOOL TYPE
This aspect of the study disaggregated faculty members' responses according to the type of Catholic school in which they taught, i.e., diocesan or private. The perceptions of faculty members from private schools regarding ownership, improvement, and results of participation were significantly more positive than the perceptions of faculty members from diocesan schools. The statistical results are summarized in Tables 7, 8, and 9.  These conclusions are assumed to be related to greater local detennination and responsibility in private schools on all aspects of school operation. Private schools have complete freedom and responsibility in all areas except religious education. Private schools are not bound by diocesan policy, nor do they share directly in diocesan resources. The locus of responsibility and decision-making is clearly the individual school, and private schools are accountable only to their boards and clients. Therefore, it was anticipated that private school faculties would be more positive on the dimensions of this study which relate to ownership, improvement, and climate results.
The findings regarding school type suggest the following recommendations for improved practice: 1. In private schools, continue to capitalize on the positive aspects of local determination. 2. In diocesan schools, seek ways to further enhance local detennination within the context of diocesan support and affiliation.

CONCLUSION
This study suggested a number of areas that would benefit from further research. These include exploration of the reasons for faculty members' perceptions about their self-study experiences; comparison of their perceptions at various points during the school improvement cycle; exploration of factors that promote or inhibit faculty ownership of the process; consideration of factors that encourage or discourage faculty evaluation of instructional effec-tiveness; and exploration of stmctures that address faculty concems regarding the time involved in the NCA evaluation process.
In 1993 the North Central Association adopted a school improvement process model that in essence combines the individualized and outcomes formats. The findings of this study support the NCAs modification of the school improvement process to include both leamer outcomes and school programs and processes. This study also affirms the current model's accommodation of individual school needs and circumstances. The more focused self-study and the provision for faculty determination of the areas targeted for improvement hold substantial promise for accomplishing NCAs goal of improving education one school at a time. The North Central Association is to be commended for its adoption of a school improvement model that directly focuses on student success and that recognizes the individuality of its member schools. Some of the questions pertain to just the SELF-STUDY part of the North Central evaluation process. Other questions refer to the entire EVALUA-TION process. These key words have been underlined to help you distinguish when you are being asked to rate your experience with just the self-study and when you are being asked to rate your experience with the entire evaluation process.

SELF-STUDY: faculty members meeting and preparing reports prior to the visit by the team of educators from other schools. EVALUATION: (1) the self-study; (2) the visit by the team of educators from other schools; and (3) the faculty considering the self-study and visiting team recommendations as ideas for school improvement.
Please circle the number which represents your rating on each aspect of the North Central process as you perceive it from participating in the last NCA evaluation at your school. Please circle only one number per item. One is the lowest rating, and seven is the highest rating.