| Mitchell J. Muehsam | Chair |
| Associate Dean, College of Business Administration | |
| Vic Sower | Executive Chair |
| Associate Professor of Management | |
| Bill Fleming | Editor |
| Professor of English | |
| Rosanne Barker | Assistant Professor of History |
| Genevieve Brown | Professor of Education |
| Chair, Department of Educational Leadership and Counseling | |
| Brian Cooper | Associate Professor of Geology |
| Barbara Corbin | Associate Professor of Music |
| Jerry Dowling | Professor of Criminal Justice |
| Roger Hanagriff | Lecturer of Agriculture |
| Janice Lange | Associate Professor, Newton Gresham Library |
| Valerie Muehsam | Assistant Professor of Business Analysis |
| Ronald Stoltenberg | Professor of Mathematics |
| Laverne Warner | Professor of Education |
| Everett Wilson | Professor of Biology |
INTRODUCTION
HISTORY OF THE OFFICE OF INSTITUTIONAL RESEARCH AND STRATEGIC PLANNING.
STRATEGIC PLANNING AT SAM HOUSTON STATE UNIVERSITY
STRATEGIC PLANNING PROCESS FOR 1997-2000
METHODOLOGY USED BY THE COMMITTEE
FINDINGS OF THE COMMITTEE ON INSTITUTIONAL EFFECTIVENESS
3.1 PLANNING AND EVALUATION: EDUCATIONAL PROGRAMS
RECOMMENDATIONS AND SUGGESTIONS: SECTION 3.1
3.2 PLANNING AND EVALUATION: ADMINISTRATIVE AND EDUCATIONAL SUPPORT SERVICES
RECOMMENDATIONS AND SUGGESTIONS: SECTION 3.2
3.3 INSTITUTIONAL RESEARCH
RECOMMENDATIONS AND SUGGESTIONS: SECTION 3.3
COMPLIANCE TABLES: SECTIONS 3.1—3.3
SUMMARY OF RECOMMENDATIONS AND SUGGESTIONS: SECTIONS 3.1—3.3
APPENDIX
APPENDIX: College of Business Administration FES Form
As a result of the 1987-88 SACS Self-Study, Sam Houston State University created an Office of Institutional Research (OIR). In response to recommendations made in the SACS Reaffirmation Committee Report and with the intent of implementing a more formal process for continuous improvement, SHSU’s Office of Institutional Research was established in July 1990. The stated mission of the office at the time of its creation was to support the needs of the administration, planning committees, accreditation reviews, and the institutional effectiveness process adopted by the University.
During the past decade the University’s institutional effectiveness process has evolved to include all academic and support units. The earliest attempts to create an institutional effectiveness process concentrated around academic units like the Department of Geography and Geology, which was one of the first units to develop its effectiveness program. In recent years, greater emphasis has been placed on encouraging all academic and support units to formalize their institutional effectiveness processes. In fact, the SACS Committee on Institutional Effectiveness found that every academic and support unit is involved in the University’s institutional effectiveness program. During the course of this self-study, representatives from all of the academic and support units have been interviewed concerning institutional effectiveness. The Committee found great variability in sophistication and enthusiasm for institutional effectiveness.
A self-study provides a "snapshot" of the institution that is beneficial in assessing current strengths and weaknesses. The members of the Committee on Institutional Effectiveness share the belief that an understanding of the process that produced the snapshot also yields valuable insights. Accordingly, the following sections describe the history of the institutional effectiveness process at SHSU and present the results of the subcommittee’s evaluation of SHSU’s current institutional effectiveness status. The results of the Committee’s interviews are available in the documentation provided to the SACS visitation team. This chapter relies upon examples of individual units’ institutional effectiveness programs as they relate to the various "must" statements in the SACS Criteria for Accreditation.
The first section of this chapter provides a history of the Office of Institutional Research and the strategic planning process at Sam Houston State University. A second section presents the methodology used by the members of the Committee on Institutional Effectiveness to make their assessment of the effectiveness process. Analysis of SHSU’s compliance with the "must" statements follows the methodology section.
During the 1987-88 SACS Self-Study, the Committee on Institutional Effectiveness concluded that Sam Houston State University does not have a comprehensive strategic planning process involving the institution as a whole, nor is there a comprehensive approach to measuring institutional effectiveness or development of an ongoing program of institutional research. (II-1) In its Reaffirmation Committee Report (October 31-November 3, 1988), the SACS visitation team agreed with the findings of the Committee on Institutional Effectiveness and recommended that the University implement an adequate procedure for planning and evaluation;
The University define its expected educational results;
The University describe how achievement of the results of its statements of expected educational results or goals will be ascertained;
The University, once it has established adequate planning and evaluation procedures, use the results of those procedures in a broad-based and continuous improvement of institutional operations; and
The institution develop and implement appropriate procedures for evaluating effectiveness of educational operations, research and public service. (12)
The visitation team further recommended that "the University use this more coordinated approach to institutional research for the purpose of conducting continuing study, analysis, and appraisal of the institution’s purposes, policies, procedures and programs" and that "the University regularly evaluate this institutional research function" (14).
The University’s needs and organizational and committee structures are dynamic in nature; accordingly, the explicit function of the Office of Institutional Research must adjust to meet the evolving needs of the University. However, since its creation, the primary role of the Office of Institutional Research has been continuously defined as overseer of the institutional effectiveness efforts, to include providing technical and clerical support to units developing and implementing institutional effectiveness programs. The second Director of the Office of Institutional Research (serving from February 1995 through August 1998) reported that a "second role of the office is coordination of questionnaire and ad hoc reporting," and a third role is "to supply information and conduct research that support planning and decision-making."
In the nearly eight years of its existence a primary function of the Director of the Office of Institutional Research has been to assist and encourage the institutional effectiveness process. The recommended process has evolved over the years as the Director has made adjustments to involve a greater number of units in the formal assessment process. The first Director developed a two-year process cycle: In the first year, the specific academic or support unit would be asked to develop a proposed institutional effectiveness plan. Starting in the second year, the unit would collect information, interpret the data, and develop recommendations for improvement. Each year thereafter, the unit would assess and report progress. After the fifth year, the unit would start the process anew. The Director of the Office of Institutional Research established a rotation whereby additional units would enter the process over a five-year period. It was envisioned that all of the University’s units would be included in the formal process by the end of the fifth year of the program. Each unit appointed a liaison with the responsibility of coordinating and communicating with the Office of Institutional Research.
In the first three years (Fall 1991 to Fall 1993), forty-two units were introduced to the process. However, not all of the units maintained a flow of information to the Office of Institutional Research. The Office reported that only twenty-three of the forty-two units were actively involved in the process during the Fall 1993 semester. (It should be noted that a unit’s failure to report to the Office of Institutional Research does not indicate that it does not participate in an institutional effectiveness program; the Committee on Institutional Effectiveness found that many units have formal and informal assessment processes that are not reported to the Office of Institutional Research.)
The first Director of the Office of Institutional Research left SHSU prior to the Fall 1994 semester to accept a position at another institution of higher learning. His successor, hired in February 1995, initiated a new institutional effectiveness program utilizing a two-year cycle: In the first year, a department delineates goals and intended outcomes and develops an assessment plan. In the second year, the department designs assessment techniques, develops the assessment tool, applies the assessment instrument, evaluates the achievement of intended outcomes, and writes a final report. An Office of Institutional Effectiveness Committee, created in 1991, has the responsibility of aiding the departments, especially in projecting goals and measurable outcomes. To assure that the goals are generated and driven by departments, however, members of this committee have no authority to set or impose goals.
Since the initiation of the two-year cycle, academic and administrative units not already participating in the institutional effectiveness process have been required to do so. As a result, all academic and administrative units are currently involved in an institutional effectiveness process. To encourage greater participation in the formal process, the appropriate deans and vice presidents, with the President of the University, were integrated to the responsibility flow chart. That is, instead of reporting to the Office of Institutional Research, academic units are now required to send copies of their reports to the appropriate dean, and support units are required to send copies of their reports to the appropriate vice president.
The President of the University has consistently stressed the importance of institutional effectiveness. In 1996, the President sent a memo to the vice presidents and academic deans not only emphasizing that institutional effectiveness is essential, but also insisting "that over the years, the cycle will greatly benefit our university by helping us maintain the overall excellence of our many programs and services."
In 1990, the Strategic Planning Committee was appointed by the President of the University and "charged . . . with the writing of new goals to guide the development and growth of Sam Houston State University." Four overarching goals asserted that SHSU will be recognized for the excellent quality of its academic and ancillary programs; project a clear and consistent image of its role as a major educational cultural resource; develop its human, financial, and physical resources; and have an inclusive, effective and integrated program of planning and management. (1996-97 Strategic Planning Committee Report
Following acceptance of the Strategic Planning Committee’s proposals, an ad hoc "Committee for the Implementation of the Strategic Plan" was charged "with giving life and substance to the four overarching university goals and with continued monitoring of the planning process" (1990-91 Strategic Planning Committee Report 7). The Committee responded by producing forty-four recommendations, the majority of which were approved, funded, and established in fact. One such recommendation was that a permanent Standing Committee for Strategic Planning (SCSP) replace the ad hoc committee. The permanent committee’s membership consisted of faculty selected by nomination from the Faculty Senate, persons nominated by the Academic Policy Council, the President of the Student Government Association, a Chair named by the President of the University, and three associate members representing the Offices of Institutional Research, Administrative Accounting, and Computer Services. The SCSP, formally established in the fall of 1994, was charged with developing environmental scans, formulating and prioritizing strategies grounded on those scans, and evaluating the success of implementation of those strategies.
The strategic planning process was designed to develop a movement from goals to objectives and then to strategies. The SCSP’s first annual report published in April of 1991 listed five goals: to provide instruction (three objectives, seven strategies); conduct research (two objectives, two strategies); provide public service (one objective, three strategies); provide for institutional support and ancillary operations (four objectives, thirteen strategies); and conduct its purchasing and contracting without favoritism, arbitrariness, or caprice to obtain the maximum value for its limited resources (three objectives, eight strategies) (1996-97 Strategic Planning Committee Report 51-53). This list of goals and corresponding objectives and strategies was sent to the President at the end of the academic year. After review by the President’s Cabinet, the President informed the SCSP that nine of the objectives had been approved; the remaining were either sent to other University committees or held for further consideration.
The SCSP membership, goals, and processes continued until the fall of 1997, when a restructuring took place "which employed broader input from all levels of the University in a recursive fashion, using extensive feedback loops from assessment of outcomes to modify new goals and objectives. It was also intended that the activities of the Committee would be integrated with, and inform the budgeting process" (1997-98 Strategic Planning Committee Final Report 2). The Committee was renamed the Strategic Planning Committee (SPC). Although the charge to the Committee remained intact, the membership of the Committee was reduced to eight persons: the Vice Presidents for Academic Affairs (Chair), Finance and Operations, and Student Services; the Chair and Chair-Elect of the Faculty Senate; the Director for Institutional Research; the Executive Director of University Advancement; and the President of the Student Government Association. In the fall of 1997, this committee adopted a systematic planning process which invites input from all levels of the University in order to encourage evaluation and accommodate modification of goals and objectives. Moreover, the Committee was given direct input to the budgeting process. The newly restructured Strategic Planning Committee has realized that some of the original timelines and processes need modifications and will adjust those accordingly as the new strategic planning process is utilized.
The strategic planning process is designed to include all areas of the University in identifying needs and implementing programs to fulfill the mission of the institution. As previously indicated, the Strategic Planning Committee’s charge remains as before: to develop environmental scans, to formulate and prioritize strategies grounded on those scans, and to evaluate the success of implementing those strategies. The process is ongoing and cyclical, with assessment and evaluation of former strategies and goals as integral parts of each year’s planning.
At present, the planning strategy is ongoing for a three-year period. The strategic planning process designed for 1998-1999 is structured as follows:
September 1998: SPC drafts environmental scan (ES) for 1999-2000; departments and colleges develop ES and respond to SPC’s ES.
October 1998: SPC develops final ES, evaluates achievements of 1997-98 and of ongoing activities of 1998-99, and drafts divisional and institutional goals, which are transmitted to departmental and college levels for responses and refinement of earlier goals and objectives.
November 1998: SPC devises divisional working goals, including measurable objectives and assessment strategies.
December 1998: SPC formulates institutional goals, which are sent to the President of the University for approval.
January 1999: SPC finalizes institutional goals and transmits them to the Budget Committee.
May 1999: Budget allocations are determined.
July 1999: Divisional and departmental goals are funded; these goals become a basis for the environmental scan for the next year.
August 1999: Divisional and departmental goals are implemented.
Important differences between strategic planning processes directed by the current SPC and those directed by the former SCSP are that
(1) strategic planning is directly tied to the budgeting process,
(2) input comes from all levels of the University,
(3) earlier timelines for decision-making can ensure implementation and accurate evaluative measures, and
(4) membership in the Committee is stable.
The Committee can also assess more readily the institutional effectiveness of the implemented strategies and of future processes.
Methodology Used by the Committee on Institutional Effectiveness to Collect Information
The process of evaluating institutional effectiveness within the eighty-four units of the University began with each Committee member’s taking responsibility for interviewing the chair, director, or institutional effectiveness coordinator of at least six units. Prior to these interviews, Committee members obtained the units’ institutional effectiveness progress reports from the Office of Institutional Research and made appointments with chairs or directors of units.
The purpose of each interview was to discuss the unit’s compliance with the institutional effectiveness structure prescribed by the University and SACS. Not only formal documentation and supporting materials, but also informal, anecdotal information was solicited as evidence of compliance with "expected educational results" (Criteria 20). Four specific areas, as prescribed by SACS, were addressed for each educational program unit:
(1) establishment of "a clearly defined purpose appropriate to collegiate education,"
(2) formulation of "educational goals consistent with the institution’s purpose,"
(3) development and implementation of "procedures to evaluate the extent to which these educational goals are being achieved," and
(4) use of "the results of these evaluations to improve educational programs, services and operations." (Criteria 20)
Four specific areas, as prescribed by SACS, were addressed for each administrative and educational support services unit:
(1) establishment of "a clearly defined purpose which supports the institution’s purpose and goals,"
(2) formulation of "goals which support the purpose of each unit,"
(3) development and implementation of "procedures to evaluate the extent to which these goals are being achieved," and
(4) use of "the results of these evaluations to improve administrative and educational support services." (Criteria 21)
The Committee on Institutional Effectiveness completed most of its interviews and written reports between October 1997 and February 1998. The Committee returned completed reports to the chairs or directors for proofing and approval.
The following sections report the Committee’s findings in the three major areas outlined in Section III of the SACS Criteria: Planning and Evaluation of Educational Programs (3.1), Planning and Evaluation of Administrative and Educational Support Services (3.2), and Institutional Research (3.3). The Committee addresses "must" statements from each of the sections and, in the order in which the statements appear in the Criteria, reports on the University’s compliance or noncompliance with these imperatives. Following each series of "must" statements and findings, the Committee makes recommendations and/or offers suggestions, according to the University’s compliance status.
The Committee on Institutional Effectiveness followed the Office of Institutional Research’s classification of educational units in its investigation of the planning and evaluation of educational programs. Table 3.1.a below lists SHSU’s four colleges and the respective academic units within each college. The table includes two units (the Center for Business and Economic Development and the Small Business Development Center) that do not offer academic courses and may not fit a classic definition of "educational program." However, it is the opinion of the Director of the Office of Institutional Research that these units perform functions which correspond more closely with educational programs than with support services. Furthermore, these units report to an academic dean and thus are included in the section on educational programs. Additionally, the University has one department, the Department of Military Science, that is not housed in a particular college; this department reports to the Associate Vice President for Academic Affairs and thus is included in the section on Educational Support Services.
|
Arts & Sciences |
Business Administration |
Criminal Justice |
Education & Applied Science |
|
Art Biological/Environmental Sciences Chemistry Dance English Foreign Languages Geography and Geology History Journalism Mathematics & Information Sciences Music Photography Physics Political Science Public Communication/Radio/Television Sociology Speech Theatre |
Accounting Economics and International Business General Business & Finance Management & Marketing Center for Business and Economic Development Small Business Development Center |
Criminal Justice |
Agriculture Family/Consumer Sciences Health Kinesiology Language, Literacy and Special Populations Library Philosophy Psychology Technology Educational Leadership & Counseling Writing Program Education Curriculum and Instruction |
As the methodology section of this chapter indicates, each educational program unit was assigned to a member of the Committee on Institutional Effectiveness, who then contacted and interviewed a representative from the unit, with the aim of evaluating the unit’s institutional effectiveness status. Table 3.1.b below lists each educational program unit and the administrative unit to which the educational program reports, then determines whether the unit’s institutional effectiveness status is acceptable. (The Committee’s reports on the individual units are available in the SACS offices, located in Building AB3.)
The Committee concluded that thirty-seven of the thirty-eight educational program units have acceptable institutional effectiveness programs in place (Table 3.1.b). The Committee has noted, however, that among the educational program units there is a wide range of sophistication and enthusiasm for institutional effectiveness. The variances may be explained partly by the different lengths of time in which units have actively participated in the institutional effectiveness process: As the history section of this chapter reports, the Office of Institutional Research, created in response to the 1987-88 SACS Self-Study, was charged with the task of assisting and encouraging the institutional effectiveness process; during the eight years of its existence the institutional effectiveness process has slowly evolved to its current form. While several educational programs have been involved in the process for seven or eight years, most of the units have been actively involved for fewer than three years. Those units which began participating more recently are, naturally, still finetuning and promoting processes which have been successfully operating in other units for some time.
| Units | Fully Acceptable1 | Not Fully Acceptable1 | |
| College of Arts & Sciences | X |
| |
| Art | X | ||
| Biology/Environmental Sciences | X | ||
| Chemistry | X | ||
| Dance | X | ||
| English | X | ||
| Foreign Languages | X | ||
| Geography/Geology | X | ||
| History | X | ||
| Journalism | X | ||
| Mathematics & Information Sciences | X | ||
| Music | X | ||
| Photography | X | ||
| Physics | X | ||
| Political Science | X | ||
| Public Communication | X | ||
| Radio/Television/Film | X | ||
| Sociology | X | ||
| Speech | X | ||
| Theater | X | ||
| College of Business | X | ||
| Accounting | X | ||
| Economics | X | ||
| General Business & Finance | X | ||
| Management & Marketing | X | ||
| Center for Business & Economic Development | X | ||
| Small Business Development Center | X | ||
| College of Criminal Justice | X | ||
| Criminal Justice | X | ||
| College of Education & Applied Sciences | X | ||
| Agriculture | X | ||
| Family/Consumer Sciences | X | ||
1Acceptability is determined by compliance with "must" statements in section 3.1 of the SACS Criteria.
Institutional Effectiveness: Academic Units (continued)
| Units | Fully Acceptable1 | Not Fully Acceptable1 | |
| Health | X | ||
| Kinesiology | X | ||
| Language, Literacy, & Special Populations | X | ||
| Library Science | X | ||
| Philosophy/Psychology | X | ||
| Technology | X | ||
| Educational Leadership & Counseling | X | ||
1Acceptability is determined by compliance with "must" statements in section 3.1 of the SACS Criteria.
While various units feature individualized institutional effectiveness processes, most of them use two systems in common: the Faculty Evaluation System (FES) and student evaluation of professors. The University requires that each faculty member be evaluated annually using the FES, which rates the faculty member’s teaching effectiveness, scholarly and artistic endeavors, professional growth and professional activities, and non-teaching activities. Furthermore, the University mandates a range of weights for each of the areas of evaluation; in accordance with the University’s stated mission, teaching has the greatest weight (40% to 60%). Academic Policy Statement 820317 provides the FES weight for full- and part-time faculty. (The FES form used by the College of Business Administration is provided as an example in the appendix to this report.) In addition to the FES, each faculty member must be evaluated annually by his or her students in at least one of the two long semesters.
Following are synopses of the Institutional Effectiveness Committee’s deliberations and conclusions about the University’s compliance with the must statements found in Section 3.1 of the Criteria for Accreditation. The "must" statements from the Criteria ask for the compliance of the institution and not of individual units per se. Accordingly, the Committee determined the institution’s overall compliance with each "must" statement by taking a composite view of the thirty-four educational program units and four educational program administrative units.
1. Educational activities of an institution include teaching, research and public service. Planning and evaluation for these activities must be systematic, broadbased, interrelated and appropriate to the institution (Criteria 20).
The Committee on Institutional Effectiveness determined that Sam Houston State University complies with this imperative.
The mission statements of all of the units support or refer clearly to the University’s mission statement. All of the educational units encourage excellence in teaching and research as well as University and public service. The nature of intellectual contributions varies according to the mission of the unit. Faculty in the Department of Art, for example, are encouraged to exhibit their works nationally and internationally, while faculty in the Department of History are encouraged to write scholarly books and articles.
2. The institution must define its expected educational results and describe its methods for analyzing the results (Criteria 20).
The Committee determined that Sam Houston State University complies with this imperative.
Each academic unit has a list of goals and outcomes. A review of the Committee’s reports on the individual academic units reveals that most of the educational units emphasize teaching more than research and service in their institutional effectiveness processes. The University’s mission emphasizes education, and the individual units, in turn, place most of the emphasis in their institutional effectiveness processes on evaluating education.
3. The institution must (1) establish a clearly defined purpose appropriate to collegiate education, (2) formulate educational goals consistent with the institution’s purpose, (3) develop and implement procedures to evaluate the extent to which these goals are being achieved, and (4) use the results of these evaluations to improve educational programs, services, and operations (Criteria 20).
Having evaluated the institutional effectiveness processes of the thirty-eight educational program units on these four points, the Committee found that the University complies with this imperative.
Thirty-seven of the thirty-eight units were deemed to have adequate institutional effectiveness processes in place (see Table 3.1.b), though the units exercise a wide variety of assessment procedures. The College of Business Administration, for example, uses a combination of alumni surveys, graduating senior surveys, capstone course surveys, and student evaluations. Partly as a result of the information garnered by these assessment measures, the College of Business Administration has increased the use of technology in the classroom; changed the curriculum to include a Management Information Systems (MIS) course; added a second required business analysis (statistics) course to the curriculum (effective Fall 1998); and created a new elective course in business presentations. (See the College of Business Administration’s AACSB Self-Study and the 1990 and 1995 alumni survey reports.)
4. The institution must develop guidelines and procedures to evaluate educational effectiveness, including the quality of student learning and of research and service (Criteria 20).
The Committee found that the University complies with this imperative.
All educational units encourage excellence in teaching, research, and service, and, as noted earlier in this section, all employ the FES in evaluating the faculty on teaching effectiveness, scholarly and artistic endeavors, professional growth and professional activities, and non-teaching activities. The units use a variety of techniques to evaluate educational effectiveness. For example, the Department of Educational Leadership and Counseling makes use of the ExCET exams and NCATE accreditation to evaluate its effectiveness; the Department of Music uses a jury system by which a panel of faculty members evaluates each student’s instrumental or vocal performance; the Dance Program enters students in regional and national meetings of the American College Dance Association; and the Department of Biological Sciences uses the Major Field Test in Biology as a measure of effectiveness.
5. The evaluation process must encompass educational goals at all academic levels and research and service functions of the institution (Criteria 20).
Having found established goals in all three of the areas addressed by this "must" statement, the Committee determined that the University complies.
While the majority of goals focus upon teaching, the FES, used by all educational units, ensures that research and service receive attention as well. The units actively encourage all three. The College of Business Administration, for example, emphasizes both teaching and intellectual contributions in its priorities, and the departments within the college list both the promotion of effective teaching and learning and the promotion of intellectual contributions by the faculty among its goals.
6. The institution must evaluate its success with respect to student achievement in relation to purpose, including, as appropriate, consideration of course completion, state licensing examinations, and job placement rates (Criteria 21).
The Committee determined that the University complies with this imperative.
The University maintains records of baccalaureate graduation/ retention/attrition rates for first-time entering freshmen and uses these records to evaluate its success in those areas. The data indicate that SHSU’s success rate in these areas is slightly lower than the average for public universities in Texas. Accordingly, the University has placed renewed emphasis on improving the retention rate; in fact, one of the President’s three major goals outlined during the Fall 1997 faculty meeting addressed retention. While the Committee found that the University complies with this criterion, it nevertheless expressed concern that information on job placement rates is not collected in a systematic fashion. Some units, like the Department of Geography and Geology, make an effort to record job placement; most departments, however, do not collect this information. The Sam Houston State University Career Services makes an effort to collect information on job placement rates, but the data it has assembled are not comprehensive. Recently, however, Career Services has begun surveying employers in an attempt to provide more detailed information on placement rates and to centralize the process of collecting such information. When certification exams are available, units do use them as evaluation tools. For example, programs in Education use the Examination for the Certification of Teachers in Texas (ExCET) exams as an outcomes measure.
RECOMMENDATIONS AND SUGGESTIONS: SECTION 3.1
The Committee on Institutional Effectiveness determined that the University complies with all of the imperatives in Section 3.1 of the Criteria; therefore, the members have no recommendations for this section. Finding the need for improvement in some areas, however, the Committee offers the following suggestions:
While the reporting of institutional effectiveness status has recently been modified with the units reporting to the deans, it is suggested that the deans annually furnish the Planning and Budget Committee with a report on the institutional effectiveness status for units in their colleges. This suggestion is designed to promote coordination between strategic planning, budgeting, and assessment. The Institutional Effectiveness Committee consists of a cross-section of University faculty, with a wide variety of academic expertise, who serve as a resource to the individual units and as a liaison between the units and the OIR. It is suggested that the Committee be restructured to consist of faculty members with assessment expertise. The members should serve as a resource to help units develop and modify assessment processes tailored to their needs. The Internal Auditor should verify the status of each unit’s institutional effectiveness program. A system should be developed to allow units to share insights and information about their institutional effectiveness processes.
3.2 PLANNING AND EVALUATION: ADMINISTRATIVE AND EDUCATIONAL SUPPORT SERVICES
The SACS Committee on Institutional Effectiveness primarily followed the Office of Institutional Research’s classification of support units in its investigation of the University’s planning and evaluation of administrative and educational support services. Table 3.2.a below lists SHSU’s four major organizational units and accompanying support units.
|
President’s Office |
Vice President for Student Services |
Vice President for Academic Affairs |
Vice President for Finance & Operations |
|
Alumni Relations |
Athletics |
Across-the-University Writing Program |
Accounting/ Telephone Services |
|
Internal Auditor |
Career Services |
Admissions |
Accounts Payable |
|
Public Communication |
Child Care |
Center for Innovative Learning |
Building Maintenance |
|
University Advancement |
Counseling Services |
Continuing Education |
Business Office |
|
|
Health Center |
Correspondence |
Business Services |
|
|
Legal Advisor |
Curriculum/Instruction |
Cashier |
|
|
Public Safety Services & Safety Officer |
Graduate Studies |
Central Store |
|
|
Recreational Sports |
Honors Program |
Computer Services |
|
|
Student Activities |
Institute for Innovative Collaborative Programs |
Contracts & Grants |
|
|
Student Center |
Institutional Research |
Custodial |
|
|
Student Life |
Library |
Facilities Planning |
|
|
|
Military Science |
Financial Aid |
|
|
|
Registrar |
Grounds |
Administrative and Educational Support Services (continued)
|
President’s Office |
Vice President for Student Services |
Vice President for Academic Affairs |
Vice President for Finance & Operations |
|
|
|
Research and Sponsored Programs |
Human Resources |
|
|
|
TRIES |
Payroll |
|
|
|
|
Post Office |
|
|
|
|
Property |
|
|
|
|
Purchasing |
|
|
|
|
Residence Life |
|
|
|
|
Sam Houston Press |
|
|
|
|
Vending |
As the methodology section of this chapter indicates, each administrative and educational support services unit was assigned to a member of the Committee on Institutional Effectiveness, who then contacted and interviewed a representative from the unit, with the aim of evaluating the unit’s institutional effectiveness status. Table 3.2.b below lists each support service and the administrative unit to which the support services unit reports, then determines whether the unit’s institutional effectiveness status is acceptable. (The Committee’s reports on the individual units are available in the SACS offices, located in Building AB3.)
The Committee concluded that fifty-two of the fifty-four administrative and educational support services units have acceptable institutional effectiveness programs in place (Table 3.2.b). While there are fifty–six units listed in the above table, two of these units are new to the institutional effectiveness process and therefore are not included in this analysis.) The Committee has noted that in most cases the support units are more recent additions to the University’s institutional effectiveness process. Thus, with many of the processes in their infancy, there is less sophistication than that found among the academic units. Furthermore, the Committee found during the interview stage of the self-study that the support services unit administrators do value a process designed for continuous improvement. There is concern, however, that they lack adequate training and resources to implement an ongoing institutional effectiveness process fully. The Committee recognizes that these support units will develop more sophisticated and thorough assessment processes as they acquire greater experience.
| Units | Fully Acceptable1 | Not Fully Acceptable1 | |
| President's Office | X | ||
| Alumni Relations | X | ||
| Internal Auditor | X | ||
| Public Communication | X | ||
| Sam Houston Museum | X | ||
| University Advancement | X | ||
| Vice President for Student Services | X | ||
| Career Services | X | ||
| Child Care | X | ||
| Counseling Services | X | ||
| Health Center | X | ||
| Legal Advisor | X | ||
| Public Safety Services & Safety Officer | X | ||
| Recreational Sports | X | ||
| Residence Life | X | ||
| Student Activities | X | ||
| Student Center | X | ||
| Student Life | X | ||
| Vice President for Academic Affairs | X | ||
| Across-the-University Writing Program | X | ||
| Admissions | X | ||
| Center for Innovative Learning | X | ||
| Continuing Education | X | ||
| Correspondence | X |
|
|
1Acceptability determined by compliance with "must" statements in section 3.2 of the Criteria.
Institutional Effectiveness: Support Units (continued)
| Units | Fully Acceptable1 | Not Fully Acceptable1 | |
| Curriculum/Instruction | X | ||
| Graduate Studies | X | ||
| Honors Program | X | ||
| Institute for Innovative Collaborative Programs | X | ||
| Institutional Research | X | ||
| Library | X | ||
| Registrar | X | ||
| Research & Sponsored Programs | X | ||
| TRIES | X | ||
| Vice President for Finance and Operations | X | ||
| Accounting/Telephone Services | X | ||
| Accounts Payable | X | ||
| Building Maintenance | X | ||
| Business Office | X | ||
| Business Services | X | ||
| Cashier | X | ||
| Central Store | X | ||
| Computer Services | X | ||
| Contracts & Grants | X | ||
| Custodial | X | ||
| Facilities Planning | X | ||
| Financial Aid | X | ||
| Grounds | X | ||
| Human Resources | X | ||
| Payroll | X | ||
| Post Office | X | ||
| Property | X | ||
| Purchasing | X | ||
| Sam Houston Press | X | ||
| Vending | X | ||
1Acceptability determined by compliance with "must" statements in section 3.2 of the Criteria.
Following are synopses of the Institutional Effectiveness Committee’s deliberations and conclusions about the University’s compliance with the must statements found in Section 3.2 of the Criteria for Accreditation. The "must" statements from the Criteria ask for the compliance of the institution and not of individual units per se. Accordingly, the Committee determined the institution’s overall compliance with each "must" statement by taking a composite view of the forty-six administrative and educational support services units and four administrative and educational support program administrative units.
1. In addition to providing evidence of planning and evaluation in its educational program, the institution must demonstrate planning and evaluation in its administrative and educational support services (Criteria 21).
The Committee determined that Sam Houston State University complies with this imperative.
The President of the University has mandated that all units comply with the institutional effectiveness requirements and has instructed the Director of the Office of Institutional Research to assist units in their efforts to comply. While individual goals are unit-specific, the support units do, for the most part, follow the planning and evaluation process suggested by the OIR. The Committee found that fifty-two support units have adequate processes in place. The Newton Gresham Library is one of many examples of educational support units strongly committed to planning and evaluation. The library seeks regular input from its users and utilizes the solicited information in making decisions about modifying its services.
2. For each administrative and educational support service unit, the institution must (1) establish a clearly defined purpose which supports the institution’s purpose and goals; (2) formulate goals which support the purpose of each unit; (3) develop and implement procedures to evaluate the extent to which these goals are being achieved in each unit; and (4) use the results of the evaluations to improve administrative and educational support services (Criteria 21).
Having evaluated the institutional effectiveness processes of the fifty administrative and educational support services units on these four points, the Committee found that the University complies with this imperative.
Fifty-two of the fifty-four units were deemed to have adequate institutional effectiveness processes in place (see Table 3.2.b). The units have clear missions, goals, and outcome measures; however, informal means of collecting data are more common among the support units than among the academic units. That having been noted, the Committee reports that support units are becoming more willing to use formal survey instruments to gather information for planning and evaluation. For example, the Physical Plant recently distributed a request for input for maintenance, renovation, and construction project needs to the deans, chairs, and department heads. This information will be used to help develop the Physical Plant’s five-year plan.
RECOMMENDATIONS AND SUGGESTIONS: SECTION 3.2
The Committee on Institutional Effectiveness found that the University complies with all of the imperatives in Section 3.2 of the Criteria; therefore, the group has no recommendations for this section. However, finding the need for improvement in several areas, the Committee offers the following suggestions:
While the reporting of institutional effectiveness status has recently been modified so that units report to the deans/directors overseeing the individual units, the committee suggests that the deans/directors annually furnish the Planning and Budget Committee with a report on the institutional effectiveness status for units that report to him or her.
Training on assessment procedures should be provided to support unit personnel.
Collection of data on job placement of graduates should be systematized and centralized.
Information about the importance and value of institutional effectiveness should be included in the orientation session for new SHSU employees.
1. Institutional research must be an integral part of the planning and evaluation process (Criteria 22). The Committee on Institutional Effectiveness found that the University complies with this imperative.
The Director of Institutional Research is a member of the University Strategic Planning Committee. The Office of Institutional Research assists organizational units in obtaining information for their planning and evaluation processes by assisting in the development of assessment instruments. One example is the assistance that OIR provided to the Theatre Department in developing a recent survey instrument.
2. Institutional Research must be effective in collecting and analyzing data and disseminating results (Criteria 22).
The Committee determined that the University does not comply fully with this imperative.
Although the OIR does routinely disseminate institutional research information, available in such publications as the Mini-Profiles report, the University currently uses a largely decentralized system of data collection, data analysis, and data preservation. For example, the Registrar routinely disseminates reports such as the 12th Day Class Report, the Semester Report, and the End of Semester Report. Computer Services personnel assist in the creation of special needs reports and databases (mailing lists for students in a particular academic program, for example). Currently, however, there is no centralized, common database available to organizational units, faculty, and staff. The decentralized databases are not generally accessible outside the originating organizational unit, and are not generally available in compatible formats. The database in the Athletics Department, for example, does not presently allow on-line access. Such restrictions affect the usefulness of the information and often result in redundant efforts to obtain information that already resides in an operational database. The Director of OIR is investigating the use of the SAS EIS system as a means of centralizing the warehousing of information in the existing databases. The director envisions a system that uses decentralized data input and verification with centralized report preparation by the OIR.
3. An institution must regularly evaluate the effectiveness of its institutional research process and use its findings for the improvement of its process (Criteria 22).
The Committee determined that the University complies with this "must" statement.
The OIR has completed its official Institutional Effectiveness cycle, the final report for which is on file in the OIR. The office has established a schedule for periodic updates of its Institutional Effectiveness report.
4. Institutions must assign administrative responsibility for conducting institutional research, allocate adequate resources, and allow access to relevant information (Criteria 22).
The Committee found that the University complies with this criterion.
The OIR is entrusted with administrative responsibility for conducting institutional research, and the Office of the Registrar is charged with responsibility for creating and distributing a wide variety of institutional resource reports. However, as acknowledged earlier in this chapter, the Committee has some concern that the OIR does not have immediate access to all of the information which exists within the decentralized databases. The decentralized structure of data collection requires that the University allocate resources to a variety of units. The Office of the Registrar has adequate resources to create, maintain, and distribute a number of institutional resource reports. Currently the resources of the OIR are split between administering the Institutional Effectiveness program and conducting institutional research. The director of the OIR has been assisted by a University-level faculty committee in administering the Institutional Effectiveness program.
RECOMMENDATIONS AND SUGGESTIONS: SECTION 3.3
The Committee on Institutional Effectiveness found that the University complies fully with all but one of the imperatives in Section 3.3 of the Criteria. To address the instance of partial compliance, the committee makes the following recommendation:
The University must undertake a study to determine the feasibility of creating a common database under the direction of the OIR and to consider the resources required to create such a database. The OIR should have the informational resources necessary to provide appropriate advice and assistance to organizational units in the design and administration of assessment instruments. A centralized database would allow the OIR to provide ready data analysis and reports in assisting organizational units, faculty, and staff with planning and decision-making processes. It should be noted that the recently hired director of the OIR has made the creation of a common database a priority and has emphasized that the OIR serve as a resource for the creation of measurement instruments.
Furthermore, the Committee makes the following suggestions:
The University should consider the implications of assigning the OIR greater responsibility for assisting organizational units in data analysis, report creation, and data warehousing. Currently the OIR often does not routinely receive copies of reports of research conducted by organizational units, which may be of interest to other organizational units.
The University should clearly define institutional effectiveness as a component of the strategic planning process.
Administrative responsibility should be incorporated into the institutional effectiveness process.
The University should consider making institutional effectiveness an explicit part of the internal audit process.
The University should consider reallocating resources to enable the OIR to take the lead role in assisting the institutional effectiveness process with consulting on assessment issues, serving as a centralized source of information to organizational units, and storing institutional effectiveness reports.
COMPLIANCE TABLES: SECTIONS 3.1—3.3
|
Imperative |
Statement of Compliance |
Supporting Documentation |
|
1. Educational activities of an institution include teaching, research and public service. Planning and evaluation for these activities must be systematic, broadbased, interrelated and appropriate to the institution. |
Compliance |
|
|
2. The institution must define its expected educational results and describe its methods for analyzing the results. |
Compliance |
Institutional Effectiveness Reports |
|
3. The institution must (1) establish a clearly defined purpose appropriate to collegiate education, (2) formulate educational goals consistent with the institution’s purpose, (3) develop and implement procedures to evaluate the extent to which these goals are being achieved, and (4) use the results of these evaluations to improve educational programs, services, and operations. |
Compliance |
Institutional Effectiveness Reports |
|
4. The institution must develop guidelines and procedures to evaluate educational effectiveness, including the quality of student learning and of research and service. |
Compliance |
Institutional Effectiveness Reports |
3.1: Planning and Evaluation: Educational Programs (continued)
|
Imperative |
Statement of Compliance |
Supporting Documentation |
|
5. The evaluation process must encompass educational goals at all academic levels and research and service functions of the institution. |
Compliance |
Institutional Effectiveness Reports College of Business Administration (COBA) Priority List COBA AACSB Self-Evaluation Report |
|
6. The institution must evaluate its success with respect to student achievement in relation to purpose, including, as appropriate, consideration of course completion, state licensing examinations, and job placement rates. |
Compliance |
Texas Higher Education Coordinating Board Date on Graduation-Retention-Attrition Rates for Public Universities, First-Time Entering Freshmen Institutional Effectiveness Reports |
3.2: Planning and Evaluation: Administrative and
Educational Support Services
|
Imperative |
Statement of Compliance |
Supporting Documentation |
|
1. In addition to providing evidence of planning and evaluation in its educational program, the institution must demonstrate planning and evaluation in its administrative and educational support services. |
Compliance |
Institutional Effectiveness Reports |
3.2: Planning and Evaluation: Administrative and
Educational Support Services (continued)
|
Imperative |
Statement of Compliance |
Supporting Documentation |
|
2. For each administrative and educational support service unit, the institution must (1) establish a clearly defined purpose which supports the institution’s purpose and goals; (2) formulate goals which support the purpose of each unit; (3) develop and implement procedures to evaluate the extent to which these goals are being achieved in each unit; and (4) use the results of the evaluations to improve administrative and educational support services. |
Compliance |
Institutional Effectiveness Reports |
3.3: Institutional Research
|
Imperative |
Statement of Compliance |
Supporting Documentation |
|
1. Institutional research must be an integral part of the planning and evaluation process. |
Compliance |
|
|
2. Institutional Research must be effective in collecting and analyzing data and disseminating results. |
Partial Compliance |
Institutional Effectiveness Reports Registrar Reports |
|
3. An institution must regularly evaluate the effectiveness of its institutional research process and use its findings for the improvement of its process. |
Compliance |
Institutional Effectiveness Reports |
|
4. Institutions must assign administrative responsibility for conducting institutional research, allocate adequate resources, and allow access to relevant information. |
Compliance |
Registrar Reports |
SUMMARY OF RECOMMENDATIONS AND SUGGESTIONS: SECTIONS 3.1—3.3
RECOMMENDATIONS:
SUGGESTIONS:The University must undertake a study to determine the feasibility of creating a common database under the direction of the OIR and to consider the resources required to create such a database. The OIR should have the informational resources necessary to provide appropriate advice and assistance to organizational units in the design and administration of assessment instruments. A centralized database would allow the OIR to provide ready data analysis and reports in assisting organizational units, faculty, and staff with planning and decision-making processes. It should be noted that the recently hired director of the OIR has made the creation of a common database a priority and has emphasized that the OIR serve as a resource for the creation of measurement instruments.
While the reporting of institutional effectiveness status has recently been modified with the units reporting to the deans, it is suggested that the deans annually furnish the Planning and Budget Committee with a report on the institutional effectiveness status for units in their colleges. This suggestion is designed to promote coordination between strategic planning, budgeting, and assessment.
The Institutional Effectiveness Committee consists of a cross-section of University faculty, with a wide variety of academic expertise, who serve as a resource to the individual units and as a liaison between the units and the OIR. It is suggested that the Committee be restructured to consist of faculty members with assessment expertise. The members should serve as a resource to help units develop and modify assessment processes tailored to their needs.
The Internal Auditor should verify the status of each unit’s institutional effectiveness program.
A system should be developed to allow units to share insights and information about their institutional effectiveness processes.
While the reporting of institutional effectiveness status has recently been modified so that units report to the deans/directors overseeing the individual units, the Committee suggests that the deans/directors annually furnish the Planning and Budget Committee with a report on the institutional effectiveness status for units that report to him or her.
Training on assessment procedures should be provided to support unit personnel.
Collection of data on job placement of graduates should be systematized and centralized.
Information about the importance and value of institutional effectiveness should be included in the orientation session for new SHSU employees.
The University should consider the implications of assigning the OIR greater responsibility for assisting organizational units in data analysis, report creation, and data warehousing. Currently the OIR often does not routinely receive copies of reports of research conducted by organizational units, which may be of interest to other organizational units.
The University should clearly define institutional effectiveness as a component of the strategic planning process.
Administrative responsibility should be incorporated into the institutional effectiveness process.
The University should consider making institutional effectiveness an explicit part of the internal audit process.
The University should consider reallocating resources to enable the OIR to take the lead role in assisting the institutional effectiveness process with consulting on assessment issues, serving as a centralized source of information to organizational units, and storing institutional effectiveness reports.