Challenge Three:  Seminole State College needs to implement its plan to assess general education skills across the cognitive, behavioral, and affective domains at the program level.  This assessment program at the program level will include direct measures of student learning, appropriate feedback loops, and a demonstration of how the information gained is being utilized to improve student learning.

Response: Challenge Three

            At the program level, Seminole State College implemented a general education assessment plan measuring educational skills and utilizing Bloom’s domains of learning taxonomy (Bloom, 1956).  This core curriculum general education Mid-Level Assessment program of student learning was expanded to include degree program assessment of student learning as well. 

            Direct measures of student learning are: 1) the institution’s comprehensive course-embedded assessment program; 2) the nationally-normed Educational Testing Service Academic Profile cognitive test; 3) regionally-based university transfer data performance indicators; and 4) technical degree program performance indicators.  These direct measures are complemented by an array of indirect measures to more effectively identify areas of instruction that may need to be addressed in the quest to improve student learning.  Informal and formal feedback loops focused on assessment give relevant constituencies critical information in a timely fashion to enable them to make appropriate changes necessary to improve student learning institution-wide.

Course-Embedded Assessment in General Education and Degree Programs

The deliberate, multi-tiered design of the Course-Embedded Assessment Program ties SSC’s comprehensive curriculum matrix of course and departmental enabling objectives to the institution’s eight general education student expectations (GESE).  The GESE, broad general education objectives, are derived from the SSC College Mission Statement.  Consequently, the Course-Embedded Assessment Program facilitates evaluation of core curriculum knowledge at the course, department/discipline, general education, and degree program levels.  Also, the focus of the Course-Embedded Assessment Program is on predetermined outcomes, objectives and expectations during each evaluation period.   In this way, evaluation proceeds selectively through the full complement of expected outcomes on a rotating basis.  Through the implementation of this course-to-department/discipline-to-general education-to-degree program evaluation process, measures of student learning in each course become the foundation for course-embedded evaluation of all of the College’s programs. 

SSC correlates general education student expectations, the GESE, directly to its Mission Statement.  All eight GESE address the diverse categories of cognitive learning, six also are concerned with affective learning, and four include behavioral learning.   Students earning associate degrees from Seminole State College must successfully complete 41 general education credit hours across the disciplines.  Each course meets one or more of the GESE.  By the time a learner successfully completes the 41 credit-hour general education requirements, he or she has reached all eight GESE outcomes.  A demonstration of how the comprehensive web of expectations achieves the goals of the general education curriculum is illustrated in the General Education Program Outcomes Course Matrix (Appendix M; Resource 3 available in the Resource Room and listed in Appendix A). 

Institutional planning to achieve course-embedded assessment began with a massive syllabus revision process.  Faculty members in divisions across the curriculum worked tenaciously to develop, identify, and implement appropriate learning outcomes.  Course-embedded assessment tools were put in place to measure these educational skills in cognitive, behavioral and affective domains utilizing Bloom’s taxonomy (Bloom, 1956).  As a standard practice, this publicly stated information is disseminated to students in each course syllabus and/or course.  Associate in Arts and Associate in Science course syllabi identify the enabling objectives, expected learner outcomes, departmental learner outcomes, and GESE in each course.  Associate in Applied Science course syllabi identify learning and/or performance outcomes specific to the relevant curriculum.   

For the 2002-2003 academic year, two of the eight GESE were evaluated: 1) students will demonstrate college-level communication skills, and 2) students will demonstrate critical thinking skills.    The Assessment of Student Learning Committee concluded that direct measures of aggregate learning increases occurred across the curriculum in both GESE evaluated.  Learning increases reported by divisions ranged from 22.71% to 35.43% in fall 2002.  The assessment process also cast more light on areas of strength and areas of concern.  To pinpoint 

areas of strength and deficiency in their respective course-embedded assessment, the faculty members are developing a linkage table in each course to align specific student performance data with specific enabling objectives, expected learner outcomes, departmental learner outcomes, and GESE.  Obtaining objective-specific direct indicators is expected to enable the faculty to correlate evaluation results from their course-embedded evaluation tools to specific student performance objectives and goals in the quest to identify and make necessary improvements in instruction and curriculum (Appendices N and O).

Nationally-Normed Assessment of General Education and Degree Programs

The Educational Testing Service Academic Profile is the College’s nationally-normed indicator of student learning.  The Academic Profile was selected as a Mid-Level, direct measure of the General Education core curriculum to test undergraduate reading, writing, critical thinking, and math skills.  Various instruments and options were reviewed before the Assessment of Student Learning Committee selected this standardized measure.  The Academic Profile makes a contribution to the assessment program in terms of the core curriculum skills measured by the test in the context of material from the humanities, social sciences, and natural sciences.  

Initially, the College piloted the Academic Profile instrument on a small sampling of student volunteers who had completed over 75% of the core general education curriculum in spring 2002.  The following spring, faculty administered the instrument to 268 students in 11 different courses selected in core curriculum areas across each of the College’s academic divisions.   The College selected this intentional sample of courses that included high numbers of SSC students who had completed 45 or more credit hours for the purpose of making Mid-Level Assessment comparisons to the national Academic Profile community college results.  Raw results were distributed to the Division Chair Council and the Assessment of Student Learning Committee.  A summary report prepared by the Assessment of Student Learning Coordinator was distributed to all faculty, the Administrative Council, and the Seminole State College Board of Regents.  The College utilizes this assessment data in its comprehensive Mid-Level Assessment strategy in conducting Departmental/Discipline, General Education, and Degree Program evaluations of student learning.

Spring 2003 results from the 138 SSC students with 45 or more credit hours were analyzed against those of sophomore students in associate degree-granting colleges across the nation.  The Academic Profile score has a range of 400-500.  The mean score of 437 for SSC students with 45 or more credit hours was slightly above the national mean score of 434.9 for associate in arts college sophomores.  Students in the SSC Mid-Level sample had a test score range of 410-485, a mode of 438, and a median of 436.  In all areas of the test, SSC students ranked near the 50th percentile (see Results Chart A in Appendix K).  Comparative application of the results is limited because the national cohort includes students who have completed 31-60 credit hours while the cohort of students with 45 or more credit hours is more appropriate for associate degree program Mid-Level Assessment.  Academic Profile representatives have verbally acknowledged and are addressing this deficiency.  This issue is prevalent among two-year institutions using this and similar standardized tests for Mid-Level Assessment. 

Regional Transfer Data Performance Indicator Reports in Assessment of

General Education and Degree Programs

Transfer Data Performance Indicator Reports from the comprehensive and senior regional universities receiving the majority of SSC’s transfer students provide numerical, regionally-based, comparative performance indicators of student learning (Nichols & Nichols, 2000).  The intended outcome of SSC transfer programs is successful performance of students at the upstream transfer institutions.  This transfer data, for assessment purposes, from baccalaureate degree-granting institutions compare SSC students in the university setting with non-transfer students.  The primary receiving institutions of East Central University, University of Central Oklahoma, and Oklahoma State University generally send these reports each semester to the College.

According to the range of results in this measure, SSC students are competitive with non-transfer students at the receiving institutions, performing slightly below the level of non-transfer students.  The majority of SSC students transfer to East Central University, a senior regional university in commuting distance of the College's service area.  At East Central University for the 2001-02 academic year, 374 students who had transferred from SSC with an average 3.0 GPA subsequently earned a 2.81.  During the same year, the average GPA for all East Central University students was 2.88; for all transfer students, 2.78; and for all other students combined, 2.69.  Compared to the total student body at East Central University, SSC students’ GPAs were only .07 below the average.  At other transfer institutions, the average decline was slightly higher.  The average GPA decline reported by the University of Central Oklahoma was 0.44.  The average GPA decline reported by Oklahoma State University was 0.41. 

SSC analyses of these results identified factors that might be expected to negatively impact the grades of transfer students.  They include cultural differences, potentially larger class sizes and more impersonal interactions, adjustments to new situations and settings, upper-level courses, and increasing financial burdens, among others.  Despite these potential factors, SSC transfer students perform competitively at their receiving institutions.  This data is disseminated to Division Chairs and to faculty by the Assessment of Student Learning Coordinator for use in multiple levels of assessment (Appendix L).

Although transfer data is usually not cast as a direct measure of student learning, Transfer Data Report Indicators were selected for this function at SSC in the manner prescribed by Nichols and Nichols (2000). In this empirical measure, the College’s primary baccalaureate transfer universities provide grade point averages of students at the time of transfer, then later the grade point averages these students attain only at the transferring institution. This data permits a calculation of the grade point average gap between the College and the transferring institution for a given student. Despite the acknowledged subjective nature of grading itself, this measure clearly demonstrates student learning by providing direct comparative indicators about a student’s ability to successfully compete and about his or her academic preparedness in the baccalaureate environment. The transfer of students from two-year degree to four-year-year degree programs is a critical component of the Seminole State College mission. By combining primary course embedded assessment and nationally normed evaluation of student learning with these more secondary Transfer Data Report Indicators, the College is able to triangulate data to determine institutional effectiveness in achieving student learning.

Technical Degree Program Performance Indicators

The technical degree program performance indicator assessment model for the Associate in Applied Science programs generally follows systematic review of a pattern of 1) meeting guidelines or objectives set by the licensing, accreditation or credentialing entities for the specific occupational program; 2) evaluating affiliate training sites; and 3) feedback from employers and graduates on students’ employment and performance in their field.  SSC offers four AAS degree programs: Nursing, Medical Laboratory Technology, Business/Information Systems, and Applied Technology.  In the Business/Information Systems and Applied Technology programs, the College also has established Cooperative Agreements with area Technology Centers leading to AAS degree completion.  Assessment of general education core curriculum in the occupational degree programs is no different from that of transfer degree programs.

The Medical Laboratory Technology degree program was evaluated by the SSC Assessment of Student Learning Committee in spring 2003.  Standards for assessment in this program are examined in terms of student performance on national certification exams reflecting current knowledge in the profession.  Modifications to the curriculum are made based on outcome measures and results from the Board of Registry National Certification Exam in medical laboratory technology.  For example, to address a weakness in student performance on blood banking concepts, the timing of the exam review was altered until after students completed their clinical rotations.  This reinforced experiential learning and resulted in improved exam scores in blood banking.  Scores improved from one percent below the national average in the year 2000, to 131% of the national average in 2001.  The College’s assessment of the degree program analyzed the direct measures required for accreditation of this program—the national certifying exam; the national accrediting standards, specific assessment of 30 focus areas—coupled with general education core curriculum evaluation.  Assessing the degree program against the College's Mission Statement, the Division Chair Council determined that Medical Laboratory Technology is fulfilling the occupational education component in that graduation rates consistently are at or above the national average and that the graduates have multiple job opportunities (Appendix P). 

Indirect Measures in Assessment of General Education and Degree Programs

The College also utilizes a variety of indirect measures to supplement and triangulate analysis of the effectiveness of student learning and students’ educational experiences.  Survey tools include the Student Feedback on Classroom Instruction, SSC Student Opinion Survey, SSC Graduate Opinion Survey, SSC Library Opinion Survey, and the Noel-Levitz Student Satisfaction Inventory and Institutional Priority Surveys.   Data from these indirect indicators information for faculty and administration about factors that enhance or inhibit student learning.  The data is widely disseminated and considered in the analysis of Course-Embedded, Departmental/Discipline, General Education, and Degree Program assessment of student learning results. 

Students take the Student Feedback on Classroom Instruction Survey each fall semester.   This assessment tool is useful to the faculty because it increases insight into students' perceptions, often directly impacting their learning.   Division Chairs distribute classroom feedback results to individual faculty members.  The SSC Student Opinion Survey given biennially is an overall assessment of the SSC student experience.  The SSC Graduate Opinion Survey, mailed to all graduates the year after their graduation, seeks opinions on the quality and relevance of instruction in their chosen career field as well as on their student experience.  The SSC Library Opinion Survey elicits the perception of users about library resources and services.   The Noel-Levitz Student Satisfaction Inventory and Institutional Priorities Survey reports were undertaken in 2001 to evaluate students' perceptions and rank the importance of institutional programs and services compared to the perceptions of faculty, staff, and administration.   The Assessment of Student Learning Committee recommends the revision of subsequent survey instruments to include questions that specifically relate to SSC learning outcomes at all levels.  When appropriately targeted on learning outcomes, such indirect indicators contribute insight and alternative perspectives on SSC’s results from direct indicators of student learning.  Results are widely disseminated to internal and external campus constituencies.  (See Resource Room Listing in Appendix A).

General Education and Degree Program Assessment

Multiple direct and indirect measures discussed previously are systematically funneled into a coordinated process of assessment of student learning at SSC.  In the course evaluation through the Mid-Level evaluation process, full-time and adjunct faculty members compile each semester’s direct individual student learning results into course-embedded assessment reports.  Instructors note the amount of gain, or decline, in student learning for the semester.  These results flow into a process of departmental/discipline analysis that is conducted by faculty members and the division chairs.  At this point, course-embedded assessment results are supplemented by the Academic Profile, Student Transfer Data Performance Indicator Reports and appropriate indirect measure results.  The subsequent departmental/discipline analyses documents and all indirect and direct data collected are used by the Assessment of Student Learning Committee to assess the general education core curriculum against pre-determined general education student expectations.  After this level of assessment is concluded, the various course-embedded assessment documents, along with the multi-level analysis documents, are forwarded to the Division Chair Council, representing all of the academic programs. The council then uses this doumentation to evaluate how well student learning in degree programs is achieving the SSC Mission Statement.   Analysis of the data is focused on determining what the evidence indicates about whether a degree program is achieving the institution’s educational mission.

The College’s Mid-Level general education core curriculum and degree program Assessment of Student Learning is incorporated in an overall assessment approach that is sequential and hierarchical.  Multiple direct and indirect measures of student learning pe