Challenge Three:  Seminole State College needs to implement its plan to assess general education skills across the cognitive, behavioral, and affective domains at the program level.  This assessment program at the program level will include direct measures of student learning, appropriate feedback loops, and a demonstration of how the information gained is being utilized to improve student learning.

Response: Challenge Three

            At the program level, Seminole State College implemented a general education assessment plan measuring educational skills and utilizing Bloom’s domains of learning taxonomy (Bloom, 1956).  This core curriculum general education Mid-Level Assessment program of student learning was expanded to include degree program assessment of student learning as well. 

            Direct measures of student learning are: 1) the institution’s comprehensive course-embedded assessment program; 2) the nationally-normed Educational Testing Service Academic Profile cognitive test; 3) regionally-based university transfer data performance indicators; and 4) technical degree program performance indicators.  These direct measures are complemented by an array of indirect measures to more effectively identify areas of instruction that may need to be addressed in the quest to improve student learning.  Informal and formal feedback loops focused on assessment give relevant constituencies critical information in a timely fashion to enable them to make appropriate changes necessary to improve student learning institution-wide.

Course-Embedded Assessment in General Education and Degree Programs

The deliberate, multi-tiered design of the Course-Embedded Assessment Program ties SSC’s comprehensive curriculum matrix of course and departmental enabling objectives to the institution’s eight general education student expectations (GESE).  The GESE, broad general education objectives, are derived from the SSC College Mission Statement.  Consequently, the Course-Embedded Assessment Program facilitates evaluation of core curriculum knowledge at the course, department/discipline, general education, and degree program levels.  Also, the focus of the Course-Embedded Assessment Program is on predetermined outcomes, objectives and expectations during each evaluation period.   In this way, evaluation proceeds selectively through the full complement of expected outcomes on a rotating basis.  Through the implementation of this course-to-department/discipline-to-general education-to-degree program evaluation process, measures of student learning in each course become the foundation for course-embedded evaluation of all of the College’s programs. 

SSC correlates general education student expectations, the GESE, directly to its Mission Statement.  All eight GESE address the diverse categories of cognitive learning, six also are concerned with affective learning, and four include behavioral learning.   Students earning associate degrees from Seminole State College must successfully complete 41 general education credit hours across the disciplines.  Each course meets one or more of the GESE.  By the time a learner successfully completes the 41 credit-hour general education requirements, he or she has reached all eight GESE outcomes.  A demonstration of how the comprehensive web of expectations achieves the goals of the general education curriculum is illustrated in the General Education Program Outcomes Course Matrix (Appendix M; Resource 3 available in the Resource Room and listed in Appendix A). 

Institutional planning to achieve course-embedded assessment began with a massive syllabus revision process.  Faculty members in divisions across the curriculum worked tenaciously to develop, identify, and implement appropriate learning outcomes.  Course-embedded assessment tools were put in place to measure these educational skills in cognitive, behavioral and affective domains utilizing Bloom’s taxonomy (Bloom, 1956).  As a standard practice, this publicly stated information is disseminated to students in each course syllabus and/or course.  Associate in Arts and Associate in Science course syllabi identify the enabling objectives, expected learner outcomes, departmental learner outcomes, and GESE in each course.  Associate in Applied Science course syllabi identify learning and/or performance outcomes specific to the relevant curriculum.   

For the 2002-2003 academic year, two of the eight GESE were evaluated: 1) students will demonstrate college-level communication skills, and 2) students will demonstrate critical thinking skills.    The Assessment of Student Learning Committee concluded that direct measures of aggregate learning increases occurred across the curriculum in both GESE evaluated.  Learning increases reported by divisions ranged from 22.71% to 35.43% in fall 2002.  The assessment process also cast more light on areas of strength and areas of concern.  To pinpoint 

areas of strength and deficiency in their respective course-embedded assessment, the faculty members are developing a linkage table in each course to align specific student performance data with specific enabling objectives, expected learner outcomes, departmental learner outcomes, and GESE.  Obtaining objective-specific direct indicators is expected to enable the faculty to correlate evaluation results from their course-embedded evaluation tools to specific student performance objectives and goals in the quest to identify and make necessary improvements in instruction and curriculum (Appendices N and O).

Nationally-Normed Assessment of General Education and Degree Programs

The Educational Testing Service Academic Profile is the College’s nationally-normed indicator of student learning.  The Academic Profile was selected as a Mid-Level, direct measure of the General Education core curriculum to test undergraduate reading, writing, critical thinking, and math skills.  Various instruments and options were reviewed before the Assessment of Student Learning Committee selected this standardized measure.  The Academic Profile makes a contribution to the assessment program in terms of the core curriculum skills measured by the test in the context of material from the humanities, social sciences, and natural sciences.  

Initially, the College piloted the Academic Profile instrument on a small sampling of student volunteers who had completed over 75% of the core general education curriculum in spring 2002.  The following spring, faculty administered the instrument to 268 students in 11 different courses selected in core curriculum areas across each of the College’s academic divisions.   The College selected this intentional sample of courses that included high numbers of SSC students who had completed 45 or more credit hours for the purpose of making Mid-Level Assessment comparisons to the national Academic Profile community college results.  Raw results were distributed to the Division Chair Council and the Assessment of Student Learning Committee.  A summary report prepared by the Assessment of Student Learning Coordinator was distributed to all faculty, the Administrative Council, and the Seminole State College Board of Regents.  The College utilizes this assessment data in its comprehensive Mid-Level Assessment strategy in conducting Departmental/Discipline, General Education, and Degree Program evaluations of student learning.

Spring 2003 results from the 138 SSC students with 45 or more credit hours were analyzed against those of sophomore students in associate degree-granting colleges across the nation.  The Academic Profile score has a range of 400-500.  The mean score of 437 for SSC students with 45 or more credit hours was slightly above the national mean score of 434.9 for associate in arts college sophomores.  Students in the SSC Mid-Level sample had a test score range of 410-485, a mode of 438, and a median of 436.  In all areas of the test, SSC students ranked near the 50th percentile (see Results Chart A in Appendix K).  Comparative application of the results is limited because the national cohort includes students who have completed 31-60 credit hours while the cohort of students with 45 or more credit hours is more appropriate for associate degree program Mid-Level Assessment.  Academic Profile representatives have verbally acknowledged and are addressing this deficiency.  This issue is prevalent among two-year institutions using this and similar standardized tests for Mid-Level Assessment. 

Regional Transfer Data Performance Indicator Reports in Assessment of

General Education and Degree Programs

Transfer Data Performance Indicator Reports from the comprehensive and senior regional universities receiving the majority of SSC’s transfer students provide numerical, regionally-based, comparative performance indicators of student learning (Nichols & Nichols, 2000).  The intended outcome of SSC transfer programs is successful performance of students at the upstream transfer institutions.  This transfer data, for assessment purposes, from baccalaureate degree-granting institutions compare SSC students in the university setting with non-transfer students.  The primary receiving institutions of East Central University, University of Central Oklahoma, and Oklahoma State University generally send these reports each semester to the College.

According to the range of results in this measure, SSC students are competitive with non-transfer students at the receiving institutions, performing slightly below the level of non-transfer students.  The majority of SSC students transfer to East Central University, a senior regional university in commuting distance of the College's service area.  At East Central University for the 2001-02 academic year, 374 students who had transferred from SSC with an average 3.0 GPA subsequently earned a 2.81.  During the same year, the average GPA for all East Central University students was 2.88; for all transfer students, 2.78; and for all other students combined, 2.69.  Compared to the total student body at East Central University, SSC students’ GPAs were only .07 below the average.  At other transfer institutions, the average decline was slightly higher.  The average GPA decline reported by the University of Central Oklahoma was 0.44.  The average GPA decline reported by Oklahoma State University was 0.41. 

SSC analyses of these results identified factors that might be expected to negatively impact the grades of transfer students.  They include cultural differences, potentially larger class sizes and more impersonal interactions, adjustments to new situations and settings, upper-level courses, and increasing financial burdens, among others.  Despite these potential factors, SSC transfer students perform competitively at their receiving institutions.  This data is disseminated to Division Chairs and to faculty by the Assessment of Student Learning Coordinator for use in multiple levels of assessment (Appendix L).

Although transfer data is usually not cast as a direct measure of student learning, Transfer Data Report Indicators were selected for this function at SSC in the manner prescribed by Nichols and Nichols (2000). In this empirical measure, the College’s primary baccalaureate transfer universities provide grade point averages of students at the time of transfer, then later the grade point averages these students attain only at the transferring institution. This data permits a calculation of the grade point average gap between the College and the transferring institution for a given student. Despite the acknowledged subjective nature of grading itself, this measure clearly demonstrates student learning by providing direct comparative indicators about a student’s ability to successfully compete and about his or her academic preparedness in the baccalaureate environment. The transfer of students from two-year degree to four-year-year degree programs is a critical component of the Seminole State College mission. By combining primary course embedded assessment and nationally normed evaluation of student learning with these more secondary Transfer Data Report Indicators, the College is able to triangulate data to determine institutional effectiveness in achieving student learning.

Technical Degree Program Performance Indicators

The technical degree program performance indicator assessment model for the Associate in Applied Science programs generally follows systematic review of a pattern of 1) meeting guidelines or objectives set by the licensing, accreditation or credentialing entities for the specific occupational program; 2) evaluating affiliate training sites; and 3) feedback from employers and graduates on students’ employment and performance in their field.  SSC offers four AAS degree programs: Nursing, Medical Laboratory Technology, Business/Information Systems, and Applied Technology.  In the Business/Information Systems and Applied Technology programs, the College also has established Cooperative Agreements with area Technology Centers leading to AAS degree completion.  Assessment of general education core curriculum in the occupational degree programs is no different from that of transfer degree programs.

The Medical Laboratory Technology degree program was evaluated by the SSC Assessment of Student Learning Committee in spring 2003.  Standards for assessment in this program are examined in terms of student performance on national certification exams reflecting current knowledge in the profession.  Modifications to the curriculum are made based on outcome measures and results from the Board of Registry National Certification Exam in medical laboratory technology.  For example, to address a weakness in student performance on blood banking concepts, the timing of the exam review was altered until after students completed their clinical rotations.  This reinforced experiential learning and resulted in improved exam scores in blood banking.  Scores improved from one percent below the national average in the year 2000, to 131% of the national average in 2001.  The College’s assessment of the degree program analyzed the direct measures required for accreditation of this program—the national certifying exam; the national accrediting standards, specific assessment of 30 focus areas—coupled with general education core curriculum evaluation.  Assessing the degree program against the College's Mission Statement, the Division Chair Council determined that Medical Laboratory Technology is fulfilling the occupational education component in that graduation rates consistently are at or above the national average and that the graduates have multiple job opportunities (Appendix P). 

Indirect Measures in Assessment of General Education and Degree Programs

The College also utilizes a variety of indirect measures to supplement and triangulate analysis of the effectiveness of student learning and students’ educational experiences.  Survey tools include the Student Feedback on Classroom Instruction, SSC Student Opinion Survey, SSC Graduate Opinion Survey, SSC Library Opinion Survey, and the Noel-Levitz Student Satisfaction Inventory and Institutional Priority Surveys.   Data from these indirect indicators information for faculty and administration about factors that enhance or inhibit student learning.  The data is widely disseminated and considered in the analysis of Course-Embedded, Departmental/Discipline, General Education, and Degree Program assessment of student learning results. 

Students take the Student Feedback on Classroom Instruction Survey each fall semester.   This assessment tool is useful to the faculty because it increases insight into students' perceptions, often directly impacting their learning.   Division Chairs distribute classroom feedback results to individual faculty members.  The SSC Student Opinion Survey given biennially is an overall assessment of the SSC student experience.  The SSC Graduate Opinion Survey, mailed to all graduates the year after their graduation, seeks opinions on the quality and relevance of instruction in their chosen career field as well as on their student experience.  The SSC Library Opinion Survey elicits the perception of users about library resources and services.   The Noel-Levitz Student Satisfaction Inventory and Institutional Priorities Survey reports were undertaken in 2001 to evaluate students' perceptions and rank the importance of institutional programs and services compared to the perceptions of faculty, staff, and administration.   The Assessment of Student Learning Committee recommends the revision of subsequent survey instruments to include questions that specifically relate to SSC learning outcomes at all levels.  When appropriately targeted on learning outcomes, such indirect indicators contribute insight and alternative perspectives on SSC’s results from direct indicators of student learning.  Results are widely disseminated to internal and external campus constituencies.  (See Resource Room Listing in Appendix A).

General Education and Degree Program Assessment

Multiple direct and indirect measures discussed previously are systematically funneled into a coordinated process of assessment of student learning at SSC.  In the course evaluation through the Mid-Level evaluation process, full-time and adjunct faculty members compile each semester’s direct individual student learning results into course-embedded assessment reports.  Instructors note the amount of gain, or decline, in student learning for the semester.  These results flow into a process of departmental/discipline analysis that is conducted by faculty members and the division chairs.  At this point, course-embedded assessment results are supplemented by the Academic Profile, Student Transfer Data Performance Indicator Reports and appropriate indirect measure results.  The subsequent departmental/discipline analyses documents and all indirect and direct data collected are used by the Assessment of Student Learning Committee to assess the general education core curriculum against pre-determined general education student expectations.  After this level of assessment is concluded, the various course-embedded assessment documents, along with the multi-level analysis documents, are forwarded to the Division Chair Council, representing all of the academic programs. The council then uses this doumentation to evaluate how well student learning in degree programs is achieving the SSC Mission Statement.   Analysis of the data is focused on determining what the evidence indicates about whether a degree program is achieving the institution’s educational mission.

The College’s Mid-Level general education core curriculum and degree program Assessment of Student Learning is incorporated in an overall assessment approach that is sequential and hierarchical.  Multiple direct and indirect measures of student learning pertinent at each level of evaluation provide additional lenses through which to view the data and triangulate a more accurate analysis.  Reporting formats are standardized for consistency and to encourage clear communication.   Evaluation formats, adapted from Nichols and Nichols (2000), make it possible to assess and analyze student learning outcomes, objectives, and expectations in departments/disciplines at the course level, then at Mid-Level in the general education core curriculum and at the degree program level.  Summary course-embedded assessment reports along with the draft and final departmental/discipline, general education, and degree program evaluation reports are widely distributed for use in improving student learning.  This information is disseminated to faculty, Division Chair Council, the Assessment of Student Learning Committee, the Administrative Council and the SSC Board of Regents.  Changes in curriculum, instruction, teaching materials, and methods of evaluation are made on an ongoing basis at each level of assessment in response to the assessment data analyses (Appendix Q).

SCC offers 22 program degrees.  There are seven Associate in Arts degree programs, eleven Associate in Science Degree programs, and four Associate in Applied Science degrees.  Assessment of degree programs is conducted on a five-year rotation, with one exception.  The Associate in Arts for General Studies is evaluated biennially because it is the degree awarded to the highest number SSC graduates.  During the 2002-03 academic year, the reviewed degree offerings of transfer and occupational degrees evaluated were Associate in Arts in Behavioral Science, Associate in Science in Business, Associate in Science in Life Science, Associate in Applied Science in Medical Laboratory Technology, and Associate in Arts for General Studies.  Evaluating these degree programs in terms of the SSC Mission Statement, the Division Chair Council determined that the College is fulfilling the transfer degree and occupational education goals.  Areas for improvement and action are noted on the evaluation documents in each program (Appendices R and S).

Assessment Feedback Loops to Improve Student Learning and Strengthen Instruction

A feedback loop is the continuous process by which the College collects, analyzes, and reports assessment data to develop and implement improvements in student learning.  SSC students, faculty, staff, administrators and regents are informed about student learning taking place through the exchange of assessment data, as noted in the previous parts of this response to Challenge Three.  The development of formal and informal feedback loops facilitates the conversion of factual data into a lively system of instructional improvement.  Assessment information is shared via Course Embedded Assessment Divisional Summary Reports, assessment office communications, assessment committee meetings, division chair and division meetings, reports to regents, memos, conversation patterns, in-class informal student surveys and comments, and instructional goals, objectives and budget process, among others.  The College fosters a campus/community culture that encourages this ongoing assessment process. 

The institution’s assessment program stimulates changes in instruction through feedback loops.   The willing and honest exchange of information in SSC’s campus-wide assessment culture inspires reflection on pedagogy, curriculum, and the general operation of the academic environment.  Affirming the concept that "Faculty need feedback on a regular, periodic basis," (North Central Association of Colleges and Schools, 1997, p.  44), Seminole State College uses both formal and informal feedback loops to link students, faculty, staff, administration, and the board of regents in an ongoing exchange of information.  Formal loops rely on the publication, dissemination, discussion and utilization of core assessment documents and results.  Informal loops include spontaneous conversations and the exchange of ideas.  SSC creates a culture of assessment of student learning that is as comfortable at lunch as at a scheduled assessment meeting.  An example of the flow of formal and informal information regarding feedback on student instruction is illustrated in the following:

·        Student written comments on the SSC Student Feedback on Classroom instruction        ;

·        In-class discussion of the SSC assessment process (a faculty-generated addition to course evaluation that adds a wealth of simple suggestions encouraging significant improvement of learning climate and techniques);

·        Casual conversations among various campus constituencies;

·        Student/faculty formal meetings in some disciplines to discuss instructional methods and curriculum; and

·        Division Chair Council; Faculty Senate; Student Government; Assessment Committee; Board of Regents meetings all include discussion of collected assessment data and pursuant results.

            There is ample and strong evidence to indicate that across the academic divisions at Seminole State College faculty members are using assessment information they gather both to improve student learning and to strengthen instruction.   Supporting evidence is contained in instructors' assessment reports, ongoing interviews of the faculty by a subcommittee of the Assessment of Student Learning Committee and reports and evaluations of instructors' teaching by supervisors and students.  As an example, Business/Information Systems Division faculty members teaching computer science and microeconomics courses responded to assessment data and changed their instruction to offer more hands-on training for students through increased online instructional opportunities.   Subsequently, division faculty documented improved post-test results.   In English Composition, instructors responded to assessment concerns seeking more instructional consistency in composition classes by preparing a new, unified curriculum.  

Feedback of Assessment Data to Divisions and Relevant Constituencies

            SSC disseminates assessment results in an efficient and timely manner, providing a clear distribution pattern to and from division chairs and to relevant constituencies on and off campus.  The Assessment of Student Learning Coordinator delivers published assessment documents to faculty and other relevant constituencies within days following publication.  Documents also are available upon request year-round from the assessment office.  

            Course-Embedded Assessment Reports and Summaries follow a semester-based schedule of deadlines for collection, analysis, reporting and publication of results.   Using faculty's Course Embedded Assessment Reports, division chairs prepare Divisional Summary Reports that are channeled to the assessment office.  In turn, the Assessment of Student Learning Coordinator compiles, analyzes and publishes the semester's Course-Embedded Assessment Report and delivers it to division chairs, assessment committee members and faculty with an Executive Summary.  Division chairs provide the Course-Embedded Divisional Summary to their faculty.  The students receive individual course-embedded assessment results as well as course and student syllabi containing assessment information in class.  Additionally, students receive institutional assessment data through their student representatives on the Assessment of Student Learning Committee (Appendices I and J).

Feedback Loops Connect Assessment Results with Institutional Resources

            The College commits its human, fiscal, and physical resources to achieve assessment goals.  Since the 2000 site visit, SSC has made a major commitment of administrative and faculty time and institutional resources to developing and implementing a viable, effective assessment program.  Faculty members, administrators, staff and students have attended numerous workshops, conferences, meetings, and training sessions focused on assessment.  Over the past three years, the College spent nearly $.34 million on salaries, activities, travel, materials, and technology directly tied to improvement of student learning.  The College's academic planning and budgeting procedures allow faculty members, through the Division Chair structure, to submit strategic planning goals and objectives and budget needs based on assessment of student learning data.  

            There is convincing proof that the investment of assessment-based dollars is improving student learning and strengthening instruction at Seminole State College.  Examples from Nursing and Geology are illustrative.  In Nursing, to address a need identified through assessment for better case planning and patient documentation found lacking in Nursing Across the Life Span (NURS 1528), the College spent hundreds of dollars on technology and related software to improve patient documentation.   Nursing faculty report students in subsequent semesters are better prepared and require less remediation than in previous years.  Ninety percent of students evaluated their documentation labs as “meeting” or “more than meeting” their learning needs since implementing the new documentation software.  In Geology, the assessment process revealed a deficiency in General Geology (GEOL 1114).  Students needed more real-life applications to strengthen learning in the course, especially in terms of reinforcing class material.   Consequently, funds were designated to provide appropriate instructional material.  The Geology Department anticipates an increase in student performance as a result of these efforts.  (Other examples of financial commitment are in Appendix T).

Response Three:  Conclusion

   In summary, Seminole State College responded to the third challenge identified by the 2000 HCL/NCA site visit

team with an effective, thorough, and evolving program of assessment.  At the general education and program level,

the College uses data obtained on student learning from direct measures of student learning across domains of

learning.  The faculty are better able to align specific student performance data from course-embedded assessment

with specific enabling objectives, expected learner outcomes, department learner outcomes, and general education

student expectations.  The implementation of the nationally-normed Academic Profile test provides an objective

measure external to the College, demonstrates the institution’s satisfactory ranking as well as notes areas to improve

in terms of general education core curriculum skills.  Regional Transfer Data Performance Indicator Reports also

provide an external measure of success of the College’s transfer program, confirming student’s solid academic

preparation and identifying areas that need to be strengthened.  Through the timely use of feedback loops,

assessment findings are widely shared, documented and utilized with appropriate constituencies to improve student

learning.