Recommendation 4 (SLO Response)

 

For over a decade the College has had a system in place, developed in house, for reporting assessment dialog (R.4.1, R.4.2). This system, also known as the “legacy system,” captured the dialog about student learning in the “Finding and Results” and “Action Changes” sections of course- and program-level reports.  These reports and associated dialogue have been the basis for program-level discussions documented in the program review reports under the following annual and comprehensive program review report sections: 3.1: “changes based on assessment findings”, 3.2: “assessment requiring further research” and Section 3 summary: “how assessment findings impact future planning.” However, the overall documentation of dialog for some areas was decentralized in cases where dialog was conducted in small department meetings or by faculty in an individual discipline. This made evaluating the overall sequence of dialog leading to student learning and planning difficult to access for people unfamiliar with the College’s legacy system (R.4.3, R.4.4, R.4.5, R.4.6, R.4.7, R.4.8, R.4.9, R.4.10, R.4.11, R.4.12, R.4.13, R.4.14). The assessment dialog leads directly to planning and includes depth of analysis, but that piece is located in the program review forms (R.4.15, R.4.16).

In November 2018, a task force began meeting with the purpose to identify a new system that would allow for a more streamlined way to capture assessment dialog, its link to programmatic planning, and the ability to re-assess in a follow-up manner to determine program improvement. The task force selected the eLumen interface to use in both assessment and curriculum management. In spring 2020, a pilot using eLumen was scheduled to take place, but due to the rapid shift to remote instruction was delayed until the fall 2020 and spring 2021 semesters. At that time, associate deans and area faculty members in six different departments volunteered to participate in the pilot. In the end, the College ended up running the pilot for two semesters to test the program, with over 100 volunteers participating (R.4.17, R.4.18, R.4.19).

In fall 2021, all instructional assessment will be done using eLumen. Key highlights of this transition from the College’s in-house system to the eLumen system include: all of the student learning outcomes for a particular course will be assessed at once, when the course comes up for evaluation on the assessment plan.  The course-level assessment process now requires instructors to complete a “reflection template” that documents specific areas of success and areas that are discussed for improvement. The results of the course-level assessments and reflection templates are reviewed and used as the basis of program review improvement plans.  This program-level step is conducted each fall prior to completion of annual program review reports.  This new system allows for dialogue that is systematically conducted on a regular basis.  This dialogue documents the successes and improvement plans for courses and programs within the program review templates.  Furthermore, these annual program assessment conferences will allow for more accessible documentation of dialogue that reflects the depth of analyzing student achievement at the course- and program-level, and that is designed to be the basis of specific course- and program-level improvement. The “action plan” tool in eLumen will document these discussions. Lastly, the reporting of results, dialogue, and improvement planning for student services assessment has been folded into the program review process. Please see the College’s response to Recommendation 6 for more information about student services assessment.

The Assessment Committee has also moved to improve outcome assessment dialogue and improvement documentation by modifying the College’s assessment process. This has facilitated important improvements, including: moving from incremental to periodic outcome assessment, so that data on all course outcomes are captured at one point in time, which allows faculty to implement improvement plans on a timely basis and to re-assess more frequently; moving from course-focused to section-focused assessment reporting, and facilitating the collection of disaggregated section data; providing the option for individual versus collective student scoring, facilitating collection of performance data by student demographics; moving to individualized faculty reflections on course assessment data; and using the eLumen platform for outcome updates, course-to-program and course-to-institutional outcome mapping, assessment planning, report generation, and the documentation of dialogue and action planning. The College has also adopted a model of the outcome assessment process which utilizes annual discipline assessment conferences where department and program faculty collaborate on the review of individual course and program assessments. This normalizes disciplinary and cross-disciplinary assessment dialogue as part of the institution’s regular activities.

The Assessment Committee and Program Review committee worked in close collaboration to modify the program review reporting system in order to integrate assessment dialogue as it relates to program plans. This was done with the additional attention of insuring that associated dialogue is well-documented and easily accessible, and that the dialogue reflects the depth of the assessment process.  Instructional divisions reflect on the results of past assessment-driven plans, report current findings, and describe any expected impact on future planning as a result of assessment.  Service areas report similar detail, with learning/service area outcomes now constituting a formal part of the program indicator analysis conducted in program review (R.4.20, R.4.21).

Analysis and Evaluation:

The College has been using an assessment reporting system that it developed “in house” to capture course-, program- and service-learning outcome assessment dialog. The dialog that is captured in the so-called “legacy tool” is then used to inform the development of programmatic and course-level planning that appears on the program review reporting form. The reporting form requires program personnel to discuss changes made based on assessment findings, evaluate the impact of the implementation of planning actions and explain how assessment dialog has informed the creation of new plans. With the transition from the “legacy tool” to eLumen, the assessment-dialog-planning-evaluation sequence will be much more streamlined, the process from assessment to improvement planning will be easier to identify, evidence of dialogue will be more accessible, and the depth of discussion will be reflected in the eLumen templates and program review reports.

Evidence Sources Below