Kinds of assessment-related actions

From a reply to a colleague's email...

You're correct that significant changes should be based on sufficient data. But what constitutes “sufficient data” may differ in every case. The primary safeguard of the integrity of our programs and our assessment procedures is that evidence-based decisions about changes are to be made by the program’s faculty.

Some programs, for some of their learning outcomes, have several semesters’, even several years’, data. But if a program only has data on a given learning outcome from a single semester, the faculty may very well decide that the appropriate “action” is to continue collecting data until there’s enough to justify doing something. In such a case, the making of that decision and the meeting and discussion from which it arose are, themselves, assessment-related actions that should be reported. More on that point below.

The Assessment Report (as distinct from the Assessment Plan) calls for reporting the assessment-related actions that were taken during a given academic year. There are several types of assessment-related actions, but they fall into two main categories and assessment results may yield evidence bearing on both types of actions:

• Changes in the instructional program for the purpose of improving students’ learning. Guided by assessment results, there’s an infinity of things that a faculty might do to improve their students’ learning and achievement of the program’s learning outcomes. These possibilities include modifying the program’s learning outcomes, themselves. Just a couple of other examples are developing a curriculum map in order to better specify the courses in which the various outcomes are addressed, and deciding on a core set of course-level learning outcomes for particular courses.
• Changes in assessment procedures. Guided by assessment results, the faculty may decide to add, delete, or modify the means by which student achievement of learning outcomes are measured. For instance they may improve a rubric, develop a plan for portfolios of student work, or figure out ways in which the measurement of some program-level learning outcomes can be embedded into certain courses.

There’s another way to think about the types of actions that should be reported in the Assessment Report. Some assessment-related actions seem more “action-like” than others, such as writing a proposal for a capstone course or developing a rubric to evaluate oral presentations. Then there are other assessment-related actions that we’re less likely to call “actions,” but they really are because they’re things that faculty do. Examples of these less action-like actions are meeting to review data or deciding that you’ll select a new means of assessment, even if you’re not going to actually select that new means until the following year. Both kinds of actions should be reported because they’re all things that faculty have done that are related to assessment.

Course-Embedded Assessment

At the last meeting of my university's Academic Assessment Committee, I said I'd look for more information about embedding assessment of program-level learning outcomes into particular courses. There's a good bit of stuff out there on this topic. Here's a paragraph I found in the University of Wisconsin's assessment manual:

Course-Embedded Assessment: Assessment practices embedded in academic courses generate information about what and how students are learning within the program and classroom environment. Course-embedded assessment takes advantage of already existing curricular offerings by using standardized data instructors already collect or by introducing new assessment measures into courses. The embedded methods most commonly used involve the development and gathering of student data based on questions placed in course assignments. These questions, intended to assess student outcomes, are incorporated or embedded into final exams, research reports, and term papers in senior-level courses. The student responses are then evaluated by two or more faculty to determine whether or not the students are achieving the prescribed educational goals and objectives of the department. This assessment is a separate process from that used by the course instructor to grade the exam, report, or term paper.

And here's a link to a paper on the subject. It was written by a faculty member in Dental Hygiene, but she addresses the topic in a general way.