The Accountability/Improvement Paradox

From today's Inside Higher Ed:

In the academic literature and public debate about assessment of student learning outcomes, it has been widely argued that tension exists between the two predominant presses for higher education assessment: the academy's internally driven efforts as a community of professional practitioners to improve their programs and practices, and calls for accountability by various policy bodies representing the “consuming public.”

...much more than merely a mismatch exists between the two perspectives; there is an inherent paradox in the relationship between assessment for accountability and for improvement. More importantly, there is an imbalance in emphasis that is contributing to a widening gap between policy makers and members of the academy with regard to their interests in and reasons for engaging in assessment. Specifically, not enough attention is being paid to the quality of measurement (and thought) in the accountability domain, which undermines the quality of assessment activity on college campuses.

The root of the paradoxical tension between forces that shape external accountability and those that promote quality improvement is the discrepancy between extrinsic and intrinsic motivations for engaging with assessment. When the question “why do assessment?” arises, often the answer is “because we have to.” Beyond this reaction to the external pressure is a more fundamental reason: professional responsibility.

And now, for the rest of the story...

New issue of Peer Review

The new issue of the AAC&U publication, Peer Review is now available.

Standards for independent study programs

Independent study programs may also serve as high-impact educational practices, when done well. Here's a discussion, from today's Inside Higher Ed, about considerations along those lines. A couple of quotes to get you going:

Academic credit should be awarded only for work that advances the education of students, within the curriculum, usually as defined by the faculty as a whole.

In situations involving individual arrangements in serious matters (including awarding academic credit), prior approval procedures and written documentation are good practice and protect everyone: the student, the faculty member, and the institution.

Conflicts of interest exist and can be addressed through disclosure and review of proposed activities by disinterested parties.

All three of these principles support the conclusion you’ve reached, which is that individual arrangements between students and faculty members should fall under the general oversight of “the faculty,” not just be idiosyncratic personal arrangements. These arrangements should be documented, reviewed to assure they meet the standards established by the faculty and also reviewed to protect all parties to them. The faculty should be protected against charges of favoritism and capricious grading. Students should be on notice of what is required of them, applicable deadlines and standards that must be met. The academic unit should have records sufficient to document the award of academic credit meeting institutional standards.

And now, for the rest of the story...

DoL releases new rules for 'educational' internships

Internships are supposed to be one of those high-impact educational practices, right? So it's of interest to Assess this! when things like this happen.

Amid increasing national attention to unpaid internships, the U.S. Department of Labor released a statement on Wednesday that clarifies employers' and colleges' roles under federal law.

The document applies a six-part test from the Fair Labor Standards Act, which derives from a 1947 Supreme Court decision involving railroad-company trainees, to modern-day interns in the for-profit sector. For interns to work for private companies without compensation, the Labor Department says, their positions must meet six criteria.

First, "the internship, even though it includes actual operation of the facilities of the employer, is similar to training which would be given in an educational environment."

And now, for the rest of the story...

Data-driven difficulties

Today's Dean Dad blog entry discusses why it's so difficult to actually do data-driven. He begins like this:

From much of the discussion of 'data-driven' reforms that take place at the national level, you'd think that all that we'd need to do is educate some administrators on the use and interpretation of data, tell them to use what they've learned, and that would be that.

If only it were that easy...

And now, for the rest of the story...

I spent the day with rubrics and spreadsheets.

Libby Gruner writes an occasional column for Inside Higher Ed.

This is not exactly the life I envisioned when I began my graduate work....

But the rubrics and the spreadsheets happen to be part of the job too, right now, and I'm learning to embrace them. The rubrics, after all, will help us to evaluate the job we're doing in the new program I'm helping to build. If we don't write them well, they're just busy work, but if we really think through them carefully we might learn something from them. (Yes, I still retain some of that youthful naïveté!) The spreadsheets are simply the easiest way I know to organize and compare the data my program is collecting — about courses, about workshops offered and taken, about summer training plans. They're not lovely, nor can I make them do everything I want them to (yet), but they are a useful tool.

And now, for the rest of the story...

à la DETC?

Inside Higher Ed also reports on an apparently bogus accreditation organization for distance education programs:

Officials at the Distance Education and Training Council, a Washington-based accrediting body, woke up Monday morning to find that the agency had acquired an unwelcome twin overseas.

The body was tipped off to the existence of a Web site for an organization with the same name, with a British Web domain and a Cyprus address, that claims to be “the leading global membership organization for the open and distance education community.”

The Washington-based council believes the organization to be an accreditation mill -- a phony accrediting body designed to lend a veneer of legitimacy to bogus institutions.

And now, for the rest of the story...

à la VSA?

Here's a story from today's Inside Higher Ed about an effort to develop a new national accountability system for community colleges that is inevitably being compared to the Voluntary System of Accountability for public four-year institutions. A voice that's prominently featured in this story and in this discussion is that of Eileen Baccus, a good friend of my university, whose very sensible comments from the story follow:

Baccus reiterated that it is important for the colleges themselves to be first to define accountability for community colleges. With growing federal and private sector interest in community colleges, she said, there will be accountability systems created. "Do we want someone else to impose these measures, or do we want to say that here are measures we can endorse?" she said.

And now, for the rest of the story...

à la Bologna?

Also from the April 18 edition of University World News, two more stories about international regional collaboration in higher education:

CARIBBEAN: Call for regional collaboration

ANGOLA: Call for increased higher education cooperation

Background & progress of Bologna Process

From the April 18 edition of University World News:

The Hungarian and Austrian Ministers for Science and Research, Education and Culture who were responsible for the 2010 Bologna ministerial meeting have released a magazine summing up the Bologna achievements and capturing many of its dimensions.

Together with the Spanish Minister of Education, now chairing the Bologna process in the capacity of holding the Presidency of the European Union, the ministers' release captures the many dimensions in the Bologna process.

The magazine is a much needed simplification of what the Bologna process is about: most notably greater accountability of degrees and study periods among European nations as a pre-condition for increased mobility during and after graduation.

.... The consensus is that the Bologna process has released new energy and a new spirit of collaboration and new goals for European higher education institutions.

Some say the Bologna process is the major impulse for university reforms ever, as expressed by E Stephen Hunt at the US Department of Education:

"The Bologna process is almost certainly the most important multinational reform of higher education undertaken since the teaching guilds and the student nations established the revolutionary concepts of the studium generale and universities in the 11th and 12th centuries," Hunt writes.

And now, for the rest of the story...

New organization for assessment professionals

Here's a post that appeared on the ASSESS listserv today:

Dear Colleagues:

I am pleased and honored to announce the formation of a new association for assessment practitioners, one that focuses on assessment practice and the factors that affect it. Designed in collaboration with a number of other assessment professionals, including Marilee Bresciani, Susan Hatfield, Ephraim Schechter, Linda Suskie, and Jon Wergin, the association will provide resources of many kinds to assessment practitioners, from assessment directors to individual faculty to graduate students.

During my tenure as Director of Assessment and Senior Scholar at the American Association for Higher Education, I had worked to organize the valuable but somewhat inchoate resources of that organization into a coherent and accessible source for information about assessment practice. Though AAHE closed before any of that work came to full fruition, I had still harbored the hope that such a source could be built. I believe that we are on the threshold of doing so.

The Association for the Assessment of Learning in Higher Education will be housed at the University of Kentucky, the current home of the ASSESS listserv, which will become a part of AALHE. Thanks to the efforts of Marsha Watson, Kentucky’s Director of Assessment, the University is providing physical and virtual space, staff and technical support, and seed money for a first conference in June 2011.

Although the website is only now being designed, our intent is to provide a wide range of resources to those who are interested in improving student learning through assessment. In addition to the ASSESS listserv, the website will offer the following features:

• A moderated, wiki-like resource area in which practitioners can post assessment tools and strategies, from successful plans through course-based rubrics;
• A threaded blogging area on selected topics in assessment, from assessing general education to the ethics of assessment practice;
• Publications, including a juried e-journal and a quarterly newsletter;
• Communities of practice, aimed at serving different populations, from Assessment Directors to graduate students, as well as separate communities of practice for each of the accreditation regions;
• Regularly scheduled virtual conversations and webinars on specific topics;
• An annual face-to-face conference, with virtual elements.

Because the association will be largely virtual, dues and expenses will be relatively low, but we are hoping that interest will be high. In addition, some of the features, like the ASSESS listserv, will be open to non-members as well. We expect to be operational in at least some of the areas listed above by June 30, 2010, and we welcome your suggestions for services other than what’s mentioned above as we go through the design-and-implementation process.

If you want to visit our website to see its development before June 30, feel free to do so; the URL is www.aalhe.org.

Robert T. Mundhenk
Visiting Scholar, The Higher Learning Commission
Former Director of Assessment and Senior Scholar, AAHE

Winter 2010 issue of AAC&U's Liberal Education

The new issue of the AAC&U publication, Liberal Education is now available.

Comparing Rubric Assessments to Standardized Tests

Here's a quote from the lead article in the new issue of AAC&U News.

When the CLA results arrived eight months later, the UC team compared the outcomes of the two assessments. “We found no statistically significant correlation between the CLA scores and the portfolio scores,” Escoe says. “In some ways, it’s a disappointing finding. If we’d found a correlation, we could tell faculty that the CLA, as an instrument, is measuring the same things that we value and that the CLA can be embedded in a course. But that didn’t happen.” There were many factors that may have contributed to the lack of correlation, she says, including the fact that the CLA is timed, while the rubric assignments are not; and that the rubric scores were diagnostic and included specific feedback, while the CLA awarded points “in a black box”: if a student referred to a specific piece of evidence in a critical-thinking question, he or she simply received one point. In addition, she says, faculty members may have had exceptionally high expectations of their honors students and assessed the e-portfolios with those high expectations in mind—leading to results that would not correlate to a computer-scored test. In the end, Escoe says, the two assessments are both useful, but for different things. The CLA can provide broad institutional data that satisfies VSA requirements, while rubric-based assessment provides better information to facilitate continuous program improvement.

And now for the rest of the story...

April issue of AAC&U News is available

The new issue of AAC&U News is now available.