I'm scheduled on a panel titled "Technology-Enabled Authentic Assessment" at the IMS Global Learning Impact Leadership Institute in May 2019 with Smart Sparrow CEO Dror Ben-Naim and Kurt Peterschmidt from ACTNext. Of course, this means I've been thinking a lot about authentic assessment. And online learning.

I'm also increasingly nervous about presenting side by side with a technology CEO and lead learning designer for AI and machine learning. I am a faculty member who created an authentic assessment project with Smart Sparrow learning designers, and I've delivered the project to over 200 students, so we will definitely be in our own lanes for this panel... You can see the project I worked on here- select "Explore Now" in the orange box to access the demo for free. It's an entire curriculum based on a single project: a Global Challenges Magazine where students research, write, and publish articles on a series of global challenges. It was the culmination of the work of over a dozen people through AASCU and Smart Sparrow and if you can't tell, I'm very proud of the outcome.

Here are some thoughts, okay really the start of a rough draft of a paper, from this panel:

What distinguishes authentic from, well, artificial or meaningless assessment, you might ask? Lots of people have weighed in on this question and here is what they say:

• A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills -- Jon Mueller

• Students complete tasks and/or a project that mirrors or actually is a real-life activity, they apply content throughout that process and in the final deliverable, and students are in charge of the process to some extent.

• The process of working on a project that mirrors a real-life activity requires higher order thinking with multiple opportunities for students to apply knowledge. It can also be iterative with peer or instructor feedback along the way.

• The primary purpose is to create direct evidence of learning through a set of tasks or project. The deliverable is evaluated by the instructor.

What is a Real-World Activity? It Depends...

This is not one of those things where 'you know it if you see it'. There are very clear criteria for authentic assessment, and perhaps the most important is this one- does it mirror or is it actually a real-world activity? You must answer yes, regardless of its design or its setting, or it is not authentic assessment. The same task can be a form of traditional assessment in one field, and authentic assessment in another. Disciplinary knowledge is critical to identifying the types of tasks and projects that are authentic, or mirror what is expected of professionals in their field. The this end, there is no magic formula for identifying exactly what kind of tasks or projects are more authentic. The trick is that is has to be real-world in the field most relevant to your course and that you convince students that it mirrors or actually is a real-world activity. Learning design can help you achieve both of these conditions, but the instructor must not forget throughout the project to maintain a high level of student trust that what they are doing really is a mirror of the real-world. Examples of projects that are authentic to real-world tasks can include websites, journals, magazines, and much more depending on the field or discipline.

Real-World Activities Enabled by Technology

Can authentic assessment be done in an online setting, you ask? Sure! But the same conditions apply. Learning designers must keep the conditions required for this type of assessment in mind and design activities that mirror the real-world in any given discipline. And really, whether an instructor uses technology to create, administer, evaluate, and disseminate authentic assessment is less important than whether it meets established criteria. What technology can do is make authentic assessment more powerful.

There are multiple benefits of leveraging technology in authentic assessment. First, technology can provide much-needed legitimacy to projects, as students become increasingly aware of how technology is used in the workplace. Why shouldn't authentic assessment follow industry practices of virtual collaboration, deliverables that live in the digital world, and other applications of technology?

Technology-enabled authentic assessment often pushes students outside their comfort zones and asks them to use technology in a much different way than they are used to, a professional setting. This is important, as it can help higher education better meet the demand in the workplace for students to "Stay(ing) current on changing technologies and their applications to the workplace" and "apply knowledge and skills to real-world settings" [1]. Applications of technology can range from leveraging AI in an online setting, the use of technology as a project management tool, or in the creation of the activity itself. It is a misconception to think that 'digital natives' can automatically translate their skills into a professional setting. Instructors are keenly aware that while students are used to technology, they are not skilled in using it in professional settings.

A Digital Magazine as Authentic Assessment

An example of technology-enabled authentic assessment is a digital media site built in the Global Challenges experience developed by AASCU and Smart Sparrow (click here for more information and select "Explore Now" in the orange box to access the demo for free). The entire course is organized around a single project: a digital Global Challenges Magazine. It was intentionally designed and built by Smart Sparrow learning designers and authors to look like a genuine digital media site, giving students and instructors have the impression throughout their entire experience that they are researching, writing, and publishing articles in a real magazine with a public audience. The magazine also includes a comment feature, a commonly seen component of real-world digital media sites, where students are reading and commenting on each other's articles for each issue. Finally, the design of the learning space was intentional, making the Global Challenges Magazine a central part of that space. Students toggle back and forth between the magazine and their "field desk", where they complete a series of lessons that prepares them to publish in the magazine. Thus, the project takes front and center throughout the entire student experience.

Students are also asked up front to create a profile for their magazine articles. This happens, in fact, in the first five screens that students see when they start the online experience.

Evaluation of this project is a mix of automatic grading built to align with ordered thinking, or Bloom's taxonomy, with manual rubrics for the project itself (i.e., students' articles and comments). The field desk, or the lessons that comprise the primary content for the course, was built to be graded automatically with a hierarchical scoring system that is tied to Bloom's taxonomy. For example, if students are asked on a screen to recall something from a previous screen, they are working at the lower order 'remember' level and are awarded the lowest possible points for that activity or interaction. Screens with interactions that require students to use higher order thinking like application or analysis are more complex and thus have higher possible scores.

Against the e-Portfolio, Most of the Time

I am adding this paragraph if only to address what I see happening on my own campus. Digital portfolios seem to have infiltrated higher education as an elixir for authentic assessment (or something else? I'm sure there are other reasons, too). However, a portfolio can only be a form of authentic assessment if it mirrors a real-world activity! That might be the case for specific industries and fields, many of them creative, but definitely not in others. To be effective, any form of authentic assessment must be intentionally designed to mirror a real world activity. A big component of this is trust between students and the instructor; students will see right through superficial attempts to create authentic assessment out of tasks and projects that do not mirror real-life expectations. If that trust is eroded, then the task or series of tasks becomes more like traditional assessment. This applies to all forms of tasks, though portfolios seem to have gained in popularity as an easy choice for instructors. But portfolios are not a guarantee for authentic assessment, and I argue that in most cases fail to meet its requirements. A project must meet the requirement that it it mirrors or actually is a real-world activity. Only then can it achieve the outcomes of authentic assessment.

Additional Resources:

-Authentic Assessment toolbox by Jon Mueller