ARTICLES

 

Curating Provocative Engagements with Assessment in Education: A Mysterious Thing

 

 

Vonzell Agosto^I; Jennifer R. Wolgemuth^II; Stephanie Green^III; Aimee Frier^IV; Sujay Sabnis^V; Michael W. Riley^VI; Jeanine Romano^VII; Jessica Kearbey^VIII

^IUniversity of South Florida, vagosto@usf.edu
^IIUniversity of South Florida, jrwolgemuth@usf.edu
^IIIUniversity of South Florida, smgreen3@mail.usf.edu
^IVUniversity of South Florida, adfrier@mail.usf.edu
^VUniversity of South Florida, svsabnis@mail.usf.edu
^VIPasco District Schools, mwriley@pasco.k12.fl.us
^VIIAACSB International, jeanine.romano@aacsb.edu
^VIIIUniversity of South Florida, jcclark2@mail.usf.edu

 

 

---

ABSTRACT

This study intervenes in commonsensical ways of exploring and understanding educational assessment within an audit culture of measurement. Drawing on sociomateriality and methodological approaches associated with material culture and narrative inquiry, we curated an exhibition based on interviews with 13 informants and the things they brought to convey their assessment experiences. Based on analysis of their narratives, we clustered our findings and organised them into a gallery of two thing-centred installations: "Assessment and Tools," and "Assessment and Arts/Crafts." Curating the gallery led us to a creative way of articulating concerns about excessive assessment into a thing-interview protocol to be used in future inquiry, involving interviews with people and things. We showcase these installations, along with interview prompts, as an online exhibition. The aim is to continue the conversation on the future of assessment in connection to purposeful, equity-oriented education.

Keywords: assessment, narrative inquiry, things, interviews, sociomaterial culture, curation

---

 

 

Introduction

According to Popham (2009), assessment consists "of a wide variety of evidence-eliciting techniques" (p. 5) whose outcomes guide the development and improvement of instructional practice and programmes. In the service of accountability, however, assessments are "measurement devices" used by governmental entities "to ascertain the effectiveness of educational endeavors" (p. 6). Despite Popham's (2009) clarity, the meanings, outcomes, and practices of assessment vary widely. Thus, we approached this study of assessment as a mysterious thing, and wondered what others' things would reveal about it. Our aim was to intervene in commonsensical understandings of the phenomenon of assessment in education while exploring the question: "What does assessment mean, and how does thinking about it through other things help us to make sense of it today?" We took up Turkle's (2007) questions about objects, which we substituted with things, to ask: "What is the power of objects [things] to move us, to forge important new ideas, or to link us to other people?" (p. 50). In other words, what do things such as a watercolour painting of a horizon, a clip from the movie, Ferris Bueller's Day Off, a Rubik's Cube, and a notebook help to communicate about assessment as some thing that informants understand and experience?

As an arts-based project, our study was largely a narrative inquiry (e.g., Kim, 2015), in that we clustered material things and stories about things into a gallery, as curators would do in a museum. The approach was one of theoretical and methodological plurality-a bricolage that explored assessment as one of the "complex political dimensions of knowledge work" (Rogers, 2012, p. 14). Our exploration of assessment reflects Lightfoot and Gustafson's (2009) call for researchers to use arts-based methods to create counterstories-transformative fictions.

 

Relevant Literature

According to Webster (2015), education in the United States (US) has come very close to the dangers that Biesta (2009) warned about: valuing what is or can be measured rather than measuring what is valued. Biesta (2009) also raised fundamental questions-"What is education for?" and "What is good education?"-to which his unequivocal answer was fairness and social justice. These warnings, enduring questions, and values provide an ethical framing for this review of relevant literature. It begins with the influence of neoliberalism and globalisation in education reform, then points to the need for critical thinking about assessment practices and purposes, and concludes with ethical concerns about the role of assessment in creating some types of people.

Neoliberalism, in its most basic form, is an ideology based on the idea that markets (more than governments) are best suited to allocate and distribute resources (Mustakova-Possardt, Basseches, Oxenberg, & Hansen, 2014). Thus, the guiding force of the market derives from the values of competition, measurement, and profit over the values of equity and justice. An argument made among proponents of neoliberalism is that it is not an ideology, "but rather an objective recognition of 'the way things work best''' to distribute goods and services (Mustakova-Possard et al., 2014, p. 30). Interests in privatisation, measurement, and accountability have now extended into education at all levels, ensuring education is fully reformed to conform to global demands and competition for goods and services provided by skilled workers.

In the US, federal funding provided by Title 1 of the Elementary and Secondary Education (ESEA) Act (1965) supported efforts to increase achievement among students who were at an economic disadvantage. Because Title I funding was bolstered in the 1980s, it was increasingly attached to performance expectations such as annual yearly progress, and linked to specific types of data reported at specific times (Fritzberg, 2004). The reauthorisation of the ESEA Act in January of 2002, referred to as No Child Left Behind (NCLB), intensified the standards and high-stakes testing movement (Fritzberg, 2004). According to Sahlberg (2011), the use of test scores, as the sole source of data, to "judge the effectiveness of individual teachers or the quality of schools is inappropriate" (p. 183). He further argued, having drawn on the work of Hargreaves and colleagues (2001, 2009), that the adoption of test-based accountability policies for schools is one of the major trends in the Global Education Reform Movement (GERM), which has been spreading since the 1980s. The most recent reauthorisation, Every Student Succeeds Act (ESSA, 2015), states more flexibility concerning assessment.

Concerns about standardised assessment being co-opted under policy regimes of evidence, and affecting many, if not all, facets of education have been validated. Studies depict how increased dependence on standardised assessment has led to the narrowing and homogenisation of educational curricula (Au, 2007, 2009, 2011; Goertz & Duffy, 2003; McNeil, 2000), dehumanisation and regulation of students (Stobart, 2008; Yazzie-Mintz & McCormick, 2012), disempowerment of teachers (Jones, Jones, & Hargrove, 2003), and the enabling of unprecedented moral-political control over (commoditised) schools by conferring or denying funds (Hursh, 2008; Johnson, Johnson, Farenga, & Ness, 2008; McKnight, 2013). While such consequences may be unintended (Carusi, 2013; Jones, Jones, & Hargrove, 2003), they nevertheless result from an overvaluation of high-stakes assessment activities, and exacerbate disparities in educational outcomes (Sternberg, 2010).

Education is interwoven with politics, economics, and cultural discourses that have narrowed education discourse around the world, and often without critical interrogation of the ways in which assessment is understood and applied in high accountability contexts (Cannella & Viruru, 2004). The absence of critical interrogation is ensured by another trend of GERM, the overemphasis on cognition related to learning, which contributes to learnification: "the transformation of everything there is to say about education in terms of learning and learners" (Biesta, 2009, p. 5). At a time when access to information has increased to allow educators and students to come under a deluge of data, the teaching of critical thinking skills needed to guide selecting, interpreting, digesting, evaluating, learning, and applying data information has decreased (Halpern, 1998). This is particularly worrisome now that "post-truth politics are more subtly seen in the manipulation of facts to suit political agendas" (Gibson, 2018, p. 25).

Assessment, and standardised testing, in particular, is pervasive in the US. It is found to linger in people's social memories of schooling and exert material influence over how they perceive themselves and recollect their school experiences (Green et al., 2015). Yet, systematic assessment practices tend to work against critical thinking (writ large) and ethical deliberation, even in democratic societies. In this study, we sought to understand how informants serving in various roles involving education assessment made sense of assessment (as a thing) through narratives and material things. We entered the conversation from an exploratory lens to consider the future of assessment and, ultimately, rectify the absence of explicit attention to questions about what is educationally desirable (Biesta, 2009).

 

Modes of Thinking and Thinging

Although we started this study from the view that assessment in education today is some thing, we took up the interrogative form of the phrase by asking: "What kind of thing is assessment?" Historically, "thing" denoted a place where people assembled to deliberate on matters of concern (Latour, 2004). We meshed the etymological history of the term with its contemporary usage in the colloquialism "It's a thing," which is a "phrase used to describe a phenomenon, often of some modern cultural significance" (haygirlhayy, 2012).

Generally, culture is a lens that determines how the world is seen, and provides a blueprint that determines how the world will be further fashioned through human effort. As a lens and blueprint, culture supplies meaning in the form of principles and categories (i.e., space, time, nature, person). Meaning is then substantiated and concretised through performances and material objects (McCracken, 1986). Material culture is built up from patterns and associations, or sets of practices within individual experiences, and carries symbolic meaning that can be both evocative and representational (Hodder, 2003). The scholarship of material culture involves the study of the beliefs, values, ideas, attitudes, and assumptions of a particular community or society at a given time through artefacts or objects (Prown, 1982). We use the term thing(s) to avoid the ethnographic baggage associated with terms such as object, materials, and artefacts (Henare, Holbraad, & Wastell, 2006). Relating through things is a driving perspective in this study, one that follows in the vein of Henare and colleagues (2006) who were interested in the blurred boundaries between subject and object-ideal and material.

Assessment has been defined as the act of making a judgment, an idea or opinion about something (assessment, n.d.). We began from the premise that assessment is something and, in being a thing, it operates on a material level in relation to other things of education-bodies, classrooms, technologies, and so forth. As such, any critique of the ethics of assessment should also take into account the ethics of assessment-things and how they structure relations shaping selves, others, and education.

Lightfoot and Gustafson (2009) encouraged the use of arts-based approaches in education and educational research to challenge assessment language used extensively in the social sciences, namely, as part of labelling and diagnosing students. These practices, they argued, tend to flatten out the ways in which students are understood, and narrow the practices researchers and educators select to understand them better. To counter such tendencies, Lightfoot and Gustafson (2009) made use of arts, primarily fiction writing, to expand narratives about students by taking into account the various metaphors students and educators use to counter totalising truths already circulating as if value-neutral. We took their call for multiple perspectives seriously, and sought perspectives from the point of view of informants in various roles and relationships with assessment. Alongside their descriptions of assessment using various objects, we included our own points of view as researchers whose methodological work was not only to conduct interviews but also to design methodological avenues through which to critique and provoke illuminating perspectives on assessment. In other words, we chose arts-based research to subvert the problematic order of assessment-things.

 

Methodology as the Construction of a Gallery Exhibition

While we situate this study in global concerns about the neoliberalism and its influence on education reform that relies heavily on standardised assessments, we acknowledge that we (educators and researchers in academia) are implicated in shaping how assessment is used and abused (Janesick, 2010)-whether in kindergarten or higher education-as actors who work within and shape the broader audit culture (Apple, 2007) in this age of measurement (Biesta, 2009, 2010). Elsewhere, we have described ourselves ideologically and ethically as follows:

We are a group of educationalists in Florida who worry about the current role and politics of assessment in the schools. . . . Focus on assessment narrows the curriculum . . . aspects of teaching that keep students connected and engaged in classrooms are being cast aside in favor of mechanistic delivery . . . production of technically savvy assessment specialists whose statistical skill is used to develop increasingly sophisticated value-added models to evaluate teacher quality. At the same time, we recognize that assessment motivates, provides valuable academic information, and aligns with pragmatic material economic and social realities in Western society. (Green et al., 2015, p. 1111)

Alignment is a condition and a value and, thus, alignment between assessment and various social realities in Western society is at times (for the) better and at other times (for the) worse, given the corrupting effects of high-stakes testing (Nichols & Berliner, 2005).

Interviews of Persons with Things

We recruited 13 informants from our professional and social circles: professors of educational assessment and measurement, elementary or secondary education teachers, special education teachers, school psychologists, school or district administrators, parents, and students. They ranged in age from approximately 16 to 70 years. The semi-structured interviews with them followed a general line of inquiry, lasted between 30 to 90 minutes and the researchers/authors worked in pairs to conduct each interview. Interviews were face-to-face except for one, which was virtual.

Our interview process was guided by what Roulston (2010) referred to as a postmodern orientation. That is, we viewed the interviews as purposeful entanglements (Marn & Wolgemuth, 2017), socially-materially situated, and ethically driven encounters between interviewers and informants. Enlisting an object (type of thing) interview approach, we asked informants to bring something that reflected their perspectives or experiences with assessment, which they either brought to or described by name in the interview. Rather than interviewing the things or bringing them into a central position, as is often done with object-interviews (Nordstrom, 2013), we typically asked about the thing once according to the protocol. We were mostly interested in how informants incorporated the things they brought into their narratives once we asked them to reflect on their first experiences with assessment, and whether assessment today is what they would have predicted. Finally, we asked informants to envision, both positively and negatively, what assessment might be in the future.

The transcribed interviews of the 13 informants produced approximately 5-9 single-spaced pages each, for a total of about 70 pages of text. We transcribed and analysed the interviews from the perspective that assessment in education can be understood through the material culture one considers and uses to construct and communicate meaning associated with it. These data served as the basis for the classification scheme we used to curate our gallery exhibition (Henare et al., 2006).

To analyse the data, two of the authors (doctoral student, faculty) reviewed transcripts separately to interpret them for narratives constructed around the term assessment. Two to four interviews were analysed and highlighted at a time. The two authors convened virtually to compare interpretations and document shared observations and analyses. They identified two broad narratives relating assessment to 1) tools and 2) arts and crafts, and proceeded to ask about the activities associated with each object, context for assessment, and experiences providing the narrative.

The next layer of analysis resulted in further organisation of the narratives into categories including assessment as a tool or assessment involving the use of tools (personal accounts that often included children, parents, and other family members and the emotions of being assessed) and arts and crafts (invoking the creative energy involved in teaching and learning). Most objects and stories were linked to standardised assessment practices. We saved the tables with the highlighted areas of text electronically, while other members of the team began identifying historical information about each of the things to situate them as part of the broader cultural narrative that informants provided about assessment in education.

We (several authors) added information about the things to the table of narrative data to support further analysis and questioning in consideration their purpose and associated practices (i.e., accounting, playing, organising data). Authors One and Two continued to analyse while writing, referring to literature, and presenting the pilot gallery in an informal works-in-progress session. The session, hosted by the Qualitative Research Advisory Group on the University of South Florida's campus, allowed students and faculty to respond, critique, and provide recommendations on work in progress. The first two authors created a mini-gallery experience in a small room using QR codes that guests accessed, using their cellular phones to listen to samples of voiced-over narrative excerpts. Feedback encouraged us to seek an Internet-based application. The final level of analysis occurred through the final drafting process with support from most of the authors, and is indicated by how we organised the findings and the subheadings we created, such as "A Multipurpose 'Out of Hand' Tool."

Curating an Exhibition: Analytical Process and Product

We constructed a gallery of transformative fictions, which includes 13 assessment-things whose modern versions originated as early as 600 BCE and as recently as 1986. Fleming (2010) argued that objects (things) can be a central part of an intellectual process for they can feed research with richness, telling us something new and allowing us to tell something new each time we interact with them. This gallery highlights the ways in which assessment in education becomes sediment(ed) in people's daily lives, routines, recollections, and perceptions of schooling and makes up the meanings, metaphors, and material objects informants used to relate (to) it. This gallery is a reminder to question the purpose of assessment, linked to what it is understood to be, and to consider its effects with regard to social justice. We were inspired, by the research process as well as interactive museums and travelling exhibitions, to curate an assessment exhibition from the gallery.

The structure of the exhibition, which is akin to those in museums or art galleries, reflects our theoretical approach to material culture for it provides people a gathering place to think through a collection of things. Traditionally, galleries and museums have served as repositories of material culture in the form of cultural artefacts (objects and documents), colonising enterprises, and educational resources (Place, Zangrando, Lea, & Lovell, 1974). Similar to museums of historically marginalised groups that communicate a deep social understanding of the role museums serve (like the recently opened U.S. Museum of African American History and Culture in Washington DC) we envision a gallery that spurs dialogues in which assessment is not merely taken as a subject of public deliberation, but is personalised, rendered palpable, and revealed as entangled in social and material experiences of schooling. We offer this potentially mobile exhibition as a type of gathering place for critical dialogue on the purposes, uses, and outcomes of educational assessment in various contexts, and for reconsideration of what assessment has meant, means, and could mean to education systems guided by the purpose of increasing fairness and social justice. We hope our assessment exhibition does more than just display things.

Our exhibition, Assessment in Education: A Mysterious Thing, is an example of how the gallery could be organised to include different collections in different contexts, for different purposes. Others could curate from the 13 collections when provoked differently-and to provoke differently. While all informants are presented herein and housed (linked) in the online gallery below (Table 1), we showcase six collections that we curated from among the installations (tools, arts/crafts). Our decisions to include these six were influenced by technical (maximum length of this article), intellectual and ethical (inclusiveness), and aesthetic (tension, richness, variety) criteria we opted to meet, and the creative energies we exerted in the process of curating (creating) as researching. The collections curated represent a diverse group of informants from among those interviewed, and which inspired us to think in different ways and use different concepts to illuminate different concerns. In Table 1, under the column, Index of Interpreted Things, we indicate with an asterisk each of the six chosen installations.

This sample of prompts, images of things, and excerpts from the transcripts narrated by a voice actress are curated in the gallery of transformative fictions at:

https://www.thinglink.com/channel/1078743100874031105

 

Gallery of Transformative Fictions

Drawn from informants' interviews, this print version of the gallery features descriptions of various objects, and is curated into two themed installations according to how they metaphorically related to assessment: "Assessment and Tools" and "Assessment and Arts/Crafts." Under "Assessment and Tools," the narratives focused on the tools used to conduct assessment, to prepare one to be assessed, and to reflect on (and further assess) the findings or outcomes of assessment. Under "Assessment and Art/Crafts," the narratives focused on what assessment should not be, what assessment should be, and what assessment is (at risk of) becoming.

Installation 1: Assessment and Tools

The things in this installation help to articulate assessment as something that involves the use of tools. The tools help in conducting, preventing, organising, preparing, reflecting on, and capturing the practices, outcomes, and understandings associated with assessment in education.

Tools tell one side in a tale of tensions.

Professor 1 brought a #2 pencil and Scantron bubble sheet. We centre the bubble sheet as a tool designed to support assessment, and the #2 pencil as the tool required to complete it. The Scantron Corporation's website provides a historical narrative about the creation of the Scantron sheet, which begins:

Scantron was founded in Los Angeles in 1972 by William E. Sanders, its first CEO, and Michael Sokolski, Executive Vice President of Engineering, who invented the Scantron Optical Mark Recognition (OMR) scanners and forms. These products have since become classroom staples around the world as students use their #2 pencils to fill in their "bubble sheets". (Scantron, n.d., para. 1)

Rather than bring something that represented what assessment meant to him, this professor brought things he thought the general public associates with assessment. He stated:

This is what people (who you run into out on the street) [think] when they think of educational assessment. They think of their number two pencil and their bubble sheet, and sitting down and taking, whether it's the FXXX or whether it's their SXX or whether it's, you know, their chemistry test or whatever. . . . So I think that's a common image people have of assessment.

He preferred to keep his definition of assessment broad and unrepresented by a single object. His definition of assessment was, "a way to gather information about people." He described how, when a student, standardised tests were given on occasion. This increased when he was a teacher during the 1980s. He contrasted the conditions under which he taught, to conditions today:

And my sense, now, is that it's just a lot more of that, to where teachers feel much more tension about what they think may be the best for a particular kid, or what they may feel is best for the group of kids, and what they feel they have to do in order to be ready for an outside assessment, which there's a lot more of.

He also shared a story about when his students communicated their appreciation of testing as a time where little else was required of them apart from taking the standardised test. After noting that a new student did not share their enthusiasm, Professor 1 asked the students, "Why did the other kid not understand that this is a good week?" Their response was, "He's all stressed out. He thinks these tests are like really important, and that you have to do really good on them." Professor 1 recalled what he realised through that conversation:

Here can be a test, which the test developer probably has the same purpose for the test, and it probably ties back to getting some information about where the kids are at, so people have a better sense, but, the actual activity is being processed very differently by different individuals at different points in time.

Professor 1 assumed most people would associate a Scantron sheet and #2 pencil with assessment because those were the main tools used to assess students on a large scale when he was in school, and assessment could be simply explained as a way to acquire information. His nostalgic narrative featured assessment-related tensions among teachers and students, and brought tension to the installation through the divergent experiences of students yesterday and today, and among those in the same classroom.

A multipurpose "out of hand" tool.

The professor of education measurement (Professor 2) brought a multipurpose utility knife to the interview-the iconic Swiss Army Knife by Victorinox:

Karl Elsener took the initiative in 1891 to form the Swiss Cutlery Guild with the main aim of producing the soldiers' knives, which the Swiss Army had, up to that time, purchased from Solingen in Germany. In October 1891, the first delivery to the Swiss Army was made. (Victorinox, n.d., para. 1)

Like this tool, according to Professor 2, assessments are often used for multiple purposes but should, rather, be created and used for a specific purpose only:

What people try to do is . . . use those data for all sorts of purposes. So it's like the assessment form then becomes the Swiss Army knife. . . . I think that it's important that our assessments have certain purposes and we should create them for that purpose in mind and use them for that purpose and don't think it's going to be useful for other purposes.

His emphasis on clarity of purpose echoes Webster (2015) on the need to recentre questions about the purposes of education and measurement. Additionally, Professor 2 suggested the use of standardised test scores has gone awry and, rather than contributing to social change, they are used to focus on simpler problems that are easier to administer than important social issues, like poverty.

If you think about the broader problems of poverty we have, all the social problems that come along with it, that it may look like a simpler solution if we attach teachers to their students' achievement, how can we do that? We do it through standardised assessments in a couple key areas that we can use as leverage to reward the good teachers and get rid of the poor teachers. So there is a hint of logic in it, it is not totally illogical but it is an isolated part of a much bigger problem and so . . . I would not have predicted that it would get this out of hand.

Even as a multipurpose Swiss Army knife, assessment is not a tool suited for mediating or correcting poverty. The professor's comments about education as a mechanism of social change and the complexity of poverty recentred the questions of the purposes of assessment and education, reconnecting both to the ethical question of whether those purposes are guided by a sense of what is socially (including economically) just.

A tool to control change confronted by menopause and motorcycle maintenance.

The literacy professor, Professor 3, referred to three things: an informal reading inventory, a major life change (sparked by a book), and constructivist strategies. We highlight the major life change, which contrasts her narrative about assessment as a form of control. She first described the informal reading inventory as a powerful tool.

Most of the time you [do] grouping in reading according to readability groups. Which, of course, is tracking. Which, you aren't supposed to do. But we do for the 'good of the kid'. So, this powerful tool has been used in ways that I wouldn't recommend. That's the problem I see. So, this accuracy and control manoeuvre with informal reading inventories is to button down a kid so you can give them controlled curriculum so you know what to expect from them. That's insecurity on the part of the teacher masquerading under accuracy. It's stupid. It's dumbing down teaching. Where is the risk-taking? Where is the discovery? Where is the learning? The teacher can't learn anything in that case. So why would you bother with a profession that can't teach you anything?

Her concerns echo others on the effects of assessment such as tracking, curricular control (Au, 2007), and declining teacher morale (Nichols & Berliner, 2005). After years of teaching using reading miscue inventories (which are identical to informal reading inventories), Professor 3 recalled how a second thing sparked a life change-her "teaching menopause." The thing was the book Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values by Robert Pirsig, described as follows by Ratner-Rosenhagen (2018):

At its most basic, it is a first-person narrative of a father who takes a motorcycle trip with his 11-year-old son and two friends, all the while journeying intellectually and existentially into his past, the philosophical questions that both enliven and torment him, and his fraught relationships with his boy, his friends, and the world. (para. 2)

Professor 3 stated:

It's a really good book. It's about how a teacher thinks and how thinking can drive you crazy. But, one of the things in Motorcycle Maintenance is something called the seed crystal. When you have a supersaturated solution in chemistry and you add one more bit of the stuff you are trying to get to go in the solution, you heat it up, and it has all the salt it can dissolve, you add one more crystal and it (the seed crystal) fractures. And so my whole world fractured when I learned this. I didn't know what to do. And all this, oh my god, we are doing all this stuff to kids, we are abusing children in the name of literacy. And for a few years, I hated what I was doing. . . . I think that the confounding part for me was the unhealthy influence of authority and curriculum telling me what I should do versus learning what I should really do with kids in terms of literacy.

Her reference to Pirsig's 1974 book is relevant to the conversation about the purpose of education in this so-called "post-truth era" given that the debates he noticed-which is most worthy of attention, truth or goodness and quantity or quality (static or dynamic)-continue. For us, it raised the question: How might those designing and conducting education come to know their purpose for doing what they do?

Installation 2: Assessment and Arts/Crafts

The things in our second installation offer multiple cleared and blurred looks at assessment, not simply to see one's perspective alone but to disrupt commonsensical understandings of assessment that neglect it as a thing of culture, politics, and policies compressed into sociomaterial assemblages.

A landscape and journey marked by celebration, what assessment should be.

The retiring district administrator described what he called a simple watercolour painting of a plains landscape. The watercolour held special significance for him because his father painted it. He explained the physical properties of the painting-moving from the foreground with lots of detail, to the background with less detail, and from warm tones to cool tones-is what represents assessment.

He described the painting as having three stages: foreground, horizon, and sky. He used each stage metaphorically to represent types of assessment (formative, level benchmarks, summative), and the colours to represent the affective qualities of engagement with assessment and the learning it indicates. In his mapping of assessment onto the painting, he described types of assessment relative to the distance between the student and an assessment and its point of origination.

In the foreground, which had more detail to it [it had the weeds], you could see it has more detail, that was your formative assessment (which is teacher-created). This type of assessment (benchmarking) is represented in the painting by the middle-ground area where the plains end and the horizon begins.

He went on to describe summative assessment as the third stage and section of the painting where the horizon leads into the sky stating, "Especially this area right here where he added a little more of the red . . . a beautiful area that was kind of a celebration."

As was done by Professor 2, this informant delineated forms of assessment. Additionally, he used the painting to focus on students' point of view and their distance from the point of the assessments' origination. For this informant, each phase in the assessment journey should be something that brings celebration rather than fear. When asked if he had predicted assessment to become what it is now he said, "regretfully, no" and stated:

It was only in Title I schools . . . and I hate to admit it but the force came from the federal government when we started to report Title I achievement to maintain our funding and those kinds of things. So I definitely would have [focused on having students map their learning]. . . Now I think it's gone too far. I think it's gone too far.

He attributed his failure to predict (about 25 years earlier) that assessment would have gone too far to his naiveté as an emerging school administrator.

Assessment as synecdoche and mereology, what assessment is and should not be.

The high school history teacher invoked a video clip from the movie, Ferris Bueller's Day Off (Chinich & Hughes, 1986). The clip is often cited in educational literature as a metaphor for classroom boredom. The following description by Dahlgren (2013) captures the cinematic effects and language used to portray the classroom scene between the teacher and students:

In an iconic scene from Ferris Bueller's Day Off (1986), an economics teacher played by Ben Stein drones on about the Smoot-Hawley Tariff Act. . . . In the scene, the teacher is shown engaging rather desperately in an attempt to engender a modicum of discussion in his classroom, imploring in a monotone: "In the 1930s.. . . Anyone? Anyone?" (pp. 8-9)

The informant picked this clip to represent his views on assessment as a teacher who proctors during testing-and finds it to be boring. From his perspective, assessment is like roll call-boring for the teacher and the students, but still a necessity. This teacher suggested creative ways of teaching and assessing, for instance by using movie clips as a way to improve student engagement. While this teacher thought creativity was needed to help low performing students, creativity in education has been a hallmark of gifted and talented programmes.

We were inspired by his reference to creativity and the thing he provided (clip), which we interpreted metaphorically as a semiotic metonym-"Ferris Bueller's Day Off" standing in for the idea of escaping schooling-and, philosophically, as an ontological (mereological) relationship (in the clip of the teacher and students) between boredom in the moment and students' disengagement from education over time. This helped us to think about assessment as relationships between parts and wholes, similar to Lightfoot and Gustafson's (2009) discussion of labelling (i.e., "at-risk"). It also inspired us to convert the clip (a graphic animation) into a looping Graphics Interchange Format (or GIF), and include it in the gallery as a reminder that high-stakes testing is a part of the whole of assessment in education.

Assessment is like a Rubik's Cube, but should not be.

According to the mother of a high school student, the Rubik's Cube is a metaphor for assessment (narrative object as metaphor) in the district in which her daughter was enrolled. We categorise the Rubik's Cube as an object of material culture and arts-based metaphor given its history of being conceptualised by a lecturer in interior design and its status as a puzzle that is more akin to creativity than the more technical or purposeful category, tool.

Erno Rubik, the creator of the Rubik's Cube, was once a lecturer in the Department of Interior Design at the Academy of Applied Arts and Crafts in Budapest (Anzelak, Frankl, & Ebner, 2006). There are six sides to the cube, with each side having a different colour. Each side is divided into nine smaller cubes that are layered into 3 x 3 (columns and rows). The aim of playing the Cube is to return it to its original state, which means restoring all six sides to their colour. The Cube is highly flexible and players have 43,252,003,274,489,586,000 possible moves (Anzelak et al., 2006).

When asked why the Rubik's Cube came to mind when thinking about assessment, the mother explicitly referred to assessment as a thing (along with "things like that"). She stated:

I think they have changed the system of assessment and testing so much that it's this thing they are trying to figure out, you know, they are just constantly trying to figure things out. And that is what you do with a Rubik's Cube. You twist it this way, with that way, what's the best way, and try to find this right pattern. So that they are all coloured, so they're all correct. And I think that is kind of what the assessment in the school system and things like that are doing, they just try all of the different ways, you know, to make this, you know-make a great student, I guess. Or make them just pass the tests, you know.

She noted the district had provided parents online support and booklets to help them understand assessment procedures, namely, standardised testing. Nevertheless, she characterised assessment as puzzling-something for educators, students, and parents to figure out through trial and error.

She further described how assessment is being conducted "on a daily basis" and "in every which way" to shape students' behaviour or alter their level of confidence. She stated:

Well, kids come in the classroom, and it's an array of people, kids, colours, different cultures, and things like that. So the teacher has to kind-of group them and put them in order, so to speak, so that it flows and they can learn and it is orderly. So with the Rubik's- again, there are lots of colours and all different patterns and lots of ways. So once you figure it out then you can solve the puzzle. You know you figured out the kids and how they work together and you seat them accordingly and teach them accordingly. But you have to figure it out before they get there or when they're there in the classroom.

In this narrative, the mother used the Cube as a metaphor for the student (people, kids) and teaching as a grouping mechanism to increase flow and order as necessary conditions for learning. Like solving the Rubik's Cube, she described teaching as an ordering process that educators have to figure out.

There are endless algorithms and strategies for use in figuring out the Rubik's Cube. The mother's narrative seems to suggest that school districts and educators are, similarly, operating with the aim of figuring out what algorithms and strategies could be applied to education to ensure the same outcome. In other words, she prompted us to consider whether educational systems could be understood as unlocking a mystery. And, if so, are there also at least 43,252,003,274,489,586,000 possible moves that can be taken in combination to figure out how to do assessment better? The mother's narrative also alluded to how assessment processes work in the teaching environment to make "a great student" or make them "pass the tests." Both options imply marking achievement, and reducing the random distribution of students through grouping (like cubes) and their differences (identities), based on assessments of varying types. In other words, assessment treated like a Rubik's Cube works to create homogeneous collections (material things, people, ideas, relationships, narratives) rather than the heterogeneous collections (our installations here).

 

Discussion Prompts

The person-interview approach and questions above centre the human and human experience as more relevant than things. Humans are ultimately, and most often, the subjects (things) of educational assessment. However, like Biesta (2009, 2010), we are concerned that contemporary assessment practices neglect humans, operating under market values that are often at odds with humanist ethics-democracy, equality, and social justice.

This study and gallery findings left us wondering what shifts would come with actively decentring the informant and the visitor in favour of the objects (both metaphorical and literal) of assessment in the form of thing-interviews. What would the things informants brought share about assessment? What does asking the object to speak for itself reveal further about the dangers of standardised assessment? What would anthropomorphising the things, using the informants' narratives, help us think and feel? By exploring assessment as a thing (that could be thought about through things), we arrived at another application of the interview and perhaps a (re)new(ed) way to engage our gallery. Below are some thing-interview question-prompts:

• Scantron sheet and #2 pencil, how does it feel after all of those years of being a couple, to think you might not ever come into contact with one another again, and may even be at risk of extinction given the rise of the computer-based testing?

• Multipurpose knife, how does it feel to be valued solely on your use in completing tasks for others?

• Dear painting, how does it feel to hang on the wall and wait for your beauty and inspiration to be perceived, rather than be given the opportunity to explicate them on your own terms?

• Media clip, by definition you are a tiny part representing a whole. How do you stay motivated to be engaging, knowing that you best serve your purpose when you provide an image that is either static or looping-either the same or more of the same?

• Rubik's Cube, what is it like to have hands touching you all over, with the intent to unlock the mystery of your puzzle-to line up all your squares so that each of your facets looks the same?

• Book, Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values, what can you tell us more about unravelling mysteries through alchemy, using both analytical and mystical knowledge, and how such knowledge can be used to transform educators who are naive about who governs their work and why?

 

Concluding Remarks and Responses to Further Open the Inquiry

We wanted our exhibit to do more than just display things. Like curators, we worried about the grouping of things, our statements about them, opportunities we might create for our visitors to engage them, and the overall message, tone, object of our installation. Ultimately, we sought to provoke, to dispel commonsensical understandings of assessment as a thing. Like Zen and the Art of Motorcycle Maintenance, we sought to disrupt assessment and assessment practices, to bring to fore the existential and ethical questions: What is assessment? What should it be used for? We cannot answer these questions-through the literature or the informants' narratives and things they brought to us-but we can, or perhaps must, engage them.

We conclude this exhibition, then, not with answers about the meanings of assessment, and what assessment should do, but with potentially transformative questions that can be taken from the overall gallery into further reflections and encounters to bring sensitivity to the ways in which assessment sediments into things such as daily life, routines, recollections, and purposes of education.

Gallery of Transformative Fictions: Some Questions or Prompts

• What thing would you bring to talk about assessment?

• How does assessment make you who you are now? How do you understand yourself through assessment?

• When is the last time you were assessed? What was it like?

• How important is assessment?

• What is the purpose of assessment?

• What is the purpose of education?

• How did you come to determine these purposes of assessment and education?

• Does assessment need to be a thing?

• What democratic relationship building opportunities are constituted through assessment performance?

• Who is to profit and what is to be gained as a result of assessment, and when?

• What aesthetic values are practised in the process of assessing what one has learned?

• How does assessment enhance and diminish well-being?

• What are the risks associated with prioritising assessment practices, and how are those risks reduced?

• What assumptions about the significance of, and relationship between, values and truth undergird assessment policies and practices?

• What other questions can or should we ask about assessment?

This study adds perspectives from those with varying points of contact with assessment, past and present, to publicised concerns about assessment being misused, overused, or abused in schooling today.

The recommendations provided by informants, and the prompts we created with the anthropomorphised things, can be used to reconsider education practice and research related to assessment and, subsequently, curriculum and instruction as sociomaterial engagements that can exacerbate inequities (Au, 2009). Our exhibition is another way to bring assessment and assessment practices into a space of critique, reflection, ethical deliberation, and out from under the cloak of a positive and positivist narrative-that uniformity, control, and global reform will secure future wealth, prosperity, and well-being for nation-states and citizens.

 

References

Anzelak, M., Frankl, G., & Ebner, W. (2006). Building up knowledge like a "Rubik Cube." Electronic Journal of Knowledge Management, 2006, S. 10-18.         [ Links ]

Apple, M. W. (2007). Education, markets, and an audit culture. International Journal of Educational Policies, 1(1), 4-19.         [ Links ]

Assessment. (n.d.). In Merriam-Webster Online. Retrieved from http://www.merriam-webster.com/dictionary/assessment

Au, W. (2007). High-stakes testing and curriculum control: A qualitative meta-synthesis. Educational Researcher, 36(5), 258-267.         [ Links ]

Au, W. (2009). Unequal by design: High-stakes testing and the standardization of inequality. New York, USA: Routledge.         [ Links ]

Au, W. (2011). Teaching under the new Taylorism: High-stakes testing and the standardization of the 21st century curriculum. Journal of Curriculum Studies, 43(1), 25-45.         [ Links ]

Biesta, G. J. J. (2009). Good education in an age of measurement: On the need to reconnect with the question of purpose in education. Educational Assessment & Evaluation Accountability, 21(1), 346. doi: 10.1007/s11092-008-9064-9        [ Links ]

Biesta, G. J. J. (2010). Good education in an age of measurement: Ethics, politics, democracy. Boulder, USA: Paradigm.         [ Links ]

Cannella, G. S., & Viruru, R. (2004). Childhood and postcolonization: Power, education, and contemporary practice. New York, USA: Routledge.         [ Links ]

Carusi, F. T. (2013). Dyst(r)opia: A tropological argument for dystopia and education. In J. A. Heybach & E. C. Sheffield (Eds.), Dystopia & education: Insights into theory, praxis, and policy in an age of utopia-gone wrong (pp. 51-68). Charlotte, USA: Information Age.         [ Links ]

Castro, J. (2013). Who invented the mirror? Live Science. Retrieved from https://www.livescience.com/34466-who-invented-mirror.html

Chinich, M. (Producer), & Hughes, J. (Director). (1986). Ferris Bueller's Day Off [Motion picture]. USA: Paramount. Retrieved from https://www.imdb.com/title/tt0091042/

Dahlgren, R. L. (2013). The attack on social studies teachers and teaching in 1970s and 1980s Hollywood movies. The Councilor: A Journal of the Social Studies, 74(2), 1-15.         [ Links ]

Early Office Museum. (n.d.) Small antique files & filing devices. Retrieved from https://www.officemuseum.com/filingequipmentsmall.htm

ESSA (2015). Every Student Succeeds Act of 2015, Pub. L. No. 114-95.

Fleming, M. (2010). Thinking through objects. In S. Pickert, H. Trischler, & C. Sichau (Eds.), The exhibition as product and generator of scholarship (pp. 33-49). Berlin, Germany: Max Planck Institute for the History of Science. Retrieved from https://www.mpiwg-berlin.mpg.de/Preprints/P399.PDF        [ Links ]

Fletcher, D. (2009, December 11). Standardized testing. Time. Retrieved from http://content.time.com/time/nation/article/0,8599,1947019,00.html

Fritzberg, G. J. (2004). No child left behind? Assessing President Bush's assessment law. Educational Foundations, 18(3/4), 7-24.         [ Links ]

Gaelic Matters. (n.d.). The meaning of Celtic Knot symbols. Retrieved from http://www.gaelicmatters.com/celtic-knot-symbols.html

Gibson, M. L. (2018). Scaffolding critical questions: Learning to read the world in a middle- school civics class in Mexico. Journal of Adolescent & Adult Literacy, 62(1), 25-34.         [ Links ]

Goertz, M., & Duffy, (2003). Mapping the landscape of high-stakes testing and accountability programs. Theory into Practice, 42(1), 4-11.         [ Links ]

Green, S., Kearby, J., Wolgemuth, J. R., Agosto, V., Romano, J., Riley, M., & Frier, A. (2015). Past, present, future of assessment in schools: A thematic narrative analysis. The Qualitative Report, 20(7), 1111-1124. Retrieved from https://nsuworks.nova.edu/tqr/vol20/iss7/11        [ Links ]

Greenbaum, H., & Rubenstein, D. (2012, January 20). The hand-held highlighter. New York Times Magazine. Retrieved from https://www.nytimes.com/2012/01/22/magazine/the-hand-held-highlighter.html

Halpern, D. F. (1998). Teaching critical thinking for transfer across domains: Disposition, skills, structure training, and metacognitive monitoring. American Psychologist, 53(4), 449-455.         [ Links ]

Hargreaves, A., Earl, L., Moore, S., & Manning, S. (2001). Learning to change: Teaching beyond subjects and standards. San Francisco, USA: Jossey-Bass.         [ Links ]

Hargreaves, A., & Shirley, D. (2009). The fourth way: The inspiring future of educational change. Thousand Oaks, USA: Corwin.         [ Links ]

haygirlhayy. (2012). It's a thing. In Urban Dictionary. Retrieved from https://www.urbandictionary.com/define.php?term=it%27s%20a%20thing

Henare, A., Holbraad, M., & Wastell, S. (Eds.). (2006). Thinking through things: Theorising artefacts in ethnographic perspective. London, UK: Routledge.         [ Links ]

Hodder, I. (2003). The interpretation of documents and material culture. Collecting and interpreting qualitative materials, 2, 155-175.         [ Links ]

Hursh, D. (2008). High-stakes testing and the decline of teaching and learning: The real crisis in education. Lanham, USA: Rowman & Littlefield.         [ Links ]

Inventions Handbook. (n.d.) Australian inventions #3. Retrieved from http://www.inventions-handbook.com/australian-inventions.html

Janesick, V. (2010). What should we know about assessment practices in urban schools? In S. Steinberg (Ed.), 19 Questions, teaching in the city (2nd ed., pp. 69-86). New York, USA: Peter Lang.         [ Links ]

Johnson, D. D., Johnson, B., Farenga, S. J., & Ness, D. (2008). Stop high-stakes testing: An appeal to America's conscience. Lanham, USA: Rowman & Littlefield.         [ Links ]

Jones, M. G., Jones, B. D., & Hargrove, T. Y. (2003). The unintended consequences of high stakes testing. Lanham, USA: Rowman & Littlefield.         [ Links ]

Kim, J. H. (2015). Understanding narrative inquiry: The crafting and analysis of stories as research. Thousand Oaks, USA: SAGE.         [ Links ]

Latour, B. (2004). Why has critique run out of steam? From matters of fact to matters of concern, Critical Enquiry, 30(2), 225-248.         [ Links ]

Lightfoot, T., & Gustafson, R. (2009). Talking in multiple voices: Using qualitative research modalities to challenge the language of assessment, diagnostics, and labeling in schools. Journal of Curriculum and Pedagogy, 6(1), 118-138.         [ Links ]

Marn, T., & Wolgemuth, J. R. (2017). Purposeful entanglements: A new materialist analysis of transformative interviews. Qualitative Inquiry, 23(5), 365-374.         [ Links ]

McCracken, G. (1986). Culture and consumption: A theoretical account of the structure and movement of the cultural meaning of consumer goods. Journal of Consumer Research, 13, 71-84.         [ Links ]

McKnight, A. N. (2013). Tests, consumerism, and the cruel drone of White Noise. In J. A. Heybach & E. C. Sheffield (Eds.), Dystopia & education: Insights into theory, praxis, and policy in an age of utopia-gone wrong (pp. 69-80). Charlotte, USA: Information Age.         [ Links ]

McNeil, L. M. (2000). Contradictions of school reform: Educational costs of standardized testing. London, UK: Routledge.         [ Links ]

Mustakova-Possardt, E., Basseches, M., Oxenberg, J., & Hansen, I. (2014). Transforming a limited social function into a viable global action agenda. In E. Mustakova-Possardt, M., Lyubansky, M., Basseches, & J. Oxenberg (Eds.), Toward a socially responsible psychology for a global era (pp. 2146). New York, USA: Springer.         [ Links ]

Nichols, S. L., & Berliner, D. C. (2005). The inevitable corruption of indicators and educators through high-stakes testing. Education Policy Research Unit, Arizona State University. Retrieved from https://nepc.colorado.edu/sites/default/files/EPSL-0503-101-EPRU.pdf

Nordstrom, S. (2013). Object-interviews: Folding, unfolding, and refolding perceptions of objects. International Journal of Qualitative Methods, 12, 237-257.         [ Links ]

Place, L. F., Zangrando, J. S., Lea, J. W., & Lovell, J. (1974). The object as subject: The role of museums and material culture. Collections in American studies, American Quarterly, 26(3), 281-294.         [ Links ]

Popham, J. W. (2009). Assessment literacy for teachers: Faddish or fundamental? Theory Into Practice, 48(1), 4-11.         [ Links ]

Prown, J. D. (1982). Mind in matter: An introduction to material culture theory and method. Winterthur Portfolio, 17(1), 1-19.         [ Links ]

Ratner-Rosenhagen, J. (2018, July 15). Zen and the art of a higher education [Review of the book Zen and the art of motorcycle maintenance: An inquiry into values, by R. M. Pirsig]. Los Angeles Review of Books. Retrieved from https://lareviewofbooks.org/article/zen-and-the-art-of-a-higher-education/

Rogers, M. (2012). Contextualizing theories and practices of bricolage research. The Qualitative Report, 17 (T&L Art, 7), 1-17. Retrieved from http://www.nova.edu/ssss/QR/QR17/rogers.pdf        [ Links ]

Roulston, K. (2010). Reflective interviewing: A guide to theory and practice. Los Angeles, USA: SAGE.         [ Links ]

Sahlberg, P. (2011). The fourth way of Finland. Journal of Educational Change, 12(2), 173-185.         [ Links ]

Scantron. (n.d.). Our beginning. Retrieved from https://www.scantron.com/company/#our-beginning

Sternberg, R. J. (2010). Assessment of gifted students for identification purposes: New techniques for a new millennium. Learning and Individual Differences, 20(4), 327-336.         [ Links ]

Stobart, G. (2008). Testing times: The uses and abuses of assessment. New York, USA: Routledge.         [ Links ]

Turkle, S. (2007). The secret power of the things we hold dear. New Scientist, 194(2607), 50-52.         [ Links ]

Webster, R. S. (2015). Valuing and desiring purposes of education to transcend miseducative measurement practices. Educational Philosophy and Theory 49(4), 331-346. . doi:10.1080/00131857.2015.1052355        [ Links ]

Victorinox. (n.d.). Victorinox Swiss Army knives info. Retrieved from http://www.pizzini.at/infosak_engl.htm

Wallop, H. (2014, May 19). Rubik's cube invention: 40 years old and never meant to be a toy. The Telegraph. Retrieved from https://www.telegraph.co.uk/technology/google/10840482/Rubiks-cube-invention-40-years-old-and-never-meant-to-be-a-toy.html

Weinberg, F. (1993). The history of the stethoscope. Canadian Family Physician, 39, 2223-2224. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2379863/        [ Links ]

Yazzie-Mintz, E., & McCormick, K. (2012). Finding the humanity in the data: Understanding, measuring, and strengthening student engagement. In S. L. Christenson, A. L. Reschly, & C. A. Wylie (Eds.), Handbook of research on student engagement (pp. 743-761). New York, USA: Springer.         [ Links ]