Research Translation

Daring to Imagine a Large-Scale Approach to Visual Arts Assessment Through a Virtual World Environment   

By:  Pamela G. Taylor

         This article discusses the idea of creating art assessment methods imbedded in technology with quantitative results.  Taylor discusses how art student knowledge and ability cannot be tested using traditional testing methods while doing the students know how justice.  In response, a virtual world called eLASTIC was created using software design that enabled 200 high school art students to web map their thinking.  Everything from compare and contrast images, to writings, recordings, images, and sound could be embedded into the eLASTIC world for the student, teacher, and classmates to view and respond to.  

         The environment of eLASTIC on a computer screen reminds me of a Sims house.  There are a series of rooms and a hallway complete with furniture and tons of frames.  The frames act as a vessel for information (images, writings, videos, sound recordings, etc.) that can be inserted by either teacher or student.  Prompts from the teacher can be submitted into this world along with “ping” responses from the student. 

         Students can connect these frames with lines or "links" of their color choice.  Each line color has a different meaning such as; appropriation, critical disruption, art history, interdisciplinary, social studies, and social justice.  In addition, each link is attached to a student or teacher comment that explains the link between the frames.  The connections between these frames remain unseen unless the user clicks "show links" or the light switch of the room is turned off.  These actions cause the links to glow, creating a web of connections between the frames.  A mind-map of sorts.

         eLASTIC can be thought of as Pinterest on steroids.  Students and teachers have the ability to “pin” or “frame” any information their heart desires, thus creating a house of artifacts. But instead of stopping there they have the ability to infinitely attach links to these frames with connections they are making between sets of information.  They can travel through their thoughts following these web maps, edit their thinking, draw new conclusions, move links around, add new information, etc.  eLASTIC is a living and changing map of thinking with mounds of student learning evidence. 

         Having such a wealth of information in one location may seem like the perfect recipe for veering off track, however, each prompt given directly relates back to art standards and rubrics.  The big idea is for students to practice interdisciplinary learning.

         Another large topic discussed in this article is the idea of computer software that mimics human grading trends amongst artwork, writings, etc.  Harvard, for example, has developed Automated Essay Scoring or AES.  This “Robo-Reader” searches for keywords, phrases, and patterns within writings and compares them to other essays.  This technology has been created to duplicate the same grading patterns used by humans.  Taylor’s thinking is to expand these categories/outlines of requirements and apply this grading software to student artworks.  Computers can already detect and measure subtle changes of the human body, such as done with MRI scans and ultrasounds.  The argument made is the strong potential of computers being able to find and assess skills such as gradations and understanding of concepts within writings.

RESEARCH PROBLEM

The problem is clearly stated in this article.  The first page states "What are students know and are able to do cannot be revealed on the kinds of traditional standardized tests used to measure students' learning proficiency levels."  From this point on the importance of the research is not only discussed throughout the article but in conclusion Taylor ends with "I dare to imagine that if we are able to expand the languages we use and the ways we describe and illustrate what art students know and are able to do, people will listen."  It is clear that Taylor's intent is not solely to improve assessment for the student and teacher, but to bring validity to art assessments that will be respected by administrators and those above. Validity that cannot be disputed and must be respected by all levels.

I believe Taylor clearly defines new terms such as AES and eLASTIC.  However, I believe she could have dived deeper into defining the different sets of links (appropriation, critical disruption, art history, interdisciplinary, social studies, and social justice) she chose for students to use within eLASTIC.  How does she define these for her students, what big ideas fit within their umbrellas?

LITERATURE REVIEW

The cited sources in this article are relatively current ranging from the oldest source in 1994 to the most recent in 2012.  Now, you also have to be aware that this article was published in January of 2014, but the research was conducted between 2009-2012.  Do I believe there could have been more recent sources incorporated, yes.  Are the sources used relevant to the research, yes.  Half of the research conducted comes from a primary source, herself.  The other half of her thinking is derived from secondary sources such as the Robo-Reader from Harvard.

RESEARCH QUESTIONS

The following research questions were present in the article:

-  What does it look like when an art student is successful?

-  Are they able to talk bout art in meaningful ways?

- What constitutes meaningful?

- Do they make connections between a work of art and their own lives in relevant ways?

and the list does on...

I found her research questions to be valid, clear, directional to her overall research goals/curiosities. 

DATA SOURCES/SAMPLING

The data source is stated to be 200 high school art students who beta tested eLASTIC, however, it is never mentioned how LONG this test was for.  The reader is never told if the research was continuously completed over the course of a semester, trimester, or year.  Or if it was completed in bursts of time.  There is no clarity in selection for the group.  The article never states that all learners participated or what characteristics or qualities the participants possessed.  The reader can assume the testing was conducted at one school, but it is never clearly stated how vast of an area was covered by this research.  I believe the sample size is decent, but more information should have been given in defining other variables mentioned above.

MEASUREMENT TOOLS

There is not an adequate description of all measurement tools provided because the results of eLASTIC are fuzzy.  There is no final conclusion, no mention of what specifically was measured, how it was measured, etc.  Instead, it was more a conversation about possibilities for what eLASTIC could become.  "Initial attempts to mind learning data were made in the 2009-2012 research study using a "statistics" feature within eLASTIC that quantified links, images, videos, and sounds along with attached notes and comments" pg. 16  This quote is as close to measurement/results/and defining findings as we get in this article.

METHODOLOGY/PROCEDURES

Examples of teacher prompts for eLASTIC and examples of student work for eLASTIC are clearly outlined.  However, I still wish more variables on the participants, the course they were in, what was being measured, and the results had been defined.  There are no statistics presented in the results of this study, just simply that it was beta tested by 200 high school students and that a "statistics" feature was implemented. 

RESULTS

The results presented are easily understandable, but I would call them more conclusions to thinking than research results.  Wonderings and dreamings of what could come from Robo-Reader and eLASTIC more so than any evident "results" of the research.

DISCUSSION

The conclusions related back directly to Taylor's original research questions.  Generalizations made by the author are relatable, understandable, and appropriate.  The research presents many doors of possibility...yet still lacks what it claimed to be searching for which is quantitative data.