In What Ways Can Learning Be Assessed Without Inhibiting Learning?

Version 2Today my post is not specifically about music or music education, but about a topic in education in general. The topic is grading; how we grade our students, different kinds of assessment, including assigning grades, and what effect these different kinds of assessment and grading have on student learning.

Most teachers are apt to teach like they were taught, and for many of us, we were taught by being graded on quizzes, tests, projects, and class participation. For the most part, grades were assigned at the end of a project, unit, or chapter, and after the test was taken or the project was turned in or presented to the class, we moved on to the next chapter or topic. Over the course of the year, we accumulated a string of grades from which a final grade was calculated and placed on a report card. From there it was either an outing to the ice cream store for a job well done, or we had a lot of explaining to do.

If any or all of this sounds familiar, it might be because you remember doing these things as a student, or it might be that you have your students do these things throughout the school year. When you or they are working on projects, taking tests, reading chapters, or listening to the teacher present material, what is your goal? If you’re like me, it was mostly to get a good grade. I’d focus on what the teacher wanted and try to do that as best I could, even if it didn’t make any sense or even if I had no other interest in what I was doing than the grade. The result of this is that I, and students all over like me, would and will still do just what they think the teacher wants, just what they have to do to get the grade they want. The problem with this is that there is no room for a student finding something they wonder about or are interested in and looking into it further. All they are likely to get for this is a lowered grade for going off topic. And because the student is doing only what they need to for the grade they want, whatever learning is taking place is generic and shallow; a kind of one size fits all.

The other problem with this is that the only thing being graded is the final result. There is no assessment of how the student is progressing, and even worse of what the student is learning, or wants to learn, along the way as he or she works on the project, studies for the test, reads the chapter, or even sits in class listening to the teacher. If it’s not on the test or in the rubric, then this other potentially valuable, interesting, relevant learning falls by the wayside.

Research in the last 10 years has shown that the most effective form of assessment is MIOSMdetailed feedback specific to individual work; more effective than even this kind of feedback combined with praise (Lipnevich & Smith, 2008). Some of the “tried and true” methods of grading students have in fact been shown to inhibit learning. For example, Canada and Hotchkiss (1989) found that “taking risks is not often rewarded in school. Students need encouragement and support, not low marks, while they try new or more demanding work.” Why would a student make the extra effort and take the initiative to delve deeper into study if their extra learning was going to result in a lower grade? They also observed that “learning is not a ‘one-shot’ deal. When the products of learning are complex and sophisticated, students need a lot of teaching, practice, and feedback before the product is evaluated.” I would even go further and say that part of the evaluation should be the growth observed as the result of that practice before doing the summative evaluation.

Now let’s say a teacher has done everything right; the teacher has given detailed feedback specific to individual work, has designed challenging, rigorous learning tasks and planned ample time for extra teaching, students to practice, and feedback to be given, understood, and applied to ongoing practice, and let’s say (as is highly likely) that students are now performing at a markedly elevated proficiency level. There is still one pitfall to be overcome. If the teacher, after all this great teaching and learning, applies the same grading standards and rubrics as before, the students, inspire of all they have done, will be unpleasantly surprised by low grades. Why? Because different methods, differently designed learning tasks, and different objects of assessment will result in different results. Students, while they have learned more, been encouraged and motivated to continue on, and have triumphed over more rigorous instruction than ever before, have probably not learned some of what was on those old assessments and tests, especially if they were heavily weighted for content knowledge above understanding. This may sound obvious, (but it must not be because the opposite happens more often than it should,) but you must be sure that you are assessing students at their proficiency of doing what you asked them to do.

You gave them feedback, so how well did they receive and apply that feedback? What growth resulted from utilizing the feedback? You asked more of them and gave them extra time to practice. What growth resulted from the added practice? To measure growth, you must pre- and post- test along the way, not just at the end. Whereas formerly you were set up to assess with vocabulary quizzes, essays, and other traditional forms of student work, when your students were doing extra practice, receiving and using feedback, and other more authentic tasks,  you weren’t concerned with them writing down definitions or even writing an essay. You were concerned with them doing something authentic and using the knowledge they would have otherwise put into that essay. But you’re no longer assessing the essay, you’re assessing what they did with the acquired learning. So you need to be sure the grade reflects the positive results you observed.  That doesn’t mean you don’t test for vocabulary or that you don’t have them write an essay. It does mean that those kinds of learning tasks are done early in the process, at the stage where they are still acquiring knowledge in preparation for doing something for which they will need that knowledge.

Acquiring knowledge is a step in the learning process that proceeds acquiring understanding. There is little point in acquiring knowledge without continuing on to gain understanding. And how do we know when a student has gained understanding? When they successfully put the knowledge to use doing something authentic to the discipline, in our case, music. And that “something” they do doesn’t have to be the same thing for every student. We can let students follow their interests by having them select the repertoire, musical ideas, projects, and so forth they will use to practice and learn within the four artistic processes of creating, performing, responding, and connecting.

Lipnevich, A. A., & Smith, J. K. (2008). Response to assessment feedback: the effects of grades, praise, and source of information. Ets Research Report Series, 2008, 1.)

Canady, R. L., & Hotchkiss, P. R. (1989). It’s a Good Score! Just a Bad Grade. Phi Delta Kappan, 71, 1, 68-71.

Using Portfolios to Improve Music Instruction

Version 2Portfolios in education are collections of student work and of documents related to those pieces of student work. They make possible the documentation of student activity and learning, the reflecting upon work even days or weeks after it is completed, and the charting of progress over time as work collected at various moments is compared in order to assess growth. Maintaining a portfolio allows more lasting and in-depth learning to take place, and makes more tangible the accomplishments of music students whose work, particularly in the performing area, can easily be lost to time after a performance is given.

In this post, I will discuss the elements of the music portfolio other than the initial student work, which includes items like notated composed music, audio or video recordings of performances, and written responses to performed music. When students are participating in a music class, it is important for them to know what they are learning, as opposed to what they are doing. The two are not the same. If a child is asked what they are learning and the child replies that he or she is learning to sing a song, or play an instrument, then the teacher has not effectively communicated the learning objective. A child is learning a song in order to gain ability or proficiency or understanding of a musical concept or skill, and if a skill, then is learning that skill in order to gain proficiency or understanding of a concept. Even if the child’s actions are recorded and placed in a portfolio, understanding is easily lost unless the child has the opportunity to draw it out of the activity.

In order to do this, reflection is a necessary part of the portfolio building process. The reflection should answer at least three questions: what did you do? how well did you do what you did? How could you improve? What did you learn from doing what you did? The responses to these questions might look something like this: What did you do? I improvised rhythms that were four beats and that sounded scary.” How well did you do what you did? I think I did well making my rhythms sound scary, but I’m not sure how many beats most of them were. I didn’t do so well making them all four beats.” How could you improve? “I could learn how to make one beat with different kinds of notes. I don’t really understand the difference between beats and notes.” What did you learn from doing what you did? “I learned that using rests makes music sound scary, and that other people don’t think my music sounds scary. I also learned that it is fun to make my own notes.”

These are actual reflections that some of my students made after doing a rhythm composition project. Think of the juncture each student was at after finishing the project. Without the reflection piece, they would have performed their composition for me, supported their creative choices in terms of conveying an expressive intent, and that would have been the end of it. But with the addition of the reflection, the students notmusic and the brain only realize what is still unclear to them (knowing how to write one beat of music with various note and rest values), but I have the opportunity to address their need, and encourage them to pursue their improvement and praise their reflection, instead of just leaving them with the knowledge that they did part of the assignment incorrectly. The reflection also brings to the front of their mind the concepts of expressive intent, note/rest values, and interpretation (by classmates), and gives them the opportunity to think about what they have done, and what they will take away from the project and apply to other situations. Because the reflections are evidence of learning and are also self-assessments, they are, along with the initial product (in this case a notated rhythm composition) student work and so legitimately belong in the portfolio.

Davidson, Crouch, & Norton identified 5 learning processes. These processes align beautifully with the 4 artistic processes of the National Core Arts Standards (NCAS); they are naturally a part of how musicians work and correlate well with the Enduring Understandings and Essential Questions found in the NCAS. The 5 processes are, listening, performing, creating, questioning, and reflecting. Davidson, Crouch, & Norton explained that, “musicians become expert listeners; they constantly work at performance; they create with instruments or as composers and improvisers; they develop a questioning frame of mind in relation to their work; and they reflect constantly on the quality of their work, both through actions and words” Davidson, L., Crouch, S., & Norton, A, 2000). The extent to which musicians engage in questioning their work, of analyzing, generating ideas, organizing those ideas, rejecting or utilizing ideas, and shaping them into a creative composition or interpretation prepares students for not only producing musical work, but also of bringing high levels of scholarship through those kinds of inquiries to other disciplines, which in turn fosters the ability to make connections between music and the other arts, and between music and other disciplines, which is also featured in the NCAS.

Because there is always intent and interpretation in woven into the work of musicians, there is also reflection upon what has been done to determine the effectiveness of conveying the intent through creation and/or interpretation. Musicians are constantly reflecting on what they have just done in order to refine what they do next; such is the process of rehearsing and preparing a performance for presentation to an audience. When these reflections are made explicit through writing them down or verbalizing them in an interview, they are apt to result in more insight and more thoughtful and informed self-assessment and correction, than when they are allowed to be nothing more than passing thoughts made during momentary pauses in practicing or composing. The first two need no elaboration; performing and listening are part and parcel of what all musicians do. Listening includes all that is included in the NCAS for responding to music, and all that is done over the course of monitoring and knowing what a musical is doing and how well they are doing it. Teachers can collect students’ written accounts of questioning and reflecting, use those accounts to deepen learning, and include them in each child’s portfolio.

Davidson, L., Crouch, S., & Norton, A, (2000). Learning through music in elementary school, Journal for Learning Through Music, Summer 2000, 56-67.

Assessment Is A Good Thing–Even For Music Teachers

Version 2There is a feeling among many music teachers, especially at this time of year when student progress must be documented, that assessment in music is a necessary evil, required by mandated teacher evaluation and/or school districts. While assessment is required by these authorities, it should not be handled merely as nuisance paperwork, because there is much value in music assessment done right. Before I go further, and lest you think I am enthralled with doing assessment, let me assure you I struggle with getting it done, and do not like the time it takes out of what I would otherwise be doing with my students.

That being said, since I began doing formal assessments, I have come to realize how much I was missing when my assessment strategy consisted of “informal” assessments. I would have students play or sing, and I would score them on a rubric of performance items such as tone, technique, and expression. There are two problems with this kind of assessment. First, it only assesses the product, not the knowledge or skill needed to produce the product. For example, a score for tone does not instruct me on what that student needs to be taught to improve tone, and it does not instruct the student on what to practice in order to improve tone. This type of assessment doesn’t break tone down into the set of skills needed to produce excellent tone on an instrument or voice. I need to know what the student needs to do in order to produce an excellent tone, and that information must come out of my assessment of tone.

The second problem with this kind of assessment is that it does not produce any student work that documents student growth. Rubric scores are not student work, anymore than the comments a Language Arts teacher writes on an essay are student work. It is the essay, not the comments that constitute the student work. In our case, it is the student’s performance that is the student work, not the rubric score. Obviously, a musical performance cannot be documented as a piece of written work, but it can be documented as an audio or video recording. Teachers who have a music room with a practice room or office attached can set up a recording device in that practice room or office. Students go into the room one at a time out of your music class and record themselves. They are assigned a number ahead of time which is recorded in your class roster. When the student goes in, they write down their number on a numbered list, so that you have a record of the order in which the students recorded themselves. Then, you can listen to the recorded performances and assess each one without knowing whose performance you are listening to, and then match up the numbers with names afterwards.

If you do not have a practice room or office, but just a single space, there are other ways to manage making recordings. One is to make the solo performances a concert for the class, with each student performing for the class while being recorded. A variation of this is to give students evaluation sheets which they must fill out on each performance. They are graded based on how close their evaluation comes to yours. For classes where students are not comfortable performing alone for their peers, the class can be given seat work to do quietly while you call one student at a time to a location in the room where you are recording them. You can still use the numbering system so that your evaluations will remain anonymous until after you have scored everyone.

To make these recorded assessments worth doing, they must target specific competencies. For example, for a second grade singing assessment, the rubric might include starting on the right pitch, maintaining the tonal center, keeping a steady beat, using a head voice throughout, and singing rhythms correctly. Notice that i-get-itgeneral criteria such as “sings with good tone” are avoided, because they do not provide useful data to improve instruction and learning. Whatever you are assessing, you must break it down into specific competencies that become clearly defined targets for the student to hit during the course of the performance. The more targets that are hit, the better the overall performance will be. You will still get a total score, but that score will be the sum of meaningful data that the student can use to learn and improve, and that you can use to inform future instruction.

There is no getting around the fact that this kind of assessment is time consuming. Because of this, I plan several classes to complete each assessment like this that I do. I typically do one-third of a class per class meeting. Meeting my classes once per week as I do, each assessment takes 3-4 weeks to complete, so if an assessment is due in June, I start it at the beginning of May. Because of the time involved, I do three of this type of assessment per year; one at the beginning of the year to set a benchmark, one a mid-year to check progress, and one at the end of the year to document student growth. This is in conformance with mandated teacher evaluation protocol.

I still do smaller assessments throughout the year. These are critical so that there are no surprises when the major assessments come around. I have students sing song fragments every class. I go around with a copy of my seating chart and jot down a rubric score for one or two competencies. Most often I am listening for accurate pitch or rhythm and use of head voice. The child sings four beats, I write down a number, then I move on to someone else. I go randomly around the room so that the children never know who will be called on next. This is not stressful for them. I make it low key and just a part of our normal routine, and they enjoy it. In fact, I think it helps develop a love of singing. They are eager to sing for me, to show me what they can do. I praise excellent performances, and say thank you for less successful ones. No negative criticism is given during this time. I usually don’t share scores with students either. I use the scores to determine what I ask particular students to do during that and future classes. If a student is struggling to sing in head voice, I may give him or her a relatively high-pitched vocal ostinato, or place him or her into a small group of singers who have really good head voices, so that the one student will begin imitating them. If I do this kind of assessment regularly, and get around to all of the elements that will be in the mid-year and final assessment, then I will have prepared my students for those major assessments well. And these little assessments take very little time, and are just part of what I do in my general music classes.

Rubrics Aren’t What They Used To Be

Version 2For music teachers assessing students, rubrics are usually the most effective means. Rubrics can be used to assess pretty much anything. That’s good for music teachers, because we assess a host of items in our students’ performances, ranging from accuracy, to tone and overall musicianship. Aspects of music making that were once thought to be not assessable because they were to subjective can, we now realize, be assessed with a well constructed rubric. But the assessment capability of a rubric is only as good as the mearueablility that is built into it. Today, I’m going to look at what makes a good rubric.

When assessing with rubrics became popular in the 1990s, it was common for rubrics to lack specificity, making precise assessment problematic. For example, as an All-State adjudicator, I was given a rubric that, for note accuracy, gave me the following choices: “The student performs…all pitches/notes accurately, most pitches/notes accurately, many pitches/notes accurately, some pitches/notes accurately, few pitches/ notes accurately.” This vague approach to rubric construction is still with us today. Many of the Model Cornerstone Assessment rubrics are similarly unclear. There is no way of knowing what differentiates “most,” from “many,” or “some” from “few.” If more than one adjudicator assess a single performance, it is likely that each adjudicator will give a different assessment because of different ideas about what each of those terms, most, many, some and few” really mean. Any rubric that uses these kinds of terms cannot give a consistent or consistently accurate assessment of student work. Said another way, it is impossible for an adjudicator to defend why he or she decided on “many” instead of “most.” A useful rubric most be specific as to what the student did or did not do.

So how could we improve this note accuracy item on this rubric? The first choice, “all pitches/notes accurately,” is fine as is. There is only one way a student can receive this score, and that is to play every note right. One mistake bumps the assessment down to the next choice. That is where we must become more specific. The simplest solution is to specify a number of mistakes at each level. It is decidedly impractical for an adjudicator to count the number of correct pitches/notes played or sung, but it is realistic for an adjudicator to count the number of errors made. Instead of “most pitches/notes accurately,” we can use, “with 1-3 incorrect pitches/notes.” Giving a range of errors captures the spirit of “most” while providing concrete parameters to the description, and where more than one adjudicator is assessing, allows the same score to be given, even if one adjudicator overlooks on or two errors. The other choices would similarly be revised, so that “many pitches/notes accurately” becomes “with 4-5 incorrect pitches/notes,” “some pitches/notes accurately” becomes “with 5-6 incorrect pitches/notes,” and “few pitches/notes accurately” becomes “with more than 6 incorrect pitches/notes.” The numbers chosen can be different from the examples given. The point is to have a small range of errors that draws clear boundaries around each choice, and describes the performance in a way that is useful to students and teachers alike.

Here’s another example, this time one that addresses response to listening for the general music teacher. One of the objectives in the National Core Arts Standards is that students will “be able to  demonstrate and describe how a response to music can be informed by a personal context.” At first, this seems like it might be one of those items that cannot be assessed. How do you measure or know a person’s personal context, let alone demonstrate how it relates to a response to music. I have recently given this some thought as I worked with my Professional Learning Community on writing a 3rd grade instructional unit for listening. We had to think creatively to come up with a solution, and we won’t really know how well it works until we’ve piloted the unit, but here’s what we came with. We decided that we could know a person’s personal context based on four sources: the person’s description of themselves (who am I), description of their friends (with whom do I choose to develop relationships), hobbies they enjoy (what interests me), and cultural background (what I have experienced in life). Students can describe these things by talking about them or writing about them, and they can demonstrate them by interpreting music or examples of music in a way that expresses each item. Thinking of music in terms of self-identity gives students the chance to thoughtfully and intentionally shape their musical world according to how they see themselves and their place amid the people and places that form their enviroment. 

So how does a rubric look that assesses a student’s ability to “demonstrate and describe how a response to music an be informed by a personal context?” One solution is given below. This assessment task was to assess students’ ability to select a musical work to which to respond, and to support their choice with evidence. This type of response and assessment correlates well with language arts common core, a definite strength for music teachers.

Criteria 4
(Exemplary)
3
(Proficient)
2
(Developing)
1
(Does Not Meet Standard)
Students will be able to  demonstrate and describe how a response to music can be informed by a personal context. Described and demonstrated (through performance example) how the selected musical work relates to self-description, description of friends, hobbies, and cultural background. Described and demonstrated (through performance example) how the selected musical work relates to self-description, description of friends, and hobbies. Described how the selected musical work relates to self-description, description of friends, and hobbies. Selected a musical work, but did not describe how it relates to self-description, description of friends, or hobbies.

Notice that as the score becomes greater, the number of things the student includes in his or her responses also becomes greater. At the lowest end, the student has still made a response, they have selected a musical work to which to respond, but have not supported their choice in any of the ways being measured. Language such as “in some detail” and in great detail” have been avoided, because, like most notes and many notes, they don’t define clear differences between one level of performance and another.

Good rubrics make clear what learning objectives are, and what students have learned or are able to do as a result of instruction. They give teachers valuable feedback that guides them in planning further instruction, and students valuable feedback that guides their further learning. Rubrics that are not specific enough to do so should be revised or replaced with rubrics that plainly state measurable items.

Developing Goals and Objectives for the Music Curriculum

2011Symposium_1_2With this post, I continue my series in music curriculum writing. From the title, you may notice the phrase “goals and objectives” indicating that there is a difference between the two. Goals are open-ended, long-range general statements that provide direction for the entire music education program, PK-12. Goals indicate the broad areas of learning the students will encounter every year they are in the program. Goals tend to include concepts, such as rhythm, pitch, and meter, and general topics as in the statement “students will understand the music of diverse cultures and musical genres.” Goals are sometimes referred to as over arching statements because of their general and long-range nature. Well written goals help teachers, administrators and others understand a concise overview of the PK-12 program.

To aid in this, goals are limited in number, and easily understood and remembered. Other characteristics of well-written goals are that they provide for continued growth through the program and beyond into adulthood, grow logically out of the music education philosophy, links concisely to a district goal, include the outcomes of music education suggested by the philosophy, are sufficiently comprehensive to form the basis for a quality PK-12 music program, provide for a lifetime of involvement with music, are realistic, manageable in number, and are conducive to generating one or more objectives.

So far, we have developed a philosophy statement, and goals which are built upon it. Next we have objectives, which are generated out of the goals. Objectives are clear, measurable statements of what students should be able to do by the end of a particular grade level or course. These objectives are not to be confused with those teachers write as part of their daily lesson plans. The later are incremental implementations of the longer-term curricular objectives. The objectives in a curriculum are end points or culminating objectives toward which the daily objectives in lesson plans help students progress. ExpectationsBecause curriculum objectives are less general than goals, they should not contain general verbs such as “understand” or “experience.” Instead, objectives should more precisely describe student behaviors; that is, things that can be observed and assessed. Verbs appropriate for use in objectives can be taken from the artistic processes, and include “describe,” “create,” “perform,” and “analyze.” Because objective describe student behavior, or what the student will do, objectives often describe the assessment that will be used when determining if students have achieved the objective.

Qualities of well written objectives are that each objective, describes observable student behaviors in assessable terms, or is paired with an existing and effective assessment strategy, is linked with a program goal and the overall philosophy, provides or is coupled with an assessment strategy, evolves sequentially from one grade level or course to the next, is attainable by most students in the grade for which it is written, and is realistic in terms of resources. There should be at least one objective for every program goal, and the number of objectives should be manageable.

Objectives should flow comfortably from one grade to the next. To achieve this, it is helpful when writing program objectives to start with grade 8 objectives. This level is a natural culminating point for the program, as students complete middle school and prepare to enter high school. It is a good point to have targets set for student learning in music. For those who elect not to take music electives in high school, grade 8 is the point at which all students have received a music education that will carry forward into their high school and adult lives. For those who do elect to take music electives in high school, the music choosing-beautiful-musicprogram at the grade 8 point should have prepared all students to pursue any musical interest they may have. Setting grade 8 as a target year puts the focus on achieving program objectives by that point. It is therefore appropriate to establish the target first, so that curriculum writers then know what they are shooting for in all the preceding grades. After grade 8, the next grade that should be completed is grade 4, because this is a mid-way point. Grade 4 is a good point at which student progress toward the grade 8 objectives can be evaluated. If the grade 4 objectives are written with this in mind, it will help keep the whole program on track. If students are on track to meet the grade 8 objectives, they should be meeting grade 4 objectives on time. After grade 8 and grade 4 objectives are written, objectives for the other grades can then be filled in more easily.

There are a few options for how to approach writing the other objectives. One is to write three sets of objectives; PK-3, 5-7, and 9-12, including separate objectives for electives such as band, chorus, orchestra, etc. This approach gives teacher multiple years to reach the objectives, and encourages reinforcing and revisiting content on more sophisticated levels from year to year. Spiral curriculum approaches are well suited to this approach. The second approach is to write a number of units designed to lead toward meeting the anchor objectives in grade 4 and grade 8 and then assign each unit to an appropriate grade level. A third approach is to combine the first two, first writing objectives for each grade or grade range, and then writing at least one common unit for each grade. While teachers would have freedom to design units and lessons throughout each year, they also would be required to teach the common units at some point in the designated grade.