Demystifying Pre-assessment

Version 2Pre-assessment can be a confusing, even upsetting thing for teachers and students. Most of the confusion and upsetting arises from a sense of unfairness; how can students be tested on something we know they don’t know? When approached in this manner, pre-assessment stirs up anxiety for teachers and students alike. Considering this, the first step to succeeding with pre-assessment is to understand what it is and what it is not.

To begin, pre-assessment results are never graded. It is critical that students understand that you are giving them the pre-assessment to find out what they know and what they don’t know about a particular topic or unit of study. The results will have no bearing on their grades, and there will be no penalty for failing the pre-assessment. Now if we just left it at that, we risk leaving the impression that the pre-assessment isn’t really all that important, and causing students not to take it seriously or even try to do well. To avoid this, it is also critical to tell them that how they do on the pre-assessment will determine what and how you teach them going forward. If they do poorly on the pre-assessment, I will be convinced that I must teach the topic thoroughly. If they did poorly because they simply didn’t take it seriously, they will be quite bored by the unnecessary instruction they will be receiving until the next assessment. Classes will be the most enjoyable and interesting if I collect accurate information on where they are with the topic, so my instruction can be appropriate to their learning needs.

Then next thing to consider about pre-assessments, is that we should not try to make these assessments “user friendly” to avoid creating test anxiety. The explanation I just discussed should alleviate most if not all test anxiety. If we populate our pre-assessment with items that we think the students already know in order to comfort students, then we are ensuring that the results will be skewed and of limited value. We will either conclude that students are more proficient than they really are, and so plan instruction that is too advanced, or, if we interpret their success on easy questions too broadly, we will be prone to avoid teaching things that they need as foundations or review in order to succeed later in the unit. For both of these reasons, the pre-assessment must be the same as the post-assessment. The whole idea is to compare how students do on two takings of the same test; one given before instruction, one given after. That is how we can determine if our instruction has been effective. If the results on both takings is statistically the same, then our instruction has not improved their proficiency; they know no more than before we taught them. If, on the other hand, they score statistically higher after instruction, then we know that they have increased their proficiency as a result of our instruction, which is what we are after. If the pre-assessment is different from the post-assessment, we have no way of knowing if the difference is due to instruction or the difference in the test. That is why the pre-assessment and post-assessment must be the same test.

Pre-assessments are also a great tool for setting student learning objectives (SLO). Not every student will turn in identical results on the pre-assessment. Items answered correctly or incorrectly will vary from student to student, as will overall scores. Each individual pre-assessment will identify in items answered incorrectly areas in which that student needs further instruction, and in items answered correctly areas in which the teacher-and-student4student is ready to move on, take on at a higher level of challenge, or mentor other students who still need more instruction. Then, when the teacher is stating how student growth will be measured, one approach can be to state it in terms of a percentage of post-assessment items that will be answered correctly that were answered incorrectly on the pre-assessment. Here again, the difficulty of assessment items is important to get right. If the pre-assessment is too easy, students will scores will be high before they have received any instruction, and the possibility of showing growth on the post-assessment will be small. Similarly, if the pre-assessment is too difficult, and instruction cannot overcome the difficulty, students will score poorly on both assessments, and again growth will be minimal. The pre-assessment/post-assessment must reflect on the one hand the expectation of rigorous instruction and on the other hand realistic learning objectives that will guide and motivate students to learn without discouraging them with excessive challenge or boring them with insufficient challenge.

At this point, many will want to see an example of a pre-assessment. Let us assume that we want to assess 5th grade students on sight singing. We have taught these students since they were in kindergarten, so we have previous assessments and experience to give us an idea of their proficiency that will enable us to create an appropriate assessment. The students have sung with fixed do solfege and Gordon rhythm syllables, and have read medium easy songs using both. With these students in mind, an assessment might have these items. First, students are presented with several pitches on a standard musical staff. Students are asked to write the correct fixed do solfege syllable under each note, and then to sing one of the items using fixed do solfege. The teacher selects which item each student sings, and the same item is never given to two consecutive students so that echoing cannot occur. The same sequence is then repeated with rhythms. Students are presented with several rhythms on a standard musical staff. They are asked to write the correct Gordon rhythm syllable under each note, and then to sing one of the items using Gordon rhythm syllables. The teacher selects which item each student sings, and the same item is never given to two consecutive students so that echoing cannot occur. Third, students are presented with four short melodies, perhaps four measures each. These melodies contain only the pitches and rhythms that were included in the previous items. The teacher selects one of the melodies for each student to sing, and the same melody is never sung by two consecutive students. The teacher scores on a rubric each student performance and each student written response. The completed rubric is attached to each student’s paper, and all scores are recorded on a spreadsheet.

Once such an assessment is given, instruction proceeds according to the results. Students may need more instruction on rhythm, pitch, or in combining the two in a melody. Instruction proceeds with the same rhythms and pitches used in the assessment. This is not to say that only sight singing is taught in music class, but that what is taught in music class will include sight singing material that uses the same rhythms and pitches. After an appropriate interval of time, the teacher gives the same assessment again, and then again enters the results on the spreadsheet that was started after the pre-assessment. Subsequent results from other administrations of the assessment are also entered on the spreadsheet. Other pre-assessments/post-assessments can and should also be given on other topics that are covered in music class, including performing on instruments, performing with voice, responses to listening including reflections, analyses and evaluations, and improvising/composing. In each case, the student is asked to do what they will be asked to do after instruction before that instruction is given, and then given the same assessment following the instruction.

Some of these assessment will be shorter and/or more informal, because we cannot give comprehensive assessments on everything we do for lack of time. Informal assessments can be as simple as having each student sing one tonal pattern from notation or after you have sung it and marking down a number representing the accuracy that one pattern was sung. Other assessments will appear more like routine classwork, such as filling out a listening guide while listening to a musical work, or filling out an evaluation scale on a popular song. The important thing is that students have the opportunity to perform the same task repeatedly over time so that results can be compared and evidence of growth can be seen and documented.

 

Specificity and Assessment

2011 Symposium2

Formative assessments are different from evaluations. The latter are meant to quantify the final results of learning, and are usually associated with summative data. The former are designed to give both teacher and student with useful feedback on how teaching and learning is going in a specific area. Because of this, it is best if formative assessments are highly specific so that the feedback they provide is clear and useful. Because summative assessments are evaluations of an end product, specific data is not necessary. All of the component parts measured in a formative assessment are applied to the task being summatively assessed. A high rating on a summative assessment indicates, among other things, that the elements formatively assessed along the way and successfully mastered by the students are indeed related to what is summatively assessed, and that the students successfully applied them. To better understand the distinction between a specific formative assessment and a more general summative assessment, let us take a look at teaching and learning singing.

If we assume that we are starting at the very beginning, students will not yet have the vocabulary or the skills to score well on a formative assessment of specific skills and techniques. Such an assessment may be given as a benchmark assessment so that growth can be measured at a later time, but it cannot be assumed that the students will show meaningful proficiency because no teaching or learning has yet taken place. Such an assessment might ask students to perform certain exercises. For example, they might be asked to hum a sustained note while pulsing from the abdomen to demonstrate control of the abdominal muscles during vocalization. They might also be asked to sing on each vowel to assess the shape of the mouth, position of the larynx, and overall resonance, or to sing a phrase that began in the lower adjustment voice, ascending into the upper adjustment voice, and then descended back down in order to assess proper use of the voice in each register.

Each of these assessment items would generate specific data on the student’s overall singing. Students and teacher alike would have data on abdominal lift, vowel formation, resonance and placement. With data this specific, the teacher and student can work with each student based on the data and individual needs. When problem areas are improved upon, overall singing, will have improved, and the students and teacher will know exactlyrecite-tjeit7 why, and will know which areas still need more attention and which ones do not, or can be strengthened further. With this assessment approach, excellent singing is being put together, skill by skill, technique by technique, until a complete singing voice is present. The singer will continue to improve upon their singing as skills and maturation progress, but all components of an excellent singing voice will be operational as the formative assessment data is utilized.

With this in mind, it is necessary for the music teacher to think through the large concepts and break them down into the individual competencies needed to gain proficiency at the the goal, be it performing, responding, creating or connecting. Teaching students musical forms and then assessing them on identifying forms isn’t enough. While some students will be able to go from “learning about” to identification, others will not. What competencies are needed to differentiate one theme from another, or to differentiate between a motif, a phrase, and a theme? Are the first four notes of Beethoven’s fifth symphony a motif or a theme? Where does the second theme begin? How do you know? These are questions students need to be able to answer after targeted instruction has been given. Certainly a listener needs to recognize contrasts between themes to realize they are separate, and the ability to hear the key change from the tonic to the dominant in sonata form is also key. These competencies need to be taught as steps along the way to fully understanding the use of themes in musical form.

One way of going about thinking through the large concepts and breaking them down into their individual competencies is to start with the end goal, and then work backwards. First, write down what you want students to be able to do and know. Next, consider where they are and what instruction will be needed to get them from where they are to the goal. Write down the steps they will need to complete. If the goal is for them to write a rhythm composition and perform it accurately, then the steps might include identify note types, chant rhythm patterns, identify how many sounds are within each beat, improvise rhythm patterns, write down rhythm patterns, select rhythm patterns to convey an expressive intent, write down rhythm patterns that convey an expressive intent, practice performing the patterns, perform the patterns. Each step along the way will be the focus of teaching and learning, and most if not all will be formativesly assessed. Students would spend more time on any given step if a formative assessment indicated they were not ready to go on. At the completion of all steps, students would perform their compositions and be assessed on the entire project on a rubric that included each step, and summarized the final product with an overall score. On a rubric that goes 1-4 and has four steps on it, the highest possible score is sixteen. I like to assign a final rubric score based on ranges of scores. A student with a rubric score of 13-16 gets a final rubric score of 4. A score of 10-12 becomes a final score of 3, 7-9 is a 2, and less than 7 translates to a final score of 1. This type of scoring system is necessary because using rubric score  to determine a percentage score distorts the results. For example, a student could have gotten 3 out of 4 points on every item but performed at a considerably higher level than 75%. The final rubric score must accurately summarize the quality of work the student has done on the goal task; it must accurately represent to what extent the student knows what you wanted him or her to know, and can do what you wanted him or her to be able to do as a result of your instruction.

Types of Assessment and Their Uses in Music Education

2011 Symposium2

We music educators sometimes have difficulty assessing our students. There are several reasons for this, including large student loads and limited instructional time, but perhaps one of the most important reasons is that assessment is something we don’t fully understand. What purpose should be achieved by using assessment? What kinds of assessments are there, and which one is best suited for a particular purpose? I will discuss the answers to these questions today.

To begin, there are two main types of assessment: formative, and summative. Our troubles with assessment often begin here, when we confuse these two assessment types. Formative assessment is used to “monitor student learning to provide ongoing feedback that teachers can use to improve their teaching and by students to improve their learning. Formative assessments help students identify their strengths and weaknesses and target areas that need work, and help teachers recognize where students are struggling and address problems immediately. Formative assessments are generally low stakes, which means that they have low or no point value.” (Carnegie Mellon) Formative assessments are not used as the basis for project, teacher-and-student4homework, test, or report card grades. Instead, they provided feedback that prepares students to be graded on future evaluations. Examples of formative assessments include asking students to:

  • submit one or two sentences identifying the main learning point of a class (This could be an “exit ticket.)
  • turn in a music composition project for early feedback
  • take a practice test
  • give a performance that is adjudicated using a performance rubric.
  • Feedback given by conductors and private tutors during rehearsals and lessons. (These should be documented so they can be referred back to.)

Rubrics tend to work well for music, because much of what is experienced and learned in music classes is subjective. Often, music teachers are not interested in responses that are “right” or “wrong” but instead are interested in to what extend certain aspects of music making occurred. For example, whereas a language arts teacher might count up the number of errors in an essay, a music teacher would probably not count up the number of wrong notes in a performance. Instead, the music teacher would find an accurate description of the success of the performance from rubric criteria that might include,  “made some errors, but they did not detract small group instructionfrom the overall enjoyment of the performance,” “made some errors, and they did detract from the overall enjoyment of the performance.” These two statement recognize that there is a factor more important than the number of errors made, and that factor is what kind of errors were made; how impactful were those errors on the performance. Priority is given to the performance being enjoyable and expressive over it being note perfect. Enjoyable is a subjective unit of measure that is made measurable by use of the rubric.

As formative assessments, assignments that are turned in are not given a final grade, but are given comments and returned to the student for further work. Support is provided to guide the student in incorporating the feedback into a more successful revision of the same assignment. This cycle of student producing work, passing it in to the teacher (or peers) for feedback, and then returning to the work for feedback is the essence of formative assessment because it walks the student through the process of shaping work into a finished product of the highest possible quality. Once the student work has been revised and put in the best possible condition, then it is ready to be assessed summatively.

Summative assessment is used to “to evaluate student learning at the end of an instructional unit by comparing it against some standard or benchmark. Summative assessments are often high stakes, which means that they have a high point value. EnsembleSummative assessments are used as the basis for project, homework, test, or report card grades. They are given at the end of a project, unit, semester, or course, and should reflect the extent to which students have met the goals and objectives of that unit, semester or course. Summative assessments are not used to provide ongoing feedback during work, they are used to provide a final evaluation of the finished work. Whereas music teachers might adjudicate their students and give them feedback on their performance that they can use as they continue to prepare for an audition (formative assessment), eventually the students will go to the audition (summative assessment), and either be accepted or not accepted, for example, into a college music degree program, or to participate in a music festival, perhaps an all-state or all-county festival.

Formative assessments tend to be more incremental in that they measure how a student is doing in stages or steps toward completing a larger work. It is a tool used during the learning process. Summative assessments tend to be more holistic in that they measure how a student has done on the completed larger work. It is a tool used at the completion of the learning process. The two types of assessment work hand in hand. When effective formative assessment has taken place, student success rates will be higher on the summative assessment, teaching will be more effective, and student learning will be significant. When formative assessment is ineffective or missing altogether, students will generally to more poorly on summative assessments because they haven’t been as well prepared, and haven’t learned as effectively without the feedback that formative assessment provides.