Formative assessments for an engineering design challenge

As part of the unit I'm creating on engineering design (specifically about how to protect an egg from breaking when the egg is dropped from a third-story window), I'd like to consider how I can integrate formative assessment.  Multiple educational researchers have identified formative assessment as an  effective strategy, including Black & Wiliam (2001), who found an effect size of between .4 and .7 of formative assessment on student achievement.  This effect size is larger than most instructional interventions, and Black & Wiliam argue that a careful integration of formative assessment into teachers' daily practice could dramatically improve our schools.

The objective I'll focus on for this exercise is, "Design a solution to the egg drop challenge, in the form of a clear, labeled drawing, and speak persuasively about the advantages of the design." The lesson for this objective will consist of three main parts.  First, I'll introduce the activity, offer direct instruction on drawing the design and speaking persuasively about it, and give examples.  Second, students will work independently to draw their designs and plan arguments for why their idea is the best.  Third, students will meet in their project teams to share their ideas.  Each student will attempt to persuade the group that his or her design is the best.

Formative assessment #1- self-assessment checklist

Self assessment is well supported as a means of formative assessment.  Researcher John Hattie is well known for having conducted meta-analyses of hundreds of educational studies.  Based on this  research, he has formulated a list of the instructional interventions that have the most and least impact on student achievement.  Near the top of the list is student self assessment, or self-reported grading (ranked 3rd) (Visible-learning.org, 2016).  Hattie has said, "the biggest effects on student learning occur when teachers become learners of their own teaching, and when students become their own teachers."  A big aspect of students becoming their own teachers is learning to evaluate their progress or self assess (Victoria dept. of ed., 2010).

John Hattie's 2011 book Visible learning for teachers points out implications of his extensive research for classroom practice.

The self assessment I would like to use for this unit is a simple checklist.  I'll invite students to do a rough sketch of their design solutions independently, and then I'll project the following checklist to help them evaluate the feasibility and functionality of their designs:

My design . . .

1) requires only the available materials.

2) can be built in the available time and with the know-how of the group.

3) includes multiple safe guards (redundancy) against egg breakage, so that if one safeguard fails, the egg will still be protected.

Students will respond to the checklist using a T-chart in their notebooks.  One one side of the chart, they will write "yes" or "no" in response to each criteria.  On the other side, they will write a supporting statement.  For example, in response to question #1, the student might write "yes" and go on to say, "My design requires a plastic grocery bag, string, five popsicle sticks, and a toilet paper tube."

The purpose of this formative assessment is to ensure quality in students' designs, before they draw their final sketches and share their ideas with their group.  Based on students' response to the checklist, I will be able to reteach how to make our designs feasible and ensure a high chance of success.  This check-in will also raise the rigor of the team's discussion about each member's design, because they will have practiced applying criteria for success.

Formative assessment #2: survey

Another formative assessment I'd like to include before students do their final drawings is a survey.  I plan to show students three examples of design drawings.  I will ask them to evaluate each of the drawings based on its clarity.  Students' survey results could be displayed immediately, for example via a Kahoot survey.  The Kahoot app allows teachers to display survey data in real time, as students respond to the questions via iPad.

The Kahoot app allows students to complete quizzes, surveys, and more via iPads, with the results projected for the whole class to see in real time.

Completing a self-assessment checklist and a survey before students do their final drawings will take significant class time.  However, I'm comfortable with using time in this way, because I believe these formative assessments will make students more aware of the learning goals and will lead to better results.  As Black & Wiliam (2001) say, "Many of the initiatives that are needed take more class time, particularly when a central purpose is to change the outlook on learning and the working methods of pupils.  Thus, teachers have to take risks in the belief that such investment of time will yield rewards in the future, whilst 'delivery' and 'coverage' with poor understanding are pointless and even harmful."  I want students to understand that creating a design sketch is really about communicating clearly, and I'm willing to take the time to accomplish this goal.

By comparing example sketches, students will see clearly the traits that make some drawings easier to understand than others.  The survey will demonstrate to students that there is a great deal of consensus among people as to which traits are most effective for communicating visually.  It will also get them thinking about the traits they would like to incorporate in their own final drawings.

Formative assessment #3: 3 do's and don'ts about speaking persuasively

This formative assessment idea comes from the article "Ten assessments you can perform in 90 seconds," on the Teach Thought website.  The assessment consists of having students list three things to do and three things not to do in reference to a given topic.  In the case of the lesson I'm planning, students will be listing three things to do and three things not to do when speaking to persuade.

As explained earlier, once students have completed their design drawings independently, they will meet with their teams, sharing their designs, and trying to persuade each other that their design is the best.  To preface this activity, I might model a right way and a wrong way to persuade, without telling students which strategies to look for.  I could then ask students to reflect on the two role plays and decide for themselves which strategies were effective and ineffective.  They will share their thinking via the "do's and don'ts" assessment.  

Conclusion

Formative assessment gives students a chance to try, get feedback, and make improvements, all before grades have been assigned.  Wormeli (2010) points out that students can learn with or without  grades, but they cannot learn without formative assessment and descriptive feedback.  Most teachers, myself included, have tended to put the emphasis on summative or graded assessments, but Wormeli makes a convincing case that formative assessment deserves the bulk of our attention.  Summative assessment is "post learning," because by that point it's often too late to intervene.  Formative assessment, however, is assessment for learning.  I will implement formative assessment much more often in my teaching going forward.

Reference list

Black, P. & Wiliam, D. (2001, Nov. 6).  Inside the black box: raising standards through classroom assessment. King's College London School of Education.  Retrieved Dec. 16, 2016, from http://weaeducation.typepad.co.uk/files/blackbox-1.pdf

Teachthought.com. (2016).  Ten assessments you can perform in 90 seconds.  Retrieved Dec. 16, 2016, from http://www.teachthought.com/pedagogy/assessment/10-assessments-you-can-perform-in-90-seconds/

Victoria Department of Education and Early Childhood Development. (2010, April). Visible learning: what's good for the goose . . ." Retrieved Dec. 16, 2016, from http://www.education.vic.gov.au/Documents/about/research/ravisiblelearning.pdf

Visible-learning.org. (2016). Hattie ranking: 195 influences and effect sizes related to student achievement. Retrieved Dec. 16, 2016, from http://visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/

Wormeli, R. (2010, Nov. 30). Formative and summative assessment. Retrieved Dec. 16, 2016, from https://www.youtube.com/watch?v=rJxFXjfB_B4

Understanding and applying standards: there's more to it than I thought

Introduction

This unit showed me that learning standards are at once more important and more helpful than I had assumed before.  My basic understanding of the standards up until now is that they let us know, in a general sense, what needed to be taught in a given year.  But until I completed the activities in this unit, I hadn't realized how much useful information is packed into each standard.  From now on, standards will be my starting point for planning, rather than an adornment once the planning process is already well underway.

Unpacking a standard

For me, learning to unpack a standard has meant learning to slow down.  In the past, I've tended to gloss over standards, picking out what seemed to be the topic statement and paying little attention to the rest.  However, this unit has shown me it's worthwhile to dig a little deeper into the language of the standard; there's a hidden richness for those who are patient enough to read carefully.  

I learned from the video "How to unpack a standard," from the Imperial County Office of Education, to analyze standards in terms of their verbs, concepts, and contexts.  By breaking standards down in this way, I found that standards often set higher expectations for students than I have tended to.  For example, the first standard in the unit I am currently planning says, "Define a simple design problem reflecting a need or a want that includes specified criteria for success and constraints on materials, time, or cost."  When I broke the standard down, my attention was drawn to the standard's verb: define.  In the past, I as the teacher have done the heavy lifting of defining the problems students needed to solve, including the criteria for success and the constraints.  But this standard says that studnets themselves should learn to define the problem, and that realization changed how I planned the unit.  Now the plan starts with a lesson on identifying design problems and describing them in detail.  This lesson will prepare students to determine the criteria for success and inherent constraints in the unit's culminating activity, the egg drop challenge. 

Backward mapping

Grant Wiggins, in his article "What is a big idea?" and his video lectures, makes the case that we should start the planning process by defining our big-picture goals.  What do we hope students will be able to do when they finish this unit, class, school year, or when they leave our school?  He points out that many teachers and educational systems seem not to know their own goals, or seem to be working from day to day without keeping the goals in mind.  He gives the example of critical and creative thinking.  Most educational systems hold the skills of critical and creative thinking to be among their major goals for students, but in the day-to-day classwork, students aren't often invited to practice critical and creative thinking.  At most schools, you can get straight As without developing these skills, he says, and I would have to agree.

Famous proponent of "backward design," Grant Wiggins, in a lecture from 2013.

My biggest takeaway from Wiggins' work (and the work of his co-author, Jay McTighe) is always to ask myself "Why?"  Instead of just covering content because it's in the standards, or doing an activity because it sounds fun, I need to ask myself, "Why this content?  Why this activity?"  If I start there, my planning and, consequently, my teaching will more likely focus on meaningful skills, taught in a way that enables students to transfer the skills to new situations.  Wiggins offers an example of how to teach a math unit on statistical measures like mean, median, and mode, with a sense of purpose.  He would start the unit with the question, "What is fair?"  because it's a question students are always raising.  "When you say what your Mom did wasn't fair, what do you mean?"  After facilitating students' reflections on questions of fairness, he would give them scenarios that involve fairness and can be addressed through mathematics (mean, median, and mode).  Students would come to see that the concepts of mean, median, and mode actually help us to answer the question, "What is fair?" 

Wiggins' example struck me, because as a 4th grade teacher I've covered mean, median, and mode.  I tried to make the unit fun by engaging the class in a simple ball game and keeping statistics on students' performance, and it was fun, but Wiggins' example gets closer to what kids really care about, and it means more in the end.  The example showed me that a backwards design approach can lead to more meaningful lesson ideas.

In terms of understanding standards and applying them to lessons, the backwards design approach has taught me to start with the standards.  Before, I started with what I thought were cool activity and project ideas and then went looking for standards that fit my ideas.  In embracing backwards design, I will start with the standards, the school's educational model, and my own big-picture goals for student learning.  Only after I've clearly defined the overall goals will I start generating unit and activity ideas.  In this way, I'll ensure a much closer match between the standards and our daily learning experiences. 

Writing objectives

To be completely honest, when I've planned lessons in the past, I've tended to use the standards as my objectives.  I just listed the standards at the top of my lesson planning template and started writing the lesson.  So it's a step forward for me to generate objectives for a lesson based on the standards.  In this unit, I've come to understand that objectives help you define the smaller steps students need to take in order to meet the standard.  The standard is complex and almost always requires multiple lessons to achieve.  Writing objectives makes you stop and think, "What exactly are the steps I will need to guide my students through in order to help them reach the expected outcomes?"  

For instance, in the standards for the unit I'm currently planning, students are asked to generate and compare multiple possible solutions to a problem based on criteria and constraints.  In writing objectives for this standard, I realized that to compare multiple solutions to a problem, students would need to practice sharing their solutions with each other and giving feedback.  They would need to develop a deep understanding of the best criteria to use to evaluate their solutions, as well as the constraints presented by the situation.  A standard I might have "covered" in a single lesson before suddenly became the source for at least two, maybe three or four, lessons.  The process of writing objectives showed me several intermediate steps I really ought to be teaching my students, and not just expecting them to pick up on their own.

One more thing

The video "Think alouds: unpacking standards" by Sarah Brown Wessling introduced me to Appendix B of the Common Core English/Language Arts standards.  In this appendix, you can find text exemplars and sample performance tasks, including for social studies and science.  These texts and performance tasks give an idea of the level of rigor expected by the CCSS, in terms of the kinds of texts students are expected to master, and the higher-level thinking that's expected of them in relation to all of their literacy activities.

Reference list

Imperial County Office of Education. (n.d.). How to unpack a standard [Online video file]. Retrieved Dec. 4, 2016, from https://www.mydigitalchalkboard.org/portal/default/Content/Viewer/Content?action=2&scId=100028&sciId=829

Wessling, S. B. (n.d.). Think alouds: unpacking the standards [Online video file]. Retrieved Dec. 11, 2016, from https://www.teachingchannel.org/videos/understanding-the-common-core-standards

Wiggins, G. (2013, Feb. 28). Understanding by Design (1 of 2) [Online video file]. Retrieved Dec. 11, 2016, from https://www.youtube.com/watch?v=4isSHf3SBuQ

Wiggins, G. (2013, Feb. 28). Understanding by Design (2 of 2). [Online video file]. Retrieved Dec. 11, 2016, from https://www.youtube.com/watch?v=vgNODvvsgxM

Wiggins, G. (2010, June 10). What is a big idea?  Big ideas: an authentic education e-journal.  Retrieved Dec. 4, 2016, from http://www.authenticeducation.org/ae_bigideas/article.lasso?artid=99

Standards and backwards mapping

Introduction

I am planning a unit centered around an egg drop challenge.  In this challenge, students will use limited supplies to construct a container that can keep an egg from breaking when dropped from a third floor window. 

(marionmac56, 2012)

For the unit, I chose the three standards from the Discipline Core Idea (DCI) of Engineering Design, from the Next Generation Science Standards (NGSS) for 5th grade.  I chose this theme and these standards because our school is in our first year of implementing the NGSS, and we are busy generating units at all grade levels that fit the standards.  Our new science scope and sequence aligns the NGSS with the curriculum of the Spanish teachers, and features interdisciplinary STEM (Science, Technology, Engineering, and Math) projects. 

My unit fits into the 5th bimester of 5th grade.  When I'm finished, I plan to share the unit with the 5th grade teachers as a starting point for their own plans.

The standards

3-5-ETS1-1: Define a simple design problem reflecting a need or a want that includes specified criteria for success and constraints on materials, time, or cost.

3-5-ETS1-2: Generate and compare multiple possible solutions to a problem based on how well each is likely to meet the criteria and constraints of the problem.

3-5-ETS1-3: Plan and carry out fair tests in which variables are controlled and failure points are considered to identify aspects of a model or prototype that can be improved.

Unit proficiencies

In order to meet these standards, students will need to develop a variety of proficiencies.  In this post, I'll elaborate on three.

To meet the first standard above (3-5-ETS1-1), students will need to be able to identify useful criteria.  The standard calls for students to define a simple design problem, including criteria for success.  Often in school, criteria are handed to students without explanation or input, so students may not understand how the criteria are generated or their usefulness.  This unit will raise students' awareness of what makes criteria useful and how they can be formed.

Below is a chart showing achievement levels related to this proficiency, which will be useful when I make the assessments:

Identify useful criteria

Excelling	Student can identify a list of five or more useful criteria for a successful design, in a variety of contexts, without teacher prompting.
Meeting	Student can identify a list of at least three useful criteria for a successful design, in multiple contexts, with minimal teacher prompting.
Progressing toward	Student can identify multiple useful criteria for a successful design, in a familiar context, with some teacher prompting.
Emerging	Student may be able to identify one or more useful criteria for a successful design, in a familiar context, with teacher prompting.

A second proficiency I would like to develop in this unit is evaluating one's own and others' ideas.  This proficiency relates to the second standard above (3-5-ETS1-2).  This standard asks students to generate and compare multiple design solutions.  The standard seems to present a perfect opportunity to develop students' feedback skills, because in comparing their designs they will naturally want to offer feedback to one another.  It makes sense to teach them how to do so effectively and politely.  Pointing out strengths and areas of improvement in a way that motivates rather than discourages is a valuable skill in a variety of contexts.  In mastering this skill, students are well on their way to becoming self-sufficient learners. 

Again, below is a chart showing achievement levels for this proficiency:

Evaluating one's own and others' ideas

Excelling	Student can offer specific, extended feedback on the strengths and areas for improvement in a design idea, in a way that motivates rather than discourages, without teacher prompting.
Meeting	Student can offer specific feedback on the strengths and areas for improvement in a design idea, while showing respect for all involved, with minimal teacher prompting.
Progressing toward	Student can offer feedback on the strengths and areas for improvement in a design idea, while showing respect for all involved, with some teacher prompting.
Emerging	Student may be able to offer feedback on a design idea, while showing respect, with teacher prompting.

A proficiency related to the third standard above (3-5-ETS1-3) is applying the results of an experiment.  Part of this standard is identifying aspects of a model or prototype that can be improved.  In order to accomplish this standard, students need to be able to work independently to first understand the experiment results and then use this understanding to figure out how to improve a design.  These skills could be transferred to any situation in which there was a need to test the effectiveness of a system and suggest improvements.  Most careers require some form of this skill.

Below is a chart showing achievement levels supporting this proficiency:

Applying the results of an experiment

Excelling	Student can draw various insightful conclusions from experiment data and apply these conclusions to the improvement of a design, without teacher prompting.
Meeting	Student can draw multiple insightful conclusions from experiment data and apply these conclusions to the improvement of a design, with minimal teacher prompting.
Progressing toward	Student can draw meaningful conclusions from experiment data and apply these conclusions to the improvement of a design, with some teacher prompting.
Emerging	Student may be able to draw conclusions from experiment data and apply these conclusions to the improvement of a design, with teacher prompting.

Assessments

This unit will include several assessments.  What follows are three examples.

1)  design problem graphic organizer

Students will work in pairs to identify an existing tool and define the design problem it addresses.  This assessment will occur early in the unit, at the end of the first lesson, and students will be provided with a graphic organizer to guide their work.  The graphic organizer will prompt them first to choose a simple tool and define the design problem that it solves.  Included in this definition of the design problem will be the criteria for success, as well as constraints on materials, time, and cost.  The graphic organizer will also prompt students to construct an argument for whether the design problem represents a need or a want.  Assessment data will be gathered through teacher observations and a review of students' graphic organizers.

How this assessment shows whether students have met the unit standard(s):

If students can identify and define a design problem on their own, they have demonstrated competency with the unit's first standard (3-5-ETS1-1).  They will then be ready to generate and compare solutions to a design problem, which will be the next step of the unit.  If students struggle to complete this activity independently, the teacher will then know that he or she needs to reteach and provide more practice before moving on, because students must possess a clear understanding of a design problem before they can suggest solutions.

2)  design drawing, sharing, and feedback

Students will offer feedback to team members' design solutions for the egg drop problem, and reach a consensus on which design or design(s) they will adapt for the team's final plan.  Assessment data will be gathered through a rubric focusing on how well students were able to offer and receive feedback and arrive at a reasonable final solution.

How this assessment shows whether students have met the unit standard(s):

The second standard (3-5-ETS1-2) asks that students generate and compare multiple solutions to a design problem, and that is exactly what this assessment asks them to do.  As part of the assessment, it will be key to find out the reasoning behind teams' final design.  Why did they choose certain design elements as opposed to others?

3)  egg drop experience presentation (taking the form of an in-class presentation, a video, a written report, etc., as students choose)

In preparation for their presentation, students will complete a reflection in which they discuss the experiment results ("Which design elements seemed to work the best?  Why do you say that?  Based on these results, how might you improve your own team's design?").  For the presentation, they will find a creative way to communicate their reflection.  Assessment data will be gathered in the form of a rubric on the effectiveness of the presentation.

How this assessment shows whether students have met the unit standard(s):

The third standard (3-5-ETS1-3) asks students to identify aspects of a model or prototype that can be improved, and this assessment tests students' ability to do just that.  Students' work can be judged on the extent to which they are able to draw reasonable conclusions from the experiment and apply these  results in meaningful ways to their own design.

Learning experiences

The above assessments are part of larger learning experiences that will equip students to succeed.  For example, the first assessment, the design problem graphic organizer, will be completed only after students have received teacher modeling and whole-class practice.

Introductory exploration of design problems

In this activity, first the teacher will show the class a simple tool (a pair of scissors, for example).  He or she will talk about this tool in terms of the following questions:

     -What problem does it solve?

     -Does it fulfill a need or a want?

     -What makes a pair of scissors good or bad?

     -What constraints do scissors makers face in terms of materials, time, and cost?

Students will then work in pairs to generate a list of more examples of simple tools (by simple, we are referring to tools that are more like a pair of scissors than a computer) that meet design problems, and the teacher will guide the class through a discussion of several of these tools, addressing the above questions for each tool and recording the class's responses in a graphic organizer on the board.

Only then, after having seen the teacher modeling and collaborated as a class, will students be asked to choose a tool that hasn't been discussed so far and complete the graphic organizer with their partner, with less teacher guidance.

How will this learning activity help students to develop knowledge and skills necessary to meet the standard(s)?

Through teacher modeling and coming up with their own examples, students will understand what a design problem is, in terms of its criteria for success and constraints, as well as whether it constitutes a need or a want.  Students will practice coming up with useful criteria for assessing a design solution.  This knowledge and these skills directly address the components of the unit's first standard (3-5-ETS1-1).

Egg protection device drawing and group critique

For this learning experience, students will work individually to create drawings of their initial solutions to the egg drop challenge.  The teacher will share an example design and invite the class to offer feedback, based on the checklist provided.  The checklist will include criteria for success (developed by the students themselves in a previous activity) and constraints on materials, time, and cost.  Students will meet in their teams, and each student will share his or her idea, arguing for why it will work.  The group will offer feedback, again based on the checklist.  The feedback will necessarily include each design's strengths and areas for improvement, and will be recorded in a graphic organizer.  Lastly, teams will review their feedback and create a final design, which will actually be used to build their egg protection device.  The final design could be based on one team member's initial design, or could be a synthesis of multiple students' ideas.

How will this learning activity help students to develop knowledge and skills necessary to meet the standard(s)?

Students will get practice thinking of a design solution, drawing it, and arguing for its advantages.  They will compare ideas in their group, evaluate each other's ideas, come up with a final design, and argue for why the final design is the best.  Their discussion will be guided by criteria for success and constraints on materials, time, and cost.  Generating and comparing design solutions are skills addressed in the second standard (3-5-ETS1-2), and referencing criteria for success and constraints further solidifies students' achievement of the first standard (3-5-ETS1-1).

Reflection on the results of the egg drop

The teacher will share a summary of the experiment results, including a chart that shows which team's designs succeeded and which did not, along with photos and video of the event.  Using this data, teams will discuss the questions "Which design elements seemed to work the best?"; "Why do you say that?"; and "Based on these results, how might you improve your own team's design?"  After recording their responses to these questions, students will develop a presentation to share their reflections.  The presentation could take the form of in-class sharing, a video, a written report, etc., depending on each group's preferences.

How will this learning activity help students to develop knowledge and skills necessary to meet the standard(s)?

This activity addresses the second half of the third standard (3-5-ETS1-3), according to which students need to identify aspects of a model that can be improved.  Using the data provided and their own observations, students will identify design elements that tended to succeed and elements that tended to fail, and try to explain why.  Finally, they will apply these reflections to their own team's design, offering suggestions for how it could be made better.

Conclusion

One of my goals for this unit is that students are aware of what they are learning.  As the English coordinator at my school, I often visit classes and get the chance to chat with students.  When I ask them, "What are you doing?" they can answer easily.  "I'm making a poster."  "I'm reading a story."  "I'm making a presentation."  But when I ask them, "What are you learning?" students often struggle to answer.  As I continue working on this unit, I want to clarify exactly what I hope students will learn, and how I can make students aware of these goals.

Reference list

marionmac56. (2012, Oct. 1). Untitled image of egg drop. Retrieved Dec. 4, 2016, from https://marionmac56.wordpress.com/tag/egg-drop/