Tag Archives: typicality

To grade or not to grade… is probably not the question?

Posted on by
1

obs in time

“Whether we grade observations or not, we need to ensure teaching is increasingly progress-orientated and outcome-orientated – concentrating the strategies that provoke progress and secure outcomes for students.”

It is probably true that we should not determine the quality of teaching based on short lesson observations, but use a triangulation of evidence that identifies the typical quality of teaching; evaluating the effectiveness of teaching for securing progress-over-time and its success in securing strong outcomes for students. (outcomes in a broad sense as well as”attainment”) 

It is also true that we need to ensure and support teachers to find their way to great progress/outcome-orientated teaching that has a deep impact on students life chances. It is this autonomy that secures the motivation and ownership to reflect and stretch toward mastery.

We should, however, explore the assumption that grading teaching denies formative development; understanding that it perhaps provides the framework for improvement. Without a descriptive continuum of effective practices there is a danger of mediocrity. Particularly as…

“Not all approaches aimed at securing progress over time are equally effective or equally well delivered”

So this begs the question how do we develop a deep understanding of what secures progress, how do we measure the quality of teaching and how do we ensure that feedback to teachers is precise, owned and liberating, such that it sparks deliberate improvement, debate and improvement? It is difficult to see how we do this without a  descriptive continuum of what matters. We can rename the continuum, but when we consider progress-over-time and outcome-orientated teaching not everything is equal…or equally well delivered. Slide1

What if we fully understood how we move from this…Slide8To this… (where teaching is progress and outcome-orientated?… and that it is this that is rewarded and developed?)

Slide16

What if we understood that measuring quality of teaching through observation alone only measures a teacher’s ability to perform a lesson by tumbling and jumping between different teaching and learning strategies to tick enough of the criteria to get them over the Good line? What if we understand the limited sample that a lesson observation provides?…(shown as the vertical line below)

jumpy progress What if we firmly framed any episode of learning or scrutiny in the context of student progress, in the past and into to the future as the measure of the typicality of teaching?… such that it becomes a measure of how:

  1. Progress-orientated the teaching has been, such that students have made good progress over time.
  2. Outcome-orientated the teaching is, such that students achieve in the future.

What if we viewed the window of observation as an opportunity to measure progress over time and future progress?… obs in time

What if we examine, reward and measure the conditions, teaching habits and approaches in lessons that give us evidence that students are making progress over time and are aimed at achieving worthwhile outcomes?

What if this involved us recognising and identifying poor proxies for learning and being smarter at evaluating what we see? (from Robert Coe)… Slide21

“every child a powerful learner” (Steve Mundy, 2015)

So given that not all approaches/strategies are equally able to leverage progress over time and that some proxies are compelling, even blinding, what should be valued to ensure that every child is a powerful learner? What if progress-orientated and outcome-orientated teaching was revealed in…

  • The quality of teacher subject knowledge, concept (and misconception) understanding as well as pedagogy understanding.
  • and that this was revealed in the quality of direct instruction and the ability to impart knowledge and to understand how students learn and make progress in their subject
  • and that this was expertly revealed in the quality of questioning that accelerates learning and unlocks understanding (perhaps the most efficacious part of the learning for progress?)
  • and that this has had the impact of increasing the quality of students answers and oracy that is beyond age-related expectation and directly improving writing.
  • and that through embedded formative assessment (within not after) – teaching plans, differentiates and intervenes to enable all abilities to make progress.
  • and that there is real clarity on the end point – students are well set to perform unusually well in assessment or exams as a result of outcome-orientated teaching.
  • and that teaching shows ambition, warmth and drive to secure progress for all; setting ambitious expectations for all students.
  • and that this is also revealed in the quality of work in books.
  • and that these highlight an appropriate amount and depth of learning as a consequence of time spent in lessons focused on writing and demonstrating learning.
  • and that teaching dwells and goes to depth at the expense of skimming content.
  • and that is informed by on-going feedback that is within and not bolt-on in lessons. And that time is committed in lessons to respond to feedback and make progress. What if we rewarded and looked for actual improvement in books from the front to back as evidence of ongoing feedback … and much less impressed by regularity of feedback sheet or dialogue that does little to improve the work?

Slide2

  • and that there is clear evidence of an ethic of excellence where students re-do and redraft work, so that they produce their very best work that they have ever done more often – something that diminishes as they progress from 3-19. Primary children often produce their best ever piece of work.

B3NFRJOCEAAQsMO.jpg-large

  • and that leads to students attitudes and approaches that demonstrate a thirst for knowledge, enquiring and knowledgeable questions.
  • …and because teaching has imparted inspiring knowledge and achieved progress over time – learning takes on its own momentum.

What if we also realised that the greatest teachers have these as habits…consistent approaches and abilities to teach with purpose, precision and consistency.

20130201-191828

What if the importance of progress over time and having the end in mind was rewarded and that observation is seen as just one part of evaluating whether the conditions are present to secure progress? What if we triangulated with data, outcomes, planning, student voice, books and other evidence? Perhaps then we would measure the quality of teaching in this way…(apologies for the grading – but not everything is equal or equally well delivered)

Slide2 …by ignoring observed performance, we reward teaching habits and approaches that have created and secured the conditions for progress and outcomes. if there is no evidence of progress over time, historic good outcomes and/or evidence of outcome-orientated teaching then teaching cannot be typically good.

What if this meant that evidence over time led to a view of the typicality of teaching and that this is stickier than when judgements are based on one-off observations? Any observed episode then simply adds to what is already known about the typical quality of teaching to secure progress and outcomes.

What if we ensured that ownership for improvement was located with individual teachers – understanding that there is also an element of earned autonomy to this freedom… Not least because you…

“Prescribe adequacy, (but) unleash greatness…”

What if this empowered teachers who then have greater freedom to explore progress-orientated and outcome-orientated approaches to teaching…

  • Purpose (secure student progress to give them a better chance in life)
  • Autonomy (you decide how you secure student progress)
  • Mastery (it is a craft not a science, be creative and innovative – seek mastery in teaching to drive progress and secure outcomes) (Dan Pink)
“We know we are in a good school when the four following things happen: Teachers talk about teaching and learning; Teachers observe each other’s practice; Teachers plan, organise, deliver, monitor and evaluate their work together; Teachers teach each other” Judith Warren, Little The Power of Organisational Setting (1981)
What if we applied our understanding of deliberate practice to support teachers to improve…deliberate practice occurs when teachers…

1. …are motivated and exert effort to improve their performance.

2. …engage in tasks that take into account their pre-existing knowledge.

3. …receive immediate informative feedback and knowledge of the results of their performance.

4. …repeatedly perform the same or similar tasks.

So that…

“practice (is) intentional, aimed at improving performance, designed for (a student’s) current skill level, (aimed at excellence), combined with immediate feedback and repetitious.” (Malcolm Gladwell)

What if observers engaged with teachers in real time, so that observers support improvement, reflection and learning during lessons?

Maybe then we would…

  • …ensure that teaching is progress-orientated and outcome-orientated.
  • ….not use short observations alone to assess quality of teaching and instead triangulate observation that evaluates progress over time and the effectiveness of the teaching to deliver outcomes.
  • …tread carefully when considering removing grades as this may reduce our ability to understand what makes the difference and to show direction of travel for teaching improvement…even without grading a descriptive continuum is required – probably divided four ways… (beginning, embedding, effective, transferrable)(or 4,3,2,1)
  • …understand that not all strategies and approaches are equal or equally well delivered.
  • …understand that the efficiency and efficacy of teaching for progress and outcomes is what matters.
  • ..release teachers to own their improvement and to consider what we know about deliberate practice as a framework/continuum to enable teachers to receive immediate feedback that informs focused, repetitious improvement.
  • …realise that what maybe considered as traditional teaching is often the most effective at securing progress over time.

March, 2015

Judge teaching over time not over 20 minutes

Posted on by
3

Progress over time

Whilst Ofsted highlight …”120. The judgement on the quality of teaching must take account of evidence of pupils’ learning and progress over time,” many schools rely heavily on brief 20 minute observations to judge the quality of teaching. This emphasises performance over systematic long term teaching impact on progress. The former encourages observation tricks and hoop-jumping the latter focuses on habits and approaches that sustain progress for each child over time.

Supporting teachers to move from Requiring Improvement to Good is often achieved by insisting on a number of non-negotiables. Teachers seek and are supported to tick-off a series of aspects of teaching and learning; they perform a 20 minute section of a lesson by tumbling and jumping between different teaching and learning strategies and approaches to ensure that they tick enough of the criteria to get them over the Good line. The consequence is that observations are high-stakes with Teachers performing a range of tricks that often hamper learning and rarely support the conditions required for students to make good or better progress over time. Teachers then carry the label of their last 20 min observation. Improving teaching needs to move much more toward rewarding teaching that has strong habits that typically create learning conditions that enable students to consistently make good progress.

jumpy progress

Student progress is not linear over time. Students make progress when the conditions are right and when they make breakthroughs in their learning (progress is more catastrophic than uniform). The blue line highlights a better description of progress over time (accepting that there will also be ‘dips’ in learning along this blue line). When teaching is good/outstanding it secures jumps in students progress because the teacher habits and typicality of approach maintains conditions for learning that promote and provoke students to make progress more often. The vertical line provides a representation of a lesson observation, scaled larger than reality, but nevertheless highlighting the tiny sample of a students journey measured by observation. We also then extrapolate the judgements made in this lesson and make the assumption that this represents a teachers performance across all of their classes all of the time … it doesn’t.

The bar below highlights how small a sample the 20 minute observation represents compared the learning over time. This sample is very likely to be unrepresentative and  hide the typicality and actual quality and effectiveness of teaching. This also only shows one class; if we place 6-8 more blue bars alongside then the sample size becomes even more unrepresentative (or ridiculous). We can be guilty of placing far to much emphasis on the 20min observed sample and place too little weight on the evidence of progress over time or use the conditions, habits and practices to extrapolate progress into the future. If we agree that what matters is the typical quality of teaching and the ability of that teaching to genererate good progress or better over time then we should look beyond the 20 minute observation to seek evidence of progress over time. We should also consider how the conditions for learning in the observation can be extrapolated (with care) to assess the likely progress of students into the future.

obs in time

We have made too much of progress in a lesson or part lesson. We have not helped ourselves at times by using phrases like, “do the students know or can they do something that they could not at the start of the observation.” In feedback we are often susceptible to making it clear to teachers that we are judging the lesson (20mins) and not them as a teacher. The reality is that measurable progress for students is unlikely in 20 minutes, but that it is possible to examine and judge the conditions present in the lesson that give us reassurance that students have, are and will make at least good progress over time…are they getting a good deal?

The first bar below shows how dominant the sample observation can be on grading a lesson and by extension the teacher that delivered the lesson. Where we weight the judgement heavily on what is observed it tempts a teacher performance; a mad rush to run through a range of strategies that are often detailed on the observation form. The consequence is the teachers teach a lot, students are busy and often bewildered, moved-on, and asked to show how much progress they think they have made. The punctuation in the lesson through questioning, AFL, modeling, peer assessment, paired work, group work, four minutes of writing…. Tick boxes on the form, but in quick combination reduce the conditions required to secure progress over time. Far better to judge the typicality of teaching and therefore the effectiveness of teaching by considering a range of evidence.

bars

If we are to reward typicality of teaching and teaching that generates good or better progress over time then the weighting of our evidence should use a range of sources. The evidence from the observation of the teaching should provide an insight into the conditions that are typical for the students over time. Much more emphasis should be placed on evidence from students, their books, evidence from panning (backwards and forwards) and the data/outcomes for this class.

With the emphasis on rewarding and promoting teaching that secures progress over time then perhaps feedback and judgements should highlight typically good or typically outstanding or typically requiring improvement or typically inadequate as more appropriate judgements on the quality of teaching. The importance of progress over time to a judgement is highlighted in this table…

table

This rewards those teachers that work hard, have good habits and the professional ability to generate that conditions in a classroom that secure good or better progress over time. This means that we should have stickier judgements for individual teachers. The first bar below highlights the range of grades that could be achieved in an academic year by a single teacher due to the high stakes nature of 20 minute observations (the observation providing a label to be carried by that teacher until the next observation). Where the emphasis is more around typicality then judgements are stickier and more reflective of the typical quality of teaching (shown by bar two). This will have the effect of polarising teaching judgements. Where teaching uses effective approaches and habits that secure progress over time the evidence will always be in books, in the planning, in the student voice, the routines, shown in the quality of feedback, in the purpose and meaningfulness of the learning journey…where this happens good and outstanding teaching becomes securer over time. Where the opposite is true, where teachers rely on the performance, mark less, plan less, have less purpose and less focus on the journey and the outcomes the typicality of teaching will require improvement…and that is likely to be true for all classes … and we are back to the importance of habits that sustain student progress over time.judgements

What would observation criteria that emphasised the importance of progress over time look like? Here is an example that places progress over time as the key determinant on the typical quality of teaching. Below this are an indication of the approaches that would contribute towards securing the conditions required for securing progress over time…

lesson obs

The form, therefore, indicates areas that may be observed that are more likely to support good progress over time. These are not prescribed or essential, neither are they a set of tick boxes to be checked to determine the grade. When the key determining judgement is progress over time, the 20 minute lesson observation is no longer a performance or does it necessitate the teacher jumping through hoops. Whilst the lesson could be all singing and all dancing it could also be students sat silently redrafting a piece of work for 20 minutes or performing or engaged in a piece of art etc. This approach frees teachers to not change practice under observation; secure in the knowledge that the observer is measuring progress secured over time not the number of tricks and hoops-jumped in 20 minutes.

Prescribe adequacy, unleash greatness…

All of which links to the trendy ideas around moving from tight to loose. To get to good it is often about being tight; using non-negotiables to raise the bar. To move through typically good to typically outstanding it is about loosening or unleashing greatness. You do this because teaching is a craft not a science; the ability to facilitate 30 teenagers often of mixed ability to make good progress requires an awareness, a professionalism and ability to keep all students progressing. Sometimes that is about engaging, inspiring, provoking responses and sometimes that is learning vocabulary, a spelling test, redrafting, testing, feeding back, being diagnostic and closing gaps. The best person to judge this interplay is the teacher. We need to recognise this and use observation to judge teaching impact over time and not on how many boxes a performer ticked in 20 minutes. This should empower teachers who then have greater freedom and…

  • Purpose (secure student progress to give them a better chance in life)
  • Autonomy (you decide how you secure student progress)
  • Mastery (it is a craft not a science, be creative and innovative – seek mastery in teaching to drive progress) (Dan Pink)

Where a teacher has taken on a class, from another teacher or at the start of the year there needs to be a shift in the emphasis placed on the different parts of the evidence. An early judgement on typicality would extrapolate the observed conditions, judge the likely impact of habits, use teacher planning and dialogue to consider the likely typicality and impact of teaching over time…and re-visit later to judge impact. The longer the teacher has a class the more the emphasis will shift to actual measurable progress over time from their starting point with the class.

As we reward progress over time (making it a determining judgement)  we need to increasingly compare actual (external) outcomes with our typical gradings of teaching. There has been a poor relationship between the quality of teaching judged through lesson observation compared to actual outcomes of the students; class performance often does not stack up well against observation data. A focus on progress over time should generate a greater link between the judgements on typicality teaching and student outcome. 

So ticking tricks in 20 minutes is mis-leading and does not focus on what matters. What matters is that students receive typically good and outstanding teaching every lesson so that they make progress over time. Those teachers that create the conditions for progress more often should be rewarded, for it is the impact of this teaching that supports students to make progress over time that counts. The freeing/loosening nature of this view on teaching and rationalising the part that observation has to play in judging teaching releases professionals to teach anyway they choose, without ticking any prescribed boxes, so long as students make progress over time and this is reflected in a triangulation of evidence then teaching is typically good and outstanding and these judgements are stickier….and that feels right.

April, 2014