Monday, May 23, 2016

Assessment validity

Over the past few weeks I have been part of several interesting discussions on the topic of cultural validity in assessment. There is so much to talk about here but I will offer just a short few thoughts now and aim to follow up with greater insight as I construct my own understanding over next few weeks.

The experience that prompted me to write this post occurred yesterday at my school. I was helping out with the assessment of some prospective students that have applied to join our school. I was asked to conduct a reading diagnostic with a student who happened to come from China. The goal of this task is to ascertain a broad range of the student's literacy and language development. This information is useful to handover to classroom teachers so that the can develop a profile of their new students as efficiently as possible. This particular diagnostic hails from North America and, as such, contains several references to North American culture and life in general. I was asked whether this particular student demonstrated comprehension of the reading material at a D level. My response was that she could read this text with fluency and was able to answer the related comprehension. This was true, she could read it with comprehension, however I explained to her the context of the details. In the story a skunk causes an array of animals to run away. I know that this is because skunks use their odour as a defensive weapon against potential threats. The reason that I know this is that I've grown up with several cultural influences via the media and in print that have allowed me to establish this in my own mind. For a 6 year old girl from rural China this is not relevant or significant for her life. I don't believe that its fair to judge a child's ability to comprehend something on content that is completely foreign to them.

Benign as the intention may be, the data gained is not useful. Had I not explained the relevance of the skunk to the story, the student would not have got the question correct and could possibly have been labelled something that she is not. I wonder how often this happens - particularly in international schools. If we take diagnostic reading programs, for example. Although teachers may have an educated, research-based opinion on what an appropriate process is to learn how to read, if the measure that is being used to assess achievement within the process is faulty then the process itself is invalid.

To quote Guillermo Solano-Flores (pg. 3):

"Tests are cultural artifacts. They are part of a complex set of  culturally established instructional and accountability practices; they are created with the intent to meet certain social needs or to comply with the mandates and legislation established in a society; they are written in the language used by those who develop them; their content is a reflection of the skills, competencies, forms of knowledge, and communication styles valued by a society - or the influential groups of that society; and they assume among test-takes full familiarity with the contexts used to frame problems, the ways in which questions are worded, and the expected ways to answer those questions."



Assessing the Cultural Validity of Assessment Practices: An introduction. In: Cultural Validity in Assessment: Addressing Linguistic and Cultural Diversity. 2011. Routledge, NY, USA.

Photo Credit: Damian Gadal via Compfight cc

Thursday, March 24, 2016

The Raised Eyebrow

Over the past couple of months I've been thinking about changing the name of this blog. The original name referenced my initiation not only into the world of blogging, but more widely into the development of a world-wide PLN, hence the 'global' aspect. Hopefully over the past five years I've grown as a learner and I feel that, although I have developed stronger opinions about certain aspects of my professional life, the understandings that I have gained have allowed me to question ideas and perspectives with a dose of healthy skepticism.

I think this is also a nod towards a useful skill for younger learners to develop. We live in a world where information is pushed under our noses faster than ever before. If you couple this with the idea that anyone can provide research to support their cause, then I think we have a need for our students to be critical consumers of information.

So there it is. The Raised Eyebrow - an inquiry into understanding.


Photo Credit: Scott SM via Compfight cc

Reasons for change

Change management is a leadership topic that is regularly discussed and analysed. There are myriad books devoted to unpacking the processes involved with change, yet it remains one of the most difficult things to define. I think that's what makes people who can deal with change more valuable. Understanding the change process, or a version of it, is an important step in being able to deal with it, and to help avoid (or manage) resistance to change from colleagues or peers. I once heard it explained this way and have found it to be a useful framework for helping to decide on which next steps to pursue in different scenarios.

Resistance to change can be described in three broad categories: I don't like it, I don't understand it, I don't like you.

I DON'T LIKE IT: There will always be a portion of people that simply don't want to change. A general outline of where people fall on the 'change acceptance continuum' can commonly look like:

Innovators - 2.5%
Early Adopters - 13.5%
Early Majority - 34%
Late Majority -  34%
Resistors -  16%

It's important to understand the possibility of that group of resistors (or laggards). I'm always disappointed when 100% of the population aren't as excited about an improvement initiative as I am, but it's a reality. You can't please everyone all the time. A way to approach this group of people was once described to me through the analogy of a garden. You need to take care of a garden and tend to the conditions that will help the plants to grow. But it's a waste of time and resources to water the rocks - they won't grow. A lot of the time in these instances the collective of the larger group will bring the resistors on board, or they will leave. It takes time, though, and it's paramount that the initiative becomes widely accepted in order for this to happen. Jim Collins' work in 'Good to Great' discussed the idea of ‘getting the right people on the bus', which suggests that one of the keys to success in an organisation is to have the right people employed to achieve your goals - and to move on anyone that is not aligned. I think that this has the propensity to be easily misinterpreted. People definitely should be aligned to an organisational vision and mission, no doubt about that. However, they may need time to fully develop an understanding and appreciation of what is being actioned. This is effectively achieved within a a culture of feedback, collaboration and trust, not of fear of being thrown off the bus. In this case, people who are viewed as resistors could actually part of the late majority group. Additionally, what looks like laziness is often exhaustion. Change is about developing automatic behaviours, changing the old ones. This requires a great deal of energy from the rational and emotional sides of people. In the end, the emotional side will always win if both are exhausted. which leads into the next category.

I DON'T UNDERSTAND IT: In their book, 'Switch', Chip and Dan Heath refer to the idea that most resistance to change is situational, not personal. They describe an experiment that gave people free buckets of wretched popcorn. Some people ate more than others - despite the fact that it was stale and didn't taste nice. This resulting data could logically be construed as showing that these people have poor eating habits and need to be educated on how to correct them. However the revelation was that those that ate more were actually given bigger buckets of popcorn. The situation guided their behaviour instead of their personal view or habit. It can be helpful for agents of change to use this idea in their dealings with people. By shaping the path of the change initiative and designing the situation to work in their favour, change events can be fast-tracked. In many cases what looks like resistance to change is actually often a lack of clarity. If people don't understand not only the imperative to change, but also the key steps along the way, then they are more likely to look at a destination that either seems to far away that they'll be exhausted by the time they get there, or they won't be able to see it at all. The 'I don't understand it' group are important because they may actually be supporters of the change but are either too tired or confused about it that they don't adopt.

I DON'T LIKE YOU: Relationships are paramount in modern society and the associated skills of collaboration and communication are constantly emphasised as being vital in order to be successful in the 21st century workforce. If change agents want initiatives to be implemented as successfully as possible then it will be helpful if they have strong relationships and support from the people around them. Michael Watkins offers this advice for leaders that are entering new organisations - failing to create momentum during the first few months virtually guarantees an uphill battle for the rest of your tenure during the job. He talks about the notion of having 3 months (or 90 days) to achieve the break even point of contributing as much value to an organisation as you consume from it. People often look to those in their social network for cues to the 'right' way of thinking and acting. If you are one of those people then you stand a much better chance of creating early adopters or early majority members in your organisation. He also talks about The Coalition Building Cycle: Gaining allies helps you recruit others, which increases your resource base, which increases the likelihood of your agenda's success, which helps you further in gaining more allies....and the cycle begins again. If you don't have good relationships with your colleagues then you are already rowing against the current without a paddle.

References and further reading:
- Watkins, M. (2003) The First 90 Days: Critical success strategies for new leaders at all levels.
- Heath, C. & Heath, D. (2010) Switch: How to change things when change is hard.
- Kotter, J. (2014) Accelerate (Also see Leading Change and A Sense of Urgency)
- Fullan, M. (2001) Leading in a Culture of Change (Also see Change Leader)
- Collins, J. (2001) Good to Great: Why some companies make the lead...and others don't.

Photo Credit: Wouter de Bruijn via Compfight cc

Wednesday, December 30, 2015

Collaboration

Yesterday I had the pleasure of catching up with a colleague who is currently on sabbatical as she completes a M.Ed (Music) degree in New York. She was back over this side of the world visiting some family and friends for her break before heading back to continue her studies. It was wonderful to hear about some of the projects she's been working on over there and how they might be put into practice when she returns to BHA next year.

One project that stuck in my mind was that of a game that she was developing as part of one of her courses. The basic idea is that as students play the game they are rewarded for improvisation based on a section of music that they are looking at. The idea here was to diverge from just replicating a set piece of music. Of course there are monsters involved too. In order for my colleague to get her idea off the ground she had to connect with a software developer and a game designer, which she found on campus. Together the three of them developed her idea and are at almost at the stage where they have a piece of software to work with. They're hoping to test it out in schools sometime throughout 2016. Their hope is that their product will be a point of difference as it will be able to be used by teachers with multiple students at a time (i.e. a class) instead of an individual app or game, which appears to be the norm with these sorts of products.

None of this would have been possible if they hadn't have worked together. My colleague isn't a gamer and doesn't know where to start when it comes to designing games. Likewise, the software developer and game designer aren't attuned to what it is like to be working with 20-30 six year olds on a regular basis. Without collaborating, the result would not be what it currently is. This is the sort of collaboration that occurs everyday in the 'real world' - multidisciplinary teams working together to solve ill-defined problems. It is not, however, the sort of collaboration that we always design in our classrooms. In many cases students are told that they have to work together with certain people, or are told that they need to be in groups of two, three or four. This is not preparing students for a life beyond school in the 21st century. Consider the PYP Exhibition - the landmark culmination for the final year of a PYP student's journey. How many times are students grouped together because they have a faint connection to something that they'd like to inquire into. 'Oh, you're interested in researching the migratory patterns of whale sharks? Great, we'll put you in the 'over-fishing' group'. Or, worse still, they're given a choice of things that 'they'd like to explore' and groups are formed that way. Collaboration in the Exhibition, or in any similar process, should be about students inquiring into their passions and, when they inevitably reach a point where they need something that they can't do for themselves, they reach out to someone else and work together with them. If they then go back to their individual work once that part of the job is done then so be it - this is a microcosm of life. Yes, in the real world you don't always get to choose who you work with. I've certainly worked with a range of different people that would not have been my initial first choice. Sometimes there are group members that simply don't contribute to the collective - either by choice or readiness. Sometimes there are learning styles or approaches to work that suit some more than others. Sure, these can happen. But ultimately I find that there is an element of natural selection that occurs with these sort of experiences and things are straightened out. Even if you are required to work with a group that you consider to be low-functioning, you are not bound by it.  There are a variety of avenues to access skills and knowledge in order to build your own understanding along the way.

The dynamic world that we live in means that it isn't feasible to expect one person to be able to manage complex tasks alone. Real collaboration requires every member of the group bringing something to the table that can be used to help the greater good. In terms of assessing this, there are structures that teachers can put in place that allow us to measure how well each person shares their specific information. This is 21st century teaching and learning.


Photo Credit: Marc_Smith via Compfight cc

Tuesday, November 10, 2015

Documentation

Over our recent break here in Korea our Early Years Coordinator at BHA attended a workshop with Fiona Zinn that explored the documentation of student learning. Our Early Years program is inspired by the Reggio approach and considers documenting student learning to be a key pillar of this. Laura returned from the workshop full of ideas that she wants to implement with the team and today she shared some of these with the specialist teachers about how we can develop the framework of our portfolios.

We consider the use of portfolios to be an important element of our assessed curriculum. They provide students and parents with valuable information on progress, feedback and an opportunity for reflection. As the IB states in Making the PYP Happen...'a portfolio is a celebration of an active mind at work. It provides a picture of each student’s progress and development over a period of time both as individual and group learners. It enables students to reflect with teachers, parents and peers in order to identify their strengths and growth as well as areas for improvement, and then to set individual goals and establish teaching and learning plans.' (pg. 50)

To develop our approach to portfolios, Laura spoke about framing them around three areas:

(1) The voice of the teacher - this could come in an explanation of the task that the students engaged in. It could also discuss the type of play that occurred during the task. This section might have more 'teacher talk' as the purpose is unpacked.

(2) The voice of the student - this section could include photos or videos of the students. An opportunity to record a reflection would also be represented here. The opportunities are vast in this section as they allow for student input into the documentation. What ownership does the student have over the learning in the sample? How are they reflected in the work?

(3) Analysis of learning - this section highlights  the skills, knowledge and conceptual understandings that are addressed in the learning experience. How are the students achieving this?

I think this is a nice framework to reference when we think about developing portfolio pieces. There is plenty of scope for creativity in the way that this is interpreted, however, the focus is firmly on the student and the learning. Laura also lead the discussion around aspects such as graphics, colours and fonts. Her explanation here was that the colour should come from the students themselves instead of from things such as banners, borders and zany fonts. We also discussed the use of images as a form of communication, especially considering that 90% of our student and parent population do not understand English easily yet.

This was a great opening discussion and one that I look forward to continuing with the team as we unpack our beliefs and understandings regarding documentation. I'm hopeful that the rest of the Junior School will be inspired by the work of this group.

Photo Credit: cambodia4kidsorg via Compfight cc