Our Journey with Assessment for Learning in Religious Education
The purpose of this article is to add to the growing conversation about assessment practices in Religious Education. At St Joseph’s in Oamaru we have a set of challenges that are quite common. We analysed our challenges, came up with a response and tested it over three years to see if it made a difference to what our students understand in Religious Education. Our data is moderated and consistent and we believe we have made a difference.
Parameters
Religious Education, unlike faith, is something we can measure. In our school timetable and reporting systems, Religious Education is expected to be equal to the core subjects that we report on for National Standards. It is my belief that we should teach it and assess it with the same rigour, moderation and reflection that we use for the National Standards subjects.
Our Challenges
The RE curriculum is a progressive curriculum which year by year adds knowledge and understandings so that in 8 years, the student can cover an extensive knowledge and experience base to help them understand the teachings of the Catholic Faith. In theory, if children didn’t learn about Mary in year 1, they can have another go in year 6 although that doesn’t help them if they missed year 1 and year 6.
At the present time this holds some challenges for students and teachers. Many students arrive at Catholic schools during their schooling with no prior religious experience. In the past, knowledge of the faith was nurtured within the parish and family. Now, many families do not have an active life in the parish or have not had the benefit of religious education themselves and the school is the sole transmitter of the faith and the content of the faith. This is by no means ideal and it does not mean we give up on making the parish the centre of religious life for the children and their families, but it is the reality.
This makes it very difficult for children to retain the vast treasury of information that is Church teaching. In our case, at St Joseph’s in Oamaru, we have a turnover of around 50 children a year entering or leaving the school and some children just coming to us for intermediate years so we do notice the effect of religious illiteracy.
Having said that, many children have 13 years of Catholic education and yet, “A general religious illiteracy has taken hold among young Catholics; unfamiliarity with the tradition that makes only the more remarkable their continuing desire to be called ‘Catholic’” (Duthie-Jung, 2011, pp. 154-155).
When I was year 7 teacher at St Joseph’s I had several children enter every year who were completely new to any form of Catholic education and I had children who had experienced their whole schooling in a Catholic school and were active members of the parish. To compare this with the curriculum area of reading: at the intermediate level one would usually expect to teach reading to children with a range of reading ages from about 8 to 16 years and in some year groups from about 5 to 16. Differentiating learning for a wide range at this level is an expected part of our practice. Translating that into Religious Education makes for a much wider achievement gap. It is not uncommon for children to enter with some maturity, perception, and developing academic rigour who have never heard of the Christian version of the Christmas story. So a year 7 class in RE not only has children who are performing at the equivalent of the 16 year old reading age, but those who not only can’t read but do not know the alphabet and have never seen a book. How do we differentiate?
I suggest that in the intermediate + years if the RE curriculum is not taught with rigor and differentiation, it leaves those with no prior religious literacy very little to latch onto while those who had a good basis switch off from a “dumbed down” curriculum. No other curriculum area faces this degree of challenge. I suggest also that it is very hard to find teachers who can cope with the level of differentiation required because it demands an in-depth understanding of the whole of the catechetical faith combined with excellent pedagogy.
Assessment for Learning - how we were doing it and why it wasn’t working
In 2012 our special character reviewer, Paul Ferris, commented on our extensive use of RE assessment data in the classroom and asked whether it went any further. Our RE assessment information helped to identify areas of weakness in the children’s understanding and areas of weakness in practice. However, we didn’t use it for anything beyond that.
We used the tests provided for each strand. We used them at the beginning as a diagnostic test and all teachers found this very disheartening and we came to the conclusion that we should just assume our students know nothing about each strand before we began it and spare them the humiliation of testing something they know nothing about. The reason for this is the new strand usually does not relate directly to the strand before it and so the children could not make connections. For instance, a student may have achieved well in year 5 for the Jesus strand which is about understanding the humanity of Jesus. The year 5 strand also contains some teaching on the divinity of Jesus which is the focus of the year 6 strand a year later, but as it was a minor focus in year 5, when that child is diagnostically tested on the year 6 Jesus strand a year later they tend not to do well. They don’t have the prior knowledge just from from their RE teaching to address the test.
Without a good diagnostic starting point, we can’t assess the impact of teaching and learning.
At the end, when tested again, most children typically did very well. An average 60-80% improvement was normal. That would indicate we were all doing very well indeed. But two or three months later, the children retained very little of that information. Part of that challenge is about maintenance and how we maintain the knowledge over time. The rest is about good pedagogy. Good pedagogy tells us that knowledge-building happens when children can relate new information to their prior knowledge and make connections between areas of knowledge. In the example of the Jesus strand, if the year 6 teacher has limited knowledge or has only taught the year 6 strand, or is teaching for the first time, they may not have the knowledge connections themselves to situate the learning in a context. They will teach the point that Jesus is the fullness of God’s revelation; the Christ aspect of Jesus, but fail to emphasise that Jesus was also fully human.
An eleven-year-old boy really illustrated for me the necessity for making connections. He was new to our school with no prior faith background. He was very enthusiastic about prayers and masses and he engaged wholeheartedly in RE. But when he answered the question, “Can you tell us some of what Jesus taught us?” his answer was, “Jesus taught us how to make shelves, side tables and cabinets.” He had grasped the idea that Jesus was a human, situated in a time and place and that he had been a carpenter. He did not have sufficient prior knowledge to make any connections as to why we might still be interested in this carpenter 2000 years later. He loved it all and accepted it as it came but couldn’t make much sense of it.
Of course that’s how we learn about God in the first place, with a child-like, even childish appreciation. The children learning at their mother’s knee learn the stories about Jesus, they hear about the birth and the humanity, alongside the miracles. My three year old son, after being at the Good Friday service, came home, took the crucifix off the wall, put it in the back of his tricycle and announced he was taking the “King” for a ride. How many connections were being made there? A three year old could identify the Triumphant King with the crucified Christ and respond with empathy, doing the best thing he could think of to help Jesus. How do we create those sort of connections with older children, who have more mature, analytical minds but only small snippets of information? How do we empower all our staff to make connections between the year levels and to make the connections sufficient to form a big picture themselves?
The Need to Make Connections
The Christian Research Association in the United States has found that young people cannot coherently articulate their own beliefs (Smith & Denton, 2005 in Hughes & Christian Research Association, 2007). We wanted to help them with that and give them some “big ideas” to help them make connections with their learning.
Another study by Hoge and Petrillo (1978) shows a positive correlation between education and understanding that translates into a negative correlation with faith. According to that study, that means that the more academic understanding students have of RE the less faith they have. Could that mean that the actual issue facing young New Zealanders is not the quality of their religious education but rather the lack of a scaffold to help them translate this knowledge into the personal meaning and relevance they feel is required of them? As post-modern young people they feel obliged to make personal sense of the world. They need a means of making connections. Whilst maintaining Catholic identity, individual attempts to make meaning are resulting in young Catholics distancing themselves from faith communities and therefore any possibility of support with developing their academic understanding into a mature faith response. I believe we need to be actively and explicitly teaching the children how to make connections within the context of their religious education.
Key Ideas
At St Joseph’s, our solution was to make “key ideas” for each strand. Over the course of a year, as a whole staff, we unpacked the theology behind each strand, using the theology and the catechetical references at the front of each strand book as a start.
To make this work we had to start making connections ourselves. Alongside this we were also working on integrating digital learning into our curriculum. As DRS at the time I started our RE blog based on flipped learning for staff http://stjoesoamarudrs.blogspot.co.nz/2014/05/holy-spirit-staff-meeting-term-2-2014-i.html?view=flipcard
We agreed that we would do a certain amount of reading and background work before arriving at meetings. Meeting time was about challenging and making connections for ourselves with the aim of all staff having a good understanding of the underlying strand theology. We also focused on tricky areas, the areas that challenge our understanding and how we would approach these areas in our teaching.
For each strand we made a graduate profile, of what children leaving our school should know and understand. If they are to have the kernel of the knowledge that sits alongside our faith, it had to be simple.
This is what we came up with for each strand:
Jesus
|
Holy Spirit
|
Sacrament
|
God
|
Church
|
Communion of Saints
|
1. Jesus was a human (as well as God).
2. Jesus came to show us how to live and to save us.
3. Jesus is God
|
1. The Holy Spirit acts in people and in the Church
2. The Holy Spirit appears in Bible stories.
3. The Holy Spirit is the third person the Trinity.
4.We have images of the Holy Spirit but these are not the Holy Spirit.
|
1. There are 7 sacraments - Baptism, Reconciliation, Eucharist, Confirmation, Marriage, Holy Orders, and Anointing of the Sick.
2. Each sacrament has its own symbols and rituals.
3. Sacraments are a sign of God’s grace in the world and in us.
4. Sacraments relate to your life journey (te wa)
|
|
1. The Church is the people united with God.
2. We all have a responsibility in the Church.
3. There are formal responsibilities in the Church.
4. The mission of the Church is to bring about the Kingdom of God on Earth.
|
1. Mary is Jesus’ mother and first disciple and model to all Christians.
2. Communion of Saints is all the souls in heaven, purgatory and God's family - the church on earth.
3. Our choices affect the type of person we become and what happens to us after we die
|
Our aim in using these key ideas was to turn the curriculum around. Many of our teachers have a full Catholic background and understanding but these teachers are becoming more rare. The first thing we turned around through addressing these key ideas was the teacher understanding. The teachers now had a broad roadmap of what they were teaching. One non-Catholic teacher with seven years experience of teaching RE said it was the first time she felt she had an understanding of what she was doing. Having taught one level for many years, it was easy to know that level really well but not know how it tied in with the bigger picture. So using key ideas, from the staff perspective has been a scaffold for understanding.
This approach is about making connections, helping the children to form a big idea into which to link their new understandings. The hope was that telling the year 7 completely new to Catholicism that Jesus was a human being who lived in Israel 2000 years ago, and He is also God, and He came to teach us how to live and to save us through sacrificing Himself on the cross, would give him the picture - but not yet the understanding. If you only get that Jesus was a carpenter you’re going down the wrong track and you’ve got the wrong starting picture. Which one of us is not on a journey of understanding about those few short statements? Which one of us actually really “gets” it and reached an endpoint where we can pass the test? It is a developing process and your understanding changes, hopefully deepens, but sometimes it is challenged, as you change. Yet a three year old can get those concepts too. Originally, when our RE curriculum was made, we may not have had to spell this out, but now we do.
To make this approach work, the first and foremost thing that we must be explicitly teaching is the key ideas for each strand. Each year when we return to the strand the key ideas are the same and in between times we refer to them. We also have to be true to the body of knowledge and in the past three years the way we have done this is to continue to teach the RE curriculum at each level. So the year 4 child will still be getting the essence of the year 4 curriculum - Jesus came to teach us how to live, but this is taught explicitly alongside Jesus is fully human and Jesus is fully divine. As we have gone on we have pulled out some examples of the Gospel stories children need to be exposed to in order to get the big idea, to sit alongside their coverage of the national RE curriculum so we can keep returning to these as maintenance. We are not taking away from the curriculum, we are adding to it.
From the children’s perspective, the proof would be in their achievement and we knew we had to give that time to show. So we dedicated three years to following the key ideas as we had agreed them and tracking achievement to see if it made any difference. We knew that from year to year previously very little was retained and connections were not being made. One of my outspoken year 7 students back in 2012 stopped me when I tried to teach the Holy Spirit strand which covers the images and action of the Holy Spirit in the Old Testament. He said, “I can’t do this, I don’t even know where this place is.” He hadn’t been at school in year one when they did the life and times of Jesus. Those who had been there were not any more able to make sense of the Old Testament in terms of geography and history. So we had to make timelines and maps and put them around the classroom and find out about the Holy Land as a real place before anyone could even imagine the Spirit of God in the Old Testament.
A year 5 boy a few years back exclaimed when we talked about modern Jerusalem, “What you mean this place is real?” He was immersed in a faith-filled family and in parish life and had been a high achiever in RE from year 1 but somehow he had decided it was all a made up story. If we’d been fully explicit in year 1 that Jesus lived in a particular time and place and was a human like ourselves and kept coming back to that point again and again, he would have known Jerusalem was a real place. There are so many connections needing to be made.
SOLO
At this point when I talk to some people about what we are doing and I start to talk about SOLO, some switch off because they don’t “do” SOLO. We all “do” SOLO. SOLO is the best structure I have come across to explain the natural learning process. It is the one promoted by John Hattie in Visible Learning, which gives us a fairly definitive meta-analysis of what makes a difference in education.
What was failing us in the way we had been teaching and assessing RE is the making of connections. There is a strong affective component to our curriculum, but given the nature of the challenges which I have described above, the children were not making connections. They were learning a discrete set of information which they could test on successfully and then instantly forget.
SOLO = Structure of Observable Learning Outcomes (Biggs & Hollis). The first part is about structural thinking and multistructural is where there are several pieces of information or strategies. Practitioners engaging with SOLO are either using things like the “hot SOLO” maps (Pam Hook, n.d.) or their own devices to explicitly scaffold children to move from knowing pieces of information to help them make connections between information, world-views, and prior knowledge in order to engage in “relational” thinking. This is when some new understanding has truly happened. This is learning. Beyond that there is “extended abstract” thinking which is when the new learning is embedded and is used in new ways, in different contexts. It is when the student has gone beyond. Here is an example of extended abstract writing about “Church” written by a year 8 student who was new to Catholic education in year 7. It shows she has taken on multistructural thinking about what we believe about Church. She is doing relational thinking as she has put it all together to make a coherent whole picture. Most importantly she is doing extended abstract thinking - she has gone beyond, she has interacted with the Holy Spirit because the wisdom she imparts in what she has written is way beyond what we taught her directly.
Written by a year 8 student, St Joseph’s Oamaru
How am I the Body of Christ?
By being the Body of Christ we are being Christ in spirit and mind by doing those little things like picking up litter which shows bits of Christ’s mind and spirit. How do we extend this in our daily life? We do things that brighten others’ day, receive the Sacraments, go to church and pray for others – simple things like this.
When and how do we continue to be the living Church?
By having and believing in our conscience, by reading the Gospel, by caring for others because of our righteousness, by trying to be leaders to others, by helping and being a role model.
Why do I believe?
Because Jesus and God show us that humanity can be so much more if we show our inner true grace; because doing something good is the true feeling that God and Jesus meant for us to feel; because our heart is not unlocked until we believe in this strong faith and because the Gospel speaks to our souls.
We choose to commit ourselves to God and Jesus but why? Because when you’re walking down the street and someone falls over - faith and our good heart shows us how to help them up. It just makes a better, more compassionate society. There is not a time limit for choosing your faith even at the end of your life you’re welcome. The price to pay for not believing is truly not being true to yourself. So that is why.
|
As a staff we did a year’s learning in SOLO in RE before we started feeling confident about using it. We realised that extended abstract thinking couldn’t be specifically planned for. In a sense it is what happens after good teaching and learning, when the Holy Spirit enters and transforms it into something else. For a year we were vigilant for examples of this and shared them with each other. Some of these examples are described on my blog and I believe some of what the children came up is inspiring: http://lorrainefrancesrees.blogspot.co.nz/2015/03/solo-development-2013.html
We are still challenged by SOLO but we know that when we use it, it helps us scaffold the natural learning process. Examples of how we might use SOLO are also on our DRS blog: http://stjoesoamarudrs.blogspot.co.nz/
Did it make a difference?
Yes it did.
We tested the children by asking open-ended questions about the key ideas. The children could then draw from the knowledge they had available to answer the question. They had to be taught how to expand and add detail so their answers would truly show their understanding - a useful skill for future high-stakes testing. For our juniors, pictures were acceptable which they then describe for teacher/older student annotation. It put all children on an equal playing field. The child who had been read Bible stories from infancy was in the same position as a child who only knew a couple of parables. I know there’s inherent difficulties with that which I will come to in our “next steps” but it is a way of finding out a bit more about what meaning the children give to their RE learning. If they can apply relational or extended abstract thinking to it, it tends to last a lot longer than the previous two or three months.
For our content knowledge, the teaching from the specific strand curriculum level provides knowledge and activities which have helped to grow understanding but the teachers also needed to be aware of the examples that we should keep coming back to, e.g:
Jesus came to show us how to live and to save us (year 2+)
a) Give an example of a Bible story where Jesus shows love (draw or write).
Possible answers:
Any of the healing stories:
- Washing of feet, obeying his parents and going home at 12
- Matthew 8:1-17, 9:18-38
- Healing crippled man and forgiving sins: Mark 2:1-12
- Healing deaf man Mark 7: 31-37
- Blind man Mark 8:22-26
- Ten lepers Luke 17: 11-19
- Washing of feet – John 13: 1-17
- Obeying his parents and going home - Luke 2: 41-52
What the data tells us
In the first year the children were post-tested after the teaching strand with the key ideas as the explicit, up-front focus. This gave us baseline data. We knew we were going to trial this for three years so we took the achievement information from the class two years ahead as the baseline data for the present class. This meant that if the present class reached that baseline (the present achievement of the class 2 years ahead) in two years we would have achieved a year’s progress each year and everything was still the same. If they didn’t reach it after two years, then our proposition to teach to the key ideas, was not helpful. If they exceeded it after two years then we knew we were on to something.
So for instance our year 3 class achieved an overall class average of 52% understanding after learning about the Holy Spirit in 2014 and our year 5 class achieved 61%. We would hope that after two further years when the 2014 year 3 cohort reached year 5, they would at least achieve an overall 61% understanding.
In fact, this class went beyond the baseline in their next year achieving 65% in 2015 and then 77% in 2016. This was typical. Most classes exceeded the benchmark they would hope to reach in two years, in their first year.
We have been able to drill down into our results in a lot more detail. We are able to tell which children are excelling and need extension and who is struggling. We found there was a strong correlation between learning difficulties in other curriculum areas and struggles with RE. We found that our “veterans” the children who had been with us all along did slightly better on average, but the difference was not significant. Not surprisingly, children who showed interest, who choose to be active in the parish and are active participants in prayer and liturgy tended to also do well. Overall though, achievement was most strongly correlated with achievement in other curriculum areas.
The data has also given useful information about teaching. We can see the ongoing areas of weakness coming through in the learning. For instance we can see from the results that the children have a good understanding of images of the Holy Spirit but less so the Holy Spirit as part of the Trinity. We can use that information to plan our future teaching and learning and factor into our professional development.
Surprisingly this has also helped us with our National Standards moderation. There is an element of teacher judgement involved in deciding whether something is relational or extended abstract thinking just as there is in making National Standards judgements. It has extended our practice of inviting others to moderate our judgements and it has identified areas where we consistently make kind, or tough judgments and helped us to smooth these out.
Reporting
In 2014 we remade our whanau reports to show each strand and we were able to give specific achievement information to parents. Our board reports now have data which grows from year to year and we can clearly see areas where we are doing well and areas for growth.
The Next Steps
The three year cycle is finished. We know that teaching with the key ideas is helping our students to make sense of their learning from year to year and we will continue to do this.
Our approach is deductive in the first instance rather than inductive. Inductive reasoning involves experiencing a variety of pieces of information and building these into knowledge; very much like the SOLO process. Deductive reasoning starts with a statement of theory or fact and demands proof – which sounds like it goes against what I have written.
In the case of religious understanding, both inductive and deductive reasoning are needed to build faith. Very few people believe God is the creator of the world because someone told them so (deductive). They much prefer the more ‘logical” or “scientific” information that it started with a big bang, forgetting to wonder what caused the big bang. People actually “get” that God made the world when they see something above and beyond in the nature of the world around them, or when they experience death and understand that life is not just a breathing body. Yet if there was no teaching available about God people would make up their own stories - which is also what happens. So it has to go both ways, through teaching and through experience.
Our approach is giving the children the statement, showing them that we believe it ourselves by our witness and giving them the information - the Bible stories, prayer/liturgical experiences and Church teaching to back it up and show its truth. Both have to happen, I don’t know of anyone who has come to an understanding of the Christian God without having any direct teaching about God. Jesus said, “Go and spread the Good News,” - yes we do it through action as St Francis so rightly said but we also have to tell - if the first witnesses did not TELL that Jesus is the Risen Lord, no-one would know.
If we had been direct disciples living with Jesus we would have come to an understanding of Jesus’ divinity through inductive reasoning as Peter did at the Transfiguration. Our body of faith has been transmitted deductively. The Church Fathers chose which writings to include in the Bible in order to convey the Church’s understanding of the truths about Jesus Christ. Our catechism tells us what the Church believes and references the many places these beliefs have been built. We don’t make up our own belief system based on our inductive reasoning. We use our experience to measure, reinforce or challenge our belief system. For this reason I think it is not sufficient to test solely for content-knowledge and that is why we are using the SOLO component. That could be compared to the affective domain but that is typically an area we cannot measure whereas when we can use SOLO to measure the degree and quality of connections.
Our children’s are getting a better idea of the contents of their faith and how to connect that to their lives and beyond through using the key ideas. Over the last year or so we have started to add more content as suggestions for what teacher’s might specifically use to help children explain their understanding, particularly when that key idea is not covered in the year’s strand.
Our next step will be to continue to build on our knowledge-base, continue with our staff formation plans where every teacher is working towards, and some have achieved the diploma in Religious Education and to integrate RE more consistently throughout the curriculum. The main focus of meaning-making with RE will be to help the children scaffold the knowledge they pick up from RE into lifestyle. By “style” I mean the way in which they live their lives. Through our integrated inquiry curriculum we will seek to help our children understand they view the world through a lens even without knowing it. We want them to become aware of the Catholic Christian lens and help them view the world with these eyes. This will be our next step towards helping our children become fully integrated Catholic Christians.
REFERENCES
Biggs, J., & Collis, K. (1982). Evaluating the quality of learning: the SOLO taxonomy. New York: Academic Press.
Duthie-Jung, C. (2011, September). Faith Amid Secularity - A Critical Exploration of Catholic Religious Identity Among Young Adult Pakeha Catholics in Aotearoa New Zealand (Doctoral thesis). Sydney College of Divinity, Sydney, Australia. Retrieved from file:///Users/Teacher/Downloads/Faith%20Amid%20Secularity%20-%20Chris%20J.%20Duthie-Jung%20(2).pdf
Hoge, D. R., & Petrillo, G. H. (1978). Development of religious thinking in adolescence: a test of Goldman’s theories. Journal for the Scientific Study of Religion, 17(2), 139–154.
Hook, P. (n.d.) http://pamhook.com/
Hughes, P. J., & Christian Research Association. (2007). Putting life together : findings from Australian youth spirituality research. Fairfield [Vic.]: Fairfield Press.
Smith, C., & Denton, M. (2005). Soul Searching : The Religious and Spiritual Lives of American Teenagers: The Religious and Spiritual Lives of American Teenagers. Oxford University Press, USA.
