You are currently browsing the tag archive for the ‘assessment’ tag.

SATs (standard assessment tests) have been very topical in the UK media over the last week. The UK Government’s pre-occupation with benchmarking schools performance and ‘school league tables’ is falling under an increasingly heavy barrage of criticism from teachers, school heads, parents and the students themselves.

BBC Radio 5 Live this morning reported that one school in England had not taught anything new since the New Year; instead it had focussed solely on preparing its students for the SATS tests. One report of one school’s alleged activities does not a scientific study make however this does appear to be a not uncommon trend over recent years.

I fully grasp the desire to assess the performance of schools, teachers and students. As a father of three bundles of boundless potential (called Toulla, Max and Matilda) I, like any normal parent want to ensure that my kids benefit from the best educational experience that my wife and I can give them. Likewise I also feel a compulsion to know how well (or not) they might be doing. Frequent, standardised ‘assessment’ would appear to be offer parents a method to make informed decisions and for governments to monitor and claim the credit for improvements in local and national education performance.

There is a reason, however, why I put the word assessment in inverted commas in the last sentence. Are we really talking about assessment or just a very simplistic and less-then useful testing of very shallow fact recollection? I fail to see how anybody can argue against the latter.

Einstein apparently once stated: “I never teach my students anything, I simply create an environment in which they can learn”. There are many people now who believe that we are not even teaching our students anything (useful), we’re just creating an environment in which we can test them.

I’ve been to the US often enough to have heard the parody of ‘No child left behind’ on many an occasion. The parody is, for those that have not heard it; “No child left untested”. Amusing at first but, considering what we are most likely doing for or, failing to do for our children; deeply worrying if we really think about it.

SATS focus on very shallow fact recollection. A secondary school History curriculum cannot possibly do anything but touch upon a truly miniscule fraction of the entire story of humanity. Therefore in order to test relative attainment of one student compared to a statistical norm, surely the test by definition has to focus only on that same tiny fraction of facts. If it (the test) did not then the probability of the test covering the miniscule amount of topics actually taught in schools would itself be miniscule and all students would score near to nothing.

The sheer quantity of tests that have to be done in every school for every age group on a regular basis also means that the tests need to be simplified for rapid throughput and to enable automation. This again leads to a culture of dumbing down of the assessment methodology and, if the assessment is dumb (yet vital to teacher’s career prospects) the likelihood is that the learning experience too will become dumb.

Do we really want to know whether our kids know the answer to “What year did WW2 start?” or, would it be more valuable to know if they knew could construct an intelligent and well-articulated response to the question; “What were the causes of WW2?”?

Remembering ‘1939’ is hardly indicative of a deep level of educational attainment. Being able to explain the concept of European ‘balance of powers’, the root causes of the rise of nationalism in Germany and elsewhere and the inherent weaknesses of the Versailles Treaty would indicate not only a sound ability to remember information but would also be a strong indicator of high level skills acquisition. Bloom’s Taxonomy may considered by many to be an overly simplistic model but ‘high level’ cognitive activities such as the ability assess, judge, compare and evaluate are surely far more useful in life than remembering numbers or dates aren’t they?

How does this relate to serious games and immersive learning simulations? In January of this year I postulated here on this blog on what I believe to be the enormous potential of this medium for delivering not just a teaching/training experience but also as a means of strong and meaningful assessment.

The obvious use of high score charts in entertainment gaming would also clearly give the UK Government the school league tables they so desperately cling to – only these ones would actually tell us something useful!

I had the pleasure of meeting up in person with Anne Derryberry of a few days ago at our offices in Coventry, UK. One of the many topics of conversation centred on the subject of assessment within learning simulations and games. Sande Chen and David Michael wrote an article for Gamasutra in October 2005 entitled “Proof of Learning: Assessment in Serious Games” in which I was referenced.

This space is at a stage where the commercial practicalities frequently don’t yet allow us to work on some of the great ideas that many of us have already. Assessment is one of the key areas in my view. I believe that immersive simulations and complex serious games offer the opportunity to address some of the major weakness of traditional eLearning and classroom instruction. Assessment is one of the most obvious of these. 

At risk of sounding like a broken record, complex sims and games when used in learning offer a tremendous opportunity for allowing an individual or a group of individuals to ‘game the skill’ in a very realistic representation of a real world environment, system, process or situation (or, even better all four!).

If that is truly achieved then in terms of assessment we could, for example assess, evaluate or benchmark: 

  • Skills and knowledge at the beginning of use (a pre-test)
  • Skills and knowledge at the end of use (post-test)
  • Patterns of behaviour (e.g. attitude to risk taking) throughout
  • Time taken to recognise a ‘red flag’ (e.g. falling sales, customer getting angry) and then how long and whether the user(s) reacted effectively and appropriately.
  • Team type indicators, behaviours within teams and ability to manage teams.
  • Ability to learn from mistakes (and how quickly)
  • Numeracy skills (e.g. ability to evaluate data in game and to draw accurate conclusions)
  • Tendencies to rely on pre-conceptions, to stereotype, to generalise or, perhaps, to exhibit prejudices (e.g. in a diversity training context).
  • Memory recall and accuracy
  • Willingness to engage with others, to share resources/information and collaborate.
  • Willingness to adopt a leadership role or, conversely, a ‘followship’ role,
  • Strategic awareness.
  • Ability to improvise and adapt under pressure.
  • Soft skills e.g. ability to interact with a virtual or real world person (through an avatar) politely, professionally and effectively.
  • Ability to coach, mentor and support others in game.
  • Personality type indicators.
  • Persistence and perseverance.
  • Willingness to seek help and advice from others. 
  • Frequency of errors and mistakes

The list above was a quick brain-dump and is by no means exhaustive – in fact I feel another lengthy white paper coming in the not too distant future – but, I think, indicates that an immersive sim or complex game can potentially offer so much more than ‘just’ knowledge acquisition or skills development.  

If taken to it’s logical conclusion (and implemented effectively) then I can quite easily see how an immersive sim could be used as the centrepiece of an organisational recruitment & retention, skills development and competency management strategy.

Depending on the efficacy of the application design then skills mastered ‘in game’ should equate more or less to actual competency, attitude and aptitude in the real world. So if you’re from IBM, Capita, PeopleSoft or any similar organisation drop me a mail and lets talk.

There’s (human capital) gold in them there simulated hills! 

twitter me

Share this blog

Bookmark and Share
%d bloggers like this: