Is it an ILS…a Serious Game..argh!

“I wonder if you (or anyone) could weigh in on the cost of serious games, and at what threshold of production/cost are we really talking about immersive simulations, rather than serious games?”

This was a comment post I received this week. As it happens, after an exchange of posts, he and I are not opposed on this subject but it was a timely reminder that there is a huge amount of accidental and deliberately created confusion in this space right now. I don’t believe that you can classify an application along the basis that it is either an ILS or a serious game. To do so would be to accept that they are entirely different beasts.

Ben Sawyer has sought to identify serious games by whether they are developed using entertainment games industry ‘game craft’. Whilst I recognize that the entertainment games sector has developed a strongly respected set of skills, principles and intellectual competency – and I do not seek to diminish that, nor do I claim that many ‘eLearning’ studios could replicate these easily – I know of many examples of ‘serious games’ that are awful in execution and ‘eLearning simulations’ which are superb. I’d also point out that this space has already delivered a multitude of examples that vary enormously in the scale (cost), the technology employed, the graphical fidelity, the ease of use, the learning design philosophies (e.g. very linear and rigid ‘instructivist’ examples and very open, exploratory ‘constructivist’ examples) and degree in which they seek to cover explict, well-defined learning topics versus loose ‘information/message promotions’. To this end I fail to see how any particular example can be easily classified as being either an ILS or a serious game.

I’ve just written a lengthy paper for the E-Learning Guild that seeks to “Demystify the design, development and deployment of Immersive Learning Simulations”. I used the term ‘ILS’ as the Guild has already done a research report that clearly found that eLearning professionals are much more comfortable with this term than ‘serious game’. This was, however, an exercise in using the terminology of your audience but in actual fact the 1,000+ Guild members whose data the conclusions were based upon clearly saw a huge variety in examples that they still considered to be ILSs.

I see ILS applications as a sub-set of a wider space, namely the serious games space. An ILS is clearly about learning whereas serious games can be used in a much wider context (e.g. marketing, brand building, advocacy, recruitment and message broadcasting). An ILS is likely to have learning objectives and approaches defined by and owned by someone from a training and development orientation whereas a non-learning application is probably going to be owned by a ‘game designer’.

We also have to recognize that there are other types of applications that people tend to see fitting in this space that may or may not use ‘game craft’ and may or may not be learning-focussed. These include ‘mini games’, advergames and quizzes. I think that mini games offer much to learning and non-learning organizational objectives and have a long way to go. I see advergame usage spreading like wild fire. I think quizzes, no matter how well-executed, are (in general) non-innovative and with limited scope for intellectual development.

There is a lot of talk about serious games delivering an ‘immersive experience’ and a perception that an ILS is not actually going to be immersive. I disagree with this as I see immersion as a quality that is used as a tool for a specific purpose not as a characteristic of the application per se. If you are designing for a disenfranchised, hard to reach community of learners than ‘game-like’ techniques (and immersion) may well be a pre-requisite. If you are designing for a highly motivated cohort of learners then there is less need to employ these techniques (and to instead focus on authentic simulation) yet this early stage analysis doesn’t by definition mean that you are embarking down an ILS or serious game route. Either way you are seeking to create a powerful experience that uses game and/or simulation techniques and which, no matter what skills, technologies or approaches are applied, still sits within the wider serious game space.

The one thing we must not do is to define an application by whether it is ‘big budget’ or whether it required C++ programming instead of Flash ActionScript. Scale and technology used do not tell us anything about the nature of the experience nor the degree of usefulness. Even if this was true, how could you possibly define a threshold above which (presumably) and application ceases to be an ILS and thus becomes a serious game?

Missing the point in serious games ‘design’

 

“The potential of using games as educational tools has been extensively Investigated……however, the marked weaknesses of these games such as the scarcity of sound pedagogy, the lack of proper player profiling, the insufficient balance between challenges and skill levels, the technologically poor and unappealing game designs, and the non-customizable nature of these games have been acknowledged and now must be addressed.“

So wrote Phaedra Mohammed and Permanand Mohan in their paper entitled “Sugar Coated Learning: Incorporating Intelligence into Principled Learning Games”. Having just spent 20 minutes banging my virtual head against a seemingly never-ending landscape of 3D office cubicles in the IBM Innov8 ‘game’, I’m heavily inclined to agree.   I’d heard so much about Innov8 and I was very keen to try it out. I liked the opening cut scene orientation insomuch as I understood my role and was excited to play it out. Therein the experience quickly deteriorated. 

“But you’re a learning guy” I already hear people saying; “games should be immersive, engaging and fun”.

I may be, at 37, older then the average gamer (whatever that actually is) but as someone who has bought 20-odd XBOX/360/PC games in the last fewyears, has played computer games since the late 70s’ and has a mini-museum of games consoles and home computers (some of which still work) I kind of think that I am not that far devolved from what ‘the kids are doing’. I also know that a bad character movement mechanic, annoyingly cumbersome camera systems and rigidly linear dialogues do not, an engaging game, make.

I confess that I gave up on Innov8 after 20 minutes so my observations are admittedly limited but surely that is a pretty key point isn’t it? If we seek to use game engagement qualities and/or the authentic and vocationally-meaningful qualities of simulations because younger generations are not engaging with traditional teaching and 1^st generation eLearning…..then having them switch off a game due to acute frustration isn’t exactly giving us the answer now is it? 

I don’t know what budget, time and technological constraints the Innov8 developers were under and, knowing how such pressures act to constrain that what we do very well myself, I have total sympathy if that is the case. Neither am I seeking to knock IBM or the development team, but when such a large multinational corporation makes so much fuss about a product like this I think we are entitled to make constructive critical comments about it. My primary problem with high profile ‘game’ products like this is that they are setting expectations around serious games that focus on eye candy and pseudo game design approaches but do so at the cost of forgetting pedagogy.   Let me give you an example. 

The game starts with a ‘to do’ list that requires you to find four documents. These documents are cunningly left on challenging-to-find locations such as …. Erm ….. office desk tops and cubicle walls. Once you have fought against the awkward character control mechanics to explore grey space after grey space you eventually find these and can read them. 

Point #1 That activity took me 15 minutes or so. If you had 1,000 students or trainees each taking that long then you have just used up 250 person hours to give the audience four 1-page documents. If the audience were first year auditors at a Big 4 firm that would equate to around US$50,000 of lost billable time! Try selling that to a Senior Partner who is looking for demonstrable performance improvement. 

These four documents are apparently going to be instrumental in my latter tasks which involves Business Process Management (or ‘BPM’ in Big Blue parlance). I actually found myself intrigued by BPM (so maybe the game did work on me as an advertising medium). I figure that I had better save these in my in-game laptop….only you cannot. The message that pops up on screen says something along the lines of:

“Mmmm this looks useful. I must remember where to find this”. 

Are you kidding me? You have given me some information that is obviously going to be useful to me later on but if I need it later I will have to run down a big staircase, navigate around a maze of office partitions and read it again. I’d rather take a screen shot and paste it in to MS Word thanks. 

Point #2 I’m a busy hard-working adult. I have a family I’d like to spend more time with if I could. I hate examples of condesending eLearning and generic training for the masses and love learning by solving problems and trying new things in a risk-free environment. Why can’t I save this and move on? Games are software. Software can write data to hard drives. Don’t make me waste my time if there is an obvious alternative. 

…..GAME OVER 

I dare say that there are many fine qualities to Innov8 and I promise to spend more time with it sometime soon uncovering these and to then provide further thoughts on the subject. In the meantime I have more pressing demands on my time.

Fig1: Identity hidden to protect the relentless in-game offerer of coffee

Pervasive & ambient technologies; shifting the focus from information to information literacy

I recently read the July 2006 edition of FutureLab’s Vision magazine – yes much of my printed ‘stuff to read’ pile really is 18 months old – and one particular article caught my attention.

It was entitled ‘Here, there and everywhere; The impact of pervasive and ambient technology in education’. The article broadly covers the 21^st century environment in which information is permanently at our finger tips from a variety of devices and through a wealth of different modes of connectivity. One quote that succinctly captures the nature of the challenge that this presents to educators is as follows: 

“We are going to be living in vastly enriched learning spaces – both inside and outside the school – and we must find a way to maximise their use for effective and engaging education”. 

FutureLab has a particular focus on learning technologies in the schools sector but it got me thinking about the nature of the information age that we live in and the way in that this also affects how we design for adult learners. 

I studied for an A-Level in Art between 1986 and 1988, a course which included the studying of the history of art. I distinctly remember my teacher describing how, just a few centuries ago, it was conceivable for a person to know just about everything known by mankind. I doubted that this was entirely true but the message, that ‘in those days’ the total sum of human knowledge was a tiny fraction of what it amounted to in 1988 is undoubtedly true. If you think about it; it is now practically impossible for one person to know (almost) everything about even a single area of human endeavour. 

The relevance to pervasive and ambient technology is thus; that with almost instantaneous availability to digitised information anywhere and at any time and with the ever increasing power of search, why do we still very frequently build learning strategies around the acquisition of information rather than the acquisition of information literacy skills? 

There is more information out there than we could ever amass and certainly more than we, as human beings, could ever absorb, retain and accurately reference. Furthermore the speed at which this global digital information repository grows is staggering and very probably exponential. If you are overwhelmed by information now – think about how many SMS, email and IM messages you get each day now compared to five years ago – consider how you will feel in another five years time! 

I am totally convinced that our education and adult learning focus should be upon building and reinforcing foundational skills around media and information literacy; giving our children, students and employees the base skills that they need to be able to locate, organise, synthesises and apply this information at the point of needing it. At the risk of ‘going all Donald Rumsfeld’, we need to know what it is we don’t know, and know how to know it. 

Search tools like Google and next generation search, semantic web and natural language processing tools like Powerset, Twine and True Knowledge will give us ever faster, easier and more accurate information finding capabilities which will, despite the prosaic scaremongering of people like Andrew Keen, become ever more relevant to those of us who know how to use them. Whilst memory capabilities are important and should certainly be nurtured and protected, it is the abilities we possess that allow us to work out what we need to know, how to find that information and how to apply it (e.g. to solve unfamiliar problems or to fulfil new tasks) which will define our performance potential and, ultimately our value as employees and employers. 

So if you buy this argument (or even if you don’t) ask yourself this; does your classroom instruction or technology-based learning strategy foster these competencies…or do you just focus on telling learners ‘stuff’ and testing whether they could recall it or not? 

Am I advocating that we in the adult learning world should suddenly scrap all of our subject domain-specific training? Of course not. It is our schools, colleges and universities that should have provided these foundational skills. It should nonetheless strongly inform our adult learning approaches.

Consider how complex the ubiquitous software tools, such as MS Vista and Office have become. Think about the bewildering depth and breadth of features these software behemoths now offer. Should we try and tell users everything that they do and exactly how to launch and appropriately use each of these? Should we then have to re-teach this with every new release of these tools? No, we should teach users how these applications are organised and structured, how the generic navigation tools work, how the in-built help features work and how to find online sources of assistance. Teach people that and they will have the ability to find what they need when they need it i.e. they will search for it intelligently, effectively and confidently. They will be able to transfer these skills to the use of other similar software applications without intense further training effort. They will be efficient, effective and productive….those “little” metrics at the heart of the decision to pay for nearly all the training activities that keeps us all gainfully employed.  

So to summarize, pervasive and ambient technology have changed the landscape that our employees live and operate in. They have already delivered a world in which the importance of enquiry skills far surpass the ability to simply memorize by rote. So why do we still very often fall back on treating learners as the stereotypical ‘vessels that need to be filled’?  That approach is now outdated, irrelevant, ineffective and down right patronizing.

Microsoft-funded study into benefits of gaming for education

Here is a post I can literally copy & paste from the source which, as it is early and my caffiene levels are still low, requires no effort on my part  🙂

######

After two and a half years, Global Kids is delighted to release the results of the independent evaluation by the Center for Children and Technology of both Playing 4 Keeps, our after school gaming program, and Ayiti, the game produced with Gamelab during the first year of the program. For the evaluation CCT observed the program and interviewed the students. To evaluate the game they looked at the results of nearly 16,000 game plays. 

Download the full report here:

In short, when we made Ayiti we wanted to learn if players would learn if the factors affecting access to education within an impoverished condition are both interdependent and exist within a dynamic system. CCT’s research found that “the central idea embedded in the game play, that no single factor accounts for success, appears to have been successfully communicated to the majority of players.”

Bingo! 

In addition, they describe how youth report that through their participation in the after school program their experiences involved:     

* Engaging in activities that require useful life skills related to communication and collaboration;    
* Learning about social issues;    
* Realizing what goes into designing and creating a good game; and    
* Gaining general computers skills. 

A gaming program that improves the lives of its participants and creates a game that has a measurable affect on the critical thinking of its players AND is an award-winning, engaging experience – nice!

Source: seriousgames.org email listserv

Using immersive simulations as assessment tools – the untapped potential of the medium

I had the pleasure of meeting up in person with Anne Derryberry of http://www.imserious.net a few days ago at our offices in Coventry, UK. One of the many topics of conversation centred on the subject of assessment within learning simulations and games. Sande Chen and David Michael wrote an article for Gamasutra in October 2005 entitled “Proof of Learning: Assessment in Serious Games” in which I was referenced.

This space is at a stage where the commercial practicalities frequently don’t yet allow us to work on some of the great ideas that many of us have already. Assessment is one of the key areas in my view. I believe that immersive simulations and complex serious games offer the opportunity to address some of the major weakness of traditional eLearning and classroom instruction. Assessment is one of the most obvious of these. 

At risk of sounding like a broken record, complex sims and games when used in learning offer a tremendous opportunity for allowing an individual or a group of individuals to ‘game the skill’ in a very realistic representation of a real world environment, system, process or situation (or, even better all four!).

If that is truly achieved then in terms of assessment we could, for example assess, evaluate or benchmark: 

• Skills and knowledge at the beginning of use (a pre-test)
• Skills and knowledge at the end of use (post-test)
• Patterns of behaviour (e.g. attitude to risk taking) throughout
• Time taken to recognise a ‘red flag’ (e.g. falling sales, customer getting angry) and then how long and whether the user(s) reacted effectively and appropriately.
• Team type indicators, behaviours within teams and ability to manage teams.
• Ability to learn from mistakes (and how quickly)
• Numeracy skills (e.g. ability to evaluate data in game and to draw accurate conclusions)
• Tendencies to rely on pre-conceptions, to stereotype, to generalise or, perhaps, to exhibit prejudices (e.g. in a diversity training context).
• Memory recall and accuracy
• Willingness to engage with others, to share resources/information and collaborate.
• Willingness to adopt a leadership role or, conversely, a ‘followship’ role,
• Strategic awareness.
• Ability to improvise and adapt under pressure.
• Soft skills e.g. ability to interact with a virtual or real world person (through an avatar) politely, professionally and effectively.
• Ability to coach, mentor and support others in game.
• Personality type indicators.
• Persistence and perseverance.
• Willingness to seek help and advice from others. 
• Frequency of errors and mistakes

The list above was a quick brain-dump and is by no means exhaustive – in fact I feel another lengthy white paper coming in the not too distant future – but, I think, indicates that an immersive sim or complex game can potentially offer so much more than ‘just’ knowledge acquisition or skills development.  

If taken to it’s logical conclusion (and implemented effectively) then I can quite easily see how an immersive sim could be used as the centrepiece of an organisational recruitment & retention, skills development and competency management strategy.

Depending on the efficacy of the application design then skills mastered ‘in game’ should equate more or less to actual competency, attitude and aptitude in the real world. So if you’re from IBM, Capita, PeopleSoft or any similar organisation drop me a mail and lets talk.

There’s (human capital) gold in them there simulated hills!