Gamification

"Erik - I remember a while back we had a talk about the rowing site you put together for your team, in particular the badges/points. Any chance you mind summarizing what has been particularly effective or ineffective in this regards?

I am planning on using some similar gamification in some sites in the next few months, but am struggling to strike a balance between effective behavioral change and gimicky bullshit. (It would be great if someone would actually study this beyond just the choice to use gamificaiton or not - I would be really curious to see the impact of each component)"

"Gamification, and things I have learned.

First, amazingly effective. Amount of time invested in training shot up by 180% (aka, nearly three times) after rolling out the system. More than that, it created a competitive culture that drove the team beyond what gamification could do. Basically, it gave the team the shot in the arm to start working hard enough that the hard work sustained itself. Ironically, the gamification aspect is less importantly now because of the level of competition on the team, but I doubt I could have short circuited to this level of success without it. So, it is best to realize that gamification is more of a stepping stone and not the objective - given our last discussion, I think this should resonate with you. The idea is not to get people extrinsically motivated to collect achievements, it is to get them into a mode where they are intrinsically motivated to keep going.

Second, it needs to be immediate and visceral. To explain, you can divide the achievements that can be attained in two ways - ones that trigger automatically, and ones I assign manually. Manual ones are like "Winning a Medal" or "Making First Varsity", basically things I cannot automate. The thing with these is, the digital reward is not really motivating them - they would far rather achieve what they reflect. They are still important - these are the most frequently requested as they tend to be the largest status symbols, but once implemented tend to be much less of a driving force. Athletes tend to work most towards the ones that are awarded automatically. In principle, the visceral response from entering a workout and getting the rewarded badge immediately tends to be what most athletes enjoy from the system. I have actually had athletes tell me they added 30 extra minutes to a workout because they knew it would score them the badge. So there has to be some kind of immediate rewards to drive the system.

Third, cheating happens but most people are honest about it. I have had plenty of athletes tell me they accidentally entered a workout incorrectly and got an award they didn't deserve - they tell me because they want it removed. The athletes recognize when someone has received an award that was undeserved and the individual athlete is usually first to remedy it. Do some people have achievements they do not deserve - sure; do I care - no. I would not recommend actually tracking progress in any meaningful way with gamification. I do not monitor athlete progress through the achievements - I use other systems. I do use the system to encourage athletes to report, track, and monitor their own progress, as well as to encourage them to workout out more with intermediate rewards. If they want to cheat the system, whatever - the purpose is to encourage interaction with the system, so as long as it does that for the majority of the users, I am happy.

Lastly, the system has to keep evolving - and it is often hard to keep up with peoples progress. The gamification drive really gets some people excited, and they become very obsessed with tracking it. When requesting new features, they most often ask for improvements for tracking achievements so they know how to tailor their workouts to maximize the rewards. This presents a problem/opportunity where I 'need to'/'can' use the achievement updates to steer them in a preferred direction. Ultimately though, it becomes about preserving momentum - people get very excited and rush down an achievement path. If the well dries up, the momentum is lost. So it becomes this game on my end of trying to keep moving the carrot in a new direction to motivate the athlete.

I have been working on this system for a while, and the system has very much evolved in response to the membership. How they interact with the system, and my attempts to stay ahead/guide their behaviour has ultimately had more to do with how the system is today than where I started with it. It has been an invaluable tool to use, and has really good potential - it is however important to realize that it is still a means to some other end. Like I said earlier, the system provides an extrinsic motivator starting out in small bites, and getting people to commit to a little more as they go along. But in the end, you still need to guide them to some intrinsic motivator. I think you have the right mindset with you project. Ultimately, the best reward of your system is when your users care less about what you can give them, and more about what they can do outside of the system."

"Embedded in your response are some key behavioral, conceptual, and software issues. People are too quick to implement some bullshit points system that have no meaning. Why in the world do I care what my supposed TripIt "travel score" is? I care about the countries I've been to and the number of miles flown - isn't that gamification enough.

Some thoughts on your points:

1. Do you have empirical evidence to back this up? Pretty impressive gains.
2. Interesting breakdown, but I view the the difference as a divide between a real world achievement and a created achievement. My names for this distinction could probably use some work, but my point is one achievement would have existed with or without your system. While I agree that the real world achievement represented in a badge is not that motivating, I think a more in your face UI/UX showing your progress towards something can fuel motivation. Granted your examples are more of discrete events and not necessarily awards that are earned gradually. Even the fact that these discrete events are consolidated into a profile is enough to fuel motivation beyond what those real world achievements could have done alone. Albeit, the increased motivation is probably relatively small, especially relative to the "created achievements" which I would guess were a core contributor to your 3x gain.
3. The fact that these badges/experiences are about status and pride in a community and there is not necessarily any other gain (monetary or otherwise) is huge. Since its purely reputation, the sweet (probably accidental) benefit is honesty. And frankly if someone was cheating to earn a digital badge that has no real world implications other than some contribution to their reputation, who cares? If you started giving away points that were redeemable for prizes, I would imagine this would change. That is the real dynamic I find so fascinating here - people are doing things they wouldn't have done otherwise for a digital badge that costs nothing (for the most part) to award. Imagine applying that to social change.
4. Moving carrot analogy is perfect. I think its impressive that you've had enough success with it to have the problem of lost momentum over achievements.

Some links:

• My initial thoughts stemmed from this article: Wired Cow Clicker
• His blogs recent post is here, I haven't had a chance to review the following links, but they look promising: Bogost Blog: Gamification is Bullshit
• Definitely going to check out this book when it comes out: The Gameful World, Authors of that book: http://spw.playbe.com/, http://codingconduct.cc/"

"Some quick answers to your points/questions:

1. I cannot quantify the effect, nor can I separate its effect from others in the development of the team. We have done a number of things to improve athlete training (improving infrastructure, investing in recruiting, secured additional coaching) all the while building the achievement system. I have been working towards allowing other programs to use the system (basically building in roster management, permissions systems, and better interfaces for coaching/admin functions). Once I am able to get teams to join, I will be able to get some more data on how the system affects performance. Ultimately, it is a non-trivial task extracting the net effect of the system, but I am currently exploring ways to assess it in hopes of one day presenting or publishing my findings.

2. Progress bars were the first thing I was asked to add. I really should have touched on how much the athletes stalk the progress page (that I have some data on, and it comes down to on average once per session - in short, everytime they log a workout, they check their progress). The feedback is what they crave, and I have had them in the past tell me that they went further or did more in a week because they knew they were close to the next level of achievement. I should also have mentioned that I am coding up a new system of achievements - I do not really have a name for them, but they are essentially achievements for collecting a specific group of achievements (some game systems refer to these as Accolades). When I was a boy scout in Canada, this was similar to our star system (collect a number of badges, as well as complete camps, and demonstrate additional proficiencies, and you are awarded a specific star - total of 5). My motivation for exploring this in my system is - like most things - to motivate training along different avenues. For example, when an athlete achieves a real world kind of achievement (i.e. making 1st varsity, racing in Boston, going to Nationals, winning an indoor race), my suspicion is that they will be more likely to endeavour to finish other achievements associated to the same accolade achievement. I am also exploring the psychology of revealing the accolades on progress bars once they have attained one of them. I have encountered some strong research indicating this approach has very significant effect on the user. The rationale is that the user 1) feels tasked to complete the accolade as one would complete an assignment they were explicitly, and 2) instills a sense of urgency to accomplish the accolade as they feel knowledge is sensitive. My hope is that I can use this system to help athletes also distribute their focus across multiple avenues of training. Someone who always lifts will lack endurance; someone who only does cardio will lack power. I hope the accolades can promote roundedness of training. In truth, these accolades are easier to program, and there benefit helps extend the efficacy of other achievements with a minimal investment of time. So, if I can use Real World badges to promote time-investment into other badges (and thus training), real world badges definitely have a use. Here is a link to one of the athlete's achievement progress page as is: http://www.illinirowing.com/wm_progress.php?user=billyhrabak

3. You have nailed the whole system on the head. This is the approach I intend to take when marketing the system to users outside of the team: this is not a workout tracking system, it is a workout motivating system. Coaches can mine the data, and I am sure Troop leaders could do the same with yours, but this is not supposed to be a spreadsheet replacement. The true value comes in the motivating effect on athletes. The secondary goal is to then to get the athletes intrinsically motivated. I try to accomplish this by giving them tools to analyze their own progress - essentially, make them researchers of their own performance. This is a lot harder, but the effect is coming out in my 4 year rowers. These guys were the ones who used to beg for new achievements, and monitor their progress towards the next badge for their profile - now, they are the ones who really getting into the analytics of training. They are the ones scrubbing through their workout history to see where/when they have been getting hung up, and are modifying their training to avoid prior pitfalls - they are basically coaching themselves. They are also the ones now asking me for new tools, different charts, improved search functions etc because their performance is what drives them. Many of the systems I have implemented a simple (displaying the distance they have rowed over time, showing them how their power has increased over the year and between years, providing a breakdown of their workout focus e.g. cardio, power, strength endurance). For your system, it may be as simple as displaying volunteer hours over time and comparing that month-to-month or year-to-year. The achievements are the gateway to getting them using the system - once they are using it, they will begin to notice the tools (specifically once they fill out with data) and will start setting their own goals. The knowledge that you did more in the past is a damn good motivator to better yourself in the future.

4. Again, the system takes rather little to get going, and it will quickly evolve. Keeping up with it is the hardest part of the job. The good news is that the evolution is mostly a refinement process. New people who come into the system will benefit from the pathways etched by previous users - there is little concern that the system outgrows the people coming in at the bottom. I would highly advocate building in a lot of tools to monitor the progress. I have been caught unaware several times when people hit certain landmarks that I did not anticipate them to hit (someone broke through 1,000,000 meters rowed before I even realized they were on it). This meant, I had to move quick to start setting up the subsequent goals which did not until that point even exist.

You thesis options all sound really cool. I have been doing quite a bit of research on human driven databases and honestly their is a profound lack of scholarly work on it. Data mining has been heralded as one of the big new jobs of the 21st century, but academia has hardly kept up with the theories needed to underpin it. I admit myself, I have been more interested in getting athletes motivate to row faster, but now I look back at my system and 1) wondering how I can mine the data for more information, and 2) wondering how much valuable data went unrecorded. I am definitely going to scour through the material you forwarded.

Anyway, this topic is something I am constantly thinking about."

You make an point interesting - the notion of "authority" in building such a system. I'm actually a little surprised that people have taken it as seriously knowing that you conjured up the watermarks yourself (not to be insulting, just that there is a level of arbitrary definition there). Granted its great that you have that level of control. (Note: in reference to my plans to virtually gamify the BSA, a group with inherent gamification)