• How Skinner Box – Operand Conditioning helps in designing Reward Schedules

 

 

Professor B.F. skinner is a renowned psychologist known for his contribution to Behavioral psychology. B.F. Skinner is often referred as the “Father of Operand Conditioning”. Operand Conditioning was studied, documented and detailed by B.F. Skinner. Operand Conditioning is a method used for shaping behavior by reinforcement and punishment. Behaviors which are reinforced have higher probability of occurrence whereas behaviors which are punished have lesser probability of occurrence.

Skinner Boxes is one overly abused psychological concepts which are present in games around the world which helps game designers in anticipating player decisions and encourage habit formation in players. Operand Condition and use of skinner boxes is staple practice in Free-To-Play game design. It is a universal practice, just pick up any top grosser and you will find the presence of this concept which is used to drive player motivation and designing reward schedule systems.

Reinforcement and Punishment are the basis of driving repetitive behavior in the desired direction by constantly rewarding the player based on unique & specific reward schedules.

 

Kinds of Reinforcement and Punishment

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Okay, Let us now list the definitions and elaborate further with clear and concise examples:-

 

• Positive

  • +VE=ADDING stimulus which can either be pleasant or unpleasant.+VE does not necessarily means it is suppose to be enjoyable

• Negative

  • -VE=SUBTRACTING stimulus which can either be pleasant or unpleasant.

  • -VE does not necessarily means it is suppose to be enjoyable.

• Reinforcement

  • An action which is caused by a PLEASANT stimulus which focuses on INCREASING or DECREASING the occurrence of the habit and promotes habit formation.

• Punishment:

  • An action which is caused by an UNPLEASANT stimulus which promotes habit formation. An action is performed to avoid consequences

 

 

The Combinations:-

 

 

• Positive Reinforcement

  • A PLEASANT stimulus is ADDED

  • Chances of behavior increase

  • For Example: Rewarding the player for Daily Logins, Rewards based on performance, Reward based on completing a task, “You just collected all the 3 stars! Here are some coins for you!”

 

• Negative Reinforcement

  • An UNPLEASANT stimulus is SUBTRACTED/removed

  • Chances of behavior increase

  • For Example: Crops wither in Farmville 2 if you do not visit them frequently to water them. If you fail to harvest them you lose your investment. To avoid this step from hindering your progress you need to water them more frequently and visit the farm regularly. Therefore the player is afraid of the unpleasant stimulus i.e. crops withering overtime which conditions the player to visit the farm and water the crops. The chance of behavior is increasing (not decreasing). It is essential to not confuse it with –VE Punishment because the chance of behavior decrease!

 

• Positive Punishment

  • An UNPLEASANT stimulus is ADDED

  • Chances of Behavior decrease

  • For Example: “If I spam the punch action, I might run out of stamina therefore I will slow down which will make me more vulnerable to enemy attacks” In the following case, the player will avoid spamming the punch action because of fear of being punished. The player is trying to avoid an unpleasant stimulus i.e. the damage dealt from enemies.

 

• Negative Punishment

  • A PLEASANT stimulus is SUBTRACTED/removed

  • Chances of Behavior decrease.

  • For Example: My chocolate will be taken away if I rant about a colleague. Chocolate in the present case is the stimulus which forces me to rant less. Chances of ranting will decrease because of my love for chocolates. I do not want to lose such a divine creation of god! Hmmmm….. Tasty!! I do not want this stimulus to be taken away from me.  

 

 

Reinforcement and Punishment are ways to condition behavior but it has nothing to do with it's effectiveness. Now the ever glorious reward schedules come into action. It is the reward schedules which further strengthens the effectiveness of a behavior. They just want to juice out all the dopamine!

 

There exits 4 major reward schedules which are used rampantly in F2P games. This is the textbook theory that makes the player glued to games and keeps them coming back constantly. The 4 reward schedules are as follows

 

 

1. Fixed Ratio

2. Fixed Interval

3. Variable Interval

4. Variable Ratio

 

 

• Ratio

  • Ratio here implies an ACTION

• Interval

  • Interval here implies an action over TIME

• Fixed

  • Fixed here implies SET no. of responses

• Variable

  • Variable here implies UNPREDICTABLE amount of responses

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1. Fixed Ratio 

 

Fixed Ratio Schedule is mostly used for defining straight forward objectives and quests. For Instance; Kill X number of zombies and gain 20 energy points, Plant 20 Crops and gain XP & Coins, Levelling up. The player can anticipate the reward because the game communicates both the objective and reward up front. Open loops [Sub-Objectives] are also an example of Fixed Ratio Schedules.

 

Game Example: Hungry Shark

 

The player will be rewarded with 125 coins if he/she eats 4 turtles.
Fixed / Set no of Responses to be performed by the player:  Collecting 4 Turtles

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2. Fixed interval

 

Rewards are offered to the player after the player has performed meaningful actions after/within fixed amount of time has passed. For Instance: Daily login bonuses, Energy points which replenish over time, Crops wither if you do not check them.

 

Game Examples: Hungry Shark, Cut the Rope 2, Chicken Boy

 

In Hungry Shark, the player is rewarded for daily login. The player has to login each day to acquire the reward. The interval factor is set i.e. each day.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In Cut The Rope 2. The player gains 1 sun power every 5 minutes.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In Chicken boy, the player gains 1 Heart (energy system) every 5 minutes

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

3. Variable Ratio

 

Reinforcement rewards the player based on random number of actions. The player expects a rewards but the player does not know when he/she will receive the reward; after 1 attempt or after 100 attempts. According to studies and research, Variable Ratio is the most effective schedule to keep the players motivated. The player keeps repeating the behavior until the reward is achieved. For Example: Gambling, Lottery Tickets, Casino Slot machines, Daily Spin in games.

 

Game Examples: Jetpack Joyride

 

The player acquires spin token which enables the player to access the machine. Not gaining a reward is a failure for  the player which the brain registers as a punishment.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4. Variable Interval

 

Reinforcement rewards the player actions over random time intervals which are not known to the player .The player is expecting a reward but does not know the point at which the reward will occur. For Instance: If you return to a game after days or even months, you are showered with gifts, coins and consumables. Identifying the pattern is impossible.

 

Game Examples: Cut The Rope 2

 

Balloons received as bonus. I wasn’t expecting it thankfully Variable Interval was implemented which saved my day.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In a nustshell

 

• Reward schedules are also referred as “Appointment Mechanics” which helps in increasing the retention rate.

• Reward Schedules are tied to wait loops in the game which defines duration of a Session (Sessioning)

• Cohort factor will increase. Player would visit the game more often.

• Reward Schedules are effective in introducing and teaching game mechanics because a reward is tied to actions in the loop.

• Strong audio and visual cues further strengthens the efficacy of the reward schedules.

 

 

 

Key Concepts:-

 

Loss Aversion, Operand Conditioning, Skinner box, Reward Schedules, Classical Conditioning, Intrinsic and Extrinsic Motivation, Instant Vs. Delayed Gratification, Appointment Mechanics, Return Trigger, Retention, Cohort