My Unique Clicker Approach

[Michael]

My Unique Clicker Approach

Some people have been noticing my non-typical use of the clicker for training so I wanted to explain it further. In this article I explain how I have transformed the clicker from a bridge to a secondary reinforcer, how this allows me to practice more tricks, how this improves performance, and how I can clicker train two parrots in the same session.

[Andromeda]

Thank you for writing about how you use the clicker as a secondary reinforcer. Before I found your videos and your blog I had read two books on clicker training birds and they both said you should fade the clicker after the trick has been learned. At some point early last year I asked you why you were still using it on established tricks and you said it was because you had made it a secondary reinforcer. That inspired me to want to use it that way in my birds.

After months of using a one click to one treat ratio and after they each knew several tricks I started chaining behaviors together and rewarding on a variable ratio. I would click for correct behavior but would only deliver a treat every 3 - 6 clicks.

I don't think I've "ruined" the clicker with my birds at all using this method. The reason why is because when I use it to teach new tricks (rewarding each time while they are learning) my birds still catch on very quickly so the clicker obviously still has meaning. Like you I believe the clicker, when used your way, helps to motivate the bird when you are using a variable ratio of reinforcement and lets the bird know that at some point it will get a treat because it is doing the right things.

I have observed a behavior in my Poi that helps support the idea that the effectiveness of the clicker has not been ruined but rather has actually been improved. When I cue a trick, if he gets a click, he will wait to see if I'm either going to cue another trick or deliver a treat. If he does not get a click (because, for example, he did a sub-par wing stretch when I cued "wings") he will often immediately repeat the cued behavior before I even have a chance to cue it again. If I don't cue a trick he doesn't get a reward; however, his eagerness to offer the trick again so quickly if he doesn't hear a click as opposed to just waiting really makes me believe the clicker is working very well as a secondary reinforcer.

I also train two birds at the same time with no problem. It has not created confusion with them or ruined the clicker in any way.

[Michael]

I'm thrilled that others have been able to make use of my special technique. I didn't exactly sit there and come up with it. It just sort of happened along the way as I was training Kili. No one else does it this way and gets criticized a lot but it sure as heck works. I'm glad it works equally effectively for others as well. My biggest use for it is for extensive VR exercise recall flights.

Some people complain that using a clicker is too much work or fuss and they want to phase it out. But for me, it's just the easiest thing. If the bird will perform 9 extra tricks or fly a few extra times just for a click (instead of filling up on treats or me having to do anything drastic), that's some really effective and easy parrot training right there! I do try to give extra praise and attention when no treats are offered... but I'm sure on the 50th flight recall or trick practice that attention isn't worth much either. Which is why I rely so much on the clicker.

I didn't mention it in the article, but it's good to practice stuff with your parrot without a clicker on occasion as well. Typically this happens inadvertently when you forgot to use it, or you have company over, or you're outside. But if it doesn't doing some stuff a few times a month without a clicker is a great additional bit of training. This way when you do want to show off for friends or don't have one handy, your bird will still respond. However, for the monotonous day to day training where you just need the bird to practice their stuff and exercise many times, my clicker strategy works best!

[Wiki]

Thank you for another well-considered article, Michael!

To me, the approach of secondary reinforcer and variable rewards seems quite intuitive. I've used this approach with my budgie at times and it works a treat. It's certainly not changed the effectiveness of the clicker at all. I use it for repetitive iterations of a trick - flight recalls and targeting (variable), putting multiple items in a container etc.

Karen Pryor's FAQ on clicker training says:

Do clickers and treats need to be used for every behavior, forever?
No. Once a behavior is learned and on cue, there’s usually no need to click, as the animal understands the behavior. Clicker trainers can maintain the behavior by replacing specially good treats with occasional and less intensive rewards including a pat or praise. Learned cues and behaviors are also maintained by real-life rewards: for example sitting quietly at the door is rewarded by opening the door so that the dog can have a walk. Clicker trainers then save clicks and treats for the next new thing they want to train.

Even when I don't explicitly reward, the click and the attention that goes with it must let Wiki feel rewarded enough to continue, and that's the purpose of the reward. I think our training buddies enjoy the interaction and attention and challenge, and that's partly their "real-life reward". We practice without a clicker on occasion, too - it's how I know whether he really enjoys doing something or not!

I think we shouldn't underestimate the capacity for understanding in our birds, and their ability to give a more subtle meaning to the clicker than just a 1:1 relationship of "click = food". There are many "old school" trainers who work with multiple performing birds and do not use clickers or even rewards when performing (yes, I even get given a hard time by them for "needing" a clicker in the first place), so I don't think we should be too precious about what is in essence a communication tool.

[Pralina]

that's interesting, because as a beginner in training after reading many many books, as well as Steve Martin's and Susan Friedman's theories on their respective websites, I did get convinced that one-click / one-treat ratio is the best way to train.... according to their teachings...

I am a bit confused right now

[Michael]

There's nothing to be confused about. If you're a beginner or haven't had success with extensive clicker training, stick to the one click one treat method. This article is about taking things a step further for people who've made a lot of progress and want to take things even further.

[KaratParrot]

Um, Micheal I believe you have that wrong. Steve Martin and Dr. Friedman don't advocate this for a reason. It's not that the clicker doesn't become a reinforcer and they certainly encourage to use VR when a behavior is well established. This is because we know that if Continuous Reinforcement (CR) is always used behavior breaks down!

So my question to you is "Are you really using VR reinforcement when clicking every time and only providing treats some of the time?"

I want you to really think about what you are doing when the behavior occurs and tell me why you think this is VR.

[rebcart]

This is because we know that if Continuous Reinforcement (CR) is always used behavior breaks down!

Are you sure? Because I think the most experienced animal trainers in the world, the Baileys, would disagree with you.

[KaratParrot]

Hey rebcart! I'm glad you brought that up! 

The Bailys are great trainers that's for sure, but they may be forgetting a little something. In that link they discuss seeing behavior of their animals breaking down when they switch from a CFR (my mistake labeling it CR! Not to be confused with Conditioned Response!) to a VR which is one reason given for not using VR. But this is well explained by Russell A. Powell, Diane G. Symbaluk, Suzanne E. MacDonald, and P. Lynne Honey in Introduction to Learning and Behavior p. 261. If the trainer expects the animal to make too many responses, without moving at the animal's pace, it is known as ratio strain. And this could well be what has happened. 

Behaviors break down on CRF if the Establishing Operations (EO) are not changed to make the reinforcement (in this case food) more valuable. For the classic slot machine example, what would happen if a small amount of money always came out?  Not as many people would play, it's not fun! And when they do play it stops when saitation hits. But what is interesting to note is what would likely happen due to different EOs. People with less money play longer and people with more money play shorter. This goes the same with birds. Hungrier birds will still train more using CRF than satiated birds. But now the bird does tricks because he has to, not necessarily because it's fun. 

Yes I am sure behavior breaks down on a CRF. People forget to consider the EOs when saying CRF works to maintain behavior.

[Michael]

I took a look at the article and see where it is coming from about VR not being necessary for the basic dog owner teaching tricks to their dog. There is a substantial difference when it comes to parrots that makes me disagree that VR isn't necessary for most pet owners. Parrots are naturally wild animals so except for the positive reinforcement taming and training that we give them, they are difficult to handle. In order to get our parrots to step up reliably, come out of the cage, flight recall to us, not bite us, etc we cannot keep giving treats every single time or the parrot will be stuffed with treats in the first 5 minutes it spends with us. The more good behavior that we can move to a diluted variable ratio reinforcement schedule, the more domesticated the parrot's behavior appears to be.

As for the clicker, yes I use the clicker on a continuous reinforcement schedule. I also give praise and attention on a pretty continuous reinforcement schedule. These are mild reinforcers and mostly secondary. The treats - which are the predominant thing the parrot is working for - are on a variable ratio reinforecement schedule. Thus I am successfully playing the best of both worlds where I can differentiate good from bad demonstration of behavior in a long chain or sequence yet get the parrot to complete all of that behavior for infrequent food rewards.

For example I can execute the following sequence with the following outcomes:
cue/response/reinforcement
wave/good/click
shake/good/click
wings/poor/no click
wave/good/click
wings/better/click
turn around/incomplete/no click
wings/good/click & treat
etc

In this manner the parrot learns success from failure throughout the sequence of behaviors (note they are not a chain because they are performed in random order and not repeated this way again) and continues trying for the sake of earning the more desirable food rewards on that variable ratio schedule. I have found this approach to be the most effective way of getting the benefits of continuous and variable ratio reinforcement schedules.

A lot of "domestication" training for our parrots ends up inadvertently being reinforced on a variable ratio. For example stepping up is occasionally rewarded with a chance to do tricks and earn treats. Other times it does nothing and other times it gets a meal and other times it gets a pesky grooming. Since it is generally rewarding but on a dilute VR, it ends up being very successful in the long run.

My approach is one I happened upon myself and not something I ever saw recommended anywhere. I am sharing it based on my own tremendous success with it. I have found the ability to train more behavior, more effectively, and maintain longer motivation using this technique than CFR or VR alone. As a side effect it has also taught my parrot to try harder when training new behavior. Sometimes the behavior I am soliciting in training something new just isn't click-worthy for a while. Since my parrot is used to clicks confirming all correct behaviors, it makes her realize she is doing things wrong and makes her keep trying to get it right. Better yet, since she is so accustomed to training in this manner, she even varies the behaviors she offers until it's right enough to earn a click. It may be a more difficult concept to achieve off the bat but it's been working phenomenally in the long run.