FNR – BubbleBoy Behaviour AI

Posted by Erin, the RobotGrrl on Monday, July 12th, 2010

This week and this Friday Night Robotics I was working on a behaviour AI for the newly refurbished BubbleBoy! It is much easier to design an AI that you know will be interactable with thew orld! Without the headbobbing capability of BubbleBoy, this effort would not be worth it.

BubbleBoy’s behaviour is primarily focused on food and water. BubbleBoy lives its anthropomorphized life just be be fed/watered! This means that BubbleBoy will want to know when it will expect to be fed, so that way it can headbob at the most optimal times. Being fed/watered is having a button pressed on BubbleBoy’s green stage area (the part with the blue and white LCD).

BubbleBoy will have three sensors, and a rudimentary measure time. The three sensors are the LDR, lid switch, and Xbee. All of these are onboard the robot (except for the Xbee which isn’t implemented yet).

Here’s a broad flowchart of the behaviour, which I will then explain below in detail.

1. Creating the Expectation

This is the observatory phase. BubbleBoy initially does not have any expectation of when to be fed, so it waits around. While it is waiting, its collecting data from all of the sensors and storing them to an array. There has to be 10 numbers in the array for each sensor before BubbleBoy can proceed to the next stage, pattern finding. Once this is fulfilled, and if BubbleBoy is fed/watered, then it goes on to find a pattern.

2. Pattern Finding

BubbleBoy is seen as a simple robot. Thereby, its pattern finding is relatively simple as well. The main idea is to check each sensor’s array and see if there is a pattern within the residuals. Meaning, if looking for a sequential pattern, the array would be iterated through (starting at i=0, stopping after i=8), and i+1 would be subtracted from i. An average of the residual change would be calculated at the same time. The array would be iterated through again, this time to count how many items are within +- 10% of the residual average. If the count is above 7, then it is said that there is a sequential pattern in there.

The same process is done for a secondary pattern, and a ‘thirdary’ pattern. Meaning, every 2nd number and 3rd number is checked to see if there is a pattern. It also goes through and checks with offsets, just incase the pattern is “even/odd/whatever”.

If there are no patterns, a random primary sensor is chosen for BubbleBoy to work with.

A problem exists in determining which pattern for which sensor to trust the most. A thirdary pattern for a photosensor is less dependable than a sequential pattern for a lid switch. This is handled in the next step.

3. Determining the Sensor to Use

The sensor with whatever pattern to follow is chosen through a Bayes Filter. This allows for a simple specification of confidence levels for a given sense resulting in a particular state through a table where all rows add up to 1.0. The sense columns are the sensor and pattern type. Meaning, there’s three sensors for every sensor, giving 9 senses in total. The states are confidence intervals. Anything >= 85 would be considered the most confident.



The numbers in bold are the ones that are more probable to be a result. A sensor in particular to look at is the Xbee for a thirdary pattern. The numbers are dispersed in such a way that it is almost a bimodal distribution. In order to understand why this is, imagine the scenario in real:

Xbee modems can communicate for up to a few meters. If BubbleBoy only receives a message every 3rd instance, it could either be a clever pattern, or it could be a miscommunication.

For this reason, the largest probability is given to the least confident state, and the second largest probability is given to the most confident state. It depends on the Bayes Filter and random roulette for what will actually be chosen.

This is just for determining what sensor is the most confident, that will provide the best result. This can mean that the chosen sensor will be followed, but it also may not. This will be explained in the next step.

4. Calculating the Cost Adjustment

The adjusted cost is determined by using the result from the previous step, and multiplying the probability by the inverse of whatever column it was situated in. In simpler terms, if the result was found in the best confidence interval state, then it would be multiplied by 5. If it was in ( 85, 70 ], it would be multiplied by 4…

This is then used as the sense for the next Bayes Filter. The state is how much of a cost to reduce the sense by.



Once the cost adjustment amount is determined, it is then subtracted from the cost of the particular sense’s cell. All of the senses are in a grid, with the most “primary” being the closest to BubbleBoy. An A* search is then used to choose the closest and least cost sensor. Once this is done, BubbleBoy can get on to entertaining its audience while waiting for food/water!

5. Following the Expectation

This is the main loop of the program. It’s where BubbleBoy is thinking in the present time about if it will be fed or not!

The key idea here is that BubbleBoy is thinking in the now. Meaning, it tracks the patterns differently than it does when it is reflecting back on them (in a “past” thought).

Sensor data is retrieved and placed into an array of size 10. The data at i is checked wither it is within +- 10% of the average in the pattern. The threshold percentage amount differs for the type of pattern, where secondary would be +- 20%, and thirdary would be +- 30%. If the data fits in to the check, then a “yes” counter is incremented.

Once the “yes” counter is >= 6, the food level will begin to decrease by (i/2)^2. At the same time, BubbleBoy will begin to show signs that he is excited to be fed/watered soon, by spinning its hat and bobbing its head.

If the “no” counter is >= 6, then it means that the expectation isn’t really working in the present thought. A flag is set to redo the expectation once BubbleBoy receives food and water.

i is then either incremented or reset to 0, depending if it hit 9 or not. (That’s a sort of obvious step)

6. Real-World Behaviors

When the time elapsed from not receiving food exceeds 150% of that of the observed elapsed time, BubbleBoy goes in to a “wallow” mode. When BubbleBoy is wallowing, it spins its hat slowly, and bobs rarely.

If the time elapsed is ( 150%, 100% ], BubbleBoy is “angry” because it did not receive its food exactly before the time elapsed. The hat will not spin, and BubbleBoy will bob side to side, and once (quickly) in the opposite axis to simulate a sort of “twitching” to all this anger!

If the time elapsed is ( 100%, 85% ], BubbleBoy is eager to please. Hat tricks will be common, same as delightful bobbing. Depending on how much food/water BubbleBoy has, it may also hoola hoop!

7. Last part of Following the Expectation

Depending on when BubbleBoy was fed, if it was in ( whatever, 100 ], the expectation will be done. Essentially, BubbleBoy is a positive/eager thinker that believes it should always be fed before the elapsed observed time. What an attitude!

If it is in [ 85, 100 ), then the expectation will be kept.

To reformulate the expectation, the previous steps are executed on the collected data. Then, everything repeats!

What will be super interesting to see, in my opinion, will be when the discrepancies occur from past thought to present thought. It will be interesting to see which sensors fare better through that transformation.

It will also be interesting to see if this actually can work on an Arduino, and not in simulation. I created a simulated version of BubbleBoy in Processing earlier in the week.

I’ll do the initial coding and testing through this simulation, mainly because I already coded the Bayes Filter Algorithm (with random roulette) in Processing from 2009 Honors Summer Research. Plus, in Processing it is very simple to communicate to an Arduino through Firmata. I can read in the data from BubbleBoy through there.

Hopefully next Friday I will have devised a test sequence to test the soon-to-be-coded AI on. This will of course be Open Source, under the Attribution-NonCommercial-ShareAlike 3.0 Unported License (BY-NC-SA).

Let me know what you think of this AI in the comment section below! (And yes I know it is very linear, but BubbleBoy doesn’t have enough DOF in the real world to spend the effort making the AI more nonlinear, since the observed result will essentially be the same!)

Posted in: Programming, Projects, Robot.

January Happenings

Posted by Erin, the RobotGrrl on Monday, February 1st, 2010

What has happened in January? Tons of stuff!

For Matlab this semester, it’s an independent project. I’m working with a friend to implement an adaptive online SLAM algorithm for an iRobot with a CMU cam and ultrasonic sensor. We want it to be able to reach a goal location even if objects are placed in front of it. I’ll be blogging more about this later, though.

The Social Robotics software that I worked on over the summer is now released under the GPLv3 license. I encourage everyone to check out the Social Robotics page if you want to learn more about the project! I am still in the process of creating the documentation and commenting for the code. As soon as it is complete, I will make a blog post. =)

Luckily for me, I took time to make detailed daily and weekly summaries. This will help a lot, plus it’s always neat to look back and see what the difficult parts were.

Did you hear/watch this year’s FIRST game animation? The game is about soccer! Team 229 has many useful links on their webpage that can fill you in.

This year I’m helping out with the website, maybe I will get to help out with some AI coding for the autonomous mode later on. It all depends on what the high school students think up!

I ended up adding a class two hours before the first lecture- Applied Statistics I. I don’t enjoy statistics very much since I have horrible memories of it from Math 536. But, once I gained access to view the class on the gradebook software, I immediately noticed two words:

SecondLife ……………… Project

Is this for real!?!?! It turned out that it is, and it is awesome! A friend and I are working on trying to figure out if there is a correlation between the virtual economy and the real economy. We’re going to focus mainly on North & South America, Europe and Australia.

Here’s a screenshot of my professor in SecondLife!

I’m taking a class on Computer Graphics. It’s really neat– I’m learning OpenGL!

OpenGL is something that I’ve wanted to learn for a while now. It’s actually quite simple when you’re given a template to work with!

Above is the first homework assignment! We were given a lot of time with it, which allowed me to play around with the code. I have to make the colours more plain before I hand it in, though.

I have no idea what I want to make with OpenGL at the moment. Maybe a moving robot? I definitely want to make some sort of game, though. (That way I can sell it on the iPhone App Store!)

That’s all for now. I’ll be blogging more about the Matlab project, since I think it’s going to be a hit!

Posted in: Animation, Art, Programming, Projects, Robot, School.

Humanoid Walking -> Easier with AI?

Posted by Erin, the RobotGrrl on Wednesday, August 5th, 2009

I have been trying for AGES now to make my humanoid walk!

The main difficulties are:
- a really good walking motion would have many frames in it
- trying to program/tweak with a robot wriggling around is tricky
- the balance changes depending on the time, wires, center of mass… so many things!

Maybe it is easier for the robot to teach itself how to walk!

I’m thinking in order to do that I would need:
- Encoders* on each DOF (* I may not need encoders if these servos do provide feedback based on location, have to check)
- AI algorithm(s)
- Data logging
- Long battery life
- Patience

Creating the AI algorithms would be fun, but it would just take a lot of trial an error. In this scenario, the robot would “find” many loopholes in the algorithm. Not only would the goal have to be precisely defined, but so would the constraints.

However, at the same time I’m wondering if there is something that I am missing when trying to create the walking motion. Check out how simple it looks on the Nao:

I’m going to try again tomorrow, but use the knee servos more. Maybe the trick is to do it like this:
Move center of mass (right/left)
Lift knee (left/right)
Move leg SLIGHTLY forward (left/right)
Lower knee (left/right)
Move center of mass (left/right)
and repeat.

I hope that one will work.

In other news, I finished another iPhone app, and it is currently in the review process. I won’t spoil too much, but it makes a *ding dong* noise.

In more other news, my summer research was completed a few weeks ago and I am currently finishing the final report. I will post more details on this when it is all complete.

In even more other news, I have been playing with the gyro sensor, and I am planning to make some sort of motion for MANOI with it. However, I have to make MANOI walk first

Posted in: MANOI, Programming, Projects, Robot.

Expelliarmus = Bayes Filter Algorithm

Posted by Erin, the RobotGrrl on Saturday, June 20th, 2009

In the final Harry Potter book, it is mentioned quite often that HP depends on this simple yet extremely functional spell called ‘Expelliarmus’.

I often find myself comparing Expelliarmus in the wizarding world to the Bayes Filter Algorithm in my world. It is such a simple method to give your program a little AI.

Before I go on any further, I just want to say that I didn’t invent the Bayes Filter Algorithm, I found it in this book. It is also said in this powerpoint. Additionally, for good measure, here’s a citation:
The Bayes Filter Algorithm I am using is based on the one in the book Probablistic Robotics. (Thrun, Sebastien, Wolfram Burgard, and Dieter Fox. Probablistic Robotics. Cambridge: The MIT Press, 2006.)
OK, now I think I have all of my bases covered.

So, the Bayes Filter algorithm uses Bayes Law, some initial and conditional probabilities, and the actual probabilities, to give you the final probability.

The first thing that happens is that the prior belief is calculated by adding up the multiplication of the conditional probability table and the initial beliefs.

After that, you add up the multiplication of the probability table and the prior belief.

You then find the normalizer by taking that summation and inversing it.

FINALLY, you can get the final belief when you multiply the normalizer and the number where you multiplied the probability table and the prior belief.

If you want, you can even take it a step further by putting log odds to it based on the final belief and the prior belief.

Here is a class I created in Java (in Processing) that is the Bayes Filter Algorithm:

The Bayes Filter Algorithm is super handy because you can check it by hand. There are no ambiguities introduced into the algorithm. Some of the algorithms, like the Kalman Filter, use covariance in the algorithm to adjust the final belief. This is great, but definitely more tricky to calculate by hand.

I favour the Bayes Filter Algorithm right now because of its ease of use, and it gives me what I want. However, for more interesting results, the Kalman Filter would be a better way to go. If I have time at the end of this project, I’ll probably implement the Kalman Filter and use it instead of the Bayes Filter Algorithm.

Posted in: Programming, Projects.

Hockey MANOI – a summary

Posted by Erin, the RobotGrrl on Saturday, January 3rd, 2009

I made a pretty good summary of Hockey MANOI in this thread at Trossen Robotics forum. I figured I should post it on my blog because it is a good summary and I broke it down into easy to read parts, so it’s like a whole JOURNEY of awesome!

Hello TRC World!

My project is a hockey playing humanoid.

It uses a MANOI AT01 kit, controlled by an Arduino (with an ATmega328) with a Wave Shield, and a SSC-32.

Humanoids have always been associated with walking or running. This project focuses on a different action for humanoids, skating. The end result of this project yields an interesting vision of the possibilities of skating robots.

Introduction

The idea came around when I was trying to make my humanoid (MANOI AT01) walk. Instead of taking a “big bite”, I decided to take a smaller bite and make it slide its feet. However, I quickly realized that when it slides its feet, it looks exactly like a newbie Canadian hockey player!

A Canadian newbie hockey player begins to skate by almost walking. Although the skater doesn’t go far, he does move forward due to the friction between the blade of the skate and the ice.

When I did notice this, I quickly grabbed some lego to create MANOI’s own version of skates, which are similar to rollerskates. I mounted the lego onto the feet using velcro.

The hockey stick I just found laying around. I had to cut a bit of it off, as it was too tall. I use tape and tie wraps to keep it mounted to MANOI.

Development

Development on this project was mainly trial and error. To get all of the motions correct so they all balance together was critical.

Instead of using the controller board that is usually used to control the MANOI, I did a major transplant and substituted it for the Arduino and SSC-32. This allows me to have much more flexibility in terms of sensing and creating motions. The H2H software was too problematic.

Usually what would happen is I would draw out a motion, on paper, that I would want to create, and I would put it into MANOI. Sometimes I got it first try, other times I didn’t. However, the cool part is that a lot of the motions stemmed from the ideas of other motions.

For instance, in the video of MANOI Skating with music (seen below), the motion where MANOI is running is actually a faster version of the sway motion! That was really surprising.

The development for the Wii nunchuck part of the code was quite easy as I had already established all of the variables and settings of when the nunchuck is tilted left or right, forwards or backwards. Instead of using real numbers for it though, I just defined a “home position” of the nunchuck, and subtracted or added numbers to the accelerometer axis, x y and z.

The music part of the wave shield was quite fun and straight forward. I looked around for the songs, and put them on a SD card which plugs into the wave shield. From there, it was just a simple method call inside of the Arduino.

Once the above developments were done, I wanted to create a version of MANOI that could sense if a ball/puck/object was there. I did this by using LDRs and LEDs.

Rest assured, I would have used IR Sensors if I had any This was the best alternative I had, though!

On the left side of the sticks the LEDs are in a yellow casing, and on the right side they are in a clear casing. There is some effect on the reading, however their values change precisely the same when an object is in front of the stick.

The black construction paper enclosure around the LDRs was required to direct the reading. Otherwise, the light from the LEDs saturated the reading and no difference was seen when an object was present or not.

I observed the change between the readings of when there was an object present, and when there was not an object present.

From this, I created a simple neuron, where if the input values succeed a predefined threshold, it will perform an action. In this case, the action would be to shoot the object.

I had to tweak the threshold a little to make it work with smaller objects, such as a roll of electrical tape.

In the video below, you will observe that it does work with a roll of electrical tape, a ball, and a spool of lead solder (the LEAD solder isn’t mine, it’s my DAD’s because he can’t use non-lead solder like the rest of us -_-;). The spool is white, which proves that the theory does work, meaning that the light that is reflected from the LEDs back into the same LDR board does not obscure the readings.

Results

Here are the videos that you can look at!

This is the first video, where I was just getting the motions down.

The program is basically a sequence of movements:

- Forward 6 times
- Shoot 3 times
- Backward 6 times
- Shoot 3 times

As you can see, the forward and backward movements both result in MANOI moving backwards. As I later found out, through trial and error, it was due to the Arduino and power cords limiting the movement of MANOI! Once they were mounted properly, it worked much better.

This is the second video, where MANOI is controlled by the Wii nunchuck!

You press Z to shoot, and you hold C and tilt to move it. MANOI can move forwards, backwards, left, right, and home.

This is there third video, where MANOI is playing a little game of hockey by himself while listening to some music. The song that you first hear is the Hockey Night in Canada theme song!

This is the last video, where MANOI can autonomously decide if he should shoot or not.

(The quality in that video is quite yucky, please check out the video on [URL="http://vimeo.com/2641041?pg=embed&sec=2641041"]vimeo[/URL] if you’d like to see it in better quality)

Conclusion

In conclusion, this project was SO much fun! The only time I didn’t enjoy it was when I was trying to hold MANOI, who was whacking me with its stick, with one hand and trying to type in some code with my other hand.

My favourite part was watching people play with the Wii version of the code. They really enjoyed it!

I also liked making the AI part too, that was pretty fun.

Next Steps

The next steps would include coding a modified version of the Bayes filter algorithm to predict if an object is in front of the stick or not.

More sensors would be fun, like three proximity sensors mounted on the front, left and right. This way MANOI could avoid opponents trying to take the ball off of it.

I would also add two more servos in the leg that would allow rotation. This would then allow me to create a more realistic skating humanoid, where there would actually be a stride.

Perhaps I could also add a camera to the head so that it could track where the ball/puck/object is.

That’s my project, I hope you like it! ^_^
You can see more of everything I mentioned at robotgrrl.com

Posted in: MANOI, Programming, Projects, Robot.