"3 Minute Thesis" Competition Presentation

I presented at the "3 Minute Thesis" Competition hosted by the Office of Graduate Studies at Saint Louis University on November 30, 2018.  Unfortunately, my presentation exceeded the 3 minute time limit by 1 second, and I was disqualified!  Here is the transcript of my presentation.

Have you ever watched a YouTube video of a cat hitching a ride on a Roomba robot vacuum cleaner and wondered if the Roomba is actually able to perform its job properly while it has a cat sitting on its back?  I blame my thesis research for the fact that I've thought about these things...

Robots find their way around a room by using sensors to perceive the world in front of them, and reconcile this view against a map of the environment to calculate their current position in the environment.  The Roomba needs an estimate of its current location in the environment so that it can plan a driving route that will help it achieve its goal of vacuuming under the couch.  This location estimate is like a "You Are Here" sticker on the map at a shopping mall.  Abnormal driving events like a cat sitting on a Roomba or a person picking up a robot and putting it down somewhere else have the same effect as if the "You Are Here" sticker were moved to some completely wrong location on the shopping mall map.  With an incorrect location estimate, the robot becomes unable to plan a path that will allow it to reach its goal.  These abnormal driving conditions are also known as Robot Kidnapping Scenarios.  These kidnapping scenarios can be detected and corrected to ensure the robot's success even under abnormal situations.   

Existing methods of Kidnap Detection have two flaws: some methods are built to detect only certain types of Kidnapping scenarios.  For example, a method might be built to detect situations in which a person picks up the robot, but the method is unable to detect that a cat is sitting on the robot.  Other methods require specific sensors or large processing power requirements, which prevents the easily implementation of these methods on just any robot configuration. 

For my thesis research, I developed a method to detect Robot Kidnapping events that is more flexible than existing methods. My detection scheme is designed to combat the weaknesses of existing methods by detecting any form of Robot Kidnapping on any robot.  To accomplish this goal, I am re-purposing some of the data that is generated in the normal course of the robot's localization process.  This re-purposing means there are no additional resource or configuration requirements to run this detection method. 

To study this problem, I ran 40 simulations of a robot driving around a room.  In the top example of the slide, the robot was driving under normal circumstances.  As you can see, the algorithm tracking the robot's location represents the robot's possible true location as a cloud of points.  Each point is associated with a confidence measure to represent the likelihood that this position is the robot's true current location.  On the slide, the orange values are the most likely, while the blue values are the least likely.  The main takeaway is that under normal circumstances, the confidences are spread across a spectrum of values. 

The lower section of the slide shows the data immediately following a Kidnapping Event, when I picked up the robot and moved it to a new location.  The localization algorithm naturally flattened out the variation in the confidence values, and this is represented in the plain green cloud on the graph. 

This insight can be translated into the 10-bin histograms as seen on the right side of the slide.  The skewed plot of the Kidnapped example was used to train an artificial neural network function to recognize the skew as an indicator that the robot has been Kidnapped.  This method is 92% accurate and can identify Kidnapping Events within 3 update cycles of the localization algorithm.

Thanks to this system,  cats can continue to enjoy Roombas, and now Roombas can get all of their work done.

ROS Addition: Piece de Resistance

I learned that ROS_INFO and ROS_DEBUG route to a ROS message topic called /rosout.  Knowing that would have saved me so much time!!!!!  Way back at the beginning, when I was trying to make an overlay of the AMCL package, I put a lot of debug statements in the AMCL code and could never figure out where they were being printed.  I later discovered the "rqt" tool, but /rosout is much easier!

The greatest part is that I can log messages to ROS_INFO and record them into my bag file by adding the /rosout topic to the list that my bag file is recording with each data run.  The list of topics being recorded is pretty long now!

This allows me to log the exact time that the robot has been kidnapped, as well as the to and from locations that the robot is being moved to.  Now I have EVERYTHING I need in the bag file - no more need for a separate .txt file to record the kidnap event, like I used to have.

I'm really happy with my apparatus for creating trial data - bag data for the win!

Adjusting the Neural Network Design

Even though my histogram ratio attribute was a good classifier for kidnap activity, I struggled to train my neural net to identify the calculation.  I wasn't doing much on the neural net's internal structure, however - I was letting Matlab handle most of it, and setting the initial layer to 15 neurons.  I probably should have been more strategic about the neural net's layers if I wanted to improve my results.

I learned a valuable lesson by discussing my problem with my research advisor.  The fact that I could perform a calculation on the histogram of the particle filter particle weights and label the results as 1 or 0 means that I'm imposing my own (unnecessary) limits on the problem (that the ratio of the tallest to 2nd tallest histograms is the solution).  If I instead give the neural network the raw histogram bin data, the neural network will have more flexibility to find its own version of the relationship between the histogram bin heights that are assigned to "Kidnap" and "Normal" events.

Now, my dataset consists of 10 variables - the heights of each of the 10 histogram bins that correspond to the weights of the particles in the particle filter.

From Colors to Histograms to Neural Networks - Part 2

In my previous blog post, I explained the method I chose for representing big, uneven histograms that appear when the particle cloud weights at a timestep are divided into 10 bins.  I use the ratio of the tallest histogram bin divided by the 2nd tallest histogram bin.

I calculated this "Histogram Bin Ratio" (HBR) for each timestep and set up my datasets for an artificial neural network in Matlab. 

Experiment 1
My first trial had Inputs of the current timestep's HBR, and outputs of the difference in meters between the robot's ground-truth pose in the Gazebo simulator and the AMCL SLAM algorithm's estimate, for X, Y, and Heading. 

Experiment 1 Dataset:
Inputs:
Ratio (current)

Outputs:
|AMCL_X - Gazebo_X|
|AMCL_Y - Gazebo_Y|
|AMCL_Heading - Gazebo_Heading|

The neural network's outputs were hopelessly off, with MSE's on the scale of 300.


Side Study
I wrote a test for my neural network setup to see if it could be trained to learn a sine function.  I found a working example that used a different Matlab NN library, and ran the same datasets through my code.  My neural network function was able to match the expected results, so I'm confident that my algorithm should be able to learn the kidnapping pattern.


Experiment 2
I scrapped the idea of using the difference between AMCL and Gazebo as the NN outputs.  Instead, I used a 0/1 coding scheme of "not post-kidnapping event" or "post-kidnapping event".  Including all the data after the kidnapping event resulted in MSEs in the 200s.

Experiment 2 Dataset:
Inputs:
Ratio (current)

Outputs:
0 or 1 (kidnapped or not)


Experiment 3
My next change was to cut off the datasets immediately following the kidnaping event's visible effects on the HBR (histogram ratio).  After the kidnapping, the HBR jumps to 5 or 18 or 40, and hovers there for a few more timesteps. Once the HBR gets back to normal, I truncate the dataset.  This worked marginally better - MSEs of the NN performance in the 100s.

Experiment 3 Dataset:
Inputs:
Ratio(current)

Outputs:
0 or 1 (kidnapped or not) but killed the dataset after the kidnapping event.


Experiment 4
Next, I added a history of 2 timestamps to the data inputs, so that the HBRs of the current timestep, the previous and the 2nd previous timesteps were represented.  This drove my NN's MSEs down to 25!


Experiment 3 Dataset:
Inputs:
Ratio(current)
Prev_Ratio
Prev_Prev_Ratio

Outputs:
0 or 1 (kidnapped or not) but killed the dataset after the kidnapping event.




HOWEVER.  Now my NN output matches  the HBR ratios (current timestep's ratio), when I expected it - and theoretically, trained it - to give me a 0 or a 1! 

The goofy part is that I could threshold the NN output around 4, and say anything over that indicates a Kidnapping.  At this rate, I don't need a neural network!

From Colors to Histograms to Neural Networks - Part 1

My previous blog post was about noticing the unique teal coloring of the particle cloud weights following a Kidnap event.  In the time since, I have:

-Used histograms for representing the color change numerically
-Experimented with arranging the datasets to train the neural network
-Tested that my neural network *should* work by training it on a standard sine function
-Found a combination for the neural network that "works" (kinda).

Histograms:

The algorithm behind coloring the particles by weight is this:
1. Come up with a range of color representations.  There should be as many different colors as there are unique weights.
2. Sort the colors from dark to light, and sort the list of unique weights least to greatest.
3. Match each unique weight to a unique color.
4. Apply this mapping to assign a color to each weights in the dataset

I needed a way to numerically recreate the message that the teal color conveys.  I applied a rule and printed out the resulting histograms:

*At each timestep, divide up the data into 10 bins.  Always 10 bins.*

The histograms at each timestep during a "normal" trial looked like this, with some variation:

The histograms following a Kidnapping event and corresponding to the teal particle cloud looked like this:

The next step was preparing the data for the neural network.  I represented the dramatic histogram of the Kidnap event by creating a ratio of the tallest histogram bin at the timestep to the 2nd tallest histogram bin.  Ratios that indicate a Kidnap event are higher (5, 18, 40) while normal ratios are 1 or 2 or less.

Here's a plot of the ratios for the whole duration of the driving trials.  It's bad resolution, but take my word for it: the "sprinkle" outliers at ratios of 20 or above are from the kidnapping trials.  There are even 3 "normal" trials that appear in the outliers, that helped me identify unintentional kidnapping occurrences during these trials.

Note: after explaining the histogram method and showing that it works...I'm still not sure why it works, and why the method appears to work exclusively for kidnapping instances.