For my Master's project, I worked under Dr. Mohammad Biglarbegian in the Intelligent and Autonomous Control for Vehicles (AICV) Lab at the University of Guelph. In this role, I led a research project that investigated the novel application of stochastic deep neural networks for path planning of mobile robots.

When this project was initially planned, there was supposed to be multiple MASc. and PhD. students working on it, but in the end I was the only grad student involved with the project. I led this large project which was funded by the Canada's National Research Council (NRC). Not only did I learn a lot of valuable technical skills, but I also strengthened my time management, organization and communication skills during the course of the project. My specific contributions to the project are explained below.

Summary

• Learning to develop various types of deep neural networks using PyTorch/Python

• Using Python to develop novel Generative Adversarial Network (GAN) and Long Short-Term Memory (LSTM) based stochastic planners that show promising results when planning paths on unseen maps

• Becoming proficient with ROS (C++/Python), and designing a trajectory tracking controller using ROS to test planned paths on simulated and real mobile robots

• Applying managerial and teamwork skills while I oversaw two undergraduate summer students who assisted me with research

• Further developing my technical writing skills by writing technical reports, progress updates, journal papers, and working on my thesis dissertation

• GitHub Repos for the project: NDM-GAN and S-LSTM

Project Details

Background

Path planners for mobile robots can be divided into two broad categories, global and local planners. Global planners assume that an environment map is known ahead of time, and then generate a path around obstacles, allowing the robot to traverse around known obstacles. Local planners can respond to dynamic or previously unseen obstacles by modifying the globally-planned path so that the robot can also traverse these new, previously unseen obstacles.

The most common type of global planners are sampling-based, which means they must first completely sample each possible point on a map to determine if the point is in free space or if it is an obstacle. The most common sampling-based global planners are Dijkstra's Algorithm, A*, and RRT*.  RRT* was developed in 2011 and is still considered a good global planner. Other than sampling-based planners, there are also many biologically-inspired planners, as well as optimization-based planners.

Since RRT* is a very common, and frequently used global planner, it was selected as the baseline for this project. Therefore, the goal of the project was to use neural networks in some way to create a planner that can perform at the same level or better than RRT* (in planning speed and planning success rate).

While there are certainly many neural network-based planners already developed, stochastic neural network-based planners are less common. A stochastic neural network is a type of neural network where random variations are introduced within the structure of the network, each time data is propagated through it. This property allows the neural network to produce varying results, even when given the exact same input. Therefore, the thesis idea for this project was to apply stochastic neural networks as path planners, so that it could be determined if the stochastic properties allowed the planners to generalize better, generate unique paths, and learn faster without mode collapse.

Literature Review

In the beginning of the project, I prepared a comprehensive literature review to completely understand the state of the field, and what deep learning technologies were currently in use, and where there would be room to improve. I researched sampling-based planners, biologically-inspired planners, and optimization-based planners.

Experimental Setup

The example environment which was created for this project was a 64x64 pixel map tensor, with varying obstacle densities. The free space was represented by zeros, and the obstacles were represented by ones. An example map is shown to the right. Maps of varying densities were created, from 20% obstacle density to 40%. Representing the maps in this way allows for the first of two path planners developed, NDM-GAN to digest and then eventually output an entire path as a two dimensional tensor (an image) rather than iteratively point by point.

Gazebo Simulations

Next I had to strengthen my ROS and Gazebo simulator skills, so that I would be able to develop and test code on a simulated and real RB-Kairos AMR, which was provided by NRC.

The RB-Kairos robot runs on ROS melodic, so I used Ubuntu 18.04 with this version of ROS and Gazebo to create the simulation environment. I modified an already available simulation package for the RB-Kairos to allow me to use Gazebo to test planners on the robot in my own environments.

I also used Gazebo to create a variety of test environments with varying sizes and containing various obstacles so that the robot could be tested on a variety of conditions. Later in the project, these Gazebo environments were also used to test the robot controllers that were developed.

Path Planner Design: NDM-GAN

Noise, Displacement, Map - GAN (NDM-GAN) was the first planner developed in this project. It is based off of the Generative Adversarial Network (GAN). It uses a three-layered tensor as input, which contains randomly generated Noise, a line connecting the desired start and goal points (Displacement), and the Map.

NDM-GAN is unique since it treats the map space as an image. Since GANs can generate an output image (path) in a single prediction, rather than generating point by point. Also unique is the inclusion of a stochastic element (noise) in a path planner, meaning that each time a path is generated between two points, it will always be unique.

The first layer of the input tensor for the generator is "Noise". Randomly generated noise is used as a part of the input to make the neural network stochastic. As mentioned previously, having stochastic elements in the network means that for a certain input, a different output can be generated each time. The goal of the project was to determine if including stochastic elements in the generator will allow the planner to generalize better, and learn faster (and with less data) than a non-stochastic planner.

The second layer fed to the generator is "Displacement", in the form of a direct line (made of ones) connecting the start and goal points. The motivation for this layer is to help introduce a notion of connectivity between the start and goal points.

The third layer fed to the generator  is the Map, where zeros represent free space and ones represent obstacles. These three layers are combined into a 3x64x64 tensor, which is then fed into the generator.

The input to the discriminator is similar to the input to the generator, except the first noise layer is replaced with either a real or "generated" path.

The structure of the discriminator and generator are visualized and included in two images on the right.

The Generator and Discriminator compete in an adversarial game, as proposed by the original GAN paper. In this game, the generator learns to produce better output (paths, in this case), and the discriminator learns to better guess whether a given path has come from the training dataset, or if it is a fake made by the generator.  Over time, the generated "fakes" from the generator start to look closer and closer to the true samples of the training set.

Path Planner Design: S-LSTM

Stochastic-LSTM (S-LSTM) is a stochastic LSTM-based planner. S-LSTM is based on another LSTM-based path planner, but then dropout is used to add a stochastic element. Dropout is when nodes within a neural network are randomly zeroed when data is propagated through the network. Each time, different nodes are dropped out. Additionally, the percentage of the nodes which are dropped out can be adjusted to achieve the desired results. 

Like mentioned previously, the benefits of including this stochastic element in the planner mean that each time a path is generated, the planner will give a new path. Therefore, if the generated path is not sufficient or complete, a new path can be generated. Additionally, having the stochastic elements means that during training, the neural-based planner is less likely to experience mode collapse.

The structure of S-LSTM is shown to the right. The first two LSTM layers process the map, as well as the current and goal point seperately. Then these two outputs from the first two layers are concatenated and fed through two final LSTM layers. These two final layers are the ones that include dropout. After being processed through these final two layers, a final, fully-connected layer of two nodes is used to make the output be of size (1,2), which gives the coordinates of the next predicted point.

Unlike NDM-GAN which runs a single time, producing a path in one shot, S-LSTM runs iteratively and produces a path point-by-point. It runs until points generated are sufficiently close to the goal point.

Controller Development

Two controllers were developed in order to control the robot so that it could follow the paths generated by NDM-GAN and S-LSTM. A Sliding Mode Controller (SMC), and a Linear PI Trajectory Tracking Controller were developed. The sliding mode controller is nonlinear, meaning in general, it should be well suited for controlling a nonlinear mechatronic system. 

The SMC developed was made to control wheel velocities, whereas the PI controller was a trajectory tracking controller. For this reason, the PI controller was most suitable for making the robot follow the planned paths. Additionally, the PI controller worked suffiently well in simulatitions so it was selected to control the physical robot. Two images on the right show two actual and desired paths traversed by the robot in simulation, proving that the PI controller is sufficient for controlling the real robot.

ROS Package Development

The two controllers developed require a series of waypoints to be sent to them as a series of waypoints. Packages such as follow_waypoints already exist in ROS to make a robot traverse a series of waypoints. However, this package caused the robot to jerkily move from waypoint to waypoint. As such, the PI controller mentioned in the previous section was developed as a ROS package.  The GitHub repo with the ROS implementation of this controller is located here: https://github.com/AICV-UofGuelph/aicv_ros_controllers

Results

The results are fully explained in my MASc. Thesis listed on the University of Guelph's Website (under embargo until February 2024). To summarize, the best performing S-LSTM model planned paths successfully 83.63% of the time on unseen maps. The best performing NDN-GAN model planned paths  successfully 93.40% of the time on unseen maps. This performance varied based on what obstacle density was present on the dataset maps. However, in general, these are pleasing results. An example path generated by NDM-GAN and S-LSTM are shown to the right. As in seen in these images, both planners are able to generate successful, collision-free paths in certain instances. 

A big benefit of using NDM-GAN and S-LSTM is the increased planning speed compared to RRT*. As mentioned previously, since RRT* is a sampling-based method, it needs to generate an occupancy grid for any map before it can generate a path. This is a timely process. After training, NDM-GAN and S-LSTM can generate paths without this need to generate a costly occupancy map. This means that NDM-GAN can generate paths up to 44x faster than RRT*, and S-LSTM can generate paths up to 379x faster!

The downside of these methods is what was mentioned previously, that they do not generate a path successfully 100% of the time. A possible solution to this problem which should be investigated in future work is the possibility of using re-planning. This means that NDM-GAN or S-LSTM could be run multiple times until a collision free path is generated. Since these planners are much faster than RRT*, they still have the possibility of being faster than RRT*.

A general qualitative trend in the output seen from both NDN-GAN, and S-LSTM is that the stochastic elements certainly allowed the output paths generated to be unique compared to the training paths. As is noted in detail in my thesis, the paths generated by these planners are often further from obstacles and more direct than the same training paths. This shows that these two stochastic neural network-based planners are learning to generalize well (not copy the training data) as well as learning the notion of connectivenss between start and goal points.

Future work should involve comparing the training times and characteristics of NDM-GAN and S-LSTM against similar but non-stochastic planners to determine if adding stochastic elements also helps to increase training speed of efficiceny.

In my final year capstone project, our group of Computer, Systems and Computing, and Biomedical engineering students embarked on a project that was centred on the distinguishing of beer cans from pop cans at the City of Guelph's Materials Recovery Facility (MRF). The MRF employs workers to manually sort beer cans from pop cans, as the beer cans can be returned to the Beer Store under Ontario's Deposit Return Program for 10 cents/can, whereas they are only worth about 2 cents as aluminum scrap. The MRF operator had a desire to automate the sorting process.

Our group created a machine learning model based on Detectron2 that was able to correctly identify returnable and non-returnable cans with over 99% accuracy. My specific contributions to this project were:

• Designing, building and installing the machine vision device in the MRF (Raspberry Pi, camera, lighting, fabrication of chassis)

• Adjusting settings in code to collect thousands of clear, blur-free images

• Downloading images from site and annotating cans of interest for use in training

• Writing code to train and evaluate the Convolutional Neural Network

This work became a conference paper, in which I was the primary author.

During my time on the University of Guelph Robotics Team (UGRT), I have hosted a few workshops. These workshops were designed to give basic technical skills to our team members, as well as introduce the wider University of Guelph engineering community to our team.

The first workshop I hosted was a soldering workshop where participants learned to solder by making a 555 Timer LED flasher circuit on perfboard. The second soldering workshop I hosted was targeted more to beginners, where the participants soldered up their own LED flashlights. The most recent workshop I hosted was a PCB CAD workshop, where participants learned to use KiCAD software to design and layout circuit boards. These workshops were well received, and helped increase enrolment on UGRT.

Lancebotics is the VEX high school robotics team, which I founded with a few of my friends when we were in grade 10. Initially, the team was formed to allow us to compete with a robot we built for a grade 10 computer class. We all enjoyed this experience and carried the team on after our class ended. 

From grade 10 to grade 12, we grew the team to over two dozen members, and were very successful. We were provincial finalists in 2015 and 2016. We went on to compete in the VEX World Championships in Louisville, Kentucky both years.

Our legacy lives on. The team is still very active and successful. They have qualified for World Championships numerous times since I left the team.