Worked from 2017-09-30 to 2020-09-30

🏷 Tags

🚦 Status

Finished

Roboboat is an international robotics competition by robonation where autonomous robotic boats compete in challenges between schools. I’ve been a part of Roboboat since my first year at University and worked on a variety of projects.

In my first year, we’ve participated in the competition in USA, Florida, but unfortunately, we did not qualify to participate due to not being able to accomplish a certain set of challenges. Since then, the University wanted us to be in a better condition for the next time we participate in the competition. Unfortunately, it was the only time we participated in the competition.

Buoy Detection

ML and OpenCV

Early on, I was part of the image detection team where we toyed with OpenCV and Tensorflow to first find possible buoys with OpenCV. We then used Tensorflow (with a trained machine learning model) to recognize if the image was a buoy or not.

As this was during my first year of University, it was not easy. Although the steps to create a machine learning model to recognize certain object seemed easy:

* Gather images of the relevant object
* Feed it to Tensorflow

It’s the first part that took a little while. There are many available sets of images online, but none of buoys and the quantity needed to create a machine learning model. We then came up with the idea extracting the images of past competitions’ video recording and other relevant videos, for example this or this . This was the first time I saw that data engineering and dataset creation was a big part of machine learning.

I downloaded all the relevant videos and extracted frames from each videos at a certain interval (every 1-4 seconds) to have the images of the competition. Once we got frames extracted, we manually cropped the buoys out of the images. We collectively cropped almost 2,000 images from 11 different videos. Our dataset was then composed of:

* 648 - Background
* 640 - Big Buoys
* 622 - Small Buoys

For anyone wanting the dataset, here is a link to it. This is fairly a small dataset. We used some techniques to multiply our dataset by flipping the images and altering the saturation, noise, resolution, hue, and brightness. This produced a dataset around 12,000 images which is not a bad amount.

When running this in a real-life scenario, there were some problems. The OpenCV part was not working correctly and the computer on the boat was not that strong to keep up with everything.

Only OpenCV

Here we removed the machine learning part and only kept the OpenCV processing. While OpenCV is very sensitive to light, we were able to tweak our OpenCV parameters using an UI built in Python showing a preview of detected buoys on a set of images.

We also had different presets based on the luminosity of the area for OpenCV to detect most of the buoys, regardless of the light.

Software/Team Lead

During my final year at Roboboat, I was the software lead in charge of administrating bi-weekly meetings with other software members of the team, where we would discuss about the teams progress, challenges, distribute tasks, and collaboratively work on some tasks together.

When the team lead at the time left, I decided to take on the role of team lead while also being the software lead and working on image detection. Unfortunately, as time progressed, there was less interest and implication from the part of students (also with COVID hitting around that timeframe).

ROS Dashboard

This is my first iteration of a ROS (Robotic Operating System) dashboard, which precedes the version that I made for Mars Rover . It was also my first time creating a React web application (since I’ve only been using React in the past to create mobile application with React Native).

It allowed us to monitor via a continuous graph of sensors information (such as the temperature of the boat), view the camera, and see an overview of ROS’s information (such as ROS parameters, topics, and nodes). It gave a general overview of the status of our boat from land.

Website

I’ve created their promotional website to showcase what Roboboat is to prospective student members and sponsors.

Open in a new tab

Testing the boat

Once we had a good base of the boat (software, electrical, and mechanical wise), we tested it in the water, more specifically in the University pool. We had to test the boat in the water to see if it was stable, if the motors were working correctly, and if the buoy detection was working correctly.