Using hardware components that were generously lent to us by Rice University's Ryon lab as well as a full-blown race course (blue tape pasted on the lab floor) we built an autonomous, lane-keeping, traffic-sign-detecting, speed-maintaining (RC) car. Our core code driving the car, is a modified version of existing Python scripts from raja_961 and from the group ReaLly_BaD_Idea. Using their code as a stepping stone we improved in various aspects such as smoothing out the steering, incorporating stop sign detection and porting it to the more powerful BeagleBone AI-64.
Components & Apps
Two webcams are attached to the front of the car, one for tracking the lane and the other to detect stop signs. They are both connected to the BeagleBone AI-64 via a USB hub. A USB WiFi adaptor is also attached to the BeagleBone to enable ssh access and remotely handle the actions of the car. OpenCV was used to enable lane-keeping and traffic sign detection.
Using a webcam and the OpenCV library, we detected lane edges (blue tape on the floor) which then enabled us to programmatically adjust our car's direction. Using the lowest possible resolution (160, 120) turned out to be completely feasible for lane keeping as well as efficient for our algorithm's runtime. We modified pre-existing turning code to make the car turn smoothly instead of just completely left or right (values 6 or 9). After a little bit of tuning using both plots as well as visually assessing the performance of the car itself, we choose a P value of 0.07 for proportional gain and a D value of 0.02 for derivative gain.
Here are the plots of our PD and PWM:
Traffic Sign Detection & Stopping
Our system is also capable of recognizing a stop sign and coming to a temporary halt before continuing. We found that a second webcam raised slightly higher than the first enables us to detect the sign from an appropriate distance. Using just a single camera led to issues performing both stop sign detection and lane-keeping. We used the pre-trained Yolo-tiny model since it proved to be much more performant than the complete version and fed it images through OpenCV. In order to keep our car running smoothly we spaced out the amount of times inference was performed to every 3 runs of our core loop. It is important to note that since we were a 3-man team we were absolved of the requirement of stopping at the red boxes placed on the floor.
We also attempt to maintain stable speed by using a speed encoder. This is enabled by our driver gpiod_driver.c which measures the time between inputs of the encoder and communicates them to our main.py file for processing. Our core file then adjusts the speed accordingly. The logic behind our code involves a translation between the timings provided by the speed encoder and the speed of the car. With a little bit of testing we derived a value (encoder_target_rotation) which we were happy with speed-wise. Whenever our encoder code segment runs we make adjustments to the car's speed in order to get closer to the target rotation. Much like traffic sign detection we also employ the trick of running this code segment every three runs of the core loop for efficiency reasons.
Performing both speed encoding and traffic light detection at the same time proved very difficult. Employing tricks like reducing the amount of times speed is adjusted and image recognition is performed led to issues in lane keeping as well as not detecting the stop sign. This is why we have added some handy booleans at the top of the file for enabling/disabling the various car functionalities.
We also found ourselves constantly retuning various parameters (P, D, base_speed, base_encoder_timing) as the batteries died and recharged for the purpose of lane-keeping. Unfortunately we couldn't find a reliable way of running the code at various battery charge levels and ensuring the same performance with the same parameters.
 User raja_961, “Autonomous Lane-Keeping Car Using Raspberry Pi and OpenCV”. Instructables. URL: https://www.instructables.com/Autonomous-Lane-Keeping-Car-U sing-Raspberry-Pi-and/
 Team ReaLly BaD Idea, "Autonomous path following car". URL: https://www.hackster.io/really-bad-idea/autonomous-path-following-car-6c4992
 fredotran, "traffic-sign-detector-yolov4". URL:https://github.com/fredotran/traffic-sign-detector-yolov4