This project is to build an autonomous vehicle using computer vision to follow a lane, stop at red, and stop at a stop sign (with ML). The project builds off of Raja _961's project which is on RPi3, and we use a similar setup with BeagleBone AI-64.
User raja_961, “Autonomous Lane-Keeping Car Using Raspberry Pi and OpenCV”. Instructables. URL: https://www.instructables.com/Autonomous-Lane-Keeping-Car-Using-Raspberry-Pi-and/
The lane to follow is blue and will stop at the stop box and stop sign for approx. 1.5 seconds. The stop boxes are constructed with red paper on the ground and the stop sign is the same as the real world stop sign. Our system utilizes machine learning to identify the real world stop sign.2 Methodology2.1 Resolution
The resolution of the camera is set to 320x240 pixels which is the second lowest resolution possible for the given camera for faster processing time. This resolution is enough to track lanes and identify the stop boxes on the ground.2.2 PID Controller
The speed and steering of the vehicle is controlled with two different types of PID controller. When choosing the p, i, and d values, we first start with a lower value and test on multiple trials and get the value.2.2.1 Speed Control
The speed calculated with the measured time intervals from the speed encoder interrupts. For speed control, the controller resides in Speed class. It uses incremental PID which uses recent three error values to calculate the change needed to correct the system. This type of PID is more robust and makes the speed more steady. The p value is chosen at 0.05, I at 0.01 and p at 0.01.2.2.2 Steering Control
The direction is calculated by the angle of the lane lines (left and right). Left is the average of negative angles and right is the average of positive angles. The current direction is the sum of left and right. For steering control, the controller resides in Direction class. This uses the location PID that uses previous, current, and cumulative error to correct the system. For PID value selection, the p chosen at 60, I and d are both at 0.2.3 Handling Stops
There are two types of stops to handle: stop box and stop sign.2.3.1 Stop Box
The stop box is identified as a red box on the ground within the lane. The red color is identified in the lower half of the vision with HSV color space. The vehicle will stop when red is over the 50% of the lower half vision.
Figure below shows the change in red pixels detected overtime. There are three peaks: the stop sign, the first red box and the final red box. The red threshold is set to 21600 which is beyond the stop sign peak to only identify stop boxes.
The stop sign is handled by the YOLOv5 algorithm. The smallest model in the family, YOLOv5n6 is chosen. The model is a TFLite model in float point 16 for better performance on a small device. When training the model, we used a road signs dataset and actual images of the stop sign on the ground and the stop boxes images make the model differentiate the stop sign from the stop box. The confidence is set to >0.5 to differentiate the stop sign.2.4 Plot2.4.1 PWM over time
The PWM shown is the duty cycle in percentage for the motor.
There are two different PID uses as shown below.
Note: frames when vehicle is stopped are ignored in speed PID.3 Outcome