We made this little autonomous vehicle using BeagleBone AI-64 for our ELEC 553 final project at Rice university to learn how we could use embedded systems and gain hands-on experience in real practical applications.
The car is able to detect the lanes and execute the lane-keeping algorithms. As a result, the car could run maintaining between two lanes. When the car detects a red or stops sign during lane keeping, the car would stop and run the image recognition. After that, the car would back to run and stop at the final red box.
For speed control, the car uses an encoder that is installed on the shaft that turns when the wheels turn. We have implemented a Linux driver that uses interrupts to read the encoder, and we calculate the speed in Python using two readings from the encoder. That allows us to set a target speed and implement a software controller to achieve the target speed.
Our work draws are built based on the following instructable:
● User raja_961, “Autonomous Lane-Keeping Car Using Raspberry Pi and OpenCV”. Instructables. URL: https://www.instructables.com/Autonomous-Lane-Keeping-Car-Using-Raspberry-Pi-and/
● The Yolo prediction URL: https://towardsdatascience.com/yolo-object-detection-with-opencv-and-python-21e50ac599e9
● Other references from a past ELEC 424 project URL: https://www.hackster.io/really-bad-idea/autonomous-path-following-car-6c4992The Resolution, Proportional gain, Derivative gain
In order to let BeagleBone process the images from the camera as soon as possible, which meant that we had to lower the resolution. We used 320x240 resolution, with the instructions:
For the steering PD controller, the proportional and derivative parameters were adjusted until we found a value that was responsive enough and let the car follow the lane. We tested this in two tracks, having success in both, even in the one that has a wider lane, which is the one shown in the video. The final parameters set were:
KP = 0.09
KD = kp*0.1Stop Box Detection
After reading the image from OpenCV, the image will filter out the color that is not considered red, and calculate the remaining part. If the remaining red part proportion is over a certain percentage, the image is considered red.Stop Sign Detection
1. For stop sign detection, we use the Yolo algorithm since the DNN module is part of the OpenCV package.
2. The Yolo algorithms need three files to predict the image. yolov3.weights is the parameter that transforms input data to the result which is our prediction. yolov3.txt contains all the objects the weights are able to predict and yolov3.cfg describes the layout of the network.
3. In the laptop the prediction for the stop sign is instant. However, when we move the same algorithms to the BeagleBone AI-64. The process takes nearly 13 seconds. As a result, the algorithms slow down the whole lane-keeping algorithm.
4. In order to work around this situation and separate the prediction and lane-keeping, the car will stop by red in red box or stop sign, and the stop will give algorithms enough time to extract the image and run the image detection algorithms. As a result, if the image is a stop sign, the car will recognize the image and stop for a while and continue moving.Plots
Two plots have been generated, in both the X-axis belongs to the frame number captured by the installed camera.
The following plot shows the values from the Proportional-Derivative controller that changes the steering of the car, along with the error signal that this PD control system receives.
This next plot shows the equivalent PWM values (in the percentage of duty cycle, around 8%), that the code is sending to the motor speed controller.
Additionally, the action of our speed control which is governed by small step increments and decrements can be shown. The encoder installed in the car sends the values to get the current speed, and the code increments or decrements the equivalent percentage sent to the motor controller to get near the target speed. The blue line shows that.
The orange line shows the steering signal that is sent to the steering servo motor, which varies between 6% and 9% depending on the correction needed to keep the car inside the lanes.