Learning-based control system for energy management in advanced vehicles

Advanced vehicles (AVs) are intended to improve vehicle safety by reducing accidents. But the demand for advanced vehicles will expand, increasing fuel consumption and emissions. AVs’ operating demand, energy consumption, and emissions are part of a complex system of many interrelated components. The collective behaviour of these elements results in the emergence of properties that cannot easily, if at all, be inferred from the properties of the individual components alone.AVs’ operating demand, energy consumption, and emissions are part of a complex system of many interrelated components. The complexity of these systems arises from the continuous fluctuation of all parameters. Controlling this complex system requires an intelligent control strategy capable of dealing with sporadic disturbances, nonlinearity, coupling, chaotic behaviour, and uncertainty. This thesis focuses on developing an Intelligent Energy Management Control System (IEMCS) that improves the fuel efficiency of AVs and reduces emissions through modelling and simulation. AVs, including conventional, electrical, and hybrid electric vehicles, will utilize gas, battery, and hybrid energy to meet the on-road power demand.   The on-road power demand must be considered based on driver behaviour, dynamical environmental conditions and vehicle specifications (DEV). The IEMCS utilizes the on-road power demand model to reduce fuel consumption and emissions. Since electric vehicles can be considered hybrid electric vehicles without engine power, this thesis constructs two IEMCS models for conventional vehicles (CVs) and hybrid electric vehicles (HEVs).   The initial results from fuzzy types I and II show that, while they can intelligently control CVs and HEVs and be used as a benchmark, they cannot communicate effectively with dynamic DEVs. Reinforcement learning (RL) algorithms aim to produce a correct answer or maximize cumulative rewards, indicating an excellent decision to execute in an unknown environment. However, there are probabilistic problems associated with them. The IEMCS CV model is constructed using RL algorithms, including Q-learning and Multi-Armed Bandit algorithms. Therefore, a novel RL algorithm based on Q-learning has been designed for CAVs to generate the required engine torque to meet the online RPD of the vehicle by controlling the air-fuel ratio, thereby improving energy efficiency. The novel RL Q-learning algorithm results show that it is a real-time self-adaptive controller that can optimally and adaptively reduce vehicle fuel consumption with relatively low operation time compared to fuzzy controllers. In the next step, a new RL algorithm called Multi-Armed Bandit (MAB) is developed to generate optimal torque for the engine and reduce CO2 and NOx emissions through multi-objective optimization based on scalarization functions simultaneously. The RL-MAB integrates with a Support Vector Machine (SVM) to classify input data for the RL algorithm, reducing operation time and modulating mass airflow to the cylinders by changing the angle of the throttle plate. The results are compared with a fuzzy controller as a benchmark, and the comparison shows that the new RL algorithm is highly promising. The IEMCS model for hybrid electric vehicles is constructed using the RL algorithm. A Self-adaptive Q-learning controller is designed for HEV to create an adaptive online energy management system. The Q-learning algorithm optimises the engine performance area, considering the battery’s State Of Charge (SOC). The cumulative reward for the Q-learning agent has been designed based on SOC and optimal engine area. By controlling the throttle angle and torque of the engine, the novel RL algorithm reduces fuel consumption and keeps SOC as stable as possible The operation time is relatively shorter, and fuel consumption is lower than the benchmark (fuzzy logic controllers I and II). In vehicle control strategy, the short operation time is crucial, and the learning-based controllers show significant results for this purpose. This thesis demonstrates that reinforcement learning methods through function approximation can produce algorithms capable of discovering effective policies for intelligent energy management systems to reduce fuel consumption and emissions without sacrificing the AV’s performance. Additionally, we demonstrate that Self-adaptive reinforcement learning through Multi-objective MAB algorithms can be naturally extended to solve and control complex systems that conventional machine learning techniques cannot address.