Closed-loop Flow Control Method Based on Deep Reinforcement Learning using a Co-flow Jet

Document Type : Regular Article

Authors

National Key Laboratory of Science and Technology on Aerodynamic Design and Research, Northwestern Polytechnical University, Xi’an, 710072, China

Abstract

A closed-loop control framework is developed for the co-flow jet (CFJ) airfoil by combining the numerical flow field environment of a CFJ0012 airfoil with a deep reinforcement learning (DRL) module called tensorforce integrated in Python. The DRL agent, which is trained through interacting with the numerical flow field environment, is capable of acquiring a policy that instructs the mass flow rate of the CFJ to make the stalled airfoil at an angle of attack (AoA) of 18 degrees reach a specific high lift coefficient set to 2.0, thereby effectively suppressing flow separation on the upper surface of the airfoil. The subsequent test shows that the policy can be implemented to find a precise jet momentum coefficient of 0.049 to make the lift coefficient of the CFJ0012 airfoil reach 2.01 with a negligible error of 0.5%. Moreover, to evaluate the generalization ability of the policy trained at an AoA of 18 degrees, two additional tests are conducted at AoAs of 16 and 20 degrees. The results show that, although using the policy gained under another AoA cannot help the lift coefficient of the airfoil reach a set target of 2 accurately, the errors are acceptable with less than 5.5%, which means the policy trained under an AoA of 18 degrees can also be applied to other AoAs to some extent. This work is helpful for the practical application of CFJ technology, as the closed-loop control framework ensures good aerodynamic performance of the CFJ airfoil, even in complex and changeable flight conditions. 

Keywords

Main Subjects

References

Ahuja, S., & Rowley, C. W. (2010). Feedback control of unstable steady states of flow past a flat plate using reduced-order estimators. Journal of Fluid Mechanics, 645, 447–478. https://doi.org/10.1017/S0022112009992655
Akbiyik, H., & Yavuz, H. (2021). Artificial neural network application for aerodynamics of an airfoil equipped with plasma actuators. Journal of Applied Fluid Mechanics, 14(4), 1165–1181. https://doi.org/10.47176/jafm.14.04.32133
Franklin, G. F., Powell, J. D., & Emami-Naeini, A. (2019). Feedback Control of Dynamic Systems (8th ed.). Boston, MA: Pearson.
Gan, W. B., Zhou, Z., Xu, X. P., & Wang, R. (2013). Delayed detached-eddy simulation and application of a coflow jet airfoil at high angle of attack. Applied Mechanics and Materials, 444–445, 270–276. https://doi.org/10.4028/www.scientific.net/AMM.444-445.270
Glauser, M., Higuchi, H., Ausseur, J., Pinier, J., & Carlson, H. (2004, June 28). Feedback control of separated flows. 2nd AIAA Flow Control Conference. 2nd AIAA Flow Control Conference, Portland, Oregon. https://doi.org/10.2514/6.2004-2521
Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. 2017 IEEE International Conference on Robotics and Automation (ICRA), 3389–3396. https://doi.org/10.1109/ICRA.2017.7989385
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Pérez, P. (2022). Deep reinforcement learning for autonomous driving: a survey. IEEE Transactions on Intelligent Transportation Systems, 23(6), 4909–4926. https://doi.org/10.1109/TITS.2021.3054625
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. https://api.semanticscholar.org/CorpusID:195908774
Lefebvre, A. M., & Zha, G. (2014, June 16). Co-flow jet airfoil trade study part I: Energy consumption and aerodynamic efficiency. 32nd AIAA Applied Aerodynamics Conference, Atlanta, GA. https://doi.org/10.2514/6.2014-2682
Lei, Z., & Zha, G. (2021). Lift enhancement of supersonic thin airfoil at low speed by co-flow jet active flow control. AIAA AVIATION 2021 FORUM. https://doi.org/10.2514/6.2021-2591
Lou, J., Chen, R., Liu, J., Bao, Y., You, Y., & Chen, Z. (2023). Aerodynamic optimization of airfoil based on deep reinforcement learning. Physics of Fluids, 35(3), 037128. https://doi.org/10.1063/5.0137002
Ma, C. Y., & Xu, H. Y. (2022). Parameter-based design and analysis of wind turbine airfoils with conformal slot co-flow jet. Journal of Applied Fluid Mechanics, 16(2), 269–283. https://doi.org/10.47176/jafm.16.02.1318
Ma, C. Y., Xu, H. Y., & Qiao, C. L. (2023). Comparative study of two combined blowing and suction flow control methods on pitching airfoils. Physics of Fluids, 35(3), 035120. https://doi.org/10.1063/5.0138962
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013, December 19). Playing atari with deep reinforcement learning. ArXiv.Org. https://arxiv.org/abs/1312.5602v1
Moshtaghzadeh, M., & Aligoodarz, M. R. (2022). Prediction of wind turbine airfoil performance using artificial neural network and CFD approaches. International Journal of Engineering and Technology Innovation, 12(4), 275–287. https://doi.org/10.46604/ijeti.2022.9735
Paris, R., Beneddine, S., & Dandois, J. (2021). Robust flow control and optimal sensor placement using deep reinforcement learning. Journal of Fluid Mechanics, 913, A25. https://doi.org/10.1017/jfm.2020.1170
Rabault, J., & Kuhnle, A. (2019). Accelerating deep reinforcement learning strategies of flow control through a multi-environment approach. Physics of Fluids, 31(9), 094105. https://doi.org/10.1063/1.5116415
Rabault, J., Kuchta, M., Jensen, A., Réglade, U., & Cerardi, N. (2019). Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control. Journal of Fluid Mechanics, 865, 281–302. https://doi.org/10.1017/jfm.2019.62
Ren, F., Rabault, J., & Tang, H. (2021a). Applying deep reinforcement learning to active flow control in weakly turbulent conditions. Physics of Fluids, 33(3), 037121. https://doi.org/10.1063/5.0037371
Ren, F., Wang, C., & Tang, H. (2021b). Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth. Physics of Fluids, 33(9), 093602. https://doi.org/10.1063/5.0060690
Samy, I., Postlethwaite, I., Gu, D. W., & Green, J. (2010). Neural-Network-Based flush air data sensing system demonstrated on a mini air vehicle. Journal of Aircraft, 47(1), 18–31. https://doi.org/10.2514/1.44157
Schulman, J., Levine, S., Abbeel, P., Jordan, M., & Moritz, P. (2015). Trust region policy optimization. Proceedings of the 32nd International Conference on Machine Learning. https://proceedings.mlr.press/v37/schulman15.htm
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017, July 20). Proximal Policy Optimization Algorithms. ArXiv.Org. https://arxiv.org/abs/1707.06347v2
Shimomura, S., Sekimoto, S., Oyama, A., Fujii, K., & Nishida, H. (2020). Closed-Loop flow separation control using the deep q network over airfoil. AIAA Journal, 58(10), 4260–4270. https://doi.org/10.2514/1.J059447
Siegel, S., Cohen, K., & McLaughlin, T. (2003, June 23). Feedback control of a circular cylinder wake in experiment and simulation (Invited). 33rd AIAA Fluid Dynamics Conference and Exhibit, Orlando, Florida. https://doi.org/10.2514/6.2003-3569
Silver, D., Huang, A., Maddison, C. Guez, A., Sifre, L., Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Kalchbrenner, N., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489. https://doi.org/10.1038/nature16961
Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3(1), 9–44. https://doi.org/10.1007/BF00115009
Tang, H., Rabault, J., Kuhnle, A., Wang, Y., & Wang, T. (2020). Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning. Physics of Fluids, 32(5), 053605. https://doi.org/10.1063/5.0006492
Wang, B., & Zha, G. C. (2011). Detached-Eddy simulation of a coflow jet airfoil at high angle of attack. Journal of Aircraft, 48(5), 1495–1502. https://doi.org/10.2514/1.C000282
Wang, B., Haddoukessouni B., Levy, J., & Zha, G. C. (2008). Numerical investigations of injection-slot-size effect on the performance of coflow jet airfoils. Journal of Aircraft, 45(6), 2084-2091. https://doi.org/10.2514/1.37441
Xu, K., & Zha, G. (2021). High control authority three-dimensional aircraft control surfaces using coflow jet. Journal of Aircraft, 58(1), 72-84. https://doi.org/10.2514/1.C035727
Yang, Y., & Zha, G. (2019, January 7). Conceptual design of the co-flow jet hybrid electric regional airplane. AIAA Scitech 2019 Forum. AIAA Scitech 2019 Forum, San Diego, California. https://doi.org/10.2514/6.2019-1584
Zha, G. C., Paxton, C. D., Conley, C. A., Wells, A., & Carroll, B. F. (2006a). Effect of injection slot size on the performance of coflow jet airfoil. Journal of aircraft, 43(4), 987-995. https://arc.aiaa.org/doi/abs/10.2514/1.16999
Zha, G., Gao W., & Paxton, C. (2006b, January 9). Numerical simulation of co-flow jet airfoil flows. 44th AIAA Aerospace Sciences Meeting and Exhibit. Reno, Nevada. https://doi.org/10.2514/6.2006-1060
Zha, G., Yang, Y., Ren, Y., & McBreen, B. (2018, June 25). Super-lift and thrusting airfoil of coflow jet actuated by micro-compressors. 2018 Flow Control Conference, Atlanta, Georgia. https://doi.org/10.2514/6.2018-3061
Zha, G. C., & Paxton, C. (2004, June 28). A novel airfoil circulation augment flow control method using CFJ. 2nd AIAA Flow Control Conference, Portland, Oregon. https://doi.org/10.2514/6.2004-2208
Zha, G. C., Carroll, B. F., Paxton, C. D., Conley, C. A., & Wells, A. (2007a). High-performance airfoil using coflow jet flow control. AIAA Journal, 45(8), 2087–2090. https://doi.org/10.2514/1.20926
Zha, G. C., Gao, W., & Paxton, C. D. (2007b). Jet effects on coflow jet airfoil performance. AIAA Journal, 45(6), 1222–1231. https://doi.org/10.2514/1.23995

Files

History

  • Received: 17 August 2023
  • Revised: 29 October 2023
  • Accepted: 18 November 2023
  • Available online: 30 January 2024

Share

How to cite

Statistics